Advertisement
Guest User

Untitled

a guest
Jun 20th, 2019
80
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.30 KB | None | 0 0
  1. from pyspark import SparkContext
  2.  
  3. sc = SparkContext.getOrCreate()
  4.  
  5. result = sc.textFile( "users.csv" ) \
  6. .map(lambda x: (x.split('|')[3],1) ) \
  7. .filter( lambda x: x[0] != 'other' ) \
  8. .reduceByKey( lambda x,y:x+y ) \
  9. .sortBy( lambda x: -x[1] ).collect()
  10.  
  11. for line in result:
  12. print line
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement