Advertisement
Guest User

Untitled

a guest
Jun 20th, 2019
63
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.35 KB | None | 0 0
  1. from pyspark import SparkContext
  2. from pyspark.sql import SparkSession
  3.  
  4. sc = SparkContext.getOrCreate()
  5. spark = SparkSession(sc)
  6.  
  7. spark.read.load( "users.csv", format="csv", sep="|" ) \
  8. .toDF( "id","age","gender","occupation","zip" ) \
  9. .createOrReplaceTempView( "users" )
  10.  
  11. spark.sql( "select gender, count(*) from users group by gender" ).show()
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement