Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- # CSCI E-63 HW3 - Problem 3
- # Author: Walter Yu
- # Description: PySpark script to connect to MySQL database, register table and display row count.
- # Create context and connect to MySQL:
- sqlContext= SQLContext(sc)
- dfm = sqlc.read.format("jdbc").option("url","jdbc:mysql://localhost/retail_db").option("driver","com.mysql.jdbc.Driver").option("dbtable","departments").option("user","xxxxx").option("password","xxxxx").load()
- # Verify schema, create view and display row count:
- dfm.printSchema()
- dfm.registerTempTable("departments")
- dfm
- dfm.count()
- dfm.collect()
Add Comment
Please, Sign In to add comment