Advertisement
Guest User

Untitled

a guest
Jul 24th, 2017
47
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.35 KB | None | 0 0
  1. df1 = spark.createDataFrame(rdd1).toDF("id", "val")
  2. df1.createOrReplaceTempView("table1")
  3. df2 = spark.createDataFrame(rdd2).toDF("id", "val")
  4. df2.createOrReplaceTempView("table2")
  5.  
  6. def callable_ssql_timeit():
  7. query="SELECT * FROM table1 LEFT JOIN table2 ON table1.id=table2.id"
  8. spark.sql(query)
  9.  
  10. print(timeit.timeit(callable_ssql_timeit, number=100))
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement