Advertisement
Guest User

Untitled

a guest
Jun 26th, 2019
87
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.60 KB | None | 0 0
  1. final_columns = ['colA', 'colB', 'colC', 'colD', 'colE']
  2.  
  3. try:
  4. sdf = sqlContext.table('test_table')
  5. except:
  6. print("test_table is empty")
  7. mySchema = StructType([ StructField("colA", StringType(), True),
  8. StructField("colB", StringType(), True),
  9. StructField("colC", StringType(), True),
  10. StructField("colD", StringType(), True),
  11. StructField("colE", DoubleType(), True) ])
  12. sdf = sqlContext.createDataFrame(spark.sparkContext.emptyRDD(),schema=mySchema)
  13.  
  14. sdf = sdf.select(final_columns)
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement