Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- final_columns = ['colA', 'colB', 'colC', 'colD', 'colE']
- try:
- sdf = sqlContext.table('test_table')
- except:
- print("test_table is empty")
- mySchema = StructType([ StructField("colA", StringType(), True),
- StructField("colB", StringType(), True),
- StructField("colC", StringType(), True),
- StructField("colD", StringType(), True),
- StructField("colE", DoubleType(), True) ])
- sdf = sqlContext.createDataFrame(spark.sparkContext.emptyRDD(),schema=mySchema)
- sdf = sdf.select(final_columns)
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement