Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- import pandas as pd
- #create a pandas df from some data
- df = pd.read_csv("some/data/test_v2.csv")
- #if SparkContext is not already initialized, run the next line
- #sc = SparkContext(conf=conf)
- #create a SQLContext
- sqlc = SQLContext(sc)
- #pass pandas dataframe into createDataFrame() function of SQLContext
- sdf = sqlc.createDataFrame(df)
Add Comment
Please, Sign In to add comment