Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- 1,2019-01-01,2019-01-01,1,2
- 1,2019-01-01,2019-01-02,3,4
- 1,2019-01-01,2019-01-03,5,6
- 2,2019-01-01,2019-01-01,1,2
- 2,2019-01-01,2019-01-02,3,4
- 2,2019-01-01,2019-01-03,5,6
- results/
- client_id=1/
- report_date=2019-01-01
- <<somename>>.csv
- client_id=2/
- report_date=2019-01-01
- <<somename>>.csv
- df.repartition(2, "customer_id", "report_date")
- .sortWithinPartitions("date", "value1")
- .write.partitionBy("customer_id", "report_date")
- .csv(...)
Add Comment
Please, Sign In to add comment