daily pastebin goal
47%
SHARE
TWEET

Untitled

a guest Jan 19th, 2019 69 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. 1,2019-01-01,2019-01-01,1,2
  2. 1,2019-01-01,2019-01-02,3,4
  3. 1,2019-01-01,2019-01-03,5,6
  4. 2,2019-01-01,2019-01-01,1,2
  5. 2,2019-01-01,2019-01-02,3,4
  6. 2,2019-01-01,2019-01-03,5,6
  7.    
  8. results/
  9.    client_id=1/
  10.       report_date=2019-01-01
  11.         <<somename>>.csv
  12.    client_id=2/
  13.       report_date=2019-01-01
  14.         <<somename>>.csv
  15.    
  16. df.repartition(2, "customer_id", "report_date")
  17.   .sortWithinPartitions("date", "value1")
  18.   .write.partitionBy("customer_id", "report_date")
  19.   .csv(...)
RAW Paste Data
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy. OK, I Understand
 
Top