daily pastebin goal
32%
SHARE
TWEET

Untitled

a guest Jun 20th, 2018 108 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. var some_df = Seq(
  2. ("A", "no"),
  3. ("B", "yes"),
  4. ("B", "yes"),
  5. ("B", "no")
  6. ).toDF(
  7. "user_id", "phone_number")
  8.    
  9. +-------+------------+
  10. |user_id|phone_number|
  11. +-------+------------+
  12. |      A|          no|
  13. |      B|         yes|
  14. |      B|         yes|
  15. |      B|          no|
  16. +-------+------------+
  17.    
  18. val omg_window = Window.partitionBy($"user_id")
  19. some_df = some_df.withColumn("my_col",
  20. count($"phone_number"==="yes").over(omg_window)).drop("phone_number")
  21. some_df.show()
  22.    
  23. +-------+------+
  24.  |user_id|my_col|
  25.  +-------+------+
  26.  |      B|     3|
  27.  |      B|     3|
  28.  |      B|     3|
  29.  |      A|     1|
  30.  +-------+------+
  31.    
  32. User Id .   my_col
  33.  B .         2
  34.  A .         0
RAW Paste Data
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy. OK, I Understand
 
Top