Guest User

Untitled

a guest
Jun 20th, 2018
280
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.66 KB | None | 0 0
  1. var some_df = Seq(
  2. ("A", "no"),
  3. ("B", "yes"),
  4. ("B", "yes"),
  5. ("B", "no")
  6. ).toDF(
  7. "user_id", "phone_number")
  8.  
  9. +-------+------------+
  10. |user_id|phone_number|
  11. +-------+------------+
  12. | A| no|
  13. | B| yes|
  14. | B| yes|
  15. | B| no|
  16. +-------+------------+
  17.  
  18. val omg_window = Window.partitionBy($"user_id")
  19. some_df = some_df.withColumn("my_col",
  20. count($"phone_number"==="yes").over(omg_window)).drop("phone_number")
  21. some_df.show()
  22.  
  23. +-------+------+
  24. |user_id|my_col|
  25. +-------+------+
  26. | B| 3|
  27. | B| 3|
  28. | B| 3|
  29. | A| 1|
  30. +-------+------+
  31.  
  32. User Id . my_col
  33. B . 2
  34. A . 0
Add Comment
Please, Sign In to add comment