Advertisement
Guest User

Untitled

a guest
Mar 28th, 2017
52
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 1.27 KB | None | 0 0
  1. +----+------+--------------------+--------+-------------+
  2. | id | name | end time | value | comment |
  3. ---------------------------------------------------------
  4. |1 |node1 |2017-03-24 08:30:00 | 5 | blabla |
  5. ---------------------------------------------------------
  6. |2 |node1 |2017-03-24 09:00:00 | 3 | blabla |
  7. ---------------------------------------------------------
  8. |3 |node1 |2017-03-24 09:30:00 | 8 | blabla |
  9. ---------------------------------------------------------
  10. |4 |node2 |2017-03-24 10:00:00 | 5 | blabla |
  11. ---------------------------------------------------------
  12. |5 |node2 |2017-03-24 10:30:00 | 3 | blabla |
  13. ---------------------------------------------------------
  14. |6 |node2 |2017-03-24 11:00:00 | 1 | blabla |
  15. ---------------------------------------------------------
  16. |7 |node2 |2017-03-24 11:30:00 | 3 | blabla |
  17. ---------------------------------------------------------
  18. |8 |node2 |2017-03-24 12:00:00 | 5 | blabla |
  19. ---------------------------------------------------------
  20.  
  21. df.groupBy(
  22. col("name"),
  23. window(col("end time"), "2 hour", "30 minute")
  24. )
  25. .agg(max("value").as("2_hour_max_value"))
  26. .filter(col("2_hour_max_value") < 6)
  27. .select("name")
  28. .distinct()
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement