Advertisement
Guest User

Untitled

a guest
Mar 18th, 2019
80
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.60 KB | None | 0 0
  1. +---+
  2. | tt|
  3. +---+
  4. | a|
  5. | b|
  6. | c|
  7. | ab|
  8. +---+
  9.  
  10. +----+-----+------+
  11. |name|alter|profit|
  12. +----+-----+------+
  13. | a| aa| 1|
  14. | b| a| 5|
  15. | c| ab| 8|
  16. +----+-----+------+
  17.  
  18. dataDF = dataDF.select("*")
  19. .join(broadcast(alterDF),
  20. col("tt") === col("Name") || col("tt") === col("alter"),
  21. "left")
  22.  
  23. +---+----+-----+------+
  24. | tt|name|alter|profit|
  25. +---+----+-----+------+
  26. | a| a| aa| 1|
  27. | a| b| a| 5| // this row is not expected.
  28. | b| b| a| 5|
  29. | c| c| ab| 8|
  30. | ab| c| ab| 8|
  31. +---+----+-----+------+
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement