Advertisement
avisrivastava254084

Untitled

Sep 30th, 2019
117
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. >>> df.show()
  2. +--------------+-------+-------+-------+
  3. |       Column1|Column2|Column3|Column4|
  4. +--------------+-------+-------+-------+
  5. |(617)-283-3811|  Salah|  Messi|   null|
  6. |   617-2833811|  Messi| Virgil|  Messi|
  7. |  617-283-3811|Ronaldo|  Messi|Ronaldo|
  8. +--------------+-------+-------+-------+
  9.  
  10. +--------------+---------+
  11. |       Column1|something|
  12. +--------------+---------+
  13. |(617)-283-3811|     true|
  14. |   617-2833811|     true|
  15. |  617-283-3811|     true|
  16. +--------------+---------+
  17.  
  18. >>> df = spark.createDataFrame(row_seq)
  19. >>> df2 = df.select("Column2", validate_format_udf("Column2").alias("something"))
  20. >>> df2.show()
  21. +-------+---------+
  22. |Column2|something|
  23. +-------+---------+
  24. |  Salah|    false|
  25. |  Messi|    false|
  26. |Ronaldo|    false|
  27. +-------+---------+
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement