Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- >>> df.show()
- +--------------+-------+-------+-------+
- | Column1|Column2|Column3|Column4|
- +--------------+-------+-------+-------+
- |(617)-283-3811| Salah| Messi| null|
- | 617-2833811| Messi| Virgil| Messi|
- | 617-283-3811|Ronaldo| Messi|Ronaldo|
- +--------------+-------+-------+-------+
- +--------------+---------+
- | Column1|something|
- +--------------+---------+
- |(617)-283-3811| true|
- | 617-2833811| true|
- | 617-283-3811| true|
- +--------------+---------+
- >>> df = spark.createDataFrame(row_seq)
- >>> df2 = df.select("Column2", validate_format_udf("Column2").alias("something"))
- >>> df2.show()
- +-------+---------+
- |Column2|something|
- +-------+---------+
- | Salah| false|
- | Messi| false|
- |Ronaldo| false|
- +-------+---------+
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement