Advertisement
Guest User

Untitled

a guest
Jul 17th, 2019
95
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.68 KB | None | 0 0
  1. {"key": { "defaultWeights":"1" }, "measures": { "m1":-0.01, "m2":-0.5.....}}
  2.  
  3. val flattened = dff.withColumn("default_weights", json_tuple(col("key"), "defaultWeights")).show
  4.  
  5. org.apache.spark.sql.AnalysisException: cannot resolve 'json_tuple(`key`, 'defaultWeights')' due to data type mismatch: json_tuple requires that all arguments are strings;;
  6. 'Project [key#6, measures#7, json_tuple(key#6, defaultWeights) AS default_weights#13]
  7. +- Relation[key#6,measures#7] json
  8.  
  9. <console>:25: error: type mismatch;
  10. found : String
  11. required: org.apache.spark.sql.Column
  12. val flattened = dff.withColumn("default_weights", json_tuple("key", "defaultWeights")).show
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement