Advertisement
Guest User

Untitled

a guest
Dec 9th, 2016
117
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.96 KB | None | 0 0
  1. Item_sale_table
  2. item_id date Sale Amount
  3. aaa 3-11 20
  4. aaa 3-12 21
  5. aaa 3-13 28
  6. ... ... ...
  7. bbb 3-11 17
  8. bbb 3-12 12
  9. ... ... ...
  10. ccc 3-11 9
  11. ... ... ...
  12.  
  13. item_id description
  14. aaa xxxx
  15. bbb xxxyx
  16. ccc zxsa
  17. ...
  18.  
  19. def ItemProcess (item_id: String, Dataset: DataFrame) = {
  20.  
  21. val item_count = Dataset.filter(Dataset("item_id").contains(item_id)).count()
  22.  
  23. println(item_id,item_count)
  24.  
  25. }
  26.  
  27. val item_count_collection = item_list.select("item_id").foreach(x => ItemProcess(x(0).toString, item_sale_table))
  28.  
  29. ERROR Executor: Exception in task 4.0 in stage 11.0 (TID 504)
  30. java.lang.NullPointerException
  31. at org.apache.spark.sql.DataFrame.resolve(DataFrame.scala:151)
  32. at org.apache.spark.sql.DataFrame.col(DataFrame.scala:664)
  33. at org.apache.spark.sql.DataFrame.apply(DataFrame.scala:652)
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement