Advertisement
Guest User

Untitled

a guest
Jun 15th, 2019
74
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 3.52 KB | None | 0 0
  1. [Stage 3:==================> (7 + 15) / 22]19/06/01 20:46:30 ERROR TaskSetManager: Total size of serialized results of 18 tasks (1030.8 MB) is bigger than spark.driver.maxResultSize (1024.0 MB)
  2. 19/06/01 20:46:30 ERROR TaskSetManager: Total size of serialized results of 19 tasks (1087.7 MB) is bigger than spark.driver.maxResultSize (1024.0 MB)
  3. 19/06/01 20:46:30 ERROR TaskSetManager: Total size of serialized results of 20 tasks (1145.6 MB) is bigger than spark.driver.maxResultSize (1024.0 MB)
  4. 19/06/01 20:46:30 ERROR TaskSetManager: Total size of serialized results of 21 tasks (1203.5 MB) is bigger than spark.driver.maxResultSize (1024.0 MB)
  5. 19/06/01 20:46:30 ERROR TaskSetManager: Total size of serialized results of 22 tasks (1261.4 MB) is bigger than spark.driver.maxResultSize (1024.0 MB)
  6. [Stage 3:====================================> (14 + 8) / 22]org.apache.spark.SparkException: Job aborted due to stage failure: Total size of serialized results of 18 tasks (1030.8 MB) is bigger than spark.driver.maxResultSize (1024.0 MB)
  7. at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1599)
  8. at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1587)
  9. at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1586)
  10. at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
  11. at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
  12. at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1586)
  13. at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:831)
  14. at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:831)
  15. at scala.Option.foreach(Option.scala:257)
  16. at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:831)
  17. at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1820)
  18. at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1769)
  19. at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1758)
  20. at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
  21. at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:642)
  22. at org.apache.spark.SparkContext.runJob(SparkContext.scala:2034)
  23. at org.apache.spark.SparkContext.runJob(SparkContext.scala:2131)
  24. at org.apache.spark.rdd.RDD$$anonfun$fold$1.apply(RDD.scala:1092)
  25. at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
  26. at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
  27. at org.apache.spark.rdd.RDD.withScope(RDD.scala:363)
  28. at org.apache.spark.rdd.RDD.fold(RDD.scala:1086)
  29. at org.apache.spark.rdd.RDD$$anonfun$treeAggregate$1.apply(RDD.scala:1155)
  30. at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
  31. at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
  32. at org.apache.spark.rdd.RDD.withScope(RDD.scala:363)
  33. at org.apache.spark.rdd.RDD.treeAggregate(RDD.scala:1131)
  34. at org.apache.spark.sql.execution.stat.StatFunctions$.multipleApproxQuantiles(StatFunctions.scala:102)
  35. at org.apache.spark.sql.DataFrameStatFunctions.approxQuantile(DataFrameStatFunctions.scala:100)
  36. at org.apache.spark.sql.DataFrameStatFunctions.approxQuantile(DataFrameStatFunctions.scala:75)
  37. ... 56 elided
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement