API tools faq
paste
Guest User

Spark 150 to 150-cdh550 stacktrace

a guest
May 10th, 2016
226
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 27.25 KB | None | 0 0
raw download clone embed print report
  1. 16/05/10 13:09:50 INFO parse.ParseDriver: Parsing command: select * from t1
  2. 16/05/10 13:09:51 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on 10.1.1.36:45048 in memory (size: 1158.0 B, free: 530.0 MB)
  3. 16/05/10 13:09:51 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on 10.1.8.109:51046 in memory (size: 1158.0 B, free: 132.8 MB)
  4. 16/05/10 13:09:51 INFO parse.ParseDriver: Parse Completed
  5. 16/05/10 13:09:51 INFO spark.ContextCleaner: Cleaned accumulator 1
  6. 16/05/10 13:09:51 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
  7. 16/05/10 13:09:51 INFO storage.MemoryStore: ensureFreeSpace(510720) called with curMem=0, maxMem=555755765
  8. 16/05/10 13:09:51 INFO storage.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 498.8 KB, free 529.5 MB)
  9. 16/05/10 13:09:51 INFO storage.MemoryStore: ensureFreeSpace(43820) called with curMem=510720, maxMem=555755765
  10. 16/05/10 13:09:51 INFO storage.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 42.8 KB, free 529.5 MB)
  11. 16/05/10 13:09:51 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on 10.1.1.36:45048 (size: 42.8 KB, free: 530.0 MB)
  12. 16/05/10 13:09:51 INFO spark.SparkContext: Created broadcast 1 from show at <console>:20
  13. 16/05/10 13:09:52 INFO mapred.FileInputFormat: Total input paths to process : 1
  14. 16/05/10 13:09:52 INFO spark.SparkContext: Starting job: show at <console>:20
  15. 16/05/10 13:09:52 INFO scheduler.DAGScheduler: Got job 1 (show at <console>:20) with 1 output partitions
  16. 16/05/10 13:09:52 INFO scheduler.DAGScheduler: Final stage: ResultStage 1(show at <console>:20)
  17. 16/05/10 13:09:52 INFO scheduler.DAGScheduler: Parents of final stage: List()
  18. 16/05/10 13:09:52 INFO scheduler.DAGScheduler: Missing parents: List()
  19. 16/05/10 13:09:52 INFO scheduler.DAGScheduler: Submitting ResultStage 1 (MapPartitionsRDD[6] at show at <console>:20), which has no missing parents
  20. 16/05/10 13:09:52 INFO storage.MemoryStore: ensureFreeSpace(19712) called with curMem=554540, maxMem=555755765
  21. 16/05/10 13:09:52 INFO storage.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 19.3 KB, free 529.5 MB)
  22. 16/05/10 13:09:52 INFO storage.MemoryStore: ensureFreeSpace(6314) called with curMem=574252, maxMem=555755765
  23. 16/05/10 13:09:52 INFO storage.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 6.2 KB, free 529.5 MB)
  24. 16/05/10 13:09:52 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on 10.1.1.36:45048 (size: 6.2 KB, free: 530.0 MB)
  25. 16/05/10 13:09:52 INFO spark.SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:861
  26. 16/05/10 13:09:52 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 1 (MapPartitionsRDD[6] at show at <console>:20)
  27. 16/05/10 13:09:52 INFO scheduler.TaskSchedulerImpl: Adding task set 1.0 with 1 tasks
  28. 16/05/10 13:09:52 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 1.0 (TID 1, 10.1.8.109, ANY, 2334 bytes)
  29. 16/05/10 13:09:52 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on 10.1.8.109:51046 (size: 6.2 KB, free: 132.8 MB)
  30. 16/05/10 13:09:52 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 1.0 (TID 1, 10.1.8.109): java.io.InvalidClassException: org.apache.spark.sql.catalyst.expressions.AttributeReference; local class incompatible: stream classdesc serialVersionUID = 370695178000872136, local class serialVersionUID = -8877631944444173448
  31. at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617)
  32. at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622)
  33. at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
  34. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
  35. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  36. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  37. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  38. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  39. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  40. at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1706)
  41. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1344)
  42. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  43. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  44. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  45. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  46. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  47. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  48. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  49. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  50. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  51. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  52. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  53. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  54. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  55. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  56. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  57. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  58. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  59. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  60. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  61. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  62. at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1706)
  63. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1344)
  64. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  65. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  66. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  67. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  68. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  69. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  70. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  71. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  72. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  73. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  74. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  75. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  76. at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
  77. at scala.collection.immutable.$colon$colon.readObject(List.scala:362)
  78. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  79. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  80. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  81. at java.lang.reflect.Method.invoke(Method.java:606)
  82. at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
  83. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
  84. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  85. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  86. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  87. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  88. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  89. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  90. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  91. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  92. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  93. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  94. at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
  95. at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:72)
  96. at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:98)
  97. at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)
  98. at org.apache.spark.scheduler.Task.run(Task.scala:88)
  99. at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
  100. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  101. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  102. at java.lang.Thread.run(Thread.java:745)
  103.  
  104. 16/05/10 13:09:52 INFO scheduler.TaskSetManager: Starting task 0.1 in stage 1.0 (TID 2, 10.1.8.109, ANY, 2334 bytes)
  105. 16/05/10 13:09:52 INFO scheduler.TaskSetManager: Lost task 0.1 in stage 1.0 (TID 2) on executor 10.1.8.109: java.io.InvalidClassException (org.apache.spark.sql.catalyst.expressions.AttributeReference; local class incompatible: stream classdesc serialVersionUID = 370695178000872136, local class serialVersionUID = -8877631944444173448) [duplicate 1]
  106. 16/05/10 13:09:52 INFO scheduler.TaskSetManager: Starting task 0.2 in stage 1.0 (TID 3, 10.1.8.109, ANY, 2334 bytes)
  107. 16/05/10 13:09:52 INFO scheduler.TaskSetManager: Lost task 0.2 in stage 1.0 (TID 3) on executor 10.1.8.109: java.io.InvalidClassException (org.apache.spark.sql.catalyst.expressions.AttributeReference; local class incompatible: stream classdesc serialVersionUID = 370695178000872136, local class serialVersionUID = -8877631944444173448) [duplicate 2]
  108. 16/05/10 13:09:52 INFO scheduler.TaskSetManager: Starting task 0.3 in stage 1.0 (TID 4, 10.1.8.109, ANY, 2334 bytes)
  109. 16/05/10 13:09:52 INFO scheduler.TaskSetManager: Lost task 0.3 in stage 1.0 (TID 4) on executor 10.1.8.109: java.io.InvalidClassException (org.apache.spark.sql.catalyst.expressions.AttributeReference; local class incompatible: stream classdesc serialVersionUID = 370695178000872136, local class serialVersionUID = -8877631944444173448) [duplicate 3]
  110. 16/05/10 13:09:52 ERROR scheduler.TaskSetManager: Task 0 in stage 1.0 failed 4 times; aborting job
  111. 16/05/10 13:09:52 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool
  112. 16/05/10 13:09:52 INFO scheduler.TaskSchedulerImpl: Cancelling stage 1
  113. 16/05/10 13:09:52 INFO scheduler.DAGScheduler: ResultStage 1 (show at <console>:20) failed in 0,210 s
  114. 16/05/10 13:09:52 INFO scheduler.DAGScheduler: Job 1 failed: show at <console>:20, took 0,240144 s
  115. org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 4, 10.1.8.109): java.io.InvalidClassException: org.apache.spark.sql.catalyst.expressions.AttributeReference; local class incompatible: stream classdesc serialVersionUID = 370695178000872136, local class serialVersionUID = -8877631944444173448
  116. at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617)
  117. at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622)
  118. at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
  119. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
  120. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  121. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  122. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  123. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  124. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  125. at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1706)
  126. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1344)
  127. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  128. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  129. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  130. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  131. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  132. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  133. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  134. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  135. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  136. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  137. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  138. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  139. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  140. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  141. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  142. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  143. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  144. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  145. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  146. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  147. at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1706)
  148. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1344)
  149. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  150. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  151. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  152. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  153. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  154. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  155. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  156. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  157. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  158. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  159. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  160. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  161. at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
  162. at scala.collection.immutable.$colon$colon.readObject(List.scala:362)
  163. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  164. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  165. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  166. at java.lang.reflect.Method.invoke(Method.java:606)
  167. at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
  168. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
  169. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  170. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  171. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  172. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  173. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  174. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  175. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  176. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  177. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  178. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  179. at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
  180. at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:72)
  181. at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:98)
  182. at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)
  183. at org.apache.spark.scheduler.Task.run(Task.scala:88)
  184. at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
  185. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  186. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  187. at java.lang.Thread.run(Thread.java:745)
  188.  
  189. Driver stacktrace:
  190. at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1280)
  191. at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1268)
  192. at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1267)
  193. at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
  194. at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
  195. at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1267)
  196. at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:697)
  197. at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:697)
  198. at scala.Option.foreach(Option.scala:236)
  199. at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:697)
  200. at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1493)
  201. at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1455)
  202. at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1444)
  203. at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
  204. at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:567)
  205. at org.apache.spark.SparkContext.runJob(SparkContext.scala:1813)
  206. at org.apache.spark.SparkContext.runJob(SparkContext.scala:1826)
  207. at org.apache.spark.SparkContext.runJob(SparkContext.scala:1839)
  208. at org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:215)
  209. at org.apache.spark.sql.execution.Limit.executeCollect(basicOperators.scala:207)
  210. at org.apache.spark.sql.DataFrame$$anonfun$collect$1.apply(DataFrame.scala:1386)
  211. at org.apache.spark.sql.DataFrame$$anonfun$collect$1.apply(DataFrame.scala:1386)
  212. at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:56)
  213. at org.apache.spark.sql.DataFrame.withNewExecutionId(DataFrame.scala:1904)
  214. at org.apache.spark.sql.DataFrame.collect(DataFrame.scala:1385)
  215. at org.apache.spark.sql.DataFrame.head(DataFrame.scala:1315)
  216. at org.apache.spark.sql.DataFrame.take(DataFrame.scala:1378)
  217. at org.apache.spark.sql.DataFrame.showString(DataFrame.scala:178)
  218. at org.apache.spark.sql.DataFrame.show(DataFrame.scala:402)
  219. at org.apache.spark.sql.DataFrame.show(DataFrame.scala:363)
  220. at org.apache.spark.sql.DataFrame.show(DataFrame.scala:371)
  221. at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:20)
  222. at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:25)
  223. at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:27)
  224. at $iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:29)
  225. at $iwC$$iwC$$iwC$$iwC.<init>(<console>:31)
  226. at $iwC$$iwC$$iwC.<init>(<console>:33)
  227. at $iwC$$iwC.<init>(<console>:35)
  228. at $iwC.<init>(<console>:37)
  229. at <init>(<console>:39)
  230. at .<init>(<console>:43)
  231. at .<clinit>(<console>)
  232. at .<init>(<console>:7)
  233. at .<clinit>(<console>)
  234. at $print(<console>)
  235. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  236. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  237. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  238. at java.lang.reflect.Method.invoke(Method.java:498)
  239. at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)
  240. at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1340)
  241. at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)
  242. at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)
  243. at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)
  244. at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)
  245. at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)
  246. at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)
  247. at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:657)
  248. at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:665)
  249. at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$loop(SparkILoop.scala:670)
  250. at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:997)
  251. at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
  252. at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
  253. at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
  254. at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)
  255. at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)
  256. at org.apache.spark.repl.Main$.main(Main.scala:31)
  257. at org.apache.spark.repl.Main.main(Main.scala)
  258. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  259. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  260. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  261. at java.lang.reflect.Method.invoke(Method.java:498)
  262. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:672)
  263. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
  264. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
  265. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120)
  266. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  267. Caused by: java.io.InvalidClassException: org.apache.spark.sql.catalyst.expressions.AttributeReference; local class incompatible: stream classdesc serialVersionUID = 370695178000872136, local class serialVersionUID = -8877631944444173448
  268. at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617)
  269. at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622)
  270. at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
  271. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
  272. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  273. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  274. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  275. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  276. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  277. at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1706)
  278. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1344)
  279. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  280. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  281. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  282. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  283. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  284. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  285. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  286. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  287. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  288. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  289. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  290. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  291. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  292. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  293. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  294. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  295. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  296. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  297. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  298. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  299. at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1706)
  300. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1344)
  301. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  302. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  303. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  304. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  305. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  306. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  307. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  308. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  309. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  310. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  311. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  312. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  313. at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
  314. at scala.collection.immutable.$colon$colon.readObject(List.scala:362)
  315. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  316. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  317. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  318. at java.lang.reflect.Method.invoke(Method.java:606)
  319. at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
  320. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
  321. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  322. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  323. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  324. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  325. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  326. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  327. at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  328. at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  329. at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  330. at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  331. at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
  332. at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:72)
  333. at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:98)
  334. at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)
  335. at org.apache.spark.scheduler.Task.run(Task.scala:88)
  336. at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
  337. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  338. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  339. at java.lang.Thread.run(Thread.java:745)
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!