API tools faq
paste
Advertisement
Guest User

broken.log

a guest
Nov 7th, 2014
244
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Java 33.84 KB | None | 0 0
raw download clone embed print report
  1. 2014-11-07 17:10:23,863 WARN  org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  2. 2014-11-07 17:10:24,815 INFO  com.sirsidynix.Transform - spark.akka.frameSize=500
  3. spark.akka.threads=6
  4. spark.app.name=1d665ef4-com.sirsidynix.Transform
  5. spark.cores.max=1
  6. spark.default.parallelism=32
  7. spark.driver.host=1.1.1.1
  8. spark.driver.port=47200
  9. spark.executor.memory=1G
  10. spark.executor.uri=
  11. spark.fileserver.uri=http://1.1.1.1:58557
  12. spark.httpBroadcast.uri=http://1.1.1.1:39980
  13. spark.master=spark://1.1.1.1:7077
  14. spark.tachyonStore.folderName=spark-15470493-b67b-4a41-a46e-65e74d2277c6
  15. spark.ui.port=0
  16. 2014-11-07 17:10:24,947 INFO  com.sirsidynix.Transform - I am doing symphony work!
  17. 2014-11-07 17:10:34,128 INFO  com.sirsidynix.Transform - spark.akka.frameSize=500
  18. spark.akka.threads=6
  19. spark.app.name=e9a478b0-com.sirsidynix.Transform
  20. spark.cores.max=1
  21. spark.default.parallelism=32
  22. spark.driver.host=1.1.1.1
  23. spark.driver.port=59928
  24. spark.executor.memory=1G
  25. spark.executor.uri=
  26. spark.fileserver.uri=http://1.1.1.1:37663
  27. spark.master=spark://1.1.1.1:7077
  28. spark.tachyonStore.folderName=spark-67075c0b-7284-42a1-8d94-6e92dd920be8
  29. spark.ui.port=0
  30. 2014-11-07 17:10:34,139 INFO  com.sirsidynix.Transform - I am doing symphony work!
  31. 2014-11-07 17:10:37,527 ERROR org.apache.spark.scheduler.TaskSchedulerImpl - Lost executor 0 on 1.1.1.1: remote Akka client disassociated
  32. 2014-11-07 17:10:37,569 WARN  org.apache.spark.scheduler.TaskSetManager - Lost TID 0 (task 0.0:0)
  33. 2014-11-07 17:10:40,927 INFO  com.sirsidynix.sym.transformers.SymCheckout -
  34. Running SymCheckout...
  35.  
  36. 2014-11-07 17:10:40,968 ERROR org.apache.spark.scheduler.TaskSchedulerImpl - Lost executor 1 on 1.1.1.1: remote Akka client disassociated
  37. 2014-11-07 17:10:40,985 WARN  org.apache.spark.scheduler.TaskSetManager - Lost TID 1 (task 0.0:0)
  38. 2014-11-07 17:10:44,444 ERROR org.apache.spark.scheduler.TaskSchedulerImpl - Lost executor 2 on 1.1.1.1: remote Akka client disassociated
  39. 2014-11-07 17:10:44,444 WARN  org.apache.spark.scheduler.TaskSetManager - Lost TID 2 (task 0.0:0)
  40. 2014-11-07 17:10:47,898 ERROR org.apache.spark.scheduler.TaskSchedulerImpl - Lost executor 3 on 1.1.1.1: remote Akka client disassociated
  41. 2014-11-07 17:10:47,898 WARN  org.apache.spark.scheduler.TaskSetManager - Lost TID 3 (task 0.0:0)
  42. 2014-11-07 17:10:47,901 ERROR org.apache.spark.scheduler.TaskSetManager - Task 0.0:0 failed 4 times; aborting job
  43. 2014-11-07 17:10:47,921 ERROR com.sirsidynix.sym.transformers.SymPolicies - Problem running  SymPolicies: Job aborted due to stage failure: Task 0.0:0 failed 4 times, most recent failure: TID 3 on host 1.1.1.1 failed for unknown reason
  44. Driver stacktrace:
  45.     org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1033)
  46.     org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1017)
  47.     org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1015)
  48.     scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
  49.     scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
  50.     org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1015)
  51.     org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:633)
  52.     org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:633)
  53.     scala.Option.foreach(Option.scala:236)
  54.     org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:633)
  55.     org.apache.spark.scheduler.DAGSchedulerEventProcessActor$$anonfun$receive$2.applyOrElse(DAGScheduler.scala:1207)
  56.     akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
  57.     akka.actor.ActorCell.invoke(ActorCell.scala:456)
  58.     akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
  59.     akka.dispatch.Mailbox.run(Mailbox.scala:219)
  60.     akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
  61.     scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  62.     scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  63.     scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  64.     scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  65.  
  66. 2014-11-07 17:10:50,941 INFO  com.sirsidynix.util.ImpLoader$ - Drop and recreate Impala table: checkout with newly transformed data in hdfs://1.1.1.1:8020/jarvis/parquet/67fc0cb1-f220-482a-8e91-c6d669aae84d/checkout Impala IP = 1.1.1.1 dbName = abcdefg
  67.  
  68. 2014-11-07 17:10:51,236 ERROR org.apache.spark.scheduler.TaskSchedulerImpl - Lost an executor 4 (already removed): remote Akka client disassociated
  69. 2014-11-07 17:10:51,984 INFO  com.sirsidynix.util.ImpLoader$ - [?1034h
  70.  
  71.  
  72. 2014-11-07 17:10:52,697 INFO  com.sirsidynix.util.ImpLoader$ - [?1034h
  73.  
  74.  
  75. 2014-11-07 17:10:53,179 ERROR com.sirsidynix.sym.transformers.SymCheckout - Problem running checkout: Nonzero exit value: 1
  76.     scala.sys.package$.error(package.scala:27)
  77.     scala.sys.process.ProcessBuilderImpl$AbstractBuilder.slurp(ProcessBuilderImpl.scala:131)
  78.     scala.sys.process.ProcessBuilderImpl$AbstractBuilder.$bang$bang(ProcessBuilderImpl.scala:101)
  79.     com.sirsidynix.util.ImpLoader$.loadImpParquet(ImpLoader.scala:16)
  80.     com.sirsidynix.sym.transformers.SymCheckout.loadImpalaParquet(SymCheckout.scala:74)
  81.     com.sirsidynix.sym.transformers.SymCheckout.doRun(SymCheckout.scala:57)
  82.     com.sirsidynix.sym.SymTransform$.run(SymTransform.scala:26)
  83.     com.sirsidynix.Transform.runJob(Transform.scala:49)
  84.     spark.jobserver.JobManagerActor$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$4.apply(JobManagerActor.scala:218)
  85.     scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
  86.     scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
  87.     akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42)
  88.     akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
  89.     scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  90.     scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  91.     scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  92.     scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  93.  
  94. 2014-11-07 17:10:53,183 INFO  com.sirsidynix.sym.transformers.SymUser -
  95. Running SymUser...
  96.  
  97. 2014-11-07 17:10:54,844 ERROR org.apache.spark.scheduler.TaskSchedulerImpl - Lost an executor 5 (already removed): remote Akka client disassociated
  98. 2014-11-07 17:10:56,982 INFO  com.sirsidynix.sym.transformers.SymCheckout -
  99. Running SymCheckout...
  100.  
  101. 2014-11-07 17:10:57,014 ERROR com.sirsidynix.sym.transformers.SymCheckout - Problem running checkout: null
  102.     com.sirsidynix.sym.transformers.SymCheckout.doRun(SymCheckout.scala:24)
  103.     com.sirsidynix.sym.SymTransform$.run(SymTransform.scala:26)
  104.     com.sirsidynix.Transform.runJob(Transform.scala:49)
  105.     spark.jobserver.JobManagerActor$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$4.apply(JobManagerActor.scala:218)
  106.     scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
  107.     scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
  108.     akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42)
  109.     akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
  110.     scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  111.     scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  112.     scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  113.     scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  114.  
  115. 2014-11-07 17:10:57,026 INFO  com.sirsidynix.sym.transformers.SymUser -
  116. Running SymUser...
  117.  
  118. 2014-11-07 17:10:57,049 ERROR com.sirsidynix.sym.transformers.SymUser - Problem running SymUsers: null
  119.     com.sirsidynix.sym.transformers.SymUser.doRun(SymUser.scala:26)
  120.     com.sirsidynix.sym.SymTransform$.run(SymTransform.scala:28)
  121.     com.sirsidynix.Transform.runJob(Transform.scala:49)
  122.     spark.jobserver.JobManagerActor$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$4.apply(JobManagerActor.scala:218)
  123.     scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
  124.     scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
  125.     akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42)
  126.     akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
  127.     scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  128.     scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  129.     scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  130.     scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  131.  
  132. 2014-11-07 17:10:57,063 INFO  com.sirsidynix.sym.transformers.SymUserAddress -
  133. Running SymUserAddress...
  134.  
  135. 2014-11-07 17:10:57,074 ERROR com.sirsidynix.sym.transformers.SymUserAddress - Problem
  136. java.lang.NullPointerException
  137.     at com.sirsidynix.sym.transformers.SymUserAddress.doRun(SymUserAddress.scala:25)
  138.     at com.sirsidynix.sym.SymTransform$.run(SymTransform.scala:30)
  139.     at com.sirsidynix.Transform.runJob(Transform.scala:49)
  140.     at spark.jobserver.JobManagerActor$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$4.apply(JobManagerActor.scala:218)
  141.     at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
  142.     at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
  143.     at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42)
  144.     at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
  145.     at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  146.     at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  147.     at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  148.     at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  149. 2014-11-07 17:10:57,077 ERROR com.sirsidynix.sym.transformers.SymUserAddress - Problem running SymUserAddress: null
  150.     com.sirsidynix.sym.transformers.SymUserAddress.doRun(SymUserAddress.scala:25)
  151.     com.sirsidynix.sym.SymTransform$.run(SymTransform.scala:30)
  152.     com.sirsidynix.Transform.runJob(Transform.scala:49)
  153.     spark.jobserver.JobManagerActor$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$4.apply(JobManagerActor.scala:218)
  154.     scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
  155.     scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
  156.     akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42)
  157.     akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
  158.     scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  159.     scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  160.     scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  161.     scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  162.  
  163. 2014-11-07 17:10:57,094 WARN  spark.jobserver.RddManagerActor - Shutting down spark.jobserver.RddManagerActor
  164. 2014-11-07 17:10:57,094 WARN  spark.jobserver.JobStatusActor - Shutting down spark.jobserver.JobStatusActor
  165. 2014-11-07 17:11:01,992 WARN  org.apache.spark.scheduler.TaskSetManager - Lost TID 2 (task 36.0:0)
  166. 2014-11-07 17:11:02,001 WARN  org.apache.spark.scheduler.TaskSetManager - Loss was due to java.net.ConnectException
  167. java.net.ConnectException: Connection refused
  168.     at java.net.PlainSocketImpl.socketConnect(Native Method)
  169.     at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
  170.     at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
  171.     at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
  172.     at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
  173.     at java.net.Socket.connect(Socket.java:579)
  174.     at java.net.Socket.connect(Socket.java:528)
  175.     at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
  176.     at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
  177.     at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
  178.     at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
  179.     at sun.net.www.http.HttpClient.New(HttpClient.java:308)
  180.     at sun.net.www.http.HttpClient.New(HttpClient.java:326)
  181.     at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:996)
  182.     at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:932)
  183.     at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:850)
  184.     at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1300)
  185.     at org.apache.spark.broadcast.HttpBroadcast$.read(HttpBroadcast.scala:196)
  186.     at org.apache.spark.broadcast.HttpBroadcast.readObject(HttpBroadcast.scala:89)
  187.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  188.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  189.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  190.     at java.lang.reflect.Method.invoke(Method.java:606)
  191.     at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
  192.     at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
  193.     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  194.     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  195.     at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  196.     at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  197.     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  198.     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  199.     at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  200.     at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  201.     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  202.     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  203.     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
  204.     at scala.collection.immutable.$colon$colon.readObject(List.scala:362)
  205.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  206.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  207.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  208.     at java.lang.reflect.Method.invoke(Method.java:606)
  209.     at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
  210.     at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
  211.     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  212.     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  213.     at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  214.     at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  215.     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  216.     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  217.     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
  218.     at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:63)
  219.     at org.apache.spark.scheduler.ShuffleMapTask$.deserializeInfo(ShuffleMapTask.scala:63)
  220.     at org.apache.spark.scheduler.ShuffleMapTask.readExternal(ShuffleMapTask.scala:135)
  221.     at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1837)
  222.     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
  223.     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  224.     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
  225.     at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:63)
  226.     at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:85)
  227.     at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:169)
  228.     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  229.     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  230.     at java.lang.Thread.run(Thread.java:745)
  231. 2014-11-07 17:11:02,018 WARN  org.apache.spark.scheduler.TaskSetManager - Lost TID 3 (task 37.0:0)
  232. 2014-11-07 17:11:02,023 WARN  org.apache.spark.scheduler.TaskSetManager - Loss was due to java.net.ConnectException
  233. java.net.ConnectException: Connection refused
  234.     at java.net.PlainSocketImpl.socketConnect(Native Method)
  235.     at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
  236.     at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
  237.     at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
  238.     at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
  239.     at java.net.Socket.connect(Socket.java:579)
  240.     at java.net.Socket.connect(Socket.java:528)
  241.     at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
  242.     at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
  243.     at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
  244.     at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
  245.     at sun.net.www.http.HttpClient.New(HttpClient.java:308)
  246.     at sun.net.www.http.HttpClient.New(HttpClient.java:326)
  247.     at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:996)
  248.     at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:932)
  249.     at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:850)
  250.     at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1300)
  251.     at org.apache.spark.broadcast.HttpBroadcast$.read(HttpBroadcast.scala:196)
  252.     at org.apache.spark.broadcast.HttpBroadcast.readObject(HttpBroadcast.scala:89)
  253.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  254.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  255.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  256.     at java.lang.reflect.Method.invoke(Method.java:606)
  257.     at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
  258.     at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
  259.     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  260.     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  261.     at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  262.     at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  263.     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  264.     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  265.     at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  266.     at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  267.     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  268.     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  269.     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
  270.     at scala.collection.immutable.$colon$colon.readObject(List.scala:362)
  271.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  272.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  273.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  274.     at java.lang.reflect.Method.invoke(Method.java:606)
  275.     at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
  276.     at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
  277.     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  278.     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  279.     at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  280.     at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  281.     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  282.     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  283.     at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  284.     at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  285.     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  286.     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  287.     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
  288.     at scala.collection.immutable.$colon$colon.readObject(List.scala:362)
  289.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  290.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  291.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  292.     at java.lang.reflect.Method.invoke(Method.java:606)
  293.     at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
  294.     at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
  295.     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  296.     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  297.     at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  298.     at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  299.     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  300.     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  301.     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
  302.     at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:63)
  303.     at org.apache.spark.scheduler.ShuffleMapTask$.deserializeInfo(ShuffleMapTask.scala:63)
  304.     at org.apache.spark.scheduler.ShuffleMapTask.readExternal(ShuffleMapTask.scala:135)
  305.     at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1837)
  306.     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
  307.     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  308.     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
  309.     at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:63)
  310.     at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:85)
  311.     at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:169)
  312.     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  313.     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  314.     at java.lang.Thread.run(Thread.java:745)
  315. 2014-11-07 17:11:02,034 WARN  org.apache.spark.scheduler.TaskSetManager - Lost TID 4 (task 36.0:0)
  316. 2014-11-07 17:11:02,055 WARN  org.apache.spark.scheduler.TaskSetManager - Lost TID 5 (task 37.0:0)
  317. 2014-11-07 17:11:02,078 WARN  org.apache.spark.scheduler.TaskSetManager - Lost TID 6 (task 36.0:0)
  318. 2014-11-07 17:11:02,099 WARN  org.apache.spark.scheduler.TaskSetManager - Lost TID 7 (task 37.0:0)
  319. 2014-11-07 17:11:02,123 WARN  org.apache.spark.scheduler.TaskSetManager - Lost TID 8 (task 36.0:0)
  320. 2014-11-07 17:11:02,123 ERROR org.apache.spark.scheduler.TaskSetManager - Task 36.0:0 failed 4 times; aborting job
  321. 2014-11-07 17:11:02,130 ERROR com.sirsidynix.sym.transformers.SymUser - Problem running SymUsers: Job aborted due to stage failure: Task 36.0:0 failed 4 times, most recent failure: Exception failure in TID 8 on host 1.1.1.1: java.net.ConnectException: Connection refused
  322.         java.net.PlainSocketImpl.socketConnect(Native Method)
  323.         java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
  324.         java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
  325.         java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
  326.         java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
  327.         java.net.Socket.connect(Socket.java:579)
  328.         java.net.Socket.connect(Socket.java:528)
  329.         sun.net.NetworkClient.doConnect(NetworkClient.java:180)
  330.         sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
  331.         sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
  332.         sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
  333.         sun.net.www.http.HttpClient.New(HttpClient.java:308)
  334.         sun.net.www.http.HttpClient.New(HttpClient.java:326)
  335.         sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:996)
  336.         sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:932)
  337.         sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:850)
  338.         sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1300)
  339.         org.apache.spark.broadcast.HttpBroadcast$.read(HttpBroadcast.scala:196)
  340.         org.apache.spark.broadcast.HttpBroadcast.readObject(HttpBroadcast.scala:89)
  341.         sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  342.         sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  343.         sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  344.         java.lang.reflect.Method.invoke(Method.java:606)
  345.         java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
  346.         java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
  347.         java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  348.         java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  349.         java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  350.         java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  351.         java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  352.         java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  353.         java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  354.         java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  355.         java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  356.         java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  357.         java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
  358.         scala.collection.immutable.$colon$colon.readObject(List.scala:362)
  359.         sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  360.         sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  361.         sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  362.         java.lang.reflect.Method.invoke(Method.java:606)
  363.         java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
  364.         java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
  365.         java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  366.         java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  367.         java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
  368.         java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
  369.         java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
  370.         java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  371.         java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
  372.         org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:63)
  373.         org.apache.spark.scheduler.ShuffleMapTask$.deserializeInfo(ShuffleMapTask.scala:63)
  374.         org.apache.spark.scheduler.ShuffleMapTask.readExternal(ShuffleMapTask.scala:135)
  375.         java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1837)
  376.         java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
  377.         java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
  378.         java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
  379.         org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:63)
  380.         org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:85)
  381.         org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:169)
  382.         java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  383.         java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  384.         java.lang.Thread.run(Thread.java:745)
  385. Driver stacktrace:
  386.     org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1033)
  387.     org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1017)
  388.     org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1015)
  389.     scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
  390.     scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
  391.     org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1015)
  392.     org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:633)
  393.     org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:633)
  394.     scala.Option.foreach(Option.scala:236)
  395.     org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:633)
  396.     org.apache.spark.scheduler.DAGSchedulerEventProcessActor$$anonfun$receive$2.applyOrElse(DAGScheduler.scala:1207)
  397.     akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
  398.     akka.actor.ActorCell.invoke(ActorCell.scala:456)
  399.     akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
  400.     akka.dispatch.Mailbox.run(Mailbox.scala:219)
  401.     akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
  402.     scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  403.     scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  404.     scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  405.     scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  406.  
  407. 2014-11-07 17:11:02,134 INFO  com.sirsidynix.sym.transformers.SymUserAddress -
  408. Running SymUserAddress...
  409.  
  410. 2014-11-07 17:11:02,176 ERROR com.sirsidynix.sym.transformers.SymUserAddress - Problem
  411. java.lang.NullPointerException
  412.     at org.apache.spark.broadcast.HttpBroadcast$.write(HttpBroadcast.scala:167)
  413.     at org.apache.spark.broadcast.HttpBroadcast.<init>(HttpBroadcast.scala:57)
  414.     at org.apache.spark.broadcast.HttpBroadcastFactory.newBroadcast(HttpBroadcastFactory.scala:35)
  415.     at org.apache.spark.broadcast.HttpBroadcastFactory.newBroadcast(HttpBroadcastFactory.scala:29)
  416.     at org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:62)
  417.     at org.apache.spark.SparkContext.broadcast(SparkContext.scala:776)
  418.     at org.apache.spark.rdd.NewHadoopRDD.<init>(NewHadoopRDD.scala:72)
  419.     at org.apache.spark.SparkContext.newAPIHadoopFile(SparkContext.scala:626)
  420.     at com.sirsidynix.util.BaseTransformer$class.readParquetFile(BaseTransformer.scala:23)
  421.     at com.sirsidynix.sym.transformers.SymUserAddress.readParquetFile(SymUserAddress.scala:16)
  422.     at com.sirsidynix.sym.transformers.SymUserAddress.buildUserAddress(SymUserAddress.scala:114)
  423.     at com.sirsidynix.sym.transformers.SymUserAddress.doRun(SymUserAddress.scala:26)
  424.     at com.sirsidynix.sym.SymTransform$.run(SymTransform.scala:30)
  425.     at com.sirsidynix.Transform.runJob(Transform.scala:49)
  426.     at spark.jobserver.JobManagerActor$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$4.apply(JobManagerActor.scala:218)
  427.     at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
  428.     at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
  429.     at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42)
  430.     at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
  431.     at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  432.     at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  433.     at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  434.     at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  435. 2014-11-07 17:11:02,188 ERROR com.sirsidynix.sym.transformers.SymUserAddress - Problem running SymUserAddress: null
  436.     org.apache.spark.broadcast.HttpBroadcast$.write(HttpBroadcast.scala:167)
  437.     org.apache.spark.broadcast.HttpBroadcast.<init>(HttpBroadcast.scala:57)
  438.     org.apache.spark.broadcast.HttpBroadcastFactory.newBroadcast(HttpBroadcastFactory.scala:35)
  439.     org.apache.spark.broadcast.HttpBroadcastFactory.newBroadcast(HttpBroadcastFactory.scala:29)
  440.     org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:62)
  441.     org.apache.spark.SparkContext.broadcast(SparkContext.scala:776)
  442.     org.apache.spark.rdd.NewHadoopRDD.<init>(NewHadoopRDD.scala:72)
  443.     org.apache.spark.SparkContext.newAPIHadoopFile(SparkContext.scala:626)
  444.     com.sirsidynix.util.BaseTransformer$class.readParquetFile(BaseTransformer.scala:23)
  445.     com.sirsidynix.sym.transformers.SymUserAddress.readParquetFile(SymUserAddress.scala:16)
  446.     com.sirsidynix.sym.transformers.SymUserAddress.buildUserAddress(SymUserAddress.scala:114)
  447.     com.sirsidynix.sym.transformers.SymUserAddress.doRun(SymUserAddress.scala:26)
  448.     com.sirsidynix.sym.SymTransform$.run(SymTransform.scala:30)
  449.     com.sirsidynix.Transform.runJob(Transform.scala:49)
  450.     spark.jobserver.JobManagerActor$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$4.apply(JobManagerActor.scala:218)
  451.     scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
  452.     scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
  453.     akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42)
  454.     akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
  455.     scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  456.     scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  457.     scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  458.     scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  459.  
  460. 2014-11-07 17:11:02,191 WARN  spark.jobserver.RddManagerActor - Shutting down spark.jobserver.RddManagerActor
  461. 2014-11-07 17:11:02,191 WARN  spark.jobserver.JobStatusActor - Shutting down spark.jobserver.JobStatusActor
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!