API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Sep 20th, 2019
99
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Python 6.11 KB | None | 0 0
raw download clone embed print report
  1. Py4JJavaError: An error occurred while calling o178.count.
  2. : org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 4.0 failed 1 times, most recent failure: Lost task 0.0 in stage 4.0 (TID 4, localhost, executor driver): java.io.FileNotFoundException: C:\Users\Miguel\AppData\Local\Temp\blockmgr-45156afc-1550-4f32-a28d-130a473ffb56\0a\shuffle_2_0_0.data.e6f4d8e9-a0dc-46e0-a199-135a0b65ba1c (El sistema no puede encontrar la ruta especificada)
  3.     at java.io.FileOutputStream.open0(Native Method)
  4.     at java.io.FileOutputStream.open(FileOutputStream.java:270)
  5.     at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
  6.     at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.writePartitionedFile(BypassMergeSortShuffleWriter.java:191)
  7.     at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:163)
  8.     at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
  9.     at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
  10.     at org.apache.spark.scheduler.Task.run(Task.scala:123)
  11.     at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
  12.     at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
  13.     at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
  14.     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  15.     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  16.     at java.lang.Thread.run(Thread.java:748)
  17.  
  18. Driver stacktrace:
  19.     at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1889)
  20.     at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1877)
  21.     at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1876)
  22.     at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
  23.     at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
  24.     at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1876)
  25.     at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
  26.     at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
  27.     at scala.Option.foreach(Option.scala:257)
  28.     at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:926)
  29.     at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2110)
  30.     at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2059)
  31.     at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2048)
  32.     at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
  33.     at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:737)
  34.     at org.apache.spark.SparkContext.runJob(SparkContext.scala:2061)
  35.     at org.apache.spark.SparkContext.runJob(SparkContext.scala:2082)
  36.     at org.apache.spark.SparkContext.runJob(SparkContext.scala:2101)
  37.     at org.apache.spark.SparkContext.runJob(SparkContext.scala:2126)
  38.     at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:945)
  39.     at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
  40.     at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
  41.     at org.apache.spark.rdd.RDD.withScope(RDD.scala:363)
  42.     at org.apache.spark.rdd.RDD.collect(RDD.scala:944)
  43.     at org.apache.spark.sql.execution.SparkPlan.executeCollect(SparkPlan.scala:299)
  44.     at org.apache.spark.sql.Dataset$$anonfun$count$1.apply(Dataset.scala:2836)
  45.     at org.apache.spark.sql.Dataset$$anonfun$count$1.apply(Dataset.scala:2835)
  46.     at org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3370)
  47.     at org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:78)
  48.     at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125)
  49.     at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73)
  50.     at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3369)
  51.     at org.apache.spark.sql.Dataset.count(Dataset.scala:2835)
  52.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  53.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  54.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  55.     at java.lang.reflect.Method.invoke(Method.java:498)
  56.     at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  57.     at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  58.     at py4j.Gateway.invoke(Gateway.java:282)
  59.     at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  60.     at py4j.commands.CallCommand.execute(CallCommand.java:79)
  61.     at py4j.GatewayConnection.run(GatewayConnection.java:238)
  62.     at java.lang.Thread.run(Thread.java:748)
  63. Caused by: java.io.FileNotFoundException: C:\Users\Miguel\AppData\Local\Temp\blockmgr-45156afc-1550-4f32-a28d-130a473ffb56\0a\shuffle_2_0_0.data.e6f4d8e9-a0dc-46e0-a199-135a0b65ba1c (El sistema no puede encontrar la ruta especificada)
  64.     at java.io.FileOutputStream.open0(Native Method)
  65.     at java.io.FileOutputStream.open(FileOutputStream.java:270)
  66.     at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
  67.     at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.writePartitionedFile(BypassMergeSortShuffleWriter.java:191)
  68.     at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:163)
  69.     at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
  70.     at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
  71.     at org.apache.spark.scheduler.Task.run(Task.scala:123)
  72.     at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
  73.     at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
  74.     at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
  75.     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  76.     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!