API tools faq
paste
Guest User

Untitled

a guest
Jul 10th, 2014
408
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 8.28 KB | None | 0 0
raw download clone embed print report
  1. 14/07/11 10:35:02 INFO SparkUI: Started SparkUI at http://172.30.92.118:4040
  2. 14/07/11 10:35:02 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  3. 14/07/11 10:35:02 ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
  4. java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
  5. at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318)
  6. at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333)
  7. at org.apache.hadoop.util.Shell.<clinit>(Shell.java:326)
  8. at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:76)
  9. at org.apache.hadoop.security.Groups.parseStaticMapping(Groups.java:93)
  10. at org.apache.hadoop.security.Groups.<init>(Groups.java:77)
  11. at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:240)
  12. at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:255)
  13. at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:283)
  14. at org.apache.spark.deploy.SparkHadoopUtil.<init>(SparkHadoopUtil.scala:36)
  15. at org.apache.spark.deploy.SparkHadoopUtil$.<init>(SparkHadoopUtil.scala:109)
  16. at org.apache.spark.deploy.SparkHadoopUtil$.<clinit>(SparkHadoopUtil.scala)
  17. at org.apache.spark.SparkContext.<init>(SparkContext.scala:228)
  18. at org.apache.spark.streaming.StreamingContext$.createNewSparkContext(StreamingContext.scala:549)
  19. at org.apache.spark.streaming.StreamingContext$.createNewSparkContext(StreamingContext.scala:561)
  20. at org.apache.spark.streaming.StreamingContext.<init>(StreamingContext.scala:91)
  21. at org.apache.spark.streaming.api.java.JavaStreamingContext.<init>(JavaStreamingContext.scala:61)
  22. at com.nec.gsd.iac.mag1c.sense.sm_core.SituationManagerCoreDriver.main(SituationManagerCoreDriver.java:31)
  23. 14/07/11 10:35:02 INFO ReceiverTracker: ReceiverTracker started
  24. 14/07/11 10:35:02 INFO ForEachDStream: metadataCleanupDelay = -1
  25. 14/07/11 10:35:02 INFO ShuffledDStream: metadataCleanupDelay = -1
  26. 14/07/11 10:35:02 INFO MappedDStream: metadataCleanupDelay = -1
  27. 14/07/11 10:35:02 INFO FlatMappedDStream: metadataCleanupDelay = -1
  28. 14/07/11 10:35:02 INFO SocketInputDStream: metadataCleanupDelay = -1
  29. 14/07/11 10:35:02 INFO SocketInputDStream: Slide time = 5000 ms
  30. 14/07/11 10:35:02 INFO SocketInputDStream: Storage level = StorageLevel(false, false, false, false, 1)
  31. 14/07/11 10:35:02 INFO SocketInputDStream: Checkpoint interval = null
  32. 14/07/11 10:35:02 INFO SocketInputDStream: Remember duration = 5000 ms
  33. 14/07/11 10:35:02 INFO SocketInputDStream: Initialized and validated org.apache.spark.streaming.dstream.SocketInputDStream@532721fd
  34. 14/07/11 10:35:02 INFO FlatMappedDStream: Slide time = 5000 ms
  35. 14/07/11 10:35:02 INFO FlatMappedDStream: Storage level = StorageLevel(false, true, false, false, 1)
  36. 14/07/11 10:35:02 INFO FlatMappedDStream: Checkpoint interval = null
  37. 14/07/11 10:35:02 INFO FlatMappedDStream: Remember duration = 5000 ms
  38. 14/07/11 10:35:02 INFO FlatMappedDStream: Initialized and validated org.apache.spark.streaming.dstream.FlatMappedDStream@410954b
  39. 14/07/11 10:35:02 INFO MappedDStream: Slide time = 5000 ms
  40. 14/07/11 10:35:02 INFO MappedDStream: Storage level = StorageLevel(false, true, false, false, 1)
  41. 14/07/11 10:35:02 INFO MappedDStream: Checkpoint interval = null
  42. 14/07/11 10:35:02 INFO MappedDStream: Remember duration = 5000 ms
  43. 14/07/11 10:35:02 INFO MappedDStream: Initialized and validated org.apache.spark.streaming.dstream.MappedDStream@7fb9f71f
  44. 14/07/11 10:35:02 INFO ShuffledDStream: Slide time = 5000 ms
  45. 14/07/11 10:35:02 INFO ShuffledDStream: Storage level = StorageLevel(false, false, false, false, 1)
  46. 14/07/11 10:35:02 INFO ShuffledDStream: Checkpoint interval = null
  47. 14/07/11 10:35:02 INFO ShuffledDStream: Remember duration = 5000 ms
  48. 14/07/11 10:35:02 INFO ShuffledDStream: Initialized and validated org.apache.spark.streaming.dstream.ShuffledDStream@3b366632
  49. 14/07/11 10:35:02 INFO ForEachDStream: Slide time = 5000 ms
  50. 14/07/11 10:35:02 INFO ForEachDStream: Storage level = StorageLevel(false, false, false, false, 1)
  51. 14/07/11 10:35:02 INFO ForEachDStream: Checkpoint interval = null
  52. 14/07/11 10:35:02 INFO ForEachDStream: Remember duration = 5000 ms
  53. 14/07/11 10:35:02 INFO ForEachDStream: Initialized and validated org.apache.spark.streaming.dstream.ForEachDStream@51f49060
  54. 14/07/11 10:35:02 INFO ReceiverTracker: Starting 1 receivers
  55. 14/07/11 10:35:02 INFO SparkContext: Starting job: runJob at ReceiverTracker.scala:275
  56. 14/07/11 10:35:02 INFO DAGScheduler: Got job 0 (runJob at ReceiverTracker.scala:275) with 1 output partitions (allowLocal=false)
  57. 14/07/11 10:35:02 INFO DAGScheduler: Final stage: Stage 0(runJob at ReceiverTracker.scala:275)
  58. 14/07/11 10:35:02 INFO DAGScheduler: Parents of final stage: List()
  59. 14/07/11 10:35:02 INFO DAGScheduler: Missing parents: List()
  60. 14/07/11 10:35:02 INFO DAGScheduler: Submitting Stage 0 (ParallelCollectionRDD[0] at makeRDD at ReceiverTracker.scala:253), which has no missing parents
  61. 14/07/11 10:35:02 INFO RecurringTimer: Started timer for JobGenerator at time 1405046105000
  62. 14/07/11 10:35:02 INFO JobGenerator: Started JobGenerator at 1405046105000 ms
  63. 14/07/11 10:35:02 INFO JobScheduler: Started JobScheduler
  64. 14/07/11 10:35:02 INFO DAGScheduler: Submitting 1 missing tasks from Stage 0 (ParallelCollectionRDD[0] at makeRDD at ReceiverTracker.scala:253)
  65. 14/07/11 10:35:02 INFO TaskSchedulerImpl: Adding task set 0.0 with 1 tasks
  66. 14/07/11 10:35:02 INFO TaskSetManager: Starting task 0.0:0 as TID 0 on executor localhost: localhost (PROCESS_LOCAL)
  67. 14/07/11 10:35:02 INFO TaskSetManager: Serialized task 0.0:0 as 1768 bytes in 1 ms
  68. 14/07/11 10:35:02 INFO Executor: Running task ID 0
  69. 14/07/11 10:35:02 INFO ReceiverSupervisorImpl: Registered receiver 0
  70. 14/07/11 10:35:02 INFO RecurringTimer: Started timer for BlockGenerator at time 1405046103000
  71. 14/07/11 10:35:02 INFO BlockGenerator: Started BlockGenerator
  72. 14/07/11 10:35:02 INFO BlockGenerator: Started block pushing thread
  73. 14/07/11 10:35:02 INFO ReceiverSupervisorImpl: Starting receiver
  74. 14/07/11 10:35:02 INFO ReceiverSupervisorImpl: Called receiver onStart
  75. 14/07/11 10:35:02 INFO SocketReceiver: Connecting to 127.0.0.1:9999
  76. 14/07/11 10:35:02 INFO SocketReceiver: Connected to 127.0.0.1:9999
  77. 14/07/11 10:35:02 INFO ReceiverTracker: Registered receiver for stream 0 from akka://spark
  78. 14/07/11 10:35:02 INFO ReceiverTracker: Registered receiver for stream 0 from akka://spark
  79. 14/07/11 10:35:05 INFO ReceiverTracker: Stream 0 received 0 blocks
  80. 14/07/11 10:35:05 INFO FlatMappedDStream: Persisting RDD 2 for time 1405046105000 ms to StorageLevel(false, true, false, false, 1) at time 1405046105000 ms
  81. 14/07/11 10:35:05 INFO MappedDStream: Persisting RDD 3 for time 1405046105000 ms to StorageLevel(false, true, false, false, 1) at time 1405046105000 ms
  82. 14/07/11 10:35:05 INFO JobScheduler: Added jobs for time 1405046105000 ms
  83. 14/07/11 10:35:05 INFO JobScheduler: Starting job streaming job 1405046105000 ms.0 from job set of time 1405046105000 ms
  84. 14/07/11 10:35:05 INFO SparkContext: Starting job: take at DStream.scala:593
  85. 14/07/11 10:35:05 INFO DAGScheduler: Registering RDD 4 (combineByKey at ShuffledDStream.scala:42)
  86. 14/07/11 10:35:05 INFO DAGScheduler: Got job 1 (take at DStream.scala:593) with 1 output partitions (allowLocal=true)
  87. 14/07/11 10:35:05 INFO DAGScheduler: Final stage: Stage 1(take at DStream.scala:593)
  88. 14/07/11 10:35:05 INFO DAGScheduler: Parents of final stage: List(Stage 2)
  89. 14/07/11 10:35:05 INFO DAGScheduler: Missing parents: List()
  90. 14/07/11 10:35:05 INFO DAGScheduler: Submitting Stage 1 (MapPartitionsRDD[6] at combineByKey at ShuffledDStream.scala:42), which has no missing parents
  91. 14/07/11 10:35:05 INFO DAGScheduler: Submitting 1 missing tasks from Stage 1 (MapPartitionsRDD[6] at combineByKey at ShuffledDStream.scala:42)
  92. 14/07/11 10:35:05 INFO TaskSchedulerImpl: Adding task set 1.0 with 1 tasks
  93. 14/07/11 10:35:10 INFO ReceiverTracker: Stream 0 received 0 blocks
  94. 14/07/11 10:35:10 INFO FlatMappedDStream: Persisting RDD 8 for time 1405046110000 ms to StorageLevel(false, true, false, false, 1) at time 1405046110000 ms
  95. 14/07/11 10:35:10 INFO MappedDStream: Persisting RDD 9 for time 1405046110000 ms to StorageLevel(false, true, false, false, 1) at time 1405046110000 ms
  96. 14/07/11 10:35:10 INFO JobScheduler: Added jobs for time 1405046110000 ms
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!