API tools faq
paste
Guest User

Untitled

a guest
Aug 25th, 2014
503
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Bash 11.07 KB | None | 0 0
raw download clone embed print report
  1. ~/spark$ ./bin/spark-shell
  2. 14/08/25 08:58:04 INFO spark.HttpServer: Starting HTTP Server
  3. 14/08/25 08:58:04 INFO server.Server: jetty-7.x.y-SNAPSHOT
  4. 14/08/25 08:58:04 INFO server.AbstractConnector: Started SocketConnector@0.0.0.0:40486
  5. Welcome to
  6.       ____              __
  7.      / __/__  ___ _____/ /__
  8.     _\ \/ _ \/ _ `/ __/  '_/
  9.   /___/ .__/\_,_/_/ /_/\_\   version 0.9.2
  10.      /_/
  11.  
  12. Using Scala version 2.10.3 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_20)
  13. Type in expressions to have them evaluated.
  14. Type :help for more information.
  15. 14/08/25 08:58:17 INFO slf4j.Slf4jLogger: Slf4jLogger started
  16. 14/08/25 08:58:17 INFO Remoting: Starting remoting
  17. 14/08/25 08:58:18 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://spark@aidan-workstation-webroot:51687]
  18. 14/08/25 08:58:18 INFO Remoting: Remoting now listens on addresses: [akka.tcp://spark@aidan-workstation-webroot:51687]
  19. 14/08/25 08:58:18 INFO spark.SparkEnv: Registering BlockManagerMaster
  20. 14/08/25 08:58:18 INFO storage.DiskBlockManager: Created local directory at /tmp/spark-local-20140825085818-9762
  21. 14/08/25 08:58:18 INFO storage.MemoryStore: MemoryStore started with capacity 294.6 MB.
  22. 14/08/25 08:58:18 INFO network.ConnectionManager: Bound socket to port 47700 with id = ConnectionManagerId(aidan-workstation-webroot,47700)
  23. 14/08/25 08:58:18 INFO storage.BlockManagerMaster: Trying to register BlockManager
  24. 14/08/25 08:58:18 INFO storage.BlockManagerMasterActor$BlockManagerInfo: Registering block manager aidan-workstation-webroot:47700 with 294.6 MB RAM
  25. 14/08/25 08:58:18 INFO storage.BlockManagerMaster: Registered BlockManager
  26. 14/08/25 08:58:18 INFO spark.HttpServer: Starting HTTP Server
  27. 14/08/25 08:58:19 INFO server.Server: jetty-7.x.y-SNAPSHOT
  28. 14/08/25 08:58:19 INFO server.AbstractConnector: Started [email protected]:50633
  29. 14/08/25 08:58:19 INFO broadcast.HttpBroadcast: Broadcast server started at http://10.17.16.43:50633
  30. 14/08/25 08:58:19 INFO spark.SparkEnv: Registering MapOutputTracker
  31. 14/08/25 08:58:19 INFO spark.HttpFileServer: HTTP File server directory is /tmp/spark-0a060f51-5f49-4420-bef6-46b2cf36b520
  32. 14/08/25 08:58:19 INFO spark.HttpServer: Starting HTTP Server
  33. 14/08/25 08:58:19 INFO server.Server: jetty-7.x.y-SNAPSHOT
  34. 14/08/25 08:58:19 INFO server.AbstractConnector: Started [email protected]:53636
  35. 14/08/25 08:58:20 INFO server.Server: jetty-7.x.y-SNAPSHOT
  36. 14/08/25 08:58:20 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/storage/rdd,null}
  37. 14/08/25 08:58:20 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/storage,null}
  38. 14/08/25 08:58:20 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/stages/stage,null}
  39. 14/08/25 08:58:20 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/stages/pool,null}
  40. 14/08/25 08:58:20 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/stages,null}
  41. 14/08/25 08:58:20 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/environment,null}
  42. 14/08/25 08:58:20 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/executors,null}
  43. 14/08/25 08:58:20 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/metrics/json,null}
  44. 14/08/25 08:58:20 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/static,null}
  45. 14/08/25 08:58:20 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/,null}
  46. 14/08/25 08:58:20 INFO server.AbstractConnector: Started [email protected]:4040
  47. 14/08/25 08:58:20 INFO ui.SparkUI: Started Spark Web UI at http://aidan-workstation-webroot:4040
  48. 14/08/25 08:58:20 INFO client.AppClient$ClientActor: Connecting to master spark://10.17.16.43:7077...
  49. 14/08/25 08:58:22 INFO cluster.SparkDeploySchedulerBackend: Connected to Spark cluster with app ID app-20140825085822-0002
  50. 14/08/25 08:58:22 INFO client.AppClient$ClientActor: Executor added: app-20140825085822-0002/0 on worker-20140825084704-ubuntu-49237 (ubuntu:49237) with 8 cores
  51. 14/08/25 08:58:22 INFO cluster.SparkDeploySchedulerBackend: Granted executor ID app-20140825085822-0002/0 on hostPort ubuntu:49237 with 8 cores, 512.0 MB RAM
  52. 14/08/25 08:58:22 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/0 is now RUNNING
  53. 14/08/25 08:58:22 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/0 is now FAILED (Command exited with code 1)
  54. 14/08/25 08:58:22 INFO cluster.SparkDeploySchedulerBackend: Executor app-20140825085822-0002/0 removed: Command exited with code 1
  55. 14/08/25 08:58:22 INFO client.AppClient$ClientActor: Executor added: app-20140825085822-0002/1 on worker-20140825084704-ubuntu-49237 (ubuntu:49237) with 8 cores
  56. 14/08/25 08:58:22 INFO cluster.SparkDeploySchedulerBackend: Granted executor ID app-20140825085822-0002/1 on hostPort ubuntu:49237 with 8 cores, 512.0 MB RAM
  57. 14/08/25 08:58:22 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/1 is now RUNNING
  58. 14/08/25 08:58:22 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  59. 14/08/25 08:58:23 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/1 is now FAILED (Command exited with code 1)
  60. 14/08/25 08:58:23 INFO cluster.SparkDeploySchedulerBackend: Executor app-20140825085822-0002/1 removed: Command exited with code 1
  61. 14/08/25 08:58:23 INFO client.AppClient$ClientActor: Executor added: app-20140825085822-0002/2 on worker-20140825084704-ubuntu-49237 (ubuntu:49237) with 8 cores
  62. 14/08/25 08:58:23 INFO cluster.SparkDeploySchedulerBackend: Granted executor ID app-20140825085822-0002/2 on hostPort ubuntu:49237 with 8 cores, 512.0 MB RAM
  63. 14/08/25 08:58:23 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/2 is now RUNNING
  64. 14/08/25 08:58:23 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/2 is now FAILED (Command exited with code 1)
  65. 14/08/25 08:58:23 INFO cluster.SparkDeploySchedulerBackend: Executor app-20140825085822-0002/2 removed: Command exited with code 1
  66. 14/08/25 08:58:23 INFO client.AppClient$ClientActor: Executor added: app-20140825085822-0002/3 on worker-20140825084704-ubuntu-49237 (ubuntu:49237) with 8 cores
  67. 14/08/25 08:58:23 INFO cluster.SparkDeploySchedulerBackend: Granted executor ID app-20140825085822-0002/3 on hostPort ubuntu:49237 with 8 cores, 512.0 MB RAM
  68. 14/08/25 08:58:23 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/3 is now RUNNING
  69. Created spark context..
  70. 14/08/25 08:58:23 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/3 is now FAILED (Command exited with code 1)
  71. 14/08/25 08:58:23 INFO cluster.SparkDeploySchedulerBackend: Executor app-20140825085822-0002/3 removed: Command exited with code 1
  72. 14/08/25 08:58:23 INFO client.AppClient$ClientActor: Executor added: app-20140825085822-0002/4 on worker-20140825084704-ubuntu-49237 (ubuntu:49237) with 8 cores
  73. 14/08/25 08:58:23 INFO cluster.SparkDeploySchedulerBackend: Granted executor ID app-20140825085822-0002/4 on hostPort ubuntu:49237 with 8 cores, 512.0 MB RAM
  74. 14/08/25 08:58:23 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/4 is now RUNNING
  75. Spark context available as sc.
  76.  
  77. scala> 14/08/25 08:58:24 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/4 is now FAILED (Command exited with code 1)
  78. 14/08/25 08:58:24 INFO cluster.SparkDeploySchedulerBackend: Executor app-20140825085822-0002/4 removed: Command exited with code 1
  79. 14/08/25 08:58:24 INFO client.AppClient$ClientActor: Executor added: app-20140825085822-0002/5 on worker-20140825084704-ubuntu-49237 (ubuntu:49237) with 8 cores
  80. 14/08/25 08:58:24 INFO cluster.SparkDeploySchedulerBackend: Granted executor ID app-20140825085822-0002/5 on hostPort ubuntu:49237 with 8 cores, 512.0 MB RAM
  81. 14/08/25 08:58:24 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/5 is now RUNNING
  82. 14/08/25 08:58:24 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/5 is now FAILED (Command exited with code 1)
  83. 14/08/25 08:58:24 INFO cluster.SparkDeploySchedulerBackend: Executor app-20140825085822-0002/5 removed: Command exited with code 1
  84. 14/08/25 08:58:24 INFO client.AppClient$ClientActor: Executor added: app-20140825085822-0002/6 on worker-20140825084704-ubuntu-49237 (ubuntu:49237) with 8 cores
  85. 14/08/25 08:58:24 INFO cluster.SparkDeploySchedulerBackend: Granted executor ID app-20140825085822-0002/6 on hostPort ubuntu:49237 with 8 cores, 512.0 MB RAM
  86. 14/08/25 08:58:24 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/6 is now RUNNING
  87. 14/08/25 08:58:24 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/6 is now FAILED (Command exited with code 1)
  88. 14/08/25 08:58:24 INFO cluster.SparkDeploySchedulerBackend: Executor app-20140825085822-0002/6 removed: Command exited with code 1
  89. 14/08/25 08:58:24 INFO client.AppClient$ClientActor: Executor added: app-20140825085822-0002/7 on worker-20140825084704-ubuntu-49237 (ubuntu:49237) with 8 cores
  90. 14/08/25 08:58:24 INFO cluster.SparkDeploySchedulerBackend: Granted executor ID app-20140825085822-0002/7 on hostPort ubuntu:49237 with 8 cores, 512.0 MB RAM
  91. 14/08/25 08:58:24 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/7 is now RUNNING
  92. 14/08/25 08:58:25 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/7 is now FAILED (Command exited with code 1)
  93. 14/08/25 08:58:25 INFO cluster.SparkDeploySchedulerBackend: Executor app-20140825085822-0002/7 removed: Command exited with code 1
  94. 14/08/25 08:58:25 INFO client.AppClient$ClientActor: Executor added: app-20140825085822-0002/8 on worker-20140825084704-ubuntu-49237 (ubuntu:49237) with 8 cores
  95. 14/08/25 08:58:25 INFO cluster.SparkDeploySchedulerBackend: Granted executor ID app-20140825085822-0002/8 on hostPort ubuntu:49237 with 8 cores, 512.0 MB RAM
  96. 14/08/25 08:58:25 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/8 is now RUNNING
  97. 14/08/25 08:58:25 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/8 is now FAILED (Command exited with code 1)
  98. 14/08/25 08:58:25 INFO cluster.SparkDeploySchedulerBackend: Executor app-20140825085822-0002/8 removed: Command exited with code 1
  99. 14/08/25 08:58:25 INFO client.AppClient$ClientActor: Executor added: app-20140825085822-0002/9 on worker-20140825084704-ubuntu-49237 (ubuntu:49237) with 8 cores
  100. 14/08/25 08:58:25 INFO cluster.SparkDeploySchedulerBackend: Granted executor ID app-20140825085822-0002/9 on hostPort ubuntu:49237 with 8 cores, 512.0 MB RAM
  101. 14/08/25 08:58:25 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/9 is now RUNNING
  102. 14/08/25 08:58:25 INFO client.AppClient$ClientActor: Executor updated: app-20140825085822-0002/9 is now FAILED (Command exited with code 1)
  103. 14/08/25 08:58:25 INFO cluster.SparkDeploySchedulerBackend: Executor app-20140825085822-0002/9 removed: Command exited with code 1
  104. 14/08/25 08:58:25 ERROR client.AppClient$ClientActor: Master removed our application: FAILED; stopping client
  105. 14/08/25 08:58:25 WARN cluster.SparkDeploySchedulerBackend: Disconnected from Spark cluster! Waiting for reconnection...
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!