Advertisement
Guest User

Untitled

a guest
Jun 30th, 2016
200
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 5.97 KB | None | 0 0
  1. ../../bin/spark-submit --verbose --class "SimpleApp" --master local[4] target/simple-project-1.0.jar
  2. Using properties file: /opt/mapr/spark/spark-1.6.1/conf/spark-defaults.conf
  3. Adding default property: spark.eventLog.enabled=true
  4. Adding default property: spark.ssl.keyStore=/opt/mapr/conf/ssl_keystore
  5. Adding default property: spark.yarn.historyServer.address=http://localhost:18080
  6. Adding default property: spark.ssl.trustStore=/opt/mapr/conf/ssl_truststore
  7. Adding default property: spark.logConf=true
  8. Adding default property: spark.ssl.keyPassword=mapr123
  9. Adding default property: spark.ssl.keyStorePassword=mapr123
  10. Adding default property: spark.sql.hive.metastore.sharedPrefixes=com.mysql.jdbc,org.postgresql,com.microsoft.sqlserver,oracle.jdbc,com.mapr.fs.shim.LibraryLoader,com.mapr.security.JNISecurity,com.mapr.fs.jni
  11. Adding default property: spark.executor.memory=2g
  12. Adding default property: spark.eventLog.dir=maprfs:///apps/spark
  13. Adding default property: spark.ssl.protocol=TLSv1.2
  14. Adding default property: spark.executor.extraClassPath=
  15. Adding default property: spark.ssl.trustStorePassword=mapr123
  16. Adding default property: spark.ssl.enabledAlgorithms=TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA
  17. Parsed arguments:
  18. master local[4]
  19. deployMode null
  20. executorMemory 2g
  21. executorCores null
  22. totalExecutorCores null
  23. propertiesFile /opt/mapr/spark/spark-1.6.1/conf/spark-defaults.conf
  24. driverMemory null
  25. driverCores null
  26. driverExtraClassPath null
  27. driverExtraLibraryPath null
  28. driverExtraJavaOptions null
  29. supervise false
  30. queue null
  31. numExecutors null
  32. files null
  33. pyFiles null
  34. archives null
  35. mainClass SimpleApp
  36. primaryResource file:/opt/mapr/spark/spark-1.6.1/project/my_example/target/simple-project-1.0.jar
  37. name SimpleApp
  38. childArgs []
  39. jars null
  40. packages null
  41. packagesExclusions null
  42. repositories null
  43. verbose true
  44.  
  45. Spark properties used, including those specified through
  46. --conf and those from the properties file /opt/mapr/spark/spark-1.6.1/conf/spark-defaults.conf:
  47. spark.executor.memory -> 2g
  48. spark.logConf -> true
  49. spark.yarn.historyServer.address -> http://localhost:18080
  50. spark.eventLog.enabled -> true
  51. spark.ssl.enabledAlgorithms -> TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA
  52. spark.ssl.trustStorePassword -> mapr123
  53. spark.ssl.keyStore -> /opt/mapr/conf/ssl_keystore
  54. spark.ssl.trustStore -> /opt/mapr/conf/ssl_truststore
  55. spark.sql.hive.metastore.sharedPrefixes -> com.mysql.jdbc,org.postgresql,com.microsoft.sqlserver,oracle.jdbc,com.mapr.fs.shim.LibraryLoader,com.mapr.security.JNISecurity,com.mapr.fs.jni
  56. spark.eventLog.dir -> maprfs:///apps/spark
  57. spark.executor.extraClassPath ->
  58. spark.ssl.protocol -> TLSv1.2
  59. spark.ssl.keyPassword -> mapr123
  60. spark.ssl.keyStorePassword -> mapr123
  61.  
  62.  
  63. Main class:
  64. SimpleApp
  65. Arguments:
  66.  
  67. System properties:
  68. spark.executor.memory -> 2g
  69. spark.logConf -> true
  70. spark.yarn.historyServer.address -> http://localhost:18080
  71. spark.eventLog.enabled -> true
  72. SPARK_SUBMIT -> true
  73. spark.app.name -> SimpleApp
  74. spark.ssl.enabledAlgorithms -> TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA
  75. spark.ssl.trustStorePassword -> mapr123
  76. spark.ssl.keyStore -> /opt/mapr/conf/ssl_keystore
  77. spark.ssl.trustStore -> /opt/mapr/conf/ssl_truststore
  78. spark.jars -> file:/opt/mapr/spark/spark-1.6.1/project/my_example/target/simple-project-1.0.jar
  79. spark.submit.deployMode -> client
  80. spark.executor.extraClassPath ->
  81. spark.eventLog.dir -> maprfs:///apps/spark
  82. spark.sql.hive.metastore.sharedPrefixes -> com.mysql.jdbc,org.postgresql,com.microsoft.sqlserver,oracle.jdbc,com.mapr.fs.shim.LibraryLoader,com.mapr.security.JNISecurity,com.mapr.fs.jni
  83. spark.master -> local[4]
  84. spark.ssl.protocol -> TLSv1.2
  85. spark.ssl.keyPassword -> mapr123
  86. spark.ssl.keyStorePassword -> mapr123
  87. Classpath elements:
  88. file:/opt/mapr/spark/spark-1.6.1/project/my_example/target/simple-project-1.0.jar
  89.  
  90.  
  91. 16/06/30 08:26:48 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  92. 16/06/30 08:26:49 WARN Utils: Your hostname, localhost resolves to a loopback address: 127.0.0.1; using 10.0.2.15 instead (on interface eth0)
  93. 16/06/30 08:26:49 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
  94. 2728
  95. END RDD
  96. Exception in thread "main" java.lang.NoClassDefFoundError: org/elasticsearch/spark/rdd/api/java/JavaEsSpark
  97. at SimpleApp.main(SimpleApp.java:72)
  98. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  99. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  100. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  101. at java.lang.reflect.Method.invoke(Method.java:606)
  102. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:742)
  103. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
  104. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
  105. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
  106. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  107. Caused by: java.lang.ClassNotFoundException: org.elasticsearch.spark.rdd.api.java.JavaEsSpark
  108. at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
  109. at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
  110. at java.security.AccessController.doPrivileged(Native Method)
  111. at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
  112. at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
  113. at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
  114. ... 10 more
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement