Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- ../../bin/spark-submit --verbose --class "SimpleApp" --master local[4] target/simple-project-1.0.jar
- Using properties file: /opt/mapr/spark/spark-1.6.1/conf/spark-defaults.conf
- Adding default property: spark.eventLog.enabled=true
- Adding default property: spark.ssl.keyStore=/opt/mapr/conf/ssl_keystore
- Adding default property: spark.yarn.historyServer.address=http://localhost:18080
- Adding default property: spark.ssl.trustStore=/opt/mapr/conf/ssl_truststore
- Adding default property: spark.logConf=true
- Adding default property: spark.ssl.keyPassword=mapr123
- Adding default property: spark.ssl.keyStorePassword=mapr123
- Adding default property: spark.sql.hive.metastore.sharedPrefixes=com.mysql.jdbc,org.postgresql,com.microsoft.sqlserver,oracle.jdbc,com.mapr.fs.shim.LibraryLoader,com.mapr.security.JNISecurity,com.mapr.fs.jni
- Adding default property: spark.executor.memory=2g
- Adding default property: spark.eventLog.dir=maprfs:///apps/spark
- Adding default property: spark.ssl.protocol=TLSv1.2
- Adding default property: spark.executor.extraClassPath=
- Adding default property: spark.ssl.trustStorePassword=mapr123
- Adding default property: spark.ssl.enabledAlgorithms=TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA
- Parsed arguments:
- master local[4]
- deployMode null
- executorMemory 2g
- executorCores null
- totalExecutorCores null
- propertiesFile /opt/mapr/spark/spark-1.6.1/conf/spark-defaults.conf
- driverMemory null
- driverCores null
- driverExtraClassPath null
- driverExtraLibraryPath null
- driverExtraJavaOptions null
- supervise false
- queue null
- numExecutors null
- files null
- pyFiles null
- archives null
- mainClass SimpleApp
- primaryResource file:/opt/mapr/spark/spark-1.6.1/project/my_example/target/simple-project-1.0.jar
- name SimpleApp
- childArgs []
- jars null
- packages null
- packagesExclusions null
- repositories null
- verbose true
- Spark properties used, including those specified through
- --conf and those from the properties file /opt/mapr/spark/spark-1.6.1/conf/spark-defaults.conf:
- spark.executor.memory -> 2g
- spark.logConf -> true
- spark.yarn.historyServer.address -> http://localhost:18080
- spark.eventLog.enabled -> true
- spark.ssl.enabledAlgorithms -> TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA
- spark.ssl.trustStorePassword -> mapr123
- spark.ssl.keyStore -> /opt/mapr/conf/ssl_keystore
- spark.ssl.trustStore -> /opt/mapr/conf/ssl_truststore
- spark.sql.hive.metastore.sharedPrefixes -> com.mysql.jdbc,org.postgresql,com.microsoft.sqlserver,oracle.jdbc,com.mapr.fs.shim.LibraryLoader,com.mapr.security.JNISecurity,com.mapr.fs.jni
- spark.eventLog.dir -> maprfs:///apps/spark
- spark.executor.extraClassPath ->
- spark.ssl.protocol -> TLSv1.2
- spark.ssl.keyPassword -> mapr123
- spark.ssl.keyStorePassword -> mapr123
- Main class:
- SimpleApp
- Arguments:
- System properties:
- spark.executor.memory -> 2g
- spark.logConf -> true
- spark.yarn.historyServer.address -> http://localhost:18080
- spark.eventLog.enabled -> true
- SPARK_SUBMIT -> true
- spark.app.name -> SimpleApp
- spark.ssl.enabledAlgorithms -> TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA
- spark.ssl.trustStorePassword -> mapr123
- spark.ssl.keyStore -> /opt/mapr/conf/ssl_keystore
- spark.ssl.trustStore -> /opt/mapr/conf/ssl_truststore
- spark.jars -> file:/opt/mapr/spark/spark-1.6.1/project/my_example/target/simple-project-1.0.jar
- spark.submit.deployMode -> client
- spark.executor.extraClassPath ->
- spark.eventLog.dir -> maprfs:///apps/spark
- spark.sql.hive.metastore.sharedPrefixes -> com.mysql.jdbc,org.postgresql,com.microsoft.sqlserver,oracle.jdbc,com.mapr.fs.shim.LibraryLoader,com.mapr.security.JNISecurity,com.mapr.fs.jni
- spark.master -> local[4]
- spark.ssl.protocol -> TLSv1.2
- spark.ssl.keyPassword -> mapr123
- spark.ssl.keyStorePassword -> mapr123
- Classpath elements:
- file:/opt/mapr/spark/spark-1.6.1/project/my_example/target/simple-project-1.0.jar
- 16/06/30 08:26:48 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
- 16/06/30 08:26:49 WARN Utils: Your hostname, localhost resolves to a loopback address: 127.0.0.1; using 10.0.2.15 instead (on interface eth0)
- 16/06/30 08:26:49 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
- 2728
- END RDD
- Exception in thread "main" java.lang.NoClassDefFoundError: org/elasticsearch/spark/rdd/api/java/JavaEsSpark
- at SimpleApp.main(SimpleApp.java:72)
- at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
- at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
- at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
- at java.lang.reflect.Method.invoke(Method.java:606)
- at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:742)
- at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
- at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
- at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
- at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
- Caused by: java.lang.ClassNotFoundException: org.elasticsearch.spark.rdd.api.java.JavaEsSpark
- at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
- at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
- at java.security.AccessController.doPrivileged(Native Method)
- at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
- at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
- at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
- ... 10 more
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement