API tools faq
paste
Advertisement
Guest User

Spark SQL errors

a guest
May 20th, 2015
787
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 26.77 KB | None | 0 0
raw download clone embed print report
  1. Welcome to
  2. ____ __
  3. / __/__ ___ _____/ /__
  4. _\ \/ _ \/ _ `/ __/ '_/
  5. /__ / .__/\_,_/_/ /_/\_\ version 1.3.0
  6. /_/
  7.  
  8. Using Python version 2.7.6 (default, Mar 22 2014 22:59:56)
  9. SparkContext available as sc, HiveContext available as sqlCtx.
  10. >>> results = sqlCtx.sql("SELECT * FROM my_table").collect()
  11. 15/05/20 06:57:07 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
  12. 15/05/20 06:57:07 INFO ObjectStore: ObjectStore, initialize called
  13. 15/05/20 06:57:08 WARN General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH-5.4.1-1.cdh5.4.1.p0.6/jars/datanucleus-core-3.2.10.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/cloudera/parcels/CDH-5.4.1-1.cdh5.4.1.p0.6/jars/datanucleus-core-3.2.2.jar."
  14. 15/05/20 06:57:08 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH-5.4.1-1.cdh5.4.1.p0.6/jars/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/cloudera/parcels/CDH-5.4.1-1.cdh5.4.1.p0.6/jars/datanucleus-api-jdo-3.2.1.jar."
  15. 15/05/20 06:57:08 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH-5.4.1-1.cdh5.4.1.p0.6/jars/datanucleus-rdbms-3.2.1.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/cloudera/parcels/CDH-5.4.1-1.cdh5.4.1.p0.6/jars/datanucleus-rdbms-3.2.9.jar."
  16. 15/05/20 06:57:08 INFO Persistence: Property datanucleus.cache.level2 unknown - will be ignored
  17. 15/05/20 06:57:08 INFO Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
  18. 15/05/20 06:57:08 WARN HiveMetaStore: Retrying creating default database after error: Error creating transactional connection factory
  19. javax.jdo.JDOFatalInternalException: Error creating transactional connection factory
  20. at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:587)
  21. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:788)
  22. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  23. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  24. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  25. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  26. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  27. at java.lang.reflect.Method.invoke(Method.java:606)
  28. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  29. at java.security.AccessController.doPrivileged(Native Method)
  30. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  31. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  32. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  33. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  34. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  35. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  36. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  37. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  38. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
  39. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
  40. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:56)
  41. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:65)
  42. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:579)
  43. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:557)
  44. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:606)
  45. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:448)
  46. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  47. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  48. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5601)
  49. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:193)
  50. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  51. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  52. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
  53. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  54. at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
  55. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1486)
  56. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:64)
  57. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:74)
  58. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2845)
  59. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2864)
  60. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:453)
  61. at org.apache.spark.sql.hive.HiveContext.sessionState$lzycompute(HiveContext.scala:229)
  62. at org.apache.spark.sql.hive.HiveContext.sessionState(HiveContext.scala:225)
  63. at org.apache.spark.sql.hive.HiveContext.hiveconf$lzycompute(HiveContext.scala:241)
  64. at org.apache.spark.sql.hive.HiveContext.hiveconf(HiveContext.scala:240)
  65. at org.apache.spark.sql.hive.HiveContext.sql(HiveContext.scala:86)
  66. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  67. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  68. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  69. at java.lang.reflect.Method.invoke(Method.java:606)
  70. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
  71. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379)
  72. at py4j.Gateway.invoke(Gateway.java:259)
  73. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
  74. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  75. at py4j.GatewayConnection.run(GatewayConnection.java:207)
  76. at java.lang.Thread.run(Thread.java:745)
  77. NestedThrowablesStackTrace:
  78. java.lang.reflect.InvocationTargetException
  79. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  80. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
  81. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  82. at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
  83. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  84. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:325)
  85. at org.datanucleus.store.AbstractStoreManager.registerConnectionFactory(AbstractStoreManager.java:282)
  86. at org.datanucleus.store.AbstractStoreManager.<init>(AbstractStoreManager.java:240)
  87. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:286)
  88. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  89. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
  90. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  91. at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
  92. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  93. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  94. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  95. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  96. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  97. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  98. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  99. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  100. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  101. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  102. at java.lang.reflect.Method.invoke(Method.java:606)
  103. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  104. at java.security.AccessController.doPrivileged(Native Method)
  105. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  106. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  107. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  108. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  109. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  110. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  111. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  112. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  113. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
  114. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
  115. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:56)
  116. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:65)
  117. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:579)
  118. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:557)
  119. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:606)
  120. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:448)
  121. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  122. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  123. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5601)
  124. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:193)
  125. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  126. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  127. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
  128. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  129. at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
  130. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1486)
  131. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:64)
  132. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:74)
  133. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2845)
  134. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2864)
  135. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:453)
  136. at org.apache.spark.sql.hive.HiveContext.sessionState$lzycompute(HiveContext.scala:229)
  137. at org.apache.spark.sql.hive.HiveContext.sessionState(HiveContext.scala:225)
  138. at org.apache.spark.sql.hive.HiveContext.hiveconf$lzycompute(HiveContext.scala:241)
  139. at org.apache.spark.sql.hive.HiveContext.hiveconf(HiveContext.scala:240)
  140. at org.apache.spark.sql.hive.HiveContext.sql(HiveContext.scala:86)
  141. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  142. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  143. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  144. at java.lang.reflect.Method.invoke(Method.java:606)
  145. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
  146. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379)
  147. at py4j.Gateway.invoke(Gateway.java:259)
  148. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
  149. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  150. at py4j.GatewayConnection.run(GatewayConnection.java:207)
  151. at java.lang.Thread.run(Thread.java:745)
  152. Caused by: java.lang.ExceptionInInitializerError
  153. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  154. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
  155. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  156. at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
  157. at java.lang.Class.newInstance(Class.java:374)
  158. at org.datanucleus.store.rdbms.connectionpool.AbstractConnectionPoolFactory.loadDriver(AbstractConnectionPoolFactory.java:47)
  159. at org.datanucleus.store.rdbms.connectionpool.BoneCPConnectionPoolFactory.createConnectionPool(BoneCPConnectionPoolFactory.java:54)
  160. at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:238)
  161. at org.datanucleus.store.rdbms.ConnectionFactoryImpl.initialiseDataSources(ConnectionFactoryImpl.java:131)
  162. at org.datanucleus.store.rdbms.ConnectionFactoryImpl.<init>(ConnectionFactoryImpl.java:85)
  163. ... 73 more
  164. Caused by: java.lang.SecurityException: sealing violation: package org.apache.derby.impl.services.locks is sealed
  165. at java.net.URLClassLoader.getAndVerifyPackage(URLClassLoader.java:388)
  166. at java.net.URLClassLoader.defineClass(URLClassLoader.java:417)
  167. at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
  168. at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
  169. at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
  170. at java.security.AccessController.doPrivileged(Native Method)
  171. at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
  172. at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
  173. at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
  174. at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
  175. at java.lang.ClassLoader.defineClass1(Native Method)
  176. at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
  177. at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
  178. at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
  179. at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
  180. at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
  181. at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
  182. at java.security.AccessController.doPrivileged(Native Method)
  183. at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
  184. at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
  185. at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
  186. at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
  187. at java.lang.Class.forName0(Native Method)
  188. at java.lang.Class.forName(Class.java:190)
  189. at org.apache.derby.impl.services.monitor.BaseMonitor.getImplementations(Unknown Source)
  190. at org.apache.derby.impl.services.monitor.BaseMonitor.getDefaultImplementations(Unknown Source)
  191. at org.apache.derby.impl.services.monitor.BaseMonitor.runWithState(Unknown Source)
  192. at org.apache.derby.impl.services.monitor.FileMonitor.<init>(Unknown Source)
  193. at org.apache.derby.iapi.services.monitor.Monitor.startMonitor(Unknown Source)
  194. at org.apache.derby.iapi.jdbc.JDBCBoot.boot(Unknown Source)
  195. at org.apache.derby.jdbc.EmbeddedDriver.boot(Unknown Source)
  196. at org.apache.derby.jdbc.EmbeddedDriver.<clinit>(Unknown Source)
  197. ... 83 more
  198. 15/05/20 06:57:08 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
  199. 15/05/20 06:57:08 INFO ObjectStore: ObjectStore, initialize called
  200. 15/05/20 06:57:08 WARN General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH-5.4.1-1.cdh5.4.1.p0.6/jars/datanucleus-core-3.2.10.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/cloudera/parcels/CDH-5.4.1-1.cdh5.4.1.p0.6/jars/datanucleus-core-3.2.2.jar."
  201. 15/05/20 06:57:08 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH-5.4.1-1.cdh5.4.1.p0.6/jars/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/cloudera/parcels/CDH-5.4.1-1.cdh5.4.1.p0.6/jars/datanucleus-api-jdo-3.2.1.jar."
  202. 15/05/20 06:57:08 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH-5.4.1-1.cdh5.4.1.p0.6/jars/datanucleus-rdbms-3.2.1.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/cloudera/parcels/CDH-5.4.1-1.cdh5.4.1.p0.6/jars/datanucleus-rdbms-3.2.9.jar."
  203. 15/05/20 06:57:08 INFO Persistence: Property datanucleus.cache.level2 unknown - will be ignored
  204. 15/05/20 06:57:08 INFO Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
  205. Traceback (most recent call last):
  206. File "<stdin>", line 1, in <module>
  207. File "/opt/cloudera/parcels/CDH-5.4.1-1.cdh5.4.1.p0.6/lib/spark/python/pyspark/sql/context.py", line 528, in sql
  208. return DataFrame(self._ssql_ctx.sql(sqlQuery), self)
  209. File "/opt/cloudera/parcels/CDH-5.4.1-1.cdh5.4.1.p0.6/lib/spark/python/lib/py4j-0.8.2.1-src.zip/py4j/java_gateway.py", line 538, in __call__
  210. File "/opt/cloudera/parcels/CDH-5.4.1-1.cdh5.4.1.p0.6/lib/spark/python/lib/py4j-0.8.2.1-src.zip/py4j/protocol.py", line 300, in get_return_value
  211. py4j.protocol.Py4JJavaError: An error occurred while calling o31.sql.
  212. : java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  213. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:472)
  214. at org.apache.spark.sql.hive.HiveContext.sessionState$lzycompute(HiveContext.scala:229)
  215. at org.apache.spark.sql.hive.HiveContext.sessionState(HiveContext.scala:225)
  216. at org.apache.spark.sql.hive.HiveContext.hiveconf$lzycompute(HiveContext.scala:241)
  217. at org.apache.spark.sql.hive.HiveContext.hiveconf(HiveContext.scala:240)
  218. at org.apache.spark.sql.hive.HiveContext.sql(HiveContext.scala:86)
  219. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  220. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  221. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  222. at java.lang.reflect.Method.invoke(Method.java:606)
  223. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
  224. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379)
  225. at py4j.Gateway.invoke(Gateway.java:259)
  226. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
  227. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  228. at py4j.GatewayConnection.run(GatewayConnection.java:207)
  229. at java.lang.Thread.run(Thread.java:745)
  230. Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  231. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1488)
  232. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:64)
  233. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:74)
  234. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2845)
  235. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2864)
  236. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:453)
  237. ... 16 more
  238. Caused by: java.lang.reflect.InvocationTargetException
  239. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  240. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
  241. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  242. at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
  243. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1486)
  244. ... 21 more
  245. Caused by: javax.jdo.JDOFatalInternalException: Error creating transactional connection factory
  246. NestedThrowables:
  247. java.lang.reflect.InvocationTargetException
  248. at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:587)
  249. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:788)
  250. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  251. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  252. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  253. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  254. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  255. at java.lang.reflect.Method.invoke(Method.java:606)
  256. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  257. at java.security.AccessController.doPrivileged(Native Method)
  258. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  259. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  260. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  261. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  262. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  263. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  264. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  265. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  266. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
  267. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
  268. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:56)
  269. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:65)
  270. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:579)
  271. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:557)
  272. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:610)
  273. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:448)
  274. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  275. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  276. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5601)
  277. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:193)
  278. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  279. ... 26 more
  280. Caused by: java.lang.reflect.InvocationTargetException
  281. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  282. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
  283. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  284. at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
  285. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  286. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:325)
  287. at org.datanucleus.store.AbstractStoreManager.registerConnectionFactory(AbstractStoreManager.java:282)
  288. at org.datanucleus.store.AbstractStoreManager.<init>(AbstractStoreManager.java:240)
  289. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:286)
  290. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  291. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
  292. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  293. at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
  294. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  295. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  296. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  297. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  298. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  299. ... 55 more
  300. Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.derby.jdbc.EmbeddedDriver
  301. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  302. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
  303. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  304. at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
  305. at java.lang.Class.newInstance(Class.java:374)
  306. at org.datanucleus.store.rdbms.connectionpool.AbstractConnectionPoolFactory.loadDriver(AbstractConnectionPoolFactory.java:47)
  307. at org.datanucleus.store.rdbms.connectionpool.BoneCPConnectionPoolFactory.createConnectionPool(BoneCPConnectionPoolFactory.java:54)
  308. at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:238)
  309. at org.datanucleus.store.rdbms.ConnectionFactoryImpl.initialiseDataSources(ConnectionFactoryImpl.java:131)
  310. at org.datanucleus.store.rdbms.ConnectionFactoryImpl.<init>(ConnectionFactoryImpl.java:85)
  311. ... 73 more
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!