API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Mar 30th, 2017
85
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 689.87 KB | None | 0 0
raw download clone embed print report
  1. Script started on Thu Mar 30 08:43:31 2017
  2. command: ./bin/spark-shell --master local[4]
  3. Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
  4. Setting default log level to "WARN".
  5. To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
  6. 17/03/30 08:43:39 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  7. 17/03/30 08:43:39 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
  8. 17/03/30 08:43:41 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/Users/james/spark/jars/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/Users/james/spark-2.1.0-bin-hadoop2.7/jars/datanucleus-api-jdo-3.2.6.jar."
  9. 17/03/30 08:43:41 WARN General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/Users/james/spark-2.1.0-bin-hadoop2.7/jars/datanucleus-core-3.2.10.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/Users/james/spark/jars/datanucleus-core-3.2.10.jar."
  10. 17/03/30 08:43:41 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/Users/james/spark-2.1.0-bin-hadoop2.7/jars/datanucleus-rdbms-3.2.9.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/Users/james/spark/jars/datanucleus-rdbms-3.2.9.jar."
  11. 17/03/30 08:43:41 ERROR Schema: Failed initialising database.
  12. Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  13. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  14. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  15. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  16. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  17. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  18. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  19. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  20. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  21. at java.security.AccessController.doPrivileged(Native Method)
  22. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  23. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  24. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  25. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  26. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  27. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  28. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  29. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  30. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  31. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  32. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  33. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  34. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  35. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  36. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  37. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  38. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  39. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  40. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  41. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  42. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  43. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  44. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  45. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  46. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  47. at java.lang.reflect.Method.invoke(Method.java:498)
  48. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  49. at java.security.AccessController.doPrivileged(Native Method)
  50. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  51. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  52. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  53. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  54. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  55. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  56. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  57. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  58. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  59. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  60. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  61. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  62. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  63. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  64. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  65. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  66. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  67. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  68. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  69. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  70. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  71. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  72. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  73. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  74. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  75. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  76. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  77. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  78. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  79. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  80. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  81. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  82. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  83. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  84. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  85. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  86. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  87. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  88. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  89. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  90. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  91. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  92. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  93. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  94. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  95. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  96. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  97. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  98. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  99. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  100. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  101. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  102. at scala.Option.getOrElse(Option.scala:121)
  103. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  104. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  105. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  106. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  107. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  108. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  109. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  110. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  111. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  112. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  113. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  114. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  115. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  116. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  117. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  118. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  119. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  120. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  121. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  122. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  123. at $line3.$read$$iw$$iw.<init>(<console>:15)
  124. at $line3.$read$$iw.<init>(<console>:42)
  125. at $line3.$read.<init>(<console>:44)
  126. at $line3.$read$.<init>(<console>:48)
  127. at $line3.$read$.<clinit>(<console>)
  128. at $line3.$eval$.$print$lzycompute(<console>:7)
  129. at $line3.$eval$.$print(<console>:6)
  130. at $line3.$eval.$print(<console>)
  131. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  132. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  133. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  134. at java.lang.reflect.Method.invoke(Method.java:498)
  135. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  136. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  137. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  138. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  139. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  140. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  141. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  142. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  143. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  144. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  145. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  146. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  147. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  148. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  149. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  150. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  151. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  152. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  153. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  154. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  155. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  156. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  157. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  158. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  159. at org.apache.spark.repl.Main$.main(Main.scala:51)
  160. at org.apache.spark.repl.Main.main(Main.scala)
  161. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  162. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  163. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  164. at java.lang.reflect.Method.invoke(Method.java:498)
  165. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  166. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  167. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  168. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  169. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  170. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  171. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  172. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  173. ... 156 more
  174. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  175. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  176. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  177. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  178. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  179. at java.security.AccessController.doPrivileged(Native Method)
  180. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  181. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  182. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  183. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  184. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  185. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  186. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  187. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  188. at java.security.AccessController.doPrivileged(Native Method)
  189. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  190. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  191. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  192. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  193. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  194. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  195. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  196. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  197. at java.security.AccessController.doPrivileged(Native Method)
  198. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  199. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  200. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  201. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  202. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  203. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  204. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  205. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  206. at java.security.AccessController.doPrivileged(Native Method)
  207. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  208. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  209. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  210. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  211. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  212. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  213. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  214. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  215. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  216. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  217. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  218. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  219. at java.security.AccessController.doPrivileged(Native Method)
  220. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  221. ... 153 more
  222. ------
  223.  
  224. org.datanucleus.exceptions.NucleusDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  225. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  226. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  227. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  228. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  229. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  230. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  231. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  232. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  233. at java.security.AccessController.doPrivileged(Native Method)
  234. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  235. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  236. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  237. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  238. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  239. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  240. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  241. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  242. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  243. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  244. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  245. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  246. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  247. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  248. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  249. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  250. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  251. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  252. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  253. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  254. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  255. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  256. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  257. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  258. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  259. at java.lang.reflect.Method.invoke(Method.java:498)
  260. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  261. at java.security.AccessController.doPrivileged(Native Method)
  262. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  263. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  264. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  265. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  266. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  267. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  268. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  269. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  270. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  271. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  272. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  273. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  274. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  275. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  276. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  277. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  278. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  279. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  280. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  281. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  282. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  283. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  284. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  285. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  286. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  287. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  288. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  289. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  290. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  291. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  292. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  293. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  294. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  295. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  296. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  297. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  298. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  299. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  300. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  301. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  302. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  303. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  304. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  305. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  306. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  307. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  308. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  309. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  310. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  311. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  312. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  313. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  314. at scala.Option.getOrElse(Option.scala:121)
  315. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  316. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  317. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  318. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  319. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  320. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  321. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  322. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  323. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  324. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  325. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  326. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  327. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  328. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  329. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  330. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  331. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  332. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  333. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  334. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  335. at $line3.$read$$iw$$iw.<init>(<console>:15)
  336. at $line3.$read$$iw.<init>(<console>:42)
  337. at $line3.$read.<init>(<console>:44)
  338. at $line3.$read$.<init>(<console>:48)
  339. at $line3.$read$.<clinit>(<console>)
  340. at $line3.$eval$.$print$lzycompute(<console>:7)
  341. at $line3.$eval$.$print(<console>:6)
  342. at $line3.$eval.$print(<console>)
  343. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  344. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  345. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  346. at java.lang.reflect.Method.invoke(Method.java:498)
  347. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  348. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  349. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  350. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  351. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  352. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  353. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  354. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  355. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  356. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  357. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  358. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  359. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  360. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  361. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  362. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  363. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  364. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  365. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  366. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  367. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  368. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  369. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  370. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  371. at org.apache.spark.repl.Main$.main(Main.scala:51)
  372. at org.apache.spark.repl.Main.main(Main.scala)
  373. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  374. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  375. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  376. at java.lang.reflect.Method.invoke(Method.java:498)
  377. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  378. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  379. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  380. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  381. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  382. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  383. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  384. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  385. ... 156 more
  386. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  387. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  388. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  389.  
  390. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  391. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  392. at java.security.AccessController.doPrivileged(Native Method)
  393. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  394. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  395. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  396. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  397. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  398. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  399. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  400. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  401. at java.security.AccessController.doPrivileged(Native Method)
  402. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  403. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  404. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  405. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  406. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  407. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  408. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  409. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  410. at java.security.AccessController.doPrivileged(Native Method)
  411. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  412. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  413. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  414. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  415. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  416. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  417. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  418. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  419. at java.security.AccessController.doPrivileged(Native Method)
  420. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  421. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  422. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  423. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  424. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  425. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  426. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  427. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  428. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  429. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  430. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  431. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  432. at java.security.AccessController.doPrivileged(Native Method)
  433. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  434. ... 153 more
  435. ------
  436.  
  437. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:516)
  438. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  439. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  440. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  441. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  442. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  443. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  444. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  445. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  446. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  447. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  448. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  449. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  450. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  451. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  452. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  453. at java.lang.reflect.Method.invoke(Method.java:498)
  454. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  455. at java.security.AccessController.doPrivileged(Native Method)
  456. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  457. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  458. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  459. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  460. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  461. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  462. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  463. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  464. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  465. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  466. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  467. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  468. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  469. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  470. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  471. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  472. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  473. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  474. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  475. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  476. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  477. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  478. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  479. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  480. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  481. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  482. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  483. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  484. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  485. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  486. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  487. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  488. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  489. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  490. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  491. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  492. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  493. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  494. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  495. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  496. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  497. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  498. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  499. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  500. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  501. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  502. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  503. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  504. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  505. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  506. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  507. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  508. at scala.Option.getOrElse(Option.scala:121)
  509. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  510. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  511. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  512. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  513. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  514. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  515. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  516. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  517. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  518. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  519. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  520. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  521. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  522. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  523. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  524. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  525. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  526. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  527. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  528. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  529. at $line3.$read$$iw$$iw.<init>(<console>:15)
  530. at $line3.$read$$iw.<init>(<console>:42)
  531. at $line3.$read.<init>(<console>:44)
  532. at $line3.$read$.<init>(<console>:48)
  533. at $line3.$read$.<clinit>(<console>)
  534. at $line3.$eval$.$print$lzycompute(<console>:7)
  535. at $line3.$eval$.$print(<console>:6)
  536. at $line3.$eval.$print(<console>)
  537. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  538. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  539. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  540. at java.lang.reflect.Method.invoke(Method.java:498)
  541. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  542. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  543. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  544. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  545. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  546. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  547. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  548. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  549. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  550. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  551. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  552. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  553. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  554. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  555. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  556. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  557. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  558. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  559. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  560. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  561. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  562. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  563. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  564. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  565. at org.apache.spark.repl.Main$.main(Main.scala:51)
  566. at org.apache.spark.repl.Main.main(Main.scala)
  567. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  568. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  569. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  570. at java.lang.reflect.Method.invoke(Method.java:498)
  571. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  572. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  573. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  574. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  575. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  576. Caused by: java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  577. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  578. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  579. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  580. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  581. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  582. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  583. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  584. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  585. at java.security.AccessController.doPrivileged(Native Method)
  586. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  587. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  588. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  589. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  590. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  591. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  592. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  593. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  594. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  595. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  596. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  597. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  598. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  599. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  600. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  601. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  602. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  603. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  604. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  605. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  606. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  607. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  608. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  609. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  610. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  611. at java.lang.reflect.Method.invoke(Method.java:498)
  612. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  613. at java.security.AccessController.doPrivileged(Native Method)
  614. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  615. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  616. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  617. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  618. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  619. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  620. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  621. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  622. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  623. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  624. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  625. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  626. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  627. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  628. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  629. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  630. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  631. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  632. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  633. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  634. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  635. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  636. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  637. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  638. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  639. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  640. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  641. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  642. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  643. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  644. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  645. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  646. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  647. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  648. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  649. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  650. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  651. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  652. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  653. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  654. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  655. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  656. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  657. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  658. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  659. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  660. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  661. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  662. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  663. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  664. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  665. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  666. at scala.Option.getOrElse(Option.scala:121)
  667. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  668. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  669. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  670. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  671. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  672. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  673. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  674. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  675. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  676. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  677. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  678. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  679. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  680. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  681. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  682. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  683. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  684. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  685. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  686. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  687. at $line3.$read$$iw$$iw.<init>(<console>:15)
  688. at $line3.$read$$iw.<init>(<console>:42)
  689. at $line3.$read.<init>(<console>:44)
  690. at $line3.$read$.<init>(<console>:48)
  691. at $line3.$read$.<clinit>(<console>)
  692. at $line3.$eval$.$print$lzycompute(<console>:7)
  693. at $line3.$eval$.$print(<console>:6)
  694. at $line3.$eval.$print(<console>)
  695.  
  696. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  697. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  698. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  699. at java.lang.reflect.Method.invoke(Method.java:498)
  700. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  701. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  702. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  703. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  704. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  705. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  706. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  707. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  708. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  709. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  710. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  711. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  712. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  713. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  714. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  715. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  716. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  717. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  718. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  719. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  720. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  721. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  722. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  723. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  724. at org.apache.spark.repl.Main$.main(Main.scala:51)
  725. at org.apache.spark.repl.Main.main(Main.scala)
  726. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  727. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  728. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  729. at java.lang.reflect.Method.invoke(Method.java:498)
  730. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  731. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  732. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  733. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  734. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  735. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  736. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  737. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  738. ... 156 more
  739. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  740. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  741. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  742. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  743. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  744. at java.security.AccessController.doPrivileged(Native Method)
  745. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  746. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  747. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  748. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  749. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  750. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  751. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  752. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  753. at java.security.AccessController.doPrivileged(Native Method)
  754. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  755. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  756. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  757. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  758. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  759. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  760. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  761. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  762. at java.security.AccessController.doPrivileged(Native Method)
  763. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  764. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  765. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  766. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  767. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  768. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  769. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  770. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  771. at java.security.AccessController.doPrivileged(Native Method)
  772. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  773. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  774. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  775. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  776. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  777. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  778. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  779. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  780. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  781. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  782. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  783. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  784. at java.security.AccessController.doPrivileged(Native Method)
  785. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  786. ... 153 more
  787. ------
  788.  
  789. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  790. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  791. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  792. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  793. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  794. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  795. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  796. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  797. ... 138 more
  798. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  799. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  800. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  801. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  802. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  803. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  804. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  805. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  806. at java.security.AccessController.doPrivileged(Native Method)
  807. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  808. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  809. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  810. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  811. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  812. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  813. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  814. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  815. ... 140 more
  816. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  817. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  818. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  819. ... 156 more
  820. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  821. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  822. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  823. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  824. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  825. at java.security.AccessController.doPrivileged(Native Method)
  826. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  827. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  828. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  829. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  830. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  831. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  832. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  833. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  834. at java.security.AccessController.doPrivileged(Native Method)
  835. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  836. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  837. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  838. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  839. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  840. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  841. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  842. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  843. at java.security.AccessController.doPrivileged(Native Method)
  844. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  845. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  846. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  847. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  848. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  849. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  850. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  851. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  852. at java.security.AccessController.doPrivileged(Native Method)
  853. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  854. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  855. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  856. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  857. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  858. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  859. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  860. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  861. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  862. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  863. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  864. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  865. at java.security.AccessController.doPrivileged(Native Method)
  866. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  867. ... 153 more
  868. Nested Throwables StackTrace:
  869. java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  870. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  871. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  872. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  873. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  874. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  875. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  876. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  877. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  878. at java.security.AccessController.doPrivileged(Native Method)
  879. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  880. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  881. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  882. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  883. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  884. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  885. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  886. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  887. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  888. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  889. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  890. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  891. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  892. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  893. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  894. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  895. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  896. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  897. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  898. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  899. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  900. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  901. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  902. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  903. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  904. at java.lang.reflect.Method.invoke(Method.java:498)
  905. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  906. at java.security.AccessController.doPrivileged(Native Method)
  907. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  908. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  909. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  910. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  911. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  912. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  913. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  914. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  915. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  916. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  917. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  918. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  919. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  920. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  921. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  922. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  923. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  924. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  925. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  926. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  927. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  928. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  929. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  930. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  931. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  932. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  933. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  934. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  935. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  936. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  937. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  938. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  939. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  940. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  941. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  942. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  943. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  944. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  945. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  946. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  947. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  948. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  949. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  950. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  951. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  952. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  953. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  954. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  955. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  956. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  957. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  958. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  959. at scala.Option.getOrElse(Option.scala:121)
  960. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  961. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  962. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  963. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  964. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  965. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  966. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  967. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  968. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  969. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  970. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  971. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  972. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  973. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  974. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  975. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  976. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  977. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  978. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  979. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  980. at $line3.$read$$iw$$iw.<init>(<console>:15)
  981. at $line3.$read$$iw.<init>(<console>:42)
  982. at $line3.$read.<init>(<console>:44)
  983. at $line3.$read$.<init>(<console>:48)
  984. at $line3.$read$.<clinit>(<console>)
  985. at $line3.$eval$.$print$lzycompute(<console>:7)
  986. at $line3.$eval$.$print(<console>:6)
  987. at $line3.$eval.$print(<console>)
  988. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  989. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  990. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  991. at java.lang.reflect.Method.invoke(Method.java:498)
  992. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  993. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  994. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  995. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  996. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  997. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  998. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  999. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  1000. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  1001. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  1002. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  1003. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  1004. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  1005. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  1006. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  1007. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  1008. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  1009. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  1010. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  1011. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  1012. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  1013. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  1014. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  1015. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  1016. at org.apache.spark.repl.Main$.main(Main.scala:51)
  1017. at org.apache.spark.repl.Main.main(Main.scala)
  1018. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1019.  
  1020. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1021. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1022. at java.lang.reflect.Method.invoke(Method.java:498)
  1023. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  1024. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  1025. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  1026. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  1027. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  1028. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  1029. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1030. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  1031. ... 156 more
  1032. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  1033. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1034. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1035. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  1036. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  1037. at java.security.AccessController.doPrivileged(Native Method)
  1038. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  1039. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  1040. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1041. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1042. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1043. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1044. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1045. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  1046. at java.security.AccessController.doPrivileged(Native Method)
  1047. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  1048. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  1049. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1050. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1051. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1052. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1053. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1054. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  1055. at java.security.AccessController.doPrivileged(Native Method)
  1056. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  1057. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  1058. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1059. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1060. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1061. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1062. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1063. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  1064. at java.security.AccessController.doPrivileged(Native Method)
  1065. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  1066. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  1067. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  1068. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1069. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1070. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  1071. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  1072. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  1073. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  1074. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  1075. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1076. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1077. at java.security.AccessController.doPrivileged(Native Method)
  1078. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  1079. ... 153 more
  1080. ------
  1081.  
  1082. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1083. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1084. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1085.  
  1086. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1087. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  1088. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  1089. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  1090. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  1091. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  1092. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1093. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1094. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1095. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1096. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  1097. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  1098. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  1099. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  1100. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  1101. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  1102. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  1103. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1104. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1105. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1106. at java.lang.reflect.Method.invoke(Method.java:498)
  1107. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  1108. at java.security.AccessController.doPrivileged(Native Method)
  1109. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  1110. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  1111. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  1112. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  1113. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  1114. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  1115. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  1116. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  1117. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  1118. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  1119. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  1120. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  1121. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  1122. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  1123. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  1124. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  1125. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  1126. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  1127. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  1128. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  1129. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  1130. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1131. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1132. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1133. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1134. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  1135. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  1136. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  1137. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  1138. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  1139. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  1140. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  1141. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  1142. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  1143. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  1144. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  1145. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1146. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1147. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1148. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1149. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  1150. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  1151. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  1152. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  1153. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1154. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1155. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1156. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1157. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  1158. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  1159. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  1160. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  1161. at scala.Option.getOrElse(Option.scala:121)
  1162. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  1163. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  1164. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  1165. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  1166. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1167. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1168. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1169. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1170. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  1171. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  1172. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  1173. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  1174. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  1175. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  1176. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  1177. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  1178. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  1179. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  1180. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  1181. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  1182. at $line3.$read$$iw$$iw.<init>(<console>:15)
  1183. at $line3.$read$$iw.<init>(<console>:42)
  1184. at $line3.$read.<init>(<console>:44)
  1185. at $line3.$read$.<init>(<console>:48)
  1186. at $line3.$read$.<clinit>(<console>)
  1187. at $line3.$eval$.$print$lzycompute(<console>:7)
  1188. at $line3.$eval$.$print(<console>:6)
  1189. at $line3.$eval.$print(<console>)
  1190. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1191. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1192. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1193. at java.lang.reflect.Method.invoke(Method.java:498)
  1194. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  1195. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  1196. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  1197. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  1198. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  1199. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  1200. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  1201. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  1202. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  1203. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  1204. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  1205. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  1206. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  1207. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  1208. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  1209. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  1210. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  1211. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  1212. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  1213. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  1214. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  1215. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  1216. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  1217. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  1218. at org.apache.spark.repl.Main$.main(Main.scala:51)
  1219. at org.apache.spark.repl.Main.main(Main.scala)
  1220. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1221. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1222. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1223. at java.lang.reflect.Method.invoke(Method.java:498)
  1224. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  1225. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  1226. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  1227. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  1228. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  1229. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  1230. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1231. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1232. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  1233. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  1234. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  1235. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1236. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1237. at java.security.AccessController.doPrivileged(Native Method)
  1238. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  1239. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1240. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1241. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  1242. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  1243. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  1244. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  1245. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  1246. ... 140 more
  1247. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  1248. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1249. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  1250. ... 156 more
  1251. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  1252. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1253. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1254. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  1255. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  1256. at java.security.AccessController.doPrivileged(Native Method)
  1257. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  1258. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  1259. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1260. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1261. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1262. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1263. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1264. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  1265.  
  1266. at java.security.AccessController.doPrivileged(Native Method)
  1267. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  1268. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  1269. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1270. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1271. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1272. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1273. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1274. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  1275. at java.security.AccessController.doPrivileged(Native Method)
  1276. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  1277. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  1278. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1279. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1280. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1281. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1282. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1283. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  1284. at java.security.AccessController.doPrivileged(Native Method)
  1285. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  1286. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  1287. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  1288. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1289. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1290. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  1291. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  1292. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  1293. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  1294. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  1295. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1296. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1297. at java.security.AccessController.doPrivileged(Native Method)
  1298. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  1299. ... 153 more
  1300. 17/03/30 08:43:41 WARN HiveMetaStore: Retrying creating default database after error: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  1301.  
  1302. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  1303. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1304. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1305. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  1306. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  1307. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  1308. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1309. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1310. at java.security.AccessController.doPrivileged(Native Method)
  1311. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  1312. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1313. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1314. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  1315. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  1316. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  1317. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  1318. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  1319. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  1320. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  1321. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  1322. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1323. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1324. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1325. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1326. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  1327. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  1328. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  1329. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  1330. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  1331. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  1332. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  1333. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1334. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1335. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1336. at java.lang.reflect.Method.invoke(Method.java:498)
  1337. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  1338. at java.security.AccessController.doPrivileged(Native Method)
  1339. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  1340. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  1341. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  1342. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  1343. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  1344. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  1345. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  1346. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  1347. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  1348. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  1349. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  1350. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  1351. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  1352. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  1353. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  1354. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  1355. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  1356. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  1357. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  1358. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  1359. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  1360. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1361. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1362. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1363. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1364. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  1365. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  1366. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  1367. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  1368. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  1369. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  1370. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  1371. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  1372. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  1373. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  1374. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  1375. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1376. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1377. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1378. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1379. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  1380. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  1381. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  1382. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  1383. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1384. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1385. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1386. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1387. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  1388. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  1389. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  1390. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  1391. at scala.Option.getOrElse(Option.scala:121)
  1392. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  1393. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  1394. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  1395. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  1396. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1397. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1398. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1399. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1400. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  1401. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  1402. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  1403. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  1404. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  1405. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  1406. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  1407. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  1408. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  1409. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  1410. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  1411. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  1412. at $line3.$read$$iw$$iw.<init>(<console>:15)
  1413. at $line3.$read$$iw.<init>(<console>:42)
  1414. at $line3.$read.<init>(<console>:44)
  1415. at $line3.$read$.<init>(<console>:48)
  1416. at $line3.$read$.<clinit>(<console>)
  1417. at $line3.$eval$.$print$lzycompute(<console>:7)
  1418. at $line3.$eval$.$print(<console>:6)
  1419. at $line3.$eval.$print(<console>)
  1420. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1421. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1422. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1423. at java.lang.reflect.Method.invoke(Method.java:498)
  1424. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  1425. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  1426. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  1427. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  1428. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  1429. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  1430. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  1431. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  1432. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  1433. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  1434. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  1435. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  1436. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  1437. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  1438. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  1439. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  1440. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  1441. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  1442. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  1443. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  1444. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  1445. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  1446. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  1447. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  1448. at org.apache.spark.repl.Main$.main(Main.scala:51)
  1449. at org.apache.spark.repl.Main.main(Main.scala)
  1450. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1451. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1452. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1453. at java.lang.reflect.Method.invoke(Method.java:498)
  1454. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  1455. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  1456. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  1457. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  1458. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  1459. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  1460. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1461. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  1462. ... 156 more
  1463. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  1464. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1465. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1466. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  1467. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  1468. at java.security.AccessController.doPrivileged(Native Method)
  1469. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  1470. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  1471. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1472. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1473. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1474. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1475. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1476. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  1477. at java.security.AccessController.doPrivileged(Native Method)
  1478. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  1479. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  1480. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1481. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1482. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1483. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1484. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1485. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  1486. at java.security.AccessController.doPrivileged(Native Method)
  1487. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  1488. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  1489. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1490. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1491. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1492. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1493. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1494. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  1495. at java.security.AccessController.doPrivileged(Native Method)
  1496. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  1497. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  1498. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  1499. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1500. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1501. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  1502. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  1503. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  1504. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  1505. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  1506. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1507. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1508. at java.security.AccessController.doPrivileged(Native Method)
  1509. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  1510. ... 153 more
  1511. ------
  1512.  
  1513.  
  1514. javax.jdo.JDOFatalDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  1515.  
  1516. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  1517. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1518. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1519. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  1520. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  1521. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  1522. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1523. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1524. at java.security.AccessController.doPrivileged(Native Method)
  1525. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  1526. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1527. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1528. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  1529. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  1530. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  1531. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  1532. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  1533. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  1534. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  1535. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  1536. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1537. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1538. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1539. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1540. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  1541. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  1542. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  1543. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  1544. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  1545. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  1546. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  1547. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1548. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1549. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1550. at java.lang.reflect.Method.invoke(Method.java:498)
  1551. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  1552. at java.security.AccessController.doPrivileged(Native Method)
  1553. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  1554. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  1555. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  1556. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  1557. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  1558. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  1559. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  1560. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  1561. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  1562. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  1563. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  1564. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  1565. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  1566. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  1567. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  1568. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  1569. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  1570. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  1571. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  1572. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  1573. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  1574. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1575. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1576. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1577. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1578. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  1579. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  1580. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  1581. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  1582. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  1583. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  1584. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  1585. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  1586. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  1587. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  1588. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  1589. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1590. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1591. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1592. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1593. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  1594. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  1595. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  1596. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  1597. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1598. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1599. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1600. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1601. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  1602. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  1603. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  1604. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  1605. at scala.Option.getOrElse(Option.scala:121)
  1606. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  1607. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  1608. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  1609. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  1610. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1611. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1612. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1613. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1614. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  1615. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  1616. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  1617. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  1618. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  1619. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  1620. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  1621. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  1622. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  1623. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  1624. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  1625. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  1626. at $line3.$read$$iw$$iw.<init>(<console>:15)
  1627. at $line3.$read$$iw.<init>(<console>:42)
  1628. at $line3.$read.<init>(<console>:44)
  1629. at $line3.$read$.<init>(<console>:48)
  1630. at $line3.$read$.<clinit>(<console>)
  1631. at $line3.$eval$.$print$lzycompute(<console>:7)
  1632. at $line3.$eval$.$print(<console>:6)
  1633. at $line3.$eval.$print(<console>)
  1634. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1635. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1636. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1637. at java.lang.reflect.Method.invoke(Method.java:498)
  1638. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  1639. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  1640. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  1641. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  1642. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  1643. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  1644. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  1645. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  1646. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  1647. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  1648. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  1649. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  1650. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  1651. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  1652. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  1653. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  1654. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  1655. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  1656. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  1657. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  1658. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  1659. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  1660. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  1661. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  1662. at org.apache.spark.repl.Main$.main(Main.scala:51)
  1663. at org.apache.spark.repl.Main.main(Main.scala)
  1664. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1665. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1666. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1667. at java.lang.reflect.Method.invoke(Method.java:498)
  1668. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  1669. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  1670. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  1671. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  1672. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  1673. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  1674. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1675. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  1676. ... 156 more
  1677. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  1678. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1679. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1680. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  1681. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  1682. at java.security.AccessController.doPrivileged(Native Method)
  1683. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  1684. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  1685. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1686. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1687. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1688. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1689. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1690. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  1691. at java.security.AccessController.doPrivileged(Native Method)
  1692. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  1693. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  1694. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1695. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1696. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1697. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1698. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1699. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  1700. at java.security.AccessController.doPrivileged(Native Method)
  1701. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  1702. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  1703. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1704. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1705. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1706. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1707. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1708. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  1709. at java.security.AccessController.doPrivileged(Native Method)
  1710. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  1711. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  1712. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  1713. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1714. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1715. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  1716. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  1717. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  1718. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  1719. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  1720. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1721. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1722. at java.security.AccessController.doPrivileged(Native Method)
  1723. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  1724. ... 153 more
  1725. ------
  1726.  
  1727. at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:436)
  1728. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:788)
  1729. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  1730. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  1731. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1732. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1733. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1734. at java.lang.reflect.Method.invoke(Method.java:498)
  1735. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  1736. at java.security.AccessController.doPrivileged(Native Method)
  1737. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  1738. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  1739. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  1740. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  1741. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  1742. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  1743. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  1744. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  1745. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  1746. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  1747. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  1748. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  1749. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  1750. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  1751. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  1752. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  1753. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  1754. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  1755. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  1756. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  1757. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  1758. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1759. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1760. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1761. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1762. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  1763. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  1764. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  1765. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  1766. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  1767. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  1768. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  1769. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  1770. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  1771. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  1772. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  1773. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1774. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1775. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1776. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1777. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  1778. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  1779. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  1780. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  1781. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1782. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1783. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1784. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1785. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  1786. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  1787. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  1788. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  1789. at scala.Option.getOrElse(Option.scala:121)
  1790. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  1791. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  1792. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  1793. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  1794. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1795. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1796. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1797. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1798. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  1799. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  1800. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  1801. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  1802. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  1803. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  1804. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  1805. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  1806. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  1807. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  1808. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  1809. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  1810. at $line3.$read$$iw$$iw.<init>(<console>:15)
  1811. at $line3.$read$$iw.<init>(<console>:42)
  1812. at $line3.$read.<init>(<console>:44)
  1813. at $line3.$read$.<init>(<console>:48)
  1814. at $line3.$read$.<clinit>(<console>)
  1815. at $line3.$eval$.$print$lzycompute(<console>:7)
  1816. at $line3.$eval$.$print(<console>:6)
  1817. at $line3.$eval.$print(<console>)
  1818. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1819. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1820. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1821. at java.lang.reflect.Method.invoke(Method.java:498)
  1822. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  1823. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  1824. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  1825. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  1826. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  1827. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  1828. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  1829. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  1830. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  1831. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  1832. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  1833. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  1834. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  1835. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  1836. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  1837. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  1838. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  1839. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  1840. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  1841. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  1842. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  1843. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  1844. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  1845. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  1846. at org.apache.spark.repl.Main$.main(Main.scala:51)
  1847. at org.apache.spark.repl.Main.main(Main.scala)
  1848. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1849. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1850. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1851. at java.lang.reflect.Method.invoke(Method.java:498)
  1852. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  1853. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  1854. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  1855. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  1856. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  1857. NestedThrowablesStackTrace:
  1858. java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  1859. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  1860. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1861. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1862. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  1863. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  1864. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  1865. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1866. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1867. at java.security.AccessController.doPrivileged(Native Method)
  1868. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  1869. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1870. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1871. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  1872. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  1873. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  1874. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  1875. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  1876. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  1877. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  1878. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  1879. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1880. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1881. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1882. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1883. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  1884. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  1885. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  1886. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  1887. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  1888. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  1889. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  1890. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1891. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1892. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1893. at java.lang.reflect.Method.invoke(Method.java:498)
  1894. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  1895. at java.security.AccessController.doPrivileged(Native Method)
  1896. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  1897. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  1898. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  1899. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  1900. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  1901. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  1902. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  1903. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  1904. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  1905. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  1906. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  1907. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  1908. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  1909. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  1910. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  1911. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  1912. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  1913. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  1914. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  1915. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  1916. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  1917. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1918. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1919. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1920. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1921. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  1922. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  1923. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  1924. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  1925. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  1926. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  1927. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  1928. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  1929. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  1930. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  1931. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  1932. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1933. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1934. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1935. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1936. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  1937. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  1938. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  1939. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  1940. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1941. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1942. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1943. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1944. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  1945. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  1946. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  1947. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  1948. at scala.Option.getOrElse(Option.scala:121)
  1949. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  1950. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  1951. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  1952. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  1953. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1954. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1955. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1956. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1957. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  1958. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  1959. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  1960. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  1961. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  1962. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  1963. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  1964. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  1965. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  1966. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  1967. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  1968. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  1969. at $line3.$read$$iw$$iw.<init>(<console>:15)
  1970. at $line3.$read$$iw.<init>(<console>:42)
  1971. at $line3.$read.<init>(<console>:44)
  1972. at $line3.$read$.<init>(<console>:48)
  1973. at $line3.$read$.<clinit>(<console>)
  1974. at $line3.$eval$.$print$lzycompute(<console>:7)
  1975. at $line3.$eval$.$print(<console>:6)
  1976. at $line3.$eval.$print(<console>)
  1977. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1978. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1979. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1980. at java.lang.reflect.Method.invoke(Method.java:498)
  1981. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  1982. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  1983. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  1984. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  1985. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  1986. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  1987. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  1988. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  1989. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  1990. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  1991. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  1992. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  1993. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  1994. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  1995. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  1996. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  1997. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  1998. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  1999. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  2000. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  2001. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  2002. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  2003. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  2004. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  2005. at org.apache.spark.repl.Main$.main(Main.scala:51)
  2006. at org.apache.spark.repl.Main.main(Main.scala)
  2007. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2008. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2009. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2010. at java.lang.reflect.Method.invoke(Method.java:498)
  2011. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  2012. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  2013. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  2014. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  2015. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  2016. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  2017. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2018. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  2019. ... 156 more
  2020. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  2021. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2022. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2023. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  2024. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  2025. at java.security.AccessController.doPrivileged(Native Method)
  2026. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  2027. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  2028. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2029. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2030. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2031. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2032. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2033. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  2034. at java.security.AccessController.doPrivileged(Native Method)
  2035. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  2036. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  2037. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2038. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2039. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2040. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2041. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2042. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  2043. at java.security.AccessController.doPrivileged(Native Method)
  2044. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  2045. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  2046. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2047. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2048. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2049. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2050. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2051. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  2052. at java.security.AccessController.doPrivileged(Native Method)
  2053. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  2054. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  2055. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  2056. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2057. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2058. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  2059. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  2060. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  2061. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  2062. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  2063. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2064. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2065. at java.security.AccessController.doPrivileged(Native Method)
  2066. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  2067. ... 153 more
  2068. ------
  2069.  
  2070. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2071. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2072. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2073. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2074. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  2075. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  2076. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  2077. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  2078. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  2079. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2080. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2081. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2082. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2083. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  2084. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  2085. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  2086. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  2087. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  2088. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  2089. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  2090. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2091. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2092. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2093. at java.lang.reflect.Method.invoke(Method.java:498)
  2094. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  2095. at java.security.AccessController.doPrivileged(Native Method)
  2096. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  2097. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  2098. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  2099. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  2100. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  2101. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  2102. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  2103. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  2104. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  2105. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  2106. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  2107. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  2108. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  2109. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  2110. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  2111. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  2112. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  2113. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  2114. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  2115. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  2116. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  2117. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2118. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2119. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2120. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2121. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  2122. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  2123. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  2124. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  2125. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  2126. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  2127. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  2128. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  2129. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  2130. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  2131. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  2132. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2133. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2134. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2135. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2136. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  2137. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  2138. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  2139. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  2140. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2141. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2142. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2143. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2144. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  2145. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  2146. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  2147. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  2148. at scala.Option.getOrElse(Option.scala:121)
  2149. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  2150. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  2151. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  2152. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  2153. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2154. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2155. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2156. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2157. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  2158. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  2159. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  2160. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  2161. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  2162. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  2163. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  2164. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  2165. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  2166. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  2167. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  2168. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  2169. at $line3.$read$$iw$$iw.<init>(<console>:15)
  2170. at $line3.$read$$iw.<init>(<console>:42)
  2171. at $line3.$read.<init>(<console>:44)
  2172. at $line3.$read$.<init>(<console>:48)
  2173. at $line3.$read$.<clinit>(<console>)
  2174. at $line3.$eval$.$print$lzycompute(<console>:7)
  2175. at $line3.$eval$.$print(<console>:6)
  2176. at $line3.$eval.$print(<console>)
  2177. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2178. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2179. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2180. at java.lang.reflect.Method.invoke(Method.java:498)
  2181. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  2182. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  2183. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  2184. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  2185. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  2186. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  2187. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  2188. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  2189. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  2190. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  2191. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  2192. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  2193. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  2194. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  2195. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  2196. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  2197. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  2198. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  2199. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  2200. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  2201. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  2202. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  2203. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  2204. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  2205. at org.apache.spark.repl.Main$.main(Main.scala:51)
  2206. at org.apache.spark.repl.Main.main(Main.scala)
  2207. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2208. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2209. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2210. at java.lang.reflect.Method.invoke(Method.java:498)
  2211. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  2212. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  2213. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  2214. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  2215. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  2216. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  2217. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2218. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2219. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  2220. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  2221. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  2222. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2223. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2224. at java.security.AccessController.doPrivileged(Native Method)
  2225. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  2226. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2227. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2228. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  2229. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  2230. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  2231. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  2232. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  2233. ... 140 more
  2234. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  2235. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2236. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  2237. ... 156 more
  2238. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  2239. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2240. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2241. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  2242. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  2243. at java.security.AccessController.doPrivileged(Native Method)
  2244. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  2245. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  2246. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2247. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2248. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2249. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2250. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2251. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  2252. at java.security.AccessController.doPrivileged(Native Method)
  2253. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  2254. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  2255. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2256. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2257. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2258. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2259. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2260. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  2261. at java.security.AccessController.doPrivileged(Native Method)
  2262. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  2263. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  2264. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2265. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2266. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2267. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2268. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2269. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  2270. at java.security.AccessController.doPrivileged(Native Method)
  2271. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  2272. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  2273. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  2274. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2275. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2276. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  2277. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  2278. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  2279. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  2280. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  2281. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2282. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2283. at java.security.AccessController.doPrivileged(Native Method)
  2284. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  2285. ... 153 more
  2286. 17/03/30 08:43:41 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/Users/james/spark/jars/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/Users/james/spark-2.1.0-bin-hadoop2.7/jars/datanucleus-api-jdo-3.2.6.jar."
  2287. 17/03/30 08:43:41 WARN General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/Users/james/spark-2.1.0-bin-hadoop2.7/jars/datanucleus-core-3.2.10.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/Users/james/spark/jars/datanucleus-core-3.2.10.jar."
  2288. 17/03/30 08:43:41 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/Users/james/spark-2.1.0-bin-hadoop2.7/jars/datanucleus-rdbms-3.2.9.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/Users/james/spark/jars/datanucleus-rdbms-3.2.9.jar."
  2289. 17/03/30 08:43:41 ERROR Schema: Failed initialising database.
  2290. Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  2291. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  2292. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2293. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2294. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  2295. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  2296. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  2297. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2298. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2299. at java.security.AccessController.doPrivileged(Native Method)
  2300. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  2301. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2302. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2303. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  2304. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  2305. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  2306. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  2307. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  2308. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  2309. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  2310. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  2311. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2312. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2313. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2314. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2315. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  2316. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  2317. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  2318. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  2319. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  2320. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  2321. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  2322. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2323. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2324. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2325. at java.lang.reflect.Method.invoke(Method.java:498)
  2326. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  2327. at java.security.AccessController.doPrivileged(Native Method)
  2328. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  2329. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  2330. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  2331. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  2332. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  2333. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  2334. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  2335. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  2336. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  2337. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  2338. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  2339. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  2340. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  2341. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  2342. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  2343. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  2344. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  2345. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  2346. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  2347. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  2348. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  2349. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2350. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2351. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2352. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2353. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  2354. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  2355. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  2356. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  2357. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  2358. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  2359. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  2360. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  2361. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  2362. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  2363. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  2364. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2365. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2366. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2367. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2368. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  2369. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  2370. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  2371. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  2372. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2373. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2374. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2375. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2376. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  2377. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  2378. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  2379. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  2380. at scala.Option.getOrElse(Option.scala:121)
  2381. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  2382. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  2383. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  2384. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  2385. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2386. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2387. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2388. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2389. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  2390. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  2391. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  2392. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  2393. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  2394. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  2395. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  2396. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  2397. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  2398. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  2399. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  2400. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  2401. at $line3.$read$$iw$$iw.<init>(<console>:15)
  2402. at $line3.$read$$iw.<init>(<console>:42)
  2403. at $line3.$read.<init>(<console>:44)
  2404. at $line3.$read$.<init>(<console>:48)
  2405. at $line3.$read$.<clinit>(<console>)
  2406. at $line3.$eval$.$print$lzycompute(<console>:7)
  2407. at $line3.$eval$.$print(<console>:6)
  2408. at $line3.$eval.$print(<console>)
  2409. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2410. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2411. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2412. at java.lang.reflect.Method.invoke(Method.java:498)
  2413. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  2414. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  2415. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  2416. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  2417. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  2418. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  2419. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  2420. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  2421. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  2422. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  2423. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  2424. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  2425. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  2426. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  2427. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  2428. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  2429. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  2430. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  2431. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  2432. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  2433. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  2434. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  2435. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  2436. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  2437. at org.apache.spark.repl.Main$.main(Main.scala:51)
  2438. at org.apache.spark.repl.Main.main(Main.scala)
  2439. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2440. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2441. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2442. at java.lang.reflect.Method.invoke(Method.java:498)
  2443. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  2444. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  2445. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  2446. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  2447. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  2448. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  2449. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2450. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  2451. ... 156 more
  2452. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  2453. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2454. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2455. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  2456. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  2457. at java.security.AccessController.doPrivileged(Native Method)
  2458. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  2459. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  2460. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2461. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2462. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2463. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2464. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2465. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  2466. at java.security.AccessController.doPrivileged(Native Method)
  2467. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  2468. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  2469. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2470. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2471. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2472. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2473. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2474. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  2475. at java.security.AccessController.doPrivileged(Native Method)
  2476. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  2477. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  2478. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2479. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2480. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2481. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2482. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2483. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  2484. at java.security.AccessController.doPrivileged(Native Method)
  2485. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  2486. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  2487. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  2488. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2489. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2490. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  2491. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  2492. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  2493. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  2494. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  2495. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2496. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2497. at java.security.AccessController.doPrivileged(Native Method)
  2498. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  2499. ... 153 more
  2500. ------
  2501.  
  2502. org.datanucleus.exceptions.NucleusDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  2503. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  2504. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2505. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2506. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  2507. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  2508. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  2509. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2510. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2511. at java.security.AccessController.doPrivileged(Native Method)
  2512. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  2513. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2514. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2515. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  2516. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  2517. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  2518. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  2519. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  2520. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  2521. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  2522. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  2523. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2524. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2525. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2526. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2527. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  2528. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  2529. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  2530. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  2531. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  2532. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  2533. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  2534. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2535. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2536. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2537. at java.lang.reflect.Method.invoke(Method.java:498)
  2538. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  2539. at java.security.AccessController.doPrivileged(Native Method)
  2540. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  2541. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  2542. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  2543. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  2544. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  2545. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  2546. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  2547. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  2548. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  2549. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  2550. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  2551. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  2552. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  2553. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  2554. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  2555. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  2556. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  2557. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  2558. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  2559. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  2560. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  2561. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2562. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2563. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2564. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2565. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  2566. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  2567. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  2568. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  2569. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  2570. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  2571. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  2572. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  2573. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  2574. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  2575. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  2576. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2577. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2578. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2579. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2580. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  2581. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  2582. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  2583. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  2584. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2585. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2586. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2587. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2588. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  2589. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  2590. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  2591. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  2592. at scala.Option.getOrElse(Option.scala:121)
  2593. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  2594. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  2595. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  2596. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  2597. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2598. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2599. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2600. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2601. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  2602. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  2603. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  2604. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  2605. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  2606. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  2607. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  2608. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  2609. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  2610. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  2611.  
  2612. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  2613. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  2614. at $line3.$read$$iw$$iw.<init>(<console>:15)
  2615. at $line3.$read$$iw.<init>(<console>:42)
  2616. at $line3.$read.<init>(<console>:44)
  2617. at $line3.$read$.<init>(<console>:48)
  2618. at $line3.$read$.<clinit>(<console>)
  2619. at $line3.$eval$.$print$lzycompute(<console>:7)
  2620. at $line3.$eval$.$print(<console>:6)
  2621. at $line3.$eval.$print(<console>)
  2622. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2623. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2624. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2625. at java.lang.reflect.Method.invoke(Method.java:498)
  2626. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  2627. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  2628. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  2629. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  2630. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  2631. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  2632. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  2633. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  2634. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  2635. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  2636. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  2637. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  2638. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  2639. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  2640. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  2641. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  2642. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  2643. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  2644. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  2645. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  2646. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  2647. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  2648. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  2649. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  2650. at org.apache.spark.repl.Main$.main(Main.scala:51)
  2651. at org.apache.spark.repl.Main.main(Main.scala)
  2652. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2653. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2654. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2655. at java.lang.reflect.Method.invoke(Method.java:498)
  2656. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  2657. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  2658. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  2659. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  2660. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  2661. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  2662. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2663. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  2664. ... 156 more
  2665. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  2666. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2667. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2668. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  2669. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  2670. at java.security.AccessController.doPrivileged(Native Method)
  2671. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  2672. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  2673. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2674. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2675. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2676. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2677. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2678. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  2679. at java.security.AccessController.doPrivileged(Native Method)
  2680. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  2681. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  2682. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2683. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2684. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2685. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2686. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2687. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  2688. at java.security.AccessController.doPrivileged(Native Method)
  2689. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  2690. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  2691. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2692. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2693. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2694. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2695. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2696. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  2697. at java.security.AccessController.doPrivileged(Native Method)
  2698. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  2699. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  2700. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  2701. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2702. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2703. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  2704. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  2705. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  2706. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  2707. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  2708. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2709. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2710. at java.security.AccessController.doPrivileged(Native Method)
  2711. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  2712. ... 153 more
  2713. ------
  2714.  
  2715. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:516)
  2716. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  2717. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2718. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2719. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2720. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2721. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  2722. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  2723. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  2724. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  2725. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  2726. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  2727. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  2728. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2729. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2730. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2731. at java.lang.reflect.Method.invoke(Method.java:498)
  2732. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  2733. at java.security.AccessController.doPrivileged(Native Method)
  2734. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  2735. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  2736. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  2737. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  2738. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  2739. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  2740. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  2741. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  2742. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  2743. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  2744. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  2745. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  2746. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  2747. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  2748. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  2749. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  2750. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  2751. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  2752. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  2753. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  2754. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  2755. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2756. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2757. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2758. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2759. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  2760. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  2761. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  2762. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  2763. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  2764. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  2765. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  2766. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  2767. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  2768. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  2769. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  2770. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2771. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2772. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2773. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2774. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  2775. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  2776. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  2777. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  2778. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2779. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2780. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2781. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2782. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  2783. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  2784. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  2785. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  2786. at scala.Option.getOrElse(Option.scala:121)
  2787. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  2788. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  2789. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  2790. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  2791. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2792. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2793. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2794. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2795. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  2796. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  2797. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  2798. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  2799. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  2800. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  2801. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  2802. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  2803. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  2804. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  2805. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  2806. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  2807. at $line3.$read$$iw$$iw.<init>(<console>:15)
  2808. at $line3.$read$$iw.<init>(<console>:42)
  2809. at $line3.$read.<init>(<console>:44)
  2810. at $line3.$read$.<init>(<console>:48)
  2811. at $line3.$read$.<clinit>(<console>)
  2812. at $line3.$eval$.$print$lzycompute(<console>:7)
  2813. at $line3.$eval$.$print(<console>:6)
  2814. at $line3.$eval.$print(<console>)
  2815. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2816. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2817. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2818. at java.lang.reflect.Method.invoke(Method.java:498)
  2819. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  2820. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  2821. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  2822. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  2823. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  2824. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  2825. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  2826. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  2827. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  2828. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  2829. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  2830. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  2831. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  2832. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  2833. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  2834. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  2835. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  2836. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  2837. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  2838. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  2839. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  2840. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  2841. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  2842. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  2843. at org.apache.spark.repl.Main$.main(Main.scala:51)
  2844. at org.apache.spark.repl.Main.main(Main.scala)
  2845. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2846. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2847. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2848. at java.lang.reflect.Method.invoke(Method.java:498)
  2849. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  2850. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  2851. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  2852. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  2853. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  2854. Caused by: java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  2855. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  2856. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2857. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2858. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  2859. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  2860. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  2861. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2862. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2863. at java.security.AccessController.doPrivileged(Native Method)
  2864. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  2865. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2866. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2867. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  2868. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  2869. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  2870. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  2871. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  2872. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  2873. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  2874. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  2875. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2876. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2877. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2878. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2879. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  2880. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  2881. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  2882. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  2883. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  2884. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  2885. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  2886. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2887. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2888. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2889. at java.lang.reflect.Method.invoke(Method.java:498)
  2890. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  2891. at java.security.AccessController.doPrivileged(Native Method)
  2892. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  2893. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  2894. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  2895. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  2896. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  2897. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  2898. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  2899. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  2900. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  2901. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  2902. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  2903. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  2904. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  2905. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  2906. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  2907. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  2908. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  2909. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  2910. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  2911. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  2912. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  2913. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2914. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2915. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2916. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2917. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  2918. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  2919. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  2920. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  2921. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  2922. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  2923. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  2924. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  2925. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  2926. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  2927. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  2928. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2929. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2930. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2931. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2932. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  2933. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  2934. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  2935. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  2936. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2937. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2938. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2939. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2940. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  2941. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  2942. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  2943. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  2944. at scala.Option.getOrElse(Option.scala:121)
  2945. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  2946. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  2947. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  2948. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  2949. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2950. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2951. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2952. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2953. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  2954. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  2955. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  2956. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  2957. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  2958. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  2959. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  2960. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  2961. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  2962. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  2963. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  2964. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  2965. at $line3.$read$$iw$$iw.<init>(<console>:15)
  2966. at $line3.$read$$iw.<init>(<console>:42)
  2967. at $line3.$read.<init>(<console>:44)
  2968. at $line3.$read$.<init>(<console>:48)
  2969. at $line3.$read$.<clinit>(<console>)
  2970. at $line3.$eval$.$print$lzycompute(<console>:7)
  2971. at $line3.$eval$.$print(<console>:6)
  2972. at $line3.$eval.$print(<console>)
  2973.  
  2974. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2975. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2976. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2977. at java.lang.reflect.Method.invoke(Method.java:498)
  2978. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  2979. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  2980. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  2981. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  2982. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  2983. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  2984. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  2985. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  2986. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  2987. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  2988. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  2989. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  2990. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  2991. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  2992. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  2993. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  2994. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  2995. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  2996. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  2997. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  2998. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  2999. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  3000. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  3001. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  3002. at org.apache.spark.repl.Main$.main(Main.scala:51)
  3003. at org.apache.spark.repl.Main.main(Main.scala)
  3004. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3005. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3006. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3007. at java.lang.reflect.Method.invoke(Method.java:498)
  3008. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  3009. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  3010. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  3011. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  3012. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  3013. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  3014. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3015. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  3016. ... 156 more
  3017. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  3018. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3019. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3020. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  3021. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  3022. at java.security.AccessController.doPrivileged(Native Method)
  3023. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  3024. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  3025. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3026. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3027. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3028. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3029. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3030. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  3031. at java.security.AccessController.doPrivileged(Native Method)
  3032. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  3033. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  3034. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3035. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3036. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3037. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3038. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3039. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  3040. at java.security.AccessController.doPrivileged(Native Method)
  3041. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  3042. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  3043. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3044. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3045. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3046. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3047. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3048. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  3049. at java.security.AccessController.doPrivileged(Native Method)
  3050. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  3051. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  3052. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  3053. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3054. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3055. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  3056. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  3057. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  3058. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  3059. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  3060. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3061. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3062. at java.security.AccessController.doPrivileged(Native Method)
  3063. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  3064. ... 153 more
  3065. ------
  3066.  
  3067. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3068. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3069. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3070. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3071. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  3072. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  3073. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  3074. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  3075. ... 138 more
  3076. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  3077. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3078. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3079. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  3080. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  3081. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  3082. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3083. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3084. at java.security.AccessController.doPrivileged(Native Method)
  3085. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  3086. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3087. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3088. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  3089. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  3090. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  3091. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  3092. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  3093. ... 140 more
  3094. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  3095. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3096. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  3097. ... 156 more
  3098. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  3099. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3100. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3101. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  3102. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  3103. at java.security.AccessController.doPrivileged(Native Method)
  3104. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  3105. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  3106. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3107. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3108. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3109. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3110. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3111. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  3112. at java.security.AccessController.doPrivileged(Native Method)
  3113. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  3114. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  3115. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3116. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3117. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3118. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3119. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3120. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  3121. at java.security.AccessController.doPrivileged(Native Method)
  3122. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  3123. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  3124. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3125. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3126. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3127. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3128. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3129. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  3130. at java.security.AccessController.doPrivileged(Native Method)
  3131. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  3132. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  3133. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  3134. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3135. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3136. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  3137. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  3138. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  3139. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  3140. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  3141. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3142. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3143. at java.security.AccessController.doPrivileged(Native Method)
  3144. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  3145. ... 153 more
  3146. Nested Throwables StackTrace:
  3147. java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  3148. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  3149. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3150. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3151. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  3152. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  3153. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  3154. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3155. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3156. at java.security.AccessController.doPrivileged(Native Method)
  3157. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  3158. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3159. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3160. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  3161. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  3162. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  3163. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  3164. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  3165. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  3166. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  3167. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  3168. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3169. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3170. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3171. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3172. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  3173. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  3174. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  3175. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  3176. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  3177. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  3178. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  3179. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3180. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3181. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3182. at java.lang.reflect.Method.invoke(Method.java:498)
  3183. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  3184. at java.security.AccessController.doPrivileged(Native Method)
  3185. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  3186. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  3187. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  3188. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  3189. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  3190. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  3191. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  3192. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  3193. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  3194. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  3195. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  3196. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  3197. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  3198. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  3199. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  3200. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  3201. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  3202. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  3203. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  3204. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  3205. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  3206. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3207. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3208. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3209. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3210. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  3211. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  3212. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  3213. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  3214. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  3215. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  3216. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  3217. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  3218. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  3219. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  3220. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  3221. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3222. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3223. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3224. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3225. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  3226. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  3227. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  3228. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  3229. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3230. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3231. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3232. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3233. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  3234. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  3235. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  3236. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  3237. at scala.Option.getOrElse(Option.scala:121)
  3238. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  3239. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  3240. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  3241. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  3242. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3243. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3244. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3245. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3246. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  3247. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  3248. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  3249. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  3250. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  3251. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  3252. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  3253. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  3254. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  3255. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  3256. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  3257. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  3258. at $line3.$read$$iw$$iw.<init>(<console>:15)
  3259. at $line3.$read$$iw.<init>(<console>:42)
  3260. at $line3.$read.<init>(<console>:44)
  3261. at $line3.$read$.<init>(<console>:48)
  3262. at $line3.$read$.<clinit>(<console>)
  3263. at $line3.$eval$.$print$lzycompute(<console>:7)
  3264. at $line3.$eval$.$print(<console>:6)
  3265. at $line3.$eval.$print(<console>)
  3266. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3267. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3268. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3269. at java.lang.reflect.Method.invoke(Method.java:498)
  3270. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  3271. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  3272. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  3273. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  3274. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  3275. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  3276. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  3277. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  3278. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  3279. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  3280. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  3281. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  3282. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  3283. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  3284. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  3285. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  3286. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  3287. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  3288. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  3289. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  3290. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  3291. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  3292. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  3293. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  3294. at org.apache.spark.repl.Main$.main(Main.scala:51)
  3295. at org.apache.spark.repl.Main.main(Main.scala)
  3296. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3297.  
  3298. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3299. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3300. at java.lang.reflect.Method.invoke(Method.java:498)
  3301. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  3302. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  3303. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  3304. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  3305. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  3306. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  3307. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3308. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  3309. ... 156 more
  3310. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  3311. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3312. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3313. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  3314. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  3315. at java.security.AccessController.doPrivileged(Native Method)
  3316. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  3317. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  3318. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3319. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3320. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3321. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3322. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3323. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  3324. at java.security.AccessController.doPrivileged(Native Method)
  3325. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  3326. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  3327. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3328. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3329. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3330. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3331. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3332. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  3333. at java.security.AccessController.doPrivileged(Native Method)
  3334. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  3335. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  3336. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3337. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3338. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3339. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3340. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3341. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  3342. at java.security.AccessController.doPrivileged(Native Method)
  3343. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  3344. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  3345. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  3346. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3347. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3348. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  3349. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  3350. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  3351. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  3352. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  3353. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3354. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3355. at java.security.AccessController.doPrivileged(Native Method)
  3356. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  3357. ... 153 more
  3358. ------
  3359.  
  3360. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3361. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3362. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3363. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3364. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  3365. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  3366. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  3367. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  3368. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  3369. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3370. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3371. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3372. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3373. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  3374. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  3375. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  3376. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  3377. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  3378. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  3379. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  3380. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3381. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3382. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3383. at java.lang.reflect.Method.invoke(Method.java:498)
  3384. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  3385. at java.security.AccessController.doPrivileged(Native Method)
  3386. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  3387. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  3388. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  3389. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  3390. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  3391. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  3392. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  3393. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  3394. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  3395. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  3396. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  3397. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  3398. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  3399. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  3400. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  3401. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  3402. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  3403. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  3404. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  3405. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  3406. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  3407. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3408. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3409. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3410. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3411. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  3412. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  3413. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  3414. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  3415. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  3416. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  3417. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  3418. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  3419. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  3420. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  3421. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  3422. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3423. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3424. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3425. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3426. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  3427. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  3428. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  3429. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  3430. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3431. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3432. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3433. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3434. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  3435. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  3436. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  3437. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  3438. at scala.Option.getOrElse(Option.scala:121)
  3439. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  3440. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  3441. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  3442. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  3443. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3444. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3445. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3446. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3447. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  3448. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  3449. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  3450. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  3451. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  3452. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  3453. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  3454. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  3455. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  3456. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  3457. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  3458. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  3459. at $line3.$read$$iw$$iw.<init>(<console>:15)
  3460. at $line3.$read$$iw.<init>(<console>:42)
  3461. at $line3.$read.<init>(<console>:44)
  3462. at $line3.$read$.<init>(<console>:48)
  3463. at $line3.$read$.<clinit>(<console>)
  3464. at $line3.$eval$.$print$lzycompute(<console>:7)
  3465. at $line3.$eval$.$print(<console>:6)
  3466. at $line3.$eval.$print(<console>)
  3467. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3468. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3469. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3470. at java.lang.reflect.Method.invoke(Method.java:498)
  3471. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  3472. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  3473. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  3474. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  3475. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  3476. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  3477. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  3478. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  3479. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  3480. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  3481. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  3482. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  3483. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  3484. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  3485. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  3486. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  3487. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  3488. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  3489. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  3490. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  3491. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  3492. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  3493. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  3494. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  3495. at org.apache.spark.repl.Main$.main(Main.scala:51)
  3496. at org.apache.spark.repl.Main.main(Main.scala)
  3497. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3498. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3499. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3500. at java.lang.reflect.Method.invoke(Method.java:498)
  3501. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  3502. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  3503. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  3504. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  3505. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  3506. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  3507. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3508. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3509. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  3510. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  3511. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  3512. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3513. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3514. at java.security.AccessController.doPrivileged(Native Method)
  3515. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  3516. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3517. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3518. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  3519. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  3520. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  3521. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  3522. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  3523. ... 140 more
  3524. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  3525. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3526. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  3527. ... 156 more
  3528. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  3529. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3530. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3531. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  3532. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  3533. at java.security.AccessController.doPrivileged(Native Method)
  3534. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  3535. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  3536. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3537. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3538. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3539. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3540. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3541. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  3542.  
  3543. at java.security.AccessController.doPrivileged(Native Method)
  3544. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  3545. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  3546. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3547. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3548. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3549. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3550. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3551. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  3552. at java.security.AccessController.doPrivileged(Native Method)
  3553. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  3554. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  3555. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3556. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3557. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3558. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3559. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3560. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  3561. at java.security.AccessController.doPrivileged(Native Method)
  3562. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  3563. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  3564. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  3565. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3566. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3567. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  3568. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  3569. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  3570. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  3571. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  3572. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3573. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3574. at java.security.AccessController.doPrivileged(Native Method)
  3575. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  3576. ... 153 more
  3577. 17/03/30 08:43:41 WARN Hive: Failed to access metastore. This class should not accessed in runtime.
  3578. org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  3579. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1236)
  3580. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  3581. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  3582. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  3583. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  3584. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3585. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3586. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3587. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3588. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  3589. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  3590. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  3591. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  3592. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3593. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3594. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3595. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3596. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  3597. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  3598. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  3599. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  3600. at scala.Option.getOrElse(Option.scala:121)
  3601. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  3602. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  3603. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  3604. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  3605. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3606. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3607. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3608. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3609. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  3610. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  3611. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  3612. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  3613. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  3614. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  3615. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  3616. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  3617. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  3618. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  3619. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  3620. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  3621. at $line3.$read$$iw$$iw.<init>(<console>:15)
  3622. at $line3.$read$$iw.<init>(<console>:42)
  3623. at $line3.$read.<init>(<console>:44)
  3624. at $line3.$read$.<init>(<console>:48)
  3625. at $line3.$read$.<clinit>(<console>)
  3626. at $line3.$eval$.$print$lzycompute(<console>:7)
  3627. at $line3.$eval$.$print(<console>:6)
  3628. at $line3.$eval.$print(<console>)
  3629. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3630. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3631. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3632. at java.lang.reflect.Method.invoke(Method.java:498)
  3633. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  3634. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  3635. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  3636. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  3637. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  3638. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  3639. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  3640. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  3641. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  3642. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  3643. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  3644. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  3645. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  3646. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  3647. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  3648. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  3649. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  3650. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  3651. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  3652. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  3653. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  3654. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  3655. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  3656. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  3657. at org.apache.spark.repl.Main$.main(Main.scala:51)
  3658. at org.apache.spark.repl.Main.main(Main.scala)
  3659. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3660. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3661. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3662. at java.lang.reflect.Method.invoke(Method.java:498)
  3663. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  3664. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  3665. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  3666. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  3667. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  3668. Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  3669. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
  3670. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  3671. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  3672. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  3673. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  3674. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  3675. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  3676. ... 88 more
  3677. Caused by: java.lang.reflect.InvocationTargetException
  3678. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3679. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3680. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3681. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3682. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  3683. ... 94 more
  3684. Caused by: javax.jdo.JDOFatalDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  3685. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  3686. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3687. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3688. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  3689. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  3690. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  3691. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3692. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3693. at java.security.AccessController.doPrivileged(Native Method)
  3694. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  3695. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3696. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3697. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  3698. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  3699. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  3700. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  3701. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  3702. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  3703. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  3704. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  3705. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3706. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3707. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3708. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3709. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  3710. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  3711. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  3712. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  3713. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  3714. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  3715. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  3716. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3717. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3718. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3719. at java.lang.reflect.Method.invoke(Method.java:498)
  3720. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  3721. at java.security.AccessController.doPrivileged(Native Method)
  3722. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  3723. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  3724. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  3725. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  3726. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  3727. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  3728. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  3729. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  3730. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  3731. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  3732. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  3733. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  3734. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  3735. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  3736. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  3737. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  3738. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  3739. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  3740. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  3741. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  3742. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  3743. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3744. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3745. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3746. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3747. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  3748. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  3749. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  3750. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  3751. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  3752. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  3753. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  3754. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  3755. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  3756. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  3757. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  3758. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3759. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3760. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3761. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3762. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  3763. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  3764. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  3765. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  3766. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3767. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3768. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3769. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3770. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  3771. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  3772. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  3773. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  3774. at scala.Option.getOrElse(Option.scala:121)
  3775. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  3776. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  3777. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  3778. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  3779. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3780. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3781. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3782. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3783. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  3784. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  3785. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  3786. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  3787. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  3788. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  3789. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  3790. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  3791. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  3792. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  3793. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  3794. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  3795. at $line3.$read$$iw$$iw.<init>(<console>:15)
  3796. at $line3.$read$$iw.<init>(<console>:42)
  3797. at $line3.$read.<init>(<console>:44)
  3798. at $line3.$read$.<init>(<console>:48)
  3799. at $line3.$read$.<clinit>(<console>)
  3800. at $line3.$eval$.$print$lzycompute(<console>:7)
  3801. at $line3.$eval$.$print(<console>:6)
  3802. at $line3.$eval.$print(<console>)
  3803. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3804. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3805. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3806. at java.lang.reflect.Method.invoke(Method.java:498)
  3807. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  3808. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  3809. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  3810. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  3811. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  3812. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  3813. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  3814. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  3815. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  3816. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  3817. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  3818. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  3819. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  3820. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  3821. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  3822. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  3823. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  3824. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  3825. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  3826. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  3827. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  3828. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  3829. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  3830. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  3831. at org.apache.spark.repl.Main$.main(Main.scala:51)
  3832. at org.apache.spark.repl.Main.main(Main.scala)
  3833. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3834. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3835. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3836. at java.lang.reflect.Method.invoke(Method.java:498)
  3837. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  3838. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  3839. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  3840. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  3841. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  3842. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  3843. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3844. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  3845. ... 156 more
  3846. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  3847. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3848. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3849. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  3850. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  3851. at java.security.AccessController.doPrivileged(Native Method)
  3852. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  3853. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  3854. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3855. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3856. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3857. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3858. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3859. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  3860. at java.security.AccessController.doPrivileged(Native Method)
  3861. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  3862. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  3863. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3864. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3865. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3866. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3867. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3868. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  3869. at java.security.AccessController.doPrivileged(Native Method)
  3870. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  3871. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  3872. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3873. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3874. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3875. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3876. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3877. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  3878. at java.security.AccessController.doPrivileged(Native Method)
  3879. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  3880. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  3881. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  3882. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3883. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3884. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  3885. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  3886. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  3887. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  3888. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  3889. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3890. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3891. at java.security.AccessController.doPrivileged(Native Method)
  3892. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  3893. ... 153 more
  3894. ------
  3895.  
  3896. NestedThrowables:
  3897. java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  3898. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  3899. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3900. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3901. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  3902. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  3903. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  3904. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3905. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3906. at java.security.AccessController.doPrivileged(Native Method)
  3907. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  3908. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3909. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3910. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  3911. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  3912. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  3913. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  3914. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  3915. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  3916. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  3917. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  3918. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3919. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3920. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3921. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3922. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  3923. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  3924. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  3925. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  3926. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  3927. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  3928. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  3929. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3930. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3931. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3932. at java.lang.reflect.Method.invoke(Method.java:498)
  3933. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  3934. at java.security.AccessController.doPrivileged(Native Method)
  3935. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  3936. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  3937. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  3938. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  3939. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  3940. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  3941. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  3942. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  3943. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  3944. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  3945. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  3946. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  3947. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  3948. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  3949. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  3950. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  3951. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  3952. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  3953. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  3954. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  3955. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  3956. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3957. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3958. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3959. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3960. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  3961. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  3962. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  3963. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  3964. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  3965. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  3966. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  3967. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  3968. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  3969. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  3970. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  3971. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3972. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3973. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3974. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3975. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  3976. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  3977. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  3978. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  3979. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3980. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3981. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3982. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3983. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  3984. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  3985. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  3986. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  3987. at scala.Option.getOrElse(Option.scala:121)
  3988. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  3989. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  3990. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  3991. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  3992. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3993. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3994. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3995. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3996. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  3997. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  3998. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  3999. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  4000. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  4001. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  4002. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  4003. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  4004. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  4005. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  4006. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  4007. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  4008. at $line3.$read$$iw$$iw.<init>(<console>:15)
  4009. at $line3.$read$$iw.<init>(<console>:42)
  4010. at $line3.$read.<init>(<console>:44)
  4011. at $line3.$read$.<init>(<console>:48)
  4012. at $line3.$read$.<clinit>(<console>)
  4013. at $line3.$eval$.$print$lzycompute(<console>:7)
  4014. at $line3.$eval$.$print(<console>:6)
  4015. at $line3.$eval.$print(<console>)
  4016. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4017. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4018. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4019. at java.lang.reflect.Method.invoke(Method.java:498)
  4020. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  4021. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  4022. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  4023. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  4024. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  4025. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  4026. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  4027. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  4028. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  4029. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  4030. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  4031. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  4032. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  4033. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  4034. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  4035. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  4036. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  4037. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  4038. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  4039. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  4040. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  4041. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  4042. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  4043. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  4044. at org.apache.spark.repl.Main$.main(Main.scala:51)
  4045. at org.apache.spark.repl.Main.main(Main.scala)
  4046. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4047. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4048. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4049. at java.lang.reflect.Method.invoke(Method.java:498)
  4050. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  4051. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  4052. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  4053. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  4054. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  4055. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  4056. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4057. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  4058. ... 156 more
  4059. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  4060. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4061. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4062. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  4063. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  4064. at java.security.AccessController.doPrivileged(Native Method)
  4065. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  4066. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  4067. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4068. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4069. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4070. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4071. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4072. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  4073. at java.security.AccessController.doPrivileged(Native Method)
  4074. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  4075. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  4076. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4077. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4078. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4079. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4080. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4081. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  4082. at java.security.AccessController.doPrivileged(Native Method)
  4083. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  4084. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  4085. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4086. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4087. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4088. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4089. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4090. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  4091. at java.security.AccessController.doPrivileged(Native Method)
  4092. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  4093. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  4094. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  4095. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4096. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4097. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  4098. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  4099. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  4100. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  4101. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  4102. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4103. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4104. at java.security.AccessController.doPrivileged(Native Method)
  4105. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  4106. ... 153 more
  4107. ------
  4108.  
  4109. at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:436)
  4110. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:788)
  4111. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  4112. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  4113. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4114. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4115. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4116. at java.lang.reflect.Method.invoke(Method.java:498)
  4117. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  4118. at java.security.AccessController.doPrivileged(Native Method)
  4119. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  4120. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  4121. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  4122. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  4123. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  4124. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  4125. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  4126. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  4127. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  4128. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  4129. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  4130. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  4131. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  4132. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  4133. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  4134. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  4135. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  4136. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  4137. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  4138. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  4139. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  4140. ... 99 more
  4141. Caused by: java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  4142. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  4143. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4144. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4145. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  4146. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  4147. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  4148. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4149. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4150. at java.security.AccessController.doPrivileged(Native Method)
  4151. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  4152. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4153. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4154. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  4155. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  4156. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  4157. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  4158. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  4159. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  4160. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  4161. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  4162. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4163. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4164. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4165. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4166. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  4167. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  4168. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  4169. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  4170. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  4171. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  4172. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  4173. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4174. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4175. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4176. at java.lang.reflect.Method.invoke(Method.java:498)
  4177. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  4178. at java.security.AccessController.doPrivileged(Native Method)
  4179. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  4180. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  4181. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  4182. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  4183. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  4184. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  4185. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  4186. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  4187. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  4188. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  4189. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  4190. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  4191. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  4192. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  4193. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  4194. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  4195. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  4196. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  4197. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  4198. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  4199. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  4200. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4201. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4202. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4203. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4204. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  4205. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  4206. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  4207. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  4208. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  4209. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  4210. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  4211. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  4212. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  4213. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  4214. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  4215. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4216. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4217. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4218. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4219. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  4220. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  4221. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  4222. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  4223. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4224. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4225. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4226. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4227. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  4228. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  4229. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  4230. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  4231. at scala.Option.getOrElse(Option.scala:121)
  4232. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  4233. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  4234. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  4235. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  4236. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4237. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4238. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4239. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4240. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  4241. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  4242. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  4243. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  4244. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  4245. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  4246. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  4247. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  4248. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  4249. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  4250. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  4251. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  4252. at $line3.$read$$iw$$iw.<init>(<console>:15)
  4253. at $line3.$read$$iw.<init>(<console>:42)
  4254. at $line3.$read.<init>(<console>:44)
  4255. at $line3.$read$.<init>(<console>:48)
  4256. at $line3.$read$.<clinit>(<console>)
  4257. at $line3.$eval$.$print$lzycompute(<console>:7)
  4258. at $line3.$eval$.$print(<console>:6)
  4259. at $line3.$eval.$print(<console>)
  4260. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4261. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4262. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4263. at java.lang.reflect.Method.invoke(Method.java:498)
  4264. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  4265. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  4266. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  4267. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  4268. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  4269. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  4270. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  4271. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  4272. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  4273. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  4274. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  4275. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  4276. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  4277. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  4278. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  4279. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  4280. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  4281. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  4282. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  4283. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  4284. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  4285. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  4286. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  4287. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  4288. at org.apache.spark.repl.Main$.main(Main.scala:51)
  4289. at org.apache.spark.repl.Main.main(Main.scala)
  4290. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4291. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4292. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4293. at java.lang.reflect.Method.invoke(Method.java:498)
  4294. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  4295. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  4296. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  4297. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  4298. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  4299. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  4300. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4301. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  4302. ... 156 more
  4303. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  4304. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4305. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4306. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  4307. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  4308. at java.security.AccessController.doPrivileged(Native Method)
  4309. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  4310. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  4311. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4312. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4313. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4314. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4315. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4316. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  4317. at java.security.AccessController.doPrivileged(Native Method)
  4318. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  4319. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  4320. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4321. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4322. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4323. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4324. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4325. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  4326. at java.security.AccessController.doPrivileged(Native Method)
  4327. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  4328. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  4329. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4330. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4331. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4332. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4333. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4334. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  4335. at java.security.AccessController.doPrivileged(Native Method)
  4336. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  4337. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  4338. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  4339. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4340. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4341. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  4342. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  4343. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  4344. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  4345. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  4346. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4347. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4348. at java.security.AccessController.doPrivileged(Native Method)
  4349. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  4350. ... 153 more
  4351. ------
  4352.  
  4353. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4354. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4355. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4356. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4357. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  4358. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  4359. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  4360. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  4361. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  4362. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4363. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4364. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4365. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4366. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  4367. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  4368. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  4369. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  4370. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  4371. ... 128 more
  4372. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  4373. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4374. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4375. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  4376. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  4377. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  4378. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4379. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4380. at java.security.AccessController.doPrivileged(Native Method)
  4381. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  4382. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4383. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4384. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  4385. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  4386. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  4387. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  4388. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  4389. ... 140 more
  4390. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  4391. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4392. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  4393. ... 156 more
  4394. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  4395. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4396. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4397. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  4398. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  4399. at java.security.AccessController.doPrivileged(Native Method)
  4400. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  4401. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  4402. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4403. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4404. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4405. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4406. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4407. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  4408. at java.security.AccessController.doPrivileged(Native Method)
  4409. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  4410. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  4411. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4412. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4413. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4414. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4415. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4416. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  4417. at java.security.AccessController.doPrivileged(Native Method)
  4418. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  4419. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  4420. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4421. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4422. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4423. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4424. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4425. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  4426. at java.security.AccessController.doPrivileged(Native Method)
  4427. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  4428. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  4429. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  4430. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4431. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4432. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  4433.  
  4434. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  4435. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  4436. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  4437. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  4438. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4439. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4440. at java.security.AccessController.doPrivileged(Native Method)
  4441. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  4442. ... 153 more
  4443. 17/03/30 08:43:41 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/Users/james/spark/jars/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/Users/james/spark-2.1.0-bin-hadoop2.7/jars/datanucleus-api-jdo-3.2.6.jar."
  4444. 17/03/30 08:43:41 WARN General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/Users/james/spark-2.1.0-bin-hadoop2.7/jars/datanucleus-core-3.2.10.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/Users/james/spark/jars/datanucleus-core-3.2.10.jar."
  4445. 17/03/30 08:43:41 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/Users/james/spark-2.1.0-bin-hadoop2.7/jars/datanucleus-rdbms-3.2.9.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/Users/james/spark/jars/datanucleus-rdbms-3.2.9.jar."
  4446. 17/03/30 08:43:41 ERROR Schema: Failed initialising database.
  4447. Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  4448. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  4449. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4450. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4451. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  4452. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  4453. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  4454. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4455. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4456. at java.security.AccessController.doPrivileged(Native Method)
  4457. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  4458. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4459. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4460. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  4461. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  4462. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  4463. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  4464. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  4465. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  4466. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  4467. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  4468. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4469. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4470. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4471. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4472. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  4473. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  4474. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  4475. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  4476. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  4477. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  4478. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  4479. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4480. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4481. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4482. at java.lang.reflect.Method.invoke(Method.java:498)
  4483. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  4484. at java.security.AccessController.doPrivileged(Native Method)
  4485. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  4486. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  4487. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  4488. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  4489. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  4490. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  4491. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  4492. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  4493. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  4494. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  4495. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  4496. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  4497. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  4498. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  4499. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  4500. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  4501. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  4502. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  4503. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  4504. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  4505. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  4506. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4507. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4508. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4509. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4510. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  4511. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  4512. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  4513. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  4514. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  4515. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  4516. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  4517. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  4518. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4519. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4520. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4521. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4522. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  4523. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  4524. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  4525. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  4526. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4527. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4528. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4529. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4530. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  4531. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  4532. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  4533. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  4534. at scala.Option.getOrElse(Option.scala:121)
  4535. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  4536. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  4537. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  4538. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  4539. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4540. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4541. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4542. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4543. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  4544. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  4545. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  4546. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  4547. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  4548. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  4549. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  4550. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  4551. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  4552. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  4553. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  4554. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  4555. at $line3.$read$$iw$$iw.<init>(<console>:15)
  4556. at $line3.$read$$iw.<init>(<console>:42)
  4557. at $line3.$read.<init>(<console>:44)
  4558. at $line3.$read$.<init>(<console>:48)
  4559. at $line3.$read$.<clinit>(<console>)
  4560. at $line3.$eval$.$print$lzycompute(<console>:7)
  4561. at $line3.$eval$.$print(<console>:6)
  4562. at $line3.$eval.$print(<console>)
  4563. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4564. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4565. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4566. at java.lang.reflect.Method.invoke(Method.java:498)
  4567. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  4568. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  4569. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  4570. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  4571. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  4572. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  4573. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  4574. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  4575. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  4576. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  4577. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  4578. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  4579. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  4580. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  4581. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  4582. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  4583. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  4584. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  4585. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  4586. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  4587. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  4588. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  4589. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  4590. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  4591. at org.apache.spark.repl.Main$.main(Main.scala:51)
  4592. at org.apache.spark.repl.Main.main(Main.scala)
  4593. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4594. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4595. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4596. at java.lang.reflect.Method.invoke(Method.java:498)
  4597. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  4598. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  4599. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  4600. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  4601. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  4602. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  4603. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4604. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  4605. ... 153 more
  4606. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  4607. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4608. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4609. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  4610. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  4611. at java.security.AccessController.doPrivileged(Native Method)
  4612. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  4613. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  4614. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4615. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4616. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4617. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4618. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4619. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  4620. at java.security.AccessController.doPrivileged(Native Method)
  4621. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  4622. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  4623. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4624. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4625. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4626. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4627. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4628. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  4629. at java.security.AccessController.doPrivileged(Native Method)
  4630. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  4631. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  4632. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4633. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4634. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4635. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4636. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4637. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  4638. at java.security.AccessController.doPrivileged(Native Method)
  4639. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  4640. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  4641. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  4642. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4643. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4644. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  4645. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  4646. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  4647. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  4648. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  4649. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4650. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4651. at java.security.AccessController.doPrivileged(Native Method)
  4652. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  4653. ... 150 more
  4654. ------
  4655.  
  4656. org.datanucleus.exceptions.NucleusDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  4657. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  4658. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4659. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4660. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  4661. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  4662. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  4663. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4664. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4665. at java.security.AccessController.doPrivileged(Native Method)
  4666. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  4667. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4668. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4669. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  4670. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  4671. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  4672. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  4673. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  4674. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  4675. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  4676. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  4677. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4678. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4679. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4680. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4681. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  4682. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  4683. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  4684. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  4685. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  4686. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  4687. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  4688. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4689. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4690. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4691. at java.lang.reflect.Method.invoke(Method.java:498)
  4692. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  4693. at java.security.AccessController.doPrivileged(Native Method)
  4694. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  4695. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  4696. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  4697. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  4698. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  4699. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  4700. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  4701. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  4702. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  4703. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  4704. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  4705. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  4706. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  4707. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  4708. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  4709. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  4710. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  4711. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  4712. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  4713. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  4714. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  4715. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4716. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4717. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4718. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4719. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  4720. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  4721. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  4722. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  4723. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  4724. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  4725. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  4726. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  4727. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4728. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4729. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4730. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4731. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  4732. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  4733. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  4734. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  4735. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4736. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4737. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4738. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4739. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  4740. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  4741. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  4742. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  4743. at scala.Option.getOrElse(Option.scala:121)
  4744. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  4745. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  4746. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  4747. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  4748. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4749. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4750. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4751. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4752. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  4753. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  4754. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  4755. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  4756. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  4757. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  4758. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  4759. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  4760. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  4761. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  4762. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  4763. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  4764. at $line3.$read$$iw$$iw.<init>(<console>:15)
  4765. at $line3.$read$$iw.<init>(<console>:42)
  4766. at $line3.$read.<init>(<console>:44)
  4767. at $line3.$read$.<init>(<console>:48)
  4768. at $line3.$read$.<clinit>(<console>)
  4769. at $line3.$eval$.$print$lzycompute(<console>:7)
  4770. at $line3.$eval$.$print(<console>:6)
  4771. at $line3.$eval.$print(<console>)
  4772. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4773. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4774. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4775. at java.lang.reflect.Method.invoke(Method.java:498)
  4776. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  4777. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  4778. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  4779. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  4780. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  4781. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  4782. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  4783. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  4784. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  4785. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  4786. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  4787. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  4788. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  4789. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  4790. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  4791. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  4792. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  4793. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  4794. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  4795. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  4796. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  4797. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  4798. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  4799. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  4800. at org.apache.spark.repl.Main$.main(Main.scala:51)
  4801. at org.apache.spark.repl.Main.main(Main.scala)
  4802. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4803. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4804. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4805. at java.lang.reflect.Method.invoke(Method.java:498)
  4806. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  4807. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  4808. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  4809. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  4810. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  4811. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  4812. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4813. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  4814. ... 153 more
  4815. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  4816. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4817. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4818. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  4819. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  4820. at java.security.AccessController.doPrivileged(Native Method)
  4821. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  4822. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  4823. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4824. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4825. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4826. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4827. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4828. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  4829. at java.security.AccessController.doPrivileged(Native Method)
  4830. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  4831. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  4832. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4833. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4834. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4835. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4836. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4837. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  4838. at java.security.AccessController.doPrivileged(Native Method)
  4839. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  4840. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  4841. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4842. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4843. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4844. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4845. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4846. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  4847. at java.security.AccessController.doPrivileged(Native Method)
  4848. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  4849. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  4850. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  4851. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4852. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4853. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  4854. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  4855. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  4856. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  4857. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  4858. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4859. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4860. at java.security.AccessController.doPrivileged(Native Method)
  4861. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  4862. ... 150 more
  4863. ------
  4864.  
  4865. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:516)
  4866. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  4867. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4868. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4869. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4870. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4871. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  4872. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  4873. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  4874. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  4875. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  4876. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  4877. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  4878. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4879. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4880. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4881. at java.lang.reflect.Method.invoke(Method.java:498)
  4882. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  4883. at java.security.AccessController.doPrivileged(Native Method)
  4884. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  4885. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  4886. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  4887. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  4888. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  4889. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  4890. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  4891. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  4892. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  4893. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  4894. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  4895. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  4896. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  4897. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  4898. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  4899. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  4900. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  4901. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  4902. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  4903. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  4904. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  4905. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4906. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4907. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4908. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4909. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  4910. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  4911. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  4912. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  4913. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  4914. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  4915. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  4916. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  4917. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4918. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4919. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4920. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4921. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  4922. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  4923. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  4924. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  4925. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4926. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4927. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4928. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4929. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  4930. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  4931. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  4932. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  4933. at scala.Option.getOrElse(Option.scala:121)
  4934. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  4935. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  4936. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  4937. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  4938. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4939. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4940. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4941. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4942. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  4943. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  4944. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  4945. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  4946. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  4947. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  4948. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  4949. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  4950. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  4951. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  4952. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  4953. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  4954. at $line3.$read$$iw$$iw.<init>(<console>:15)
  4955. at $line3.$read$$iw.<init>(<console>:42)
  4956. at $line3.$read.<init>(<console>:44)
  4957. at $line3.$read$.<init>(<console>:48)
  4958. at $line3.$read$.<clinit>(<console>)
  4959. at $line3.$eval$.$print$lzycompute(<console>:7)
  4960. at $line3.$eval$.$print(<console>:6)
  4961. at $line3.$eval.$print(<console>)
  4962. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4963. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4964. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4965. at java.lang.reflect.Method.invoke(Method.java:498)
  4966. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  4967. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  4968. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  4969. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  4970. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  4971. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  4972. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  4973. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  4974. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  4975. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  4976. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  4977. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  4978. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  4979. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  4980. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  4981. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  4982. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  4983. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  4984. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  4985. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  4986. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  4987. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  4988. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  4989. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  4990. at org.apache.spark.repl.Main$.main(Main.scala:51)
  4991. at org.apache.spark.repl.Main.main(Main.scala)
  4992. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4993. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4994. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4995. at java.lang.reflect.Method.invoke(Method.java:498)
  4996. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  4997. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  4998. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  4999. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  5000. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  5001. Caused by: java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  5002. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  5003. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5004. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5005. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  5006. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  5007. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  5008. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5009. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5010. at java.security.AccessController.doPrivileged(Native Method)
  5011. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  5012. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5013. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5014. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  5015. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  5016. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  5017. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  5018. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  5019. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  5020. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  5021. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  5022. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5023. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5024. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5025. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5026. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  5027. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  5028. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  5029. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  5030. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  5031. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  5032. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  5033. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5034. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5035. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5036. at java.lang.reflect.Method.invoke(Method.java:498)
  5037. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  5038. at java.security.AccessController.doPrivileged(Native Method)
  5039. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  5040. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  5041. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  5042. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  5043. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  5044. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  5045. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  5046. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  5047. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  5048. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  5049. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  5050. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  5051. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  5052. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  5053. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  5054. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  5055. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  5056. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  5057. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  5058. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  5059. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  5060. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5061. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5062. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5063. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5064. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  5065. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  5066. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  5067. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  5068. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  5069. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  5070. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  5071. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  5072. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5073. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5074. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5075. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5076. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  5077. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  5078. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  5079. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  5080. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5081. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5082. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5083. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5084. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  5085. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  5086. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  5087. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  5088. at scala.Option.getOrElse(Option.scala:121)
  5089. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  5090. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  5091. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  5092. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  5093. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5094. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5095. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5096. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5097. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  5098. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  5099. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  5100. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  5101. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  5102. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  5103. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  5104. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  5105. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  5106. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  5107. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  5108. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  5109. at $line3.$read$$iw$$iw.<init>(<console>:15)
  5110. at $line3.$read$$iw.<init>(<console>:42)
  5111. at $line3.$read.<init>(<console>:44)
  5112. at $line3.$read$.<init>(<console>:48)
  5113. at $line3.$read$.<clinit>(<console>)
  5114. at $line3.$eval$.$print$lzycompute(<console>:7)
  5115. at $line3.$eval$.$print(<console>:6)
  5116. at $line3.$eval.$print(<console>)
  5117. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5118. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5119. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5120. at java.lang.reflect.Method.invoke(Method.java:498)
  5121. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  5122. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  5123. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  5124. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  5125. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  5126. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  5127. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  5128. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  5129. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  5130. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  5131. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  5132. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  5133. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  5134. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  5135. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  5136. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  5137. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  5138. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  5139. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  5140. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  5141. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  5142. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  5143. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  5144. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  5145. at org.apache.spark.repl.Main$.main(Main.scala:51)
  5146. at org.apache.spark.repl.Main.main(Main.scala)
  5147. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5148. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5149. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5150. at java.lang.reflect.Method.invoke(Method.java:498)
  5151. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  5152. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  5153. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  5154. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  5155. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  5156. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  5157. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5158. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  5159. ... 153 more
  5160. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  5161. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5162. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5163. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  5164. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  5165. at java.security.AccessController.doPrivileged(Native Method)
  5166. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  5167. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  5168. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5169. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5170. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5171. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5172. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5173. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  5174. at java.security.AccessController.doPrivileged(Native Method)
  5175. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  5176. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  5177. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5178. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5179. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5180. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5181. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5182. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  5183. at java.security.AccessController.doPrivileged(Native Method)
  5184. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  5185. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  5186. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5187. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5188. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5189. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5190. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5191. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  5192. at java.security.AccessController.doPrivileged(Native Method)
  5193. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  5194. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  5195. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  5196. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5197. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5198. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  5199. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  5200. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  5201. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  5202. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  5203. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5204. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5205. at java.security.AccessController.doPrivileged(Native Method)
  5206. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  5207. ... 150 more
  5208. ------
  5209.  
  5210. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5211. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5212. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5213. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5214. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  5215. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  5216. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  5217. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  5218. ... 135 more
  5219. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  5220. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5221. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5222. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  5223. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  5224. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  5225. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5226. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5227. at java.security.AccessController.doPrivileged(Native Method)
  5228. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  5229. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5230. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5231. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  5232. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  5233. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  5234. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  5235. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  5236. ... 137 more
  5237. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  5238. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5239. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  5240. ... 153 more
  5241. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  5242. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5243. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5244. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  5245. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  5246. at java.security.AccessController.doPrivileged(Native Method)
  5247. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  5248. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  5249. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5250. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5251. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5252. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5253. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5254. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  5255. at java.security.AccessController.doPrivileged(Native Method)
  5256. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  5257. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  5258. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5259. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5260. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5261. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5262. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5263. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  5264. at java.security.AccessController.doPrivileged(Native Method)
  5265. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  5266. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  5267. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5268. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5269. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5270. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5271. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5272. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  5273. at java.security.AccessController.doPrivileged(Native Method)
  5274. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  5275. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  5276. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  5277. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5278. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5279. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  5280. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  5281. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  5282. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  5283. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  5284. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5285. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5286. at java.security.AccessController.doPrivileged(Native Method)
  5287. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  5288. ... 150 more
  5289. Nested Throwables StackTrace:
  5290. java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  5291. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  5292. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5293. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5294. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  5295. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  5296. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  5297. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5298. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5299. at java.security.AccessController.doPrivileged(Native Method)
  5300. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  5301. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5302. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5303. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  5304. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  5305. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  5306. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  5307. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  5308. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  5309. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  5310. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  5311. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5312. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5313. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5314. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5315. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  5316. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  5317. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  5318. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  5319. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  5320. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  5321. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  5322. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5323. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5324. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5325. at java.lang.reflect.Method.invoke(Method.java:498)
  5326. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  5327. at java.security.AccessController.doPrivileged(Native Method)
  5328. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  5329. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  5330. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  5331. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  5332. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  5333. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  5334. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  5335. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  5336. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  5337. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  5338. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  5339. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  5340. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  5341. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  5342. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  5343. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  5344. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  5345. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  5346. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  5347. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  5348. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  5349. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5350. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5351. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5352. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5353. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  5354. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  5355. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  5356. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  5357. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  5358. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  5359. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  5360. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  5361. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5362. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5363. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5364. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5365. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  5366. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  5367. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  5368. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  5369. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5370. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5371. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5372. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5373. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  5374. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  5375. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  5376. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  5377. at scala.Option.getOrElse(Option.scala:121)
  5378. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  5379. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  5380. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  5381. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  5382. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5383. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5384. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5385. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5386. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  5387. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  5388. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  5389. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  5390. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  5391. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  5392. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  5393. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  5394. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  5395. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  5396. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  5397. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  5398. at $line3.$read$$iw$$iw.<init>(<console>:15)
  5399. at $line3.$read$$iw.<init>(<console>:42)
  5400. at $line3.$read.<init>(<console>:44)
  5401. at $line3.$read$.<init>(<console>:48)
  5402. at $line3.$read$.<clinit>(<console>)
  5403. at $line3.$eval$.$print$lzycompute(<console>:7)
  5404. at $line3.$eval$.$print(<console>:6)
  5405. at $line3.$eval.$print(<console>)
  5406. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5407. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5408. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5409. at java.lang.reflect.Method.invoke(Method.java:498)
  5410. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  5411. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  5412. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  5413. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  5414. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  5415. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  5416. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  5417. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  5418. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  5419. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  5420. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  5421. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  5422. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  5423. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  5424.  
  5425. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  5426. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  5427. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  5428. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  5429. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  5430. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  5431. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  5432. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  5433. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  5434. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  5435. at org.apache.spark.repl.Main$.main(Main.scala:51)
  5436. at org.apache.spark.repl.Main.main(Main.scala)
  5437. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5438. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5439. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5440. at java.lang.reflect.Method.invoke(Method.java:498)
  5441. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  5442. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  5443. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  5444. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  5445. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  5446. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  5447. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5448. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  5449. ... 153 more
  5450. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  5451. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5452. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5453. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  5454. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  5455. at java.security.AccessController.doPrivileged(Native Method)
  5456. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  5457. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  5458. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5459. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5460. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5461. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5462. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5463. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  5464. at java.security.AccessController.doPrivileged(Native Method)
  5465. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  5466. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  5467. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5468. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5469. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5470. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5471. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5472. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  5473. at java.security.AccessController.doPrivileged(Native Method)
  5474. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  5475. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  5476. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5477. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5478. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5479. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5480. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5481. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  5482. at java.security.AccessController.doPrivileged(Native Method)
  5483. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  5484. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  5485. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  5486. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5487. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5488. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  5489. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  5490. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  5491. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  5492. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  5493. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5494. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5495. at java.security.AccessController.doPrivileged(Native Method)
  5496. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  5497. ... 150 more
  5498. ------
  5499.  
  5500. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5501. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5502. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5503. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5504. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  5505. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  5506. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  5507. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  5508. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  5509. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5510. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5511. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5512. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5513. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  5514. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  5515. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  5516. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  5517. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  5518. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  5519. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  5520. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5521. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5522. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5523. at java.lang.reflect.Method.invoke(Method.java:498)
  5524. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  5525. at java.security.AccessController.doPrivileged(Native Method)
  5526. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  5527. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  5528. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  5529. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  5530. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  5531. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  5532. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  5533. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  5534. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  5535. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  5536. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  5537. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  5538. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  5539. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  5540. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  5541. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  5542. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  5543. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  5544. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  5545. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  5546. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  5547. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5548. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5549. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5550. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5551. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  5552. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  5553. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  5554. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  5555. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  5556. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  5557. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  5558. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  5559. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5560. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5561. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5562. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5563. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  5564. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  5565. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  5566. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  5567. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5568. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5569. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5570. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5571. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  5572. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  5573. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  5574. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  5575. at scala.Option.getOrElse(Option.scala:121)
  5576. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  5577. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  5578. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  5579. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  5580. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5581. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5582. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5583. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5584. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  5585. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  5586. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  5587. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  5588. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  5589. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  5590. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  5591. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  5592. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  5593. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  5594. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  5595. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  5596. at $line3.$read$$iw$$iw.<init>(<console>:15)
  5597. at $line3.$read$$iw.<init>(<console>:42)
  5598. at $line3.$read.<init>(<console>:44)
  5599. at $line3.$read$.<init>(<console>:48)
  5600. at $line3.$read$.<clinit>(<console>)
  5601. at $line3.$eval$.$print$lzycompute(<console>:7)
  5602. at $line3.$eval$.$print(<console>:6)
  5603. at $line3.$eval.$print(<console>)
  5604. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5605. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5606. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5607. at java.lang.reflect.Method.invoke(Method.java:498)
  5608. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  5609. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  5610. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  5611. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  5612. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  5613. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  5614. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  5615. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  5616. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  5617. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  5618. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  5619. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  5620. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  5621. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  5622. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  5623. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  5624. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  5625. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  5626. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  5627. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  5628. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  5629. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  5630. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  5631. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  5632. at org.apache.spark.repl.Main$.main(Main.scala:51)
  5633. at org.apache.spark.repl.Main.main(Main.scala)
  5634. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5635. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5636. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5637. at java.lang.reflect.Method.invoke(Method.java:498)
  5638. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  5639. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  5640. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  5641. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  5642. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  5643. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  5644. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5645. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5646. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  5647. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  5648. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  5649. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5650. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5651. at java.security.AccessController.doPrivileged(Native Method)
  5652. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  5653. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5654. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5655. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  5656. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  5657. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  5658. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  5659. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  5660. ... 137 more
  5661. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  5662. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5663. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  5664. ... 153 more
  5665. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  5666. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5667. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5668. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  5669. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  5670. at java.security.AccessController.doPrivileged(Native Method)
  5671. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  5672. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  5673. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5674. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5675. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5676. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5677. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5678. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  5679. at java.security.AccessController.doPrivileged(Native Method)
  5680. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  5681. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  5682. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5683. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5684. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5685. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5686. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5687. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  5688. at java.security.AccessController.doPrivileged(Native Method)
  5689. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  5690. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  5691. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5692. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5693. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5694. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5695. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5696. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  5697. at java.security.AccessController.doPrivileged(Native Method)
  5698. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  5699. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  5700. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  5701. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5702. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5703. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  5704. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  5705. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  5706. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  5707. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  5708. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5709. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5710. at java.security.AccessController.doPrivileged(Native Method)
  5711. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  5712. ... 150 more
  5713. 17/03/30 08:43:41 WARN HiveMetaStore: Retrying creating default database after error: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  5714.  
  5715. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  5716. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5717. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5718. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  5719. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  5720. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  5721. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5722. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5723. at java.security.AccessController.doPrivileged(Native Method)
  5724. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  5725. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5726. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5727. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  5728. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  5729. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  5730. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  5731. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  5732. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  5733. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  5734. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  5735. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5736. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5737. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5738. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5739. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  5740. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  5741. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  5742. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  5743. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  5744. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  5745. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  5746. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5747. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5748. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5749. at java.lang.reflect.Method.invoke(Method.java:498)
  5750. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  5751. at java.security.AccessController.doPrivileged(Native Method)
  5752. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  5753. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  5754. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  5755. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  5756. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  5757. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  5758. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  5759. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  5760. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  5761. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  5762. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  5763. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  5764. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  5765. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  5766. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  5767. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  5768. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  5769. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  5770. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  5771. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  5772. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  5773. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5774. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5775. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5776. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5777. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  5778. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  5779. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  5780. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  5781. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  5782. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  5783. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  5784. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  5785. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5786. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5787. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5788. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5789. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  5790. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  5791. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  5792. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  5793. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5794. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5795. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5796. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5797. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  5798. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  5799. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  5800. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  5801. at scala.Option.getOrElse(Option.scala:121)
  5802. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  5803. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  5804. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  5805. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  5806. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5807. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5808. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5809. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5810. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  5811. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  5812. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  5813. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  5814. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  5815. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  5816. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  5817. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  5818. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  5819. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  5820. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  5821. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  5822. at $line3.$read$$iw$$iw.<init>(<console>:15)
  5823. at $line3.$read$$iw.<init>(<console>:42)
  5824. at $line3.$read.<init>(<console>:44)
  5825. at $line3.$read$.<init>(<console>:48)
  5826. at $line3.$read$.<clinit>(<console>)
  5827. at $line3.$eval$.$print$lzycompute(<console>:7)
  5828. at $line3.$eval$.$print(<console>:6)
  5829. at $line3.$eval.$print(<console>)
  5830. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5831. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5832. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5833. at java.lang.reflect.Method.invoke(Method.java:498)
  5834. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  5835. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  5836. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  5837. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  5838. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  5839. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  5840. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  5841. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  5842. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  5843. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  5844. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  5845. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  5846. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  5847. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  5848. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  5849. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  5850. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  5851. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  5852. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  5853. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  5854. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  5855. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  5856. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  5857. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  5858. at org.apache.spark.repl.Main$.main(Main.scala:51)
  5859. at org.apache.spark.repl.Main.main(Main.scala)
  5860. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5861. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5862. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5863. at java.lang.reflect.Method.invoke(Method.java:498)
  5864. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  5865. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  5866. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  5867. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  5868. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  5869. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  5870. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5871. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  5872. ... 153 more
  5873. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  5874. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5875. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5876. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  5877. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  5878. at java.security.AccessController.doPrivileged(Native Method)
  5879. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  5880. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  5881. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5882. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5883. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5884. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5885. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5886. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  5887. at java.security.AccessController.doPrivileged(Native Method)
  5888. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  5889. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  5890. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5891. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5892. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5893. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5894. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5895. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  5896. at java.security.AccessController.doPrivileged(Native Method)
  5897. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  5898. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  5899. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5900. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5901. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5902. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5903. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5904. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  5905. at java.security.AccessController.doPrivileged(Native Method)
  5906. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  5907. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  5908. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  5909. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5910. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5911. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  5912. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  5913. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  5914. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  5915. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  5916. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5917. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5918. at java.security.AccessController.doPrivileged(Native Method)
  5919. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  5920. ... 150 more
  5921. ------
  5922.  
  5923.  
  5924. javax.jdo.JDOFatalDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  5925. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  5926. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5927. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5928. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  5929. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  5930. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  5931. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5932. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5933. at java.security.AccessController.doPrivileged(Native Method)
  5934. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  5935. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5936. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5937. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  5938. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  5939. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  5940. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  5941. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  5942. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  5943. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  5944. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  5945. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5946. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5947. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5948. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5949. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  5950. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  5951. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  5952. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  5953. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  5954. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  5955. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  5956. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5957. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5958. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5959. at java.lang.reflect.Method.invoke(Method.java:498)
  5960. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  5961. at java.security.AccessController.doPrivileged(Native Method)
  5962. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  5963. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  5964. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  5965. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  5966. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  5967. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  5968. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  5969. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  5970. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  5971. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  5972. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  5973. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  5974. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  5975. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  5976. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  5977. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  5978. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  5979. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  5980. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  5981. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  5982. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  5983. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5984. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5985. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5986. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5987. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  5988. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  5989. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  5990. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  5991. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  5992. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  5993. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  5994. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  5995. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5996. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5997. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5998. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5999. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  6000. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  6001. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  6002. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  6003. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6004. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6005. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6006. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6007. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  6008. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  6009. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  6010. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  6011. at scala.Option.getOrElse(Option.scala:121)
  6012. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  6013. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  6014. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  6015. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  6016. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6017. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6018. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6019. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6020. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  6021. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  6022. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  6023. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  6024. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  6025. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  6026. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  6027. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  6028. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  6029. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  6030. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  6031. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  6032. at $line3.$read$$iw$$iw.<init>(<console>:15)
  6033. at $line3.$read$$iw.<init>(<console>:42)
  6034. at $line3.$read.<init>(<console>:44)
  6035. at $line3.$read$.<init>(<console>:48)
  6036. at $line3.$read$.<clinit>(<console>)
  6037. at $line3.$eval$.$print$lzycompute(<console>:7)
  6038. at $line3.$eval$.$print(<console>:6)
  6039. at $line3.$eval.$print(<console>)
  6040. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6041. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6042. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6043. at java.lang.reflect.Method.invoke(Method.java:498)
  6044. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  6045. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  6046. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  6047. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  6048. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  6049. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  6050. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  6051. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  6052. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  6053. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  6054. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  6055. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  6056. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  6057. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  6058. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  6059. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  6060. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  6061. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  6062. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  6063. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  6064. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  6065. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  6066. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  6067. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  6068. at org.apache.spark.repl.Main$.main(Main.scala:51)
  6069. at org.apache.spark.repl.Main.main(Main.scala)
  6070. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6071. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6072. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6073. at java.lang.reflect.Method.invoke(Method.java:498)
  6074. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  6075. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  6076. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  6077. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  6078. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  6079. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  6080. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6081. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  6082. ... 153 more
  6083. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  6084. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6085. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6086. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  6087. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  6088. at java.security.AccessController.doPrivileged(Native Method)
  6089. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  6090. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  6091. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6092. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6093. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6094. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6095. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6096. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  6097. at java.security.AccessController.doPrivileged(Native Method)
  6098. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  6099. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  6100. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6101. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6102. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6103. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6104. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6105. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  6106. at java.security.AccessController.doPrivileged(Native Method)
  6107. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  6108. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  6109. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6110. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6111. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6112. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6113. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6114. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  6115. at java.security.AccessController.doPrivileged(Native Method)
  6116. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  6117. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  6118. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  6119. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6120. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6121. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  6122. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  6123. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  6124. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  6125. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  6126. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  6127. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  6128. at java.security.AccessController.doPrivileged(Native Method)
  6129. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  6130. ... 150 more
  6131. ------
  6132.  
  6133. at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:436)
  6134. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:788)
  6135. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  6136. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  6137. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6138. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6139. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6140. at java.lang.reflect.Method.invoke(Method.java:498)
  6141. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  6142. at java.security.AccessController.doPrivileged(Native Method)
  6143. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  6144. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  6145. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  6146. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  6147. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  6148. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  6149. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  6150. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  6151. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  6152. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  6153. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  6154. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  6155. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  6156. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  6157. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  6158. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  6159. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  6160. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  6161. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  6162. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  6163. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  6164. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6165. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6166. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6167. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6168. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  6169. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  6170. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  6171. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  6172. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  6173. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  6174. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  6175. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  6176. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6177. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6178. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6179. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6180. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  6181. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  6182. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  6183. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  6184. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6185. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6186. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6187. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6188. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  6189. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  6190. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  6191. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  6192. at scala.Option.getOrElse(Option.scala:121)
  6193. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  6194. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  6195. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  6196. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  6197. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6198. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6199. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6200. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6201. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  6202. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  6203. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  6204. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  6205. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  6206. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  6207. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  6208. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  6209. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  6210. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  6211. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  6212. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  6213. at $line3.$read$$iw$$iw.<init>(<console>:15)
  6214. at $line3.$read$$iw.<init>(<console>:42)
  6215. at $line3.$read.<init>(<console>:44)
  6216. at $line3.$read$.<init>(<console>:48)
  6217. at $line3.$read$.<clinit>(<console>)
  6218. at $line3.$eval$.$print$lzycompute(<console>:7)
  6219. at $line3.$eval$.$print(<console>:6)
  6220. at $line3.$eval.$print(<console>)
  6221. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6222. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6223. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6224. at java.lang.reflect.Method.invoke(Method.java:498)
  6225. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  6226. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  6227. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  6228. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  6229. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  6230. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  6231. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  6232. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  6233. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  6234. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  6235. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  6236. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  6237. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  6238. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  6239. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  6240. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  6241. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  6242. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  6243. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  6244. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  6245. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  6246. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  6247. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  6248. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  6249. at org.apache.spark.repl.Main$.main(Main.scala:51)
  6250. at org.apache.spark.repl.Main.main(Main.scala)
  6251. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6252. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6253. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6254. at java.lang.reflect.Method.invoke(Method.java:498)
  6255. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  6256. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  6257. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  6258. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  6259. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  6260. NestedThrowablesStackTrace:
  6261. java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  6262. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  6263. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  6264. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  6265. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  6266. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  6267. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  6268. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  6269. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  6270. at java.security.AccessController.doPrivileged(Native Method)
  6271. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  6272. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  6273. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  6274. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  6275. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  6276. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  6277. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  6278. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  6279. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  6280. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  6281. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  6282. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6283. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6284. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6285. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6286. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  6287. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  6288. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  6289. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  6290. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  6291. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  6292. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  6293. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6294. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6295. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6296. at java.lang.reflect.Method.invoke(Method.java:498)
  6297. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  6298. at java.security.AccessController.doPrivileged(Native Method)
  6299. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  6300. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  6301. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  6302. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  6303. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  6304. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  6305. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  6306. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  6307. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  6308. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  6309. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  6310. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  6311. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  6312. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  6313. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  6314. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  6315. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  6316. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  6317. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  6318. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  6319. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  6320. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6321. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6322. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6323. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6324. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  6325. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  6326. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  6327. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  6328. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  6329. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  6330. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  6331. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  6332. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6333. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6334. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6335. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6336. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  6337. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  6338. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  6339. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  6340. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6341. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6342. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6343. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6344. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  6345. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  6346. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  6347. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  6348. at scala.Option.getOrElse(Option.scala:121)
  6349. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  6350. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  6351. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  6352. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  6353. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6354. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6355. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6356. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6357. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  6358. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  6359. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  6360. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  6361. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  6362. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  6363. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  6364. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  6365. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  6366. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  6367. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  6368. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  6369. at $line3.$read$$iw$$iw.<init>(<console>:15)
  6370. at $line3.$read$$iw.<init>(<console>:42)
  6371. at $line3.$read.<init>(<console>:44)
  6372. at $line3.$read$.<init>(<console>:48)
  6373. at $line3.$read$.<clinit>(<console>)
  6374. at $line3.$eval$.$print$lzycompute(<console>:7)
  6375. at $line3.$eval$.$print(<console>:6)
  6376. at $line3.$eval.$print(<console>)
  6377. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6378. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6379. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6380. at java.lang.reflect.Method.invoke(Method.java:498)
  6381. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  6382. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  6383. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  6384. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  6385. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  6386. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  6387. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  6388. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  6389. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  6390. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  6391. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  6392. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  6393. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  6394. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  6395. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  6396. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  6397. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  6398. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  6399. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  6400. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  6401. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  6402. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  6403. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  6404. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  6405. at org.apache.spark.repl.Main$.main(Main.scala:51)
  6406. at org.apache.spark.repl.Main.main(Main.scala)
  6407. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6408. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6409. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6410. at java.lang.reflect.Method.invoke(Method.java:498)
  6411. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  6412. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  6413. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  6414. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  6415. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  6416. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  6417. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6418. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  6419. ... 153 more
  6420. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  6421. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6422. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6423. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  6424. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  6425. at java.security.AccessController.doPrivileged(Native Method)
  6426. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  6427. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  6428. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6429. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6430. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6431. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6432. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6433. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  6434. at java.security.AccessController.doPrivileged(Native Method)
  6435. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  6436. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  6437. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6438. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6439. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6440. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6441. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6442. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  6443. at java.security.AccessController.doPrivileged(Native Method)
  6444. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  6445. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  6446. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6447. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6448. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6449. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6450. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6451. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  6452. at java.security.AccessController.doPrivileged(Native Method)
  6453. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  6454. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  6455. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  6456. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6457. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6458. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  6459. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  6460. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  6461. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  6462. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  6463. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  6464. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  6465. at java.security.AccessController.doPrivileged(Native Method)
  6466. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  6467. ... 150 more
  6468. ------
  6469.  
  6470. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6471. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6472. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6473. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6474. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  6475. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  6476. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  6477. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  6478. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  6479. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6480. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6481. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6482. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6483. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  6484. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  6485. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  6486. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  6487. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  6488. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  6489. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  6490. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6491. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6492. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6493. at java.lang.reflect.Method.invoke(Method.java:498)
  6494. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  6495. at java.security.AccessController.doPrivileged(Native Method)
  6496. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  6497. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  6498. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  6499. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  6500. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  6501. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  6502. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  6503. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  6504. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  6505. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  6506. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  6507. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  6508. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  6509. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  6510. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  6511. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  6512. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  6513. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  6514. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  6515. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  6516. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  6517. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6518. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6519. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6520. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6521. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  6522. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  6523. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  6524. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  6525. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  6526. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  6527. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  6528. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  6529. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6530. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6531. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6532. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6533. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  6534. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  6535. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  6536. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  6537. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6538. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6539. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6540. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6541. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  6542. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  6543. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  6544. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  6545. at scala.Option.getOrElse(Option.scala:121)
  6546. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  6547. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  6548. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  6549. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  6550. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6551. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6552. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6553. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6554. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  6555. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  6556. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  6557. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  6558. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  6559. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  6560. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  6561. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  6562. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  6563. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  6564. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  6565. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  6566. at $line3.$read$$iw$$iw.<init>(<console>:15)
  6567. at $line3.$read$$iw.<init>(<console>:42)
  6568. at $line3.$read.<init>(<console>:44)
  6569. at $line3.$read$.<init>(<console>:48)
  6570. at $line3.$read$.<clinit>(<console>)
  6571. at $line3.$eval$.$print$lzycompute(<console>:7)
  6572. at $line3.$eval$.$print(<console>:6)
  6573. at $line3.$eval.$print(<console>)
  6574. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6575. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6576. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6577. at java.lang.reflect.Method.invoke(Method.java:498)
  6578. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  6579. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  6580. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  6581. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  6582. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  6583. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  6584. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  6585. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  6586. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  6587. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  6588. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  6589. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  6590. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  6591. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  6592. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  6593. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  6594. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  6595. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  6596. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  6597. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  6598. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  6599. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  6600. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  6601. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  6602. at org.apache.spark.repl.Main$.main(Main.scala:51)
  6603. at org.apache.spark.repl.Main.main(Main.scala)
  6604. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6605. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6606. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6607. at java.lang.reflect.Method.invoke(Method.java:498)
  6608. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  6609. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  6610. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  6611. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  6612. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  6613. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  6614. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  6615. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  6616. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  6617. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  6618. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  6619. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  6620. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  6621. at java.security.AccessController.doPrivileged(Native Method)
  6622. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  6623. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  6624. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  6625. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  6626. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  6627. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  6628. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  6629. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  6630. ... 137 more
  6631. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  6632. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6633. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  6634. ... 153 more
  6635. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  6636. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6637. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6638. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  6639. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  6640. at java.security.AccessController.doPrivileged(Native Method)
  6641. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  6642. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  6643. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6644. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6645. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6646. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6647. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6648. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  6649. at java.security.AccessController.doPrivileged(Native Method)
  6650. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  6651. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  6652. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6653. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6654. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6655. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6656. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6657. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  6658. at java.security.AccessController.doPrivileged(Native Method)
  6659. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  6660. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  6661. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6662. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6663. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6664. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6665. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6666. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  6667. at java.security.AccessController.doPrivileged(Native Method)
  6668. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  6669. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  6670. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  6671. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6672. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6673. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  6674. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  6675. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  6676. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  6677. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  6678. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  6679. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  6680. at java.security.AccessController.doPrivileged(Native Method)
  6681. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  6682. ... 150 more
  6683. 17/03/30 08:43:41 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/Users/james/spark/jars/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/Users/james/spark-2.1.0-bin-hadoop2.7/jars/datanucleus-api-jdo-3.2.6.jar."
  6684. 17/03/30 08:43:41 WARN General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/Users/james/spark-2.1.0-bin-hadoop2.7/jars/datanucleus-core-3.2.10.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/Users/james/spark/jars/datanucleus-core-3.2.10.jar."
  6685. 17/03/30 08:43:41 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/Users/james/spark-2.1.0-bin-hadoop2.7/jars/datanucleus-rdbms-3.2.9.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/Users/james/spark/jars/datanucleus-rdbms-3.2.9.jar."
  6686. 17/03/30 08:43:41 ERROR Schema: Failed initialising database.
  6687. Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  6688. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  6689. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  6690. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  6691. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  6692. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  6693. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  6694. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  6695. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  6696. at java.security.AccessController.doPrivileged(Native Method)
  6697. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  6698. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  6699. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  6700. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  6701. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  6702. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  6703. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  6704. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  6705. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  6706. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  6707. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  6708. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6709. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6710. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6711. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6712. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  6713. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  6714. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  6715. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  6716. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  6717. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  6718. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  6719. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6720. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6721. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6722. at java.lang.reflect.Method.invoke(Method.java:498)
  6723. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  6724. at java.security.AccessController.doPrivileged(Native Method)
  6725. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  6726. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  6727. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  6728. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  6729. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  6730. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  6731. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  6732. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  6733. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  6734. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  6735. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  6736. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  6737. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  6738. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  6739. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  6740. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  6741. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  6742. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  6743. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  6744. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  6745. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  6746. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6747. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6748. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6749. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6750. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  6751. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  6752. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  6753. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  6754. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  6755. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  6756. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  6757. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  6758. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6759. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6760. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6761. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6762. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  6763. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  6764. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  6765. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  6766. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6767. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6768. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6769. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6770. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  6771. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  6772. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  6773. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  6774. at scala.Option.getOrElse(Option.scala:121)
  6775. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  6776. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  6777. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  6778. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  6779. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6780. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6781. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6782. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6783. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  6784. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  6785. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  6786. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  6787. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  6788. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  6789. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  6790. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  6791. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  6792. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  6793. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  6794. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  6795. at $line3.$read$$iw$$iw.<init>(<console>:15)
  6796. at $line3.$read$$iw.<init>(<console>:42)
  6797. at $line3.$read.<init>(<console>:44)
  6798. at $line3.$read$.<init>(<console>:48)
  6799. at $line3.$read$.<clinit>(<console>)
  6800. at $line3.$eval$.$print$lzycompute(<console>:7)
  6801. at $line3.$eval$.$print(<console>:6)
  6802. at $line3.$eval.$print(<console>)
  6803. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6804. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6805. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6806. at java.lang.reflect.Method.invoke(Method.java:498)
  6807. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  6808. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  6809. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  6810. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  6811. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  6812. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  6813. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  6814. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  6815. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  6816. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  6817. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  6818. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  6819. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  6820. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  6821. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  6822. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  6823. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  6824. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  6825. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  6826. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  6827. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  6828. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  6829. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  6830. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  6831. at org.apache.spark.repl.Main$.main(Main.scala:51)
  6832. at org.apache.spark.repl.Main.main(Main.scala)
  6833. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6834. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6835. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6836. at java.lang.reflect.Method.invoke(Method.java:498)
  6837. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  6838. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  6839. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  6840. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  6841. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  6842. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  6843. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6844. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  6845. ... 153 more
  6846. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  6847. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6848. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6849. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  6850. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  6851. at java.security.AccessController.doPrivileged(Native Method)
  6852. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  6853. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  6854. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6855. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6856. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6857. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6858. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6859. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  6860. at java.security.AccessController.doPrivileged(Native Method)
  6861. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  6862. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  6863. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6864. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6865. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6866. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6867. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6868. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  6869. at java.security.AccessController.doPrivileged(Native Method)
  6870. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  6871. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  6872. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6873. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6874. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6875. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6876. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6877. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  6878. at java.security.AccessController.doPrivileged(Native Method)
  6879. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  6880. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  6881. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  6882. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6883. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6884. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  6885. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  6886. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  6887. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  6888. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  6889. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  6890. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  6891. at java.security.AccessController.doPrivileged(Native Method)
  6892. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  6893. ... 150 more
  6894. ------
  6895.  
  6896. org.datanucleus.exceptions.NucleusDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  6897. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  6898. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  6899. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  6900. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  6901. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  6902. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  6903. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  6904. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  6905. at java.security.AccessController.doPrivileged(Native Method)
  6906. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  6907. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  6908. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  6909. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  6910. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  6911. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  6912. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  6913. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  6914. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  6915. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  6916. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  6917. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6918. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6919. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6920. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6921. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  6922. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  6923. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  6924. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  6925. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  6926. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  6927. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  6928. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6929. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6930. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6931. at java.lang.reflect.Method.invoke(Method.java:498)
  6932. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  6933. at java.security.AccessController.doPrivileged(Native Method)
  6934. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  6935. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  6936. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  6937. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  6938. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  6939. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  6940. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  6941. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  6942. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  6943. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  6944. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  6945. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  6946. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  6947. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  6948. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  6949. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  6950. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  6951. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  6952. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  6953. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  6954. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  6955. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6956. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6957. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6958. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6959. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  6960. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  6961. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  6962. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  6963. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  6964. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  6965. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  6966. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  6967. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6968. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6969. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6970. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6971. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  6972. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  6973. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  6974. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  6975. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6976. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6977. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6978. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6979. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  6980. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  6981. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  6982. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  6983. at scala.Option.getOrElse(Option.scala:121)
  6984. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  6985. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  6986. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  6987. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  6988. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6989. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6990. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6991. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6992. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  6993. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  6994. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  6995. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  6996. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  6997. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  6998. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  6999. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  7000. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  7001. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  7002. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  7003. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  7004. at $line3.$read$$iw$$iw.<init>(<console>:15)
  7005. at $line3.$read$$iw.<init>(<console>:42)
  7006. at $line3.$read.<init>(<console>:44)
  7007. at $line3.$read$.<init>(<console>:48)
  7008. at $line3.$read$.<clinit>(<console>)
  7009. at $line3.$eval$.$print$lzycompute(<console>:7)
  7010. at $line3.$eval$.$print(<console>:6)
  7011. at $line3.$eval.$print(<console>)
  7012. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  7013. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  7014. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  7015. at java.lang.reflect.Method.invoke(Method.java:498)
  7016. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  7017. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  7018. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  7019. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  7020. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  7021. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  7022. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  7023. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  7024. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  7025. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  7026. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  7027. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  7028. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  7029. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  7030. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  7031. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  7032. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  7033. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  7034. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  7035. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  7036. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  7037. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  7038. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  7039. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  7040. at org.apache.spark.repl.Main$.main(Main.scala:51)
  7041. at org.apache.spark.repl.Main.main(Main.scala)
  7042. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  7043. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  7044. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  7045. at java.lang.reflect.Method.invoke(Method.java:498)
  7046. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  7047. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  7048. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  7049. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  7050. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  7051. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  7052. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  7053. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  7054. ... 153 more
  7055. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  7056. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  7057. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  7058. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  7059. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  7060. at java.security.AccessController.doPrivileged(Native Method)
  7061. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  7062. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  7063. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7064. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7065. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  7066. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  7067. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  7068. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  7069. at java.security.AccessController.doPrivileged(Native Method)
  7070. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  7071. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  7072. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7073. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7074. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  7075. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  7076. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  7077. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  7078. at java.security.AccessController.doPrivileged(Native Method)
  7079. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  7080. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  7081. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7082. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7083. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  7084. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  7085. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  7086. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  7087. at java.security.AccessController.doPrivileged(Native Method)
  7088. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  7089. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  7090. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  7091. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7092. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7093. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  7094. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  7095. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  7096. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  7097. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  7098. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  7099. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  7100. at java.security.AccessController.doPrivileged(Native Method)
  7101. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  7102. ... 150 more
  7103. ------
  7104.  
  7105. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:516)
  7106. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  7107. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7108. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7109. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7110. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7111. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  7112. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  7113. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  7114. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  7115. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  7116. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  7117. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  7118. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  7119. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  7120. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  7121. at java.lang.reflect.Method.invoke(Method.java:498)
  7122. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  7123. at java.security.AccessController.doPrivileged(Native Method)
  7124. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  7125. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  7126. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  7127. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  7128. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  7129. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  7130. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  7131. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  7132. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  7133. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  7134. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  7135. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  7136. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  7137. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  7138. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  7139. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  7140. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  7141. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  7142. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  7143. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  7144. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  7145. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7146. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7147. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7148. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7149. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  7150. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  7151. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  7152. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  7153. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  7154. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  7155. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  7156. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  7157. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7158. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7159. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7160. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7161. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  7162. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  7163. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  7164. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  7165. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7166. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7167. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7168. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7169. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  7170. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  7171. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  7172. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  7173. at scala.Option.getOrElse(Option.scala:121)
  7174. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  7175. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  7176. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  7177. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  7178. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7179. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7180. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7181. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7182. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  7183. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  7184. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  7185. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  7186. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  7187. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  7188. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  7189. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  7190. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  7191. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  7192. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  7193. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  7194. at $line3.$read$$iw$$iw.<init>(<console>:15)
  7195. at $line3.$read$$iw.<init>(<console>:42)
  7196. at $line3.$read.<init>(<console>:44)
  7197. at $line3.$read$.<init>(<console>:48)
  7198. at $line3.$read$.<clinit>(<console>)
  7199. at $line3.$eval$.$print$lzycompute(<console>:7)
  7200. at $line3.$eval$.$print(<console>:6)
  7201. at $line3.$eval.$print(<console>)
  7202. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  7203. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  7204. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  7205. at java.lang.reflect.Method.invoke(Method.java:498)
  7206. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  7207. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  7208. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  7209. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  7210. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  7211. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  7212. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  7213. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  7214. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  7215. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  7216. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  7217. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  7218. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  7219. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  7220. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  7221. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  7222. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  7223. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  7224. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  7225. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  7226. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  7227. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  7228. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  7229. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  7230. at org.apache.spark.repl.Main$.main(Main.scala:51)
  7231. at org.apache.spark.repl.Main.main(Main.scala)
  7232. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  7233. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  7234. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  7235. at java.lang.reflect.Method.invoke(Method.java:498)
  7236. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  7237. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  7238. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  7239. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  7240. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  7241. Caused by: java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  7242. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  7243. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  7244. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  7245. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  7246. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  7247. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  7248. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  7249. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  7250. at java.security.AccessController.doPrivileged(Native Method)
  7251. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  7252. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  7253. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  7254. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  7255. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  7256. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  7257. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  7258. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  7259. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  7260. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  7261. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  7262. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7263. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7264. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7265. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7266. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  7267. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  7268. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  7269. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  7270. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  7271. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  7272. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  7273. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  7274. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  7275. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  7276. at java.lang.reflect.Method.invoke(Method.java:498)
  7277. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  7278. at java.security.AccessController.doPrivileged(Native Method)
  7279. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  7280. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  7281. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  7282. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  7283. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  7284. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  7285. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  7286. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  7287. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  7288. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  7289. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  7290. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  7291. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  7292. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  7293. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  7294. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  7295. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  7296. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  7297. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  7298. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  7299. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  7300. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7301. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7302. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7303. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7304. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  7305. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  7306. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  7307. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  7308. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  7309. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  7310. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  7311. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  7312. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7313. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7314. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7315. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7316. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  7317. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  7318. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  7319. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  7320. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7321. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7322. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7323. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7324. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  7325. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  7326. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  7327. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  7328. at scala.Option.getOrElse(Option.scala:121)
  7329. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  7330. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  7331. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  7332. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  7333. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7334. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7335. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7336. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7337. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  7338. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  7339. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  7340. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  7341. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  7342. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  7343. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  7344. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  7345. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  7346. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  7347. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  7348. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  7349. at $line3.$read$$iw$$iw.<init>(<console>:15)
  7350. at $line3.$read$$iw.<init>(<console>:42)
  7351. at $line3.$read.<init>(<console>:44)
  7352. at $line3.$read$.<init>(<console>:48)
  7353. at $line3.$read$.<clinit>(<console>)
  7354. at $line3.$eval$.$print$lzycompute(<console>:7)
  7355. at $line3.$eval$.$print(<console>:6)
  7356. at $line3.$eval.$print(<console>)
  7357. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  7358. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  7359. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  7360. at java.lang.reflect.Method.invoke(Method.java:498)
  7361. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  7362. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  7363. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  7364. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  7365. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  7366. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  7367. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  7368. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  7369. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  7370. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  7371. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  7372. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  7373. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  7374. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  7375. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  7376. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  7377. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  7378. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  7379. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  7380. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  7381. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  7382. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  7383. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  7384. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  7385. at org.apache.spark.repl.Main$.main(Main.scala:51)
  7386. at org.apache.spark.repl.Main.main(Main.scala)
  7387. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  7388. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  7389. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  7390. at java.lang.reflect.Method.invoke(Method.java:498)
  7391. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  7392. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  7393. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  7394. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  7395. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  7396. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  7397. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  7398. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  7399. ... 153 more
  7400. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  7401. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  7402. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  7403. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  7404. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  7405. at java.security.AccessController.doPrivileged(Native Method)
  7406. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  7407. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  7408. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7409. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7410. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  7411. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  7412. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  7413. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  7414. at java.security.AccessController.doPrivileged(Native Method)
  7415. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  7416. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  7417. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7418. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7419. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  7420. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  7421. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  7422. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  7423. at java.security.AccessController.doPrivileged(Native Method)
  7424. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  7425. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  7426. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7427. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7428. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  7429. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  7430. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  7431. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  7432. at java.security.AccessController.doPrivileged(Native Method)
  7433. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  7434. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  7435. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  7436. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7437. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7438. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  7439. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  7440. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  7441. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  7442. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  7443. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  7444. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  7445. at java.security.AccessController.doPrivileged(Native Method)
  7446. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  7447. ... 150 more
  7448. ------
  7449.  
  7450. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7451. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7452. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7453. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7454. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  7455. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  7456. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  7457. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  7458. ... 135 more
  7459. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  7460. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  7461. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  7462. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  7463. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  7464. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  7465. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  7466. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  7467. at java.security.AccessController.doPrivileged(Native Method)
  7468. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  7469. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  7470. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  7471. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  7472. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  7473. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  7474. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  7475. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  7476. ... 137 more
  7477. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  7478. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  7479. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  7480. ... 153 more
  7481. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  7482. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  7483. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  7484. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  7485. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  7486. at java.security.AccessController.doPrivileged(Native Method)
  7487. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  7488. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  7489. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7490. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7491. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  7492. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  7493. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  7494. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  7495. at java.security.AccessController.doPrivileged(Native Method)
  7496. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  7497. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  7498. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7499. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7500. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  7501. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  7502. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  7503. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  7504. at java.security.AccessController.doPrivileged(Native Method)
  7505. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  7506. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  7507. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7508. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7509. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  7510. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  7511. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  7512. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  7513. at java.security.AccessController.doPrivileged(Native Method)
  7514. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  7515. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  7516. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  7517. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7518. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7519. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  7520. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  7521. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  7522. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  7523. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  7524. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  7525. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  7526. at java.security.AccessController.doPrivileged(Native Method)
  7527. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  7528. ... 150 more
  7529. Nested Throwables StackTrace:
  7530. java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  7531. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  7532. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  7533. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  7534. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  7535. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  7536. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  7537. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  7538. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  7539. at java.security.AccessController.doPrivileged(Native Method)
  7540. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  7541. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  7542. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  7543. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  7544. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  7545. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  7546. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  7547. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  7548. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  7549. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  7550. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  7551. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7552. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7553. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7554. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7555. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  7556. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  7557. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  7558. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  7559. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  7560. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  7561. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  7562. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  7563. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  7564. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  7565. at java.lang.reflect.Method.invoke(Method.java:498)
  7566. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  7567. at java.security.AccessController.doPrivileged(Native Method)
  7568. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  7569. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  7570. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  7571. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  7572. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  7573. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  7574. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  7575. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  7576. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  7577. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  7578. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  7579. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  7580. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  7581. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  7582. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  7583. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  7584. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  7585. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  7586. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  7587. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  7588. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  7589. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7590. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7591. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7592. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7593. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  7594. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  7595. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  7596. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  7597. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  7598. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  7599. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  7600. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  7601. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7602. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7603. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7604. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7605. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  7606. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  7607. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  7608. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  7609. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7610. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7611. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7612. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7613. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  7614. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  7615. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  7616. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  7617. at scala.Option.getOrElse(Option.scala:121)
  7618. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  7619. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  7620. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  7621. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  7622. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7623. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7624. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7625. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7626. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  7627. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  7628. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  7629. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  7630. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  7631. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  7632. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  7633. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  7634. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  7635. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  7636. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  7637. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  7638. at $line3.$read$$iw$$iw.<init>(<console>:15)
  7639. at $line3.$read$$iw.<init>(<console>:42)
  7640. at $line3.$read.<init>(<console>:44)
  7641. at $line3.$read$.<init>(<console>:48)
  7642. at $line3.$read$.<clinit>(<console>)
  7643. at $line3.$eval$.$print$lzycompute(<console>:7)
  7644. at $line3.$eval$.$print(<console>:6)
  7645. at $line3.$eval.$print(<console>)
  7646. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  7647. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  7648. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  7649. at java.lang.reflect.Method.invoke(Method.java:498)
  7650. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  7651. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  7652. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  7653. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  7654. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  7655. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  7656. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  7657. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  7658. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  7659. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  7660. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  7661. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  7662. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  7663. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  7664.  
  7665. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  7666. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  7667. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  7668. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  7669. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  7670. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  7671. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  7672. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  7673. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  7674. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  7675. at org.apache.spark.repl.Main$.main(Main.scala:51)
  7676. at org.apache.spark.repl.Main.main(Main.scala)
  7677. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  7678. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  7679. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  7680. at java.lang.reflect.Method.invoke(Method.java:498)
  7681. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  7682. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  7683. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  7684. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  7685. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  7686. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  7687. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  7688. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  7689. ... 153 more
  7690. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  7691. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  7692. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  7693. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  7694. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  7695. at java.security.AccessController.doPrivileged(Native Method)
  7696. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  7697. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  7698. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7699. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7700. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  7701. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  7702. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  7703. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  7704. at java.security.AccessController.doPrivileged(Native Method)
  7705. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  7706. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  7707. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7708. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7709. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  7710. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  7711. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  7712. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  7713. at java.security.AccessController.doPrivileged(Native Method)
  7714. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  7715. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  7716. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7717. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7718. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  7719. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  7720. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  7721. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  7722. at java.security.AccessController.doPrivileged(Native Method)
  7723. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  7724. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  7725. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  7726. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7727. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7728. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  7729. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  7730. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  7731. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  7732. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  7733. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  7734. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  7735. at java.security.AccessController.doPrivileged(Native Method)
  7736. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  7737. ... 150 more
  7738. ------
  7739.  
  7740. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7741. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7742. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7743. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7744. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  7745. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  7746. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  7747. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  7748. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  7749. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7750. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7751. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7752. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7753. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  7754. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  7755. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  7756. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  7757. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  7758. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  7759. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  7760. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  7761. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  7762. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  7763. at java.lang.reflect.Method.invoke(Method.java:498)
  7764. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  7765. at java.security.AccessController.doPrivileged(Native Method)
  7766. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  7767. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  7768. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  7769. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  7770. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  7771. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  7772. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  7773. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  7774. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  7775. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  7776. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  7777. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  7778. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  7779. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  7780. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  7781. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  7782. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  7783. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  7784. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  7785. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  7786. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  7787. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7788. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7789. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7790. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7791. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  7792. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  7793. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  7794. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  7795. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  7796. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  7797. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  7798. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  7799. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7800. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7801. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7802. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7803. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  7804. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  7805. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  7806. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  7807. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7808. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7809. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7810. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7811. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  7812. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  7813. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  7814. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  7815. at scala.Option.getOrElse(Option.scala:121)
  7816. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  7817. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  7818. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  7819. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  7820. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7821. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7822. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7823. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7824. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  7825. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  7826. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  7827. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  7828. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  7829. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  7830. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  7831. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  7832. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  7833. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  7834. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  7835. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  7836. at $line3.$read$$iw$$iw.<init>(<console>:15)
  7837. at $line3.$read$$iw.<init>(<console>:42)
  7838. at $line3.$read.<init>(<console>:44)
  7839. at $line3.$read$.<init>(<console>:48)
  7840. at $line3.$read$.<clinit>(<console>)
  7841. at $line3.$eval$.$print$lzycompute(<console>:7)
  7842. at $line3.$eval$.$print(<console>:6)
  7843. at $line3.$eval.$print(<console>)
  7844. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  7845. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  7846. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  7847. at java.lang.reflect.Method.invoke(Method.java:498)
  7848. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  7849. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  7850. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  7851. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  7852. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  7853. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  7854. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  7855. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  7856. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  7857. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  7858. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  7859. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  7860. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  7861. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  7862. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  7863. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  7864. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  7865. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  7866. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  7867. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  7868. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  7869. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  7870. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  7871. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  7872. at org.apache.spark.repl.Main$.main(Main.scala:51)
  7873. at org.apache.spark.repl.Main.main(Main.scala)
  7874. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  7875. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  7876. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  7877. at java.lang.reflect.Method.invoke(Method.java:498)
  7878. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  7879. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  7880. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  7881. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  7882. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  7883. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  7884. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  7885. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  7886. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  7887. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  7888. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  7889. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  7890. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  7891. at java.security.AccessController.doPrivileged(Native Method)
  7892. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  7893. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  7894. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  7895. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  7896. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  7897. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  7898. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  7899. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  7900. ... 137 more
  7901. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  7902. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  7903. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  7904. ... 153 more
  7905. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  7906. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  7907. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  7908. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  7909. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  7910. at java.security.AccessController.doPrivileged(Native Method)
  7911. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  7912. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  7913. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7914. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7915. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  7916. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  7917. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  7918. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  7919. at java.security.AccessController.doPrivileged(Native Method)
  7920. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  7921. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  7922. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7923. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7924. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  7925. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  7926. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  7927. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  7928. at java.security.AccessController.doPrivileged(Native Method)
  7929. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  7930. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  7931. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7932. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7933. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  7934. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  7935. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  7936. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  7937. at java.security.AccessController.doPrivileged(Native Method)
  7938. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  7939. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  7940. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  7941. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  7942. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  7943. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  7944. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  7945. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  7946. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  7947. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  7948. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  7949. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  7950. at java.security.AccessController.doPrivileged(Native Method)
  7951. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  7952. ... 150 more
  7953. java.lang.IllegalArgumentException: Error while instantiating 'org.apache.spark.sql.hive.HiveSessionState':
  7954. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:981)
  7955. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  7956. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  7957. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  7958. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  7959. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  7960. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  7961. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  7962. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  7963. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  7964. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  7965. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  7966. ... 47 elided
  7967. Caused by: java.lang.reflect.InvocationTargetException: java.lang.IllegalArgumentException: Error while instantiating 'org.apache.spark.sql.hive.HiveExternalCatalog':
  7968. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7969. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7970. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7971. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7972. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  7973. ... 58 more
  7974. Caused by: java.lang.IllegalArgumentException: Error while instantiating 'org.apache.spark.sql.hive.HiveExternalCatalog':
  7975. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:169)
  7976. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  7977. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  7978. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  7979. at scala.Option.getOrElse(Option.scala:121)
  7980. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  7981. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  7982. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  7983. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  7984. ... 63 more
  7985. Caused by: java.lang.reflect.InvocationTargetException: java.lang.reflect.InvocationTargetException: java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  7986. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7987. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7988. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7989. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7990. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  7991. ... 71 more
  7992. Caused by: java.lang.reflect.InvocationTargetException: java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  7993. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  7994. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  7995. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  7996. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  7997. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  7998. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  7999. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  8000. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  8001. ... 76 more
  8002. Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  8003. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
  8004. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  8005. ... 84 more
  8006. Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  8007. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
  8008. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  8009. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  8010. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  8011. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  8012. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  8013. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  8014. ... 85 more
  8015. Caused by: java.lang.reflect.InvocationTargetException: javax.jdo.JDOFatalDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  8016. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  8017. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  8018. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  8019. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  8020. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  8021. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  8022. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  8023. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  8024. at java.security.AccessController.doPrivileged(Native Method)
  8025. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  8026. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  8027. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  8028. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  8029. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  8030. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  8031. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  8032. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  8033. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  8034. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  8035. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  8036. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8037. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8038. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8039. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8040. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  8041. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  8042. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  8043. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  8044. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  8045. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  8046. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  8047. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  8048. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  8049. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  8050. at java.lang.reflect.Method.invoke(Method.java:498)
  8051. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  8052. at java.security.AccessController.doPrivileged(Native Method)
  8053. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  8054. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  8055. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  8056. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  8057. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  8058. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  8059. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  8060. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  8061. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  8062. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  8063. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  8064. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  8065. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  8066. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  8067. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  8068. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  8069. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  8070. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  8071. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  8072. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  8073. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  8074. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8075. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8076. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8077. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8078. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  8079. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  8080. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  8081. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  8082. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  8083. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  8084. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  8085. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  8086. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8087. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8088. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8089. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8090. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  8091. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  8092. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  8093. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  8094. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8095. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8096. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8097. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8098. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  8099. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  8100. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  8101. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  8102. at scala.Option.getOrElse(Option.scala:121)
  8103. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  8104. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  8105. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  8106. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  8107. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8108. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8109. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8110. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8111. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  8112. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  8113. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  8114. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  8115. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  8116. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  8117. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  8118. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  8119. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  8120. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  8121. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  8122. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  8123. at <init>(<console>:15)
  8124. at <init>(<console>:42)
  8125. at <init>(<console>:44)
  8126. at .<init>(<console>:48)
  8127. at .<clinit>(<console>)
  8128. at .$print$lzycompute(<console>:7)
  8129. at .$print(<console>:6)
  8130. at $print(<console>)
  8131. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  8132. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  8133. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  8134. at java.lang.reflect.Method.invoke(Method.java:498)
  8135. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  8136. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  8137. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  8138. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  8139. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  8140. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  8141. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  8142. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  8143. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  8144. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  8145. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  8146. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  8147. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  8148. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  8149. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  8150. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  8151. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  8152. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  8153. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  8154. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  8155. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  8156. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  8157. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  8158. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  8159. at org.apache.spark.repl.Main$.main(Main.scala:51)
  8160. at org.apache.spark.repl.Main.main(Main.scala)
  8161. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  8162. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  8163. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  8164. at java.lang.reflect.Method.invoke(Method.java:498)
  8165. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  8166. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  8167. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  8168. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  8169. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  8170. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  8171. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  8172. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  8173. ... 153 more
  8174. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  8175. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  8176. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  8177. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  8178. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  8179. at java.security.AccessController.doPrivileged(Native Method)
  8180. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  8181. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  8182. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  8183. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  8184. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  8185. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  8186. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  8187. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  8188. at java.security.AccessController.doPrivileged(Native Method)
  8189. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  8190. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  8191. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  8192. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  8193. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  8194. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  8195. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  8196. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  8197. at java.security.AccessController.doPrivileged(Native Method)
  8198. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  8199. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  8200. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  8201. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  8202. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  8203. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  8204. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  8205. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  8206. at java.security.AccessController.doPrivileged(Native Method)
  8207. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  8208. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  8209. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  8210. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  8211. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  8212. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  8213. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  8214. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  8215. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  8216. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  8217. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  8218. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  8219. at java.security.AccessController.doPrivileged(Native Method)
  8220. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  8221. ... 150 more
  8222. ------
  8223.  
  8224. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8225. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8226. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8227. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8228. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  8229. ... 91 more
  8230. Caused by: javax.jdo.JDOFatalDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  8231. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  8232. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  8233. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  8234. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  8235. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  8236. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  8237. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  8238. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  8239. at java.security.AccessController.doPrivileged(Native Method)
  8240. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  8241. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  8242. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  8243. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  8244. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  8245. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  8246. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  8247. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  8248. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  8249. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  8250. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  8251. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8252. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8253. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8254. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8255. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  8256. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  8257. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  8258. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  8259. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  8260. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  8261. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  8262. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  8263. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  8264. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  8265. at java.lang.reflect.Method.invoke(Method.java:498)
  8266. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  8267. at java.security.AccessController.doPrivileged(Native Method)
  8268. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  8269. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  8270. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  8271. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  8272. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  8273. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  8274. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  8275. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  8276. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  8277. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  8278. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  8279. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  8280. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  8281. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  8282. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  8283. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  8284. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  8285. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  8286. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  8287. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  8288. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  8289. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8290. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8291. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8292. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8293. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  8294. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  8295. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  8296. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  8297. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  8298. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  8299. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  8300. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  8301. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8302. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8303. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8304. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8305. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  8306. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  8307. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  8308. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  8309. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8310. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8311. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8312. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8313. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  8314. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  8315. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  8316. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  8317. at scala.Option.getOrElse(Option.scala:121)
  8318. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  8319. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  8320. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  8321. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  8322. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8323. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8324. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8325. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8326. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  8327. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  8328. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  8329. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  8330. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  8331. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  8332. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  8333. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  8334. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  8335. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  8336. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  8337. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  8338. at <init>(<console>:15)
  8339. at <init>(<console>:42)
  8340. at <init>(<console>:44)
  8341. at .<init>(<console>:48)
  8342. at .<clinit>(<console>)
  8343. at .$print$lzycompute(<console>:7)
  8344. at .$print(<console>:6)
  8345. at $print(<console>)
  8346. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  8347. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  8348. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  8349. at java.lang.reflect.Method.invoke(Method.java:498)
  8350. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  8351. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  8352. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  8353. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  8354. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  8355. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  8356. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  8357. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  8358. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  8359. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  8360. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  8361. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  8362. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  8363. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  8364. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  8365. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  8366. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  8367. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  8368. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  8369. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  8370. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  8371. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  8372. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  8373. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  8374. at org.apache.spark.repl.Main$.main(Main.scala:51)
  8375. at org.apache.spark.repl.Main.main(Main.scala)
  8376. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  8377. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  8378. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  8379. at java.lang.reflect.Method.invoke(Method.java:498)
  8380. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  8381. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  8382. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  8383. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  8384. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  8385. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  8386. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  8387. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  8388. ... 153 more
  8389. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  8390. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  8391. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  8392. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  8393. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  8394. at java.security.AccessController.doPrivileged(Native Method)
  8395. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  8396. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  8397. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  8398. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  8399. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  8400. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  8401. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  8402. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  8403. at java.security.AccessController.doPrivileged(Native Method)
  8404. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  8405. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  8406. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  8407. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  8408. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  8409. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  8410. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  8411. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  8412. at java.security.AccessController.doPrivileged(Native Method)
  8413. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  8414. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  8415. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  8416. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  8417. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  8418. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  8419. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  8420. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  8421. at java.security.AccessController.doPrivileged(Native Method)
  8422. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  8423. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  8424. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  8425. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  8426. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  8427. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  8428. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  8429. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  8430. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  8431. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  8432. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  8433. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  8434. at java.security.AccessController.doPrivileged(Native Method)
  8435. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  8436. ... 150 more
  8437. ------
  8438.  
  8439. at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:436)
  8440. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:788)
  8441. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  8442. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  8443. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  8444. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  8445. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  8446. at java.lang.reflect.Method.invoke(Method.java:498)
  8447. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  8448. at java.security.AccessController.doPrivileged(Native Method)
  8449. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  8450. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  8451. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  8452. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  8453. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  8454. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  8455. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  8456. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  8457. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  8458. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  8459. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  8460. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  8461. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  8462. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  8463. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  8464. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  8465. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  8466. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  8467. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  8468. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  8469. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  8470. ... 96 more
  8471. Caused by: java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  8472. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  8473. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  8474. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  8475. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  8476. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  8477. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  8478. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  8479. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  8480. at java.security.AccessController.doPrivileged(Native Method)
  8481. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  8482. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  8483. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  8484. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  8485. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  8486. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  8487. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  8488. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  8489. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  8490. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  8491. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  8492. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8493. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8494. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8495. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8496. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  8497. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  8498. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  8499. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  8500. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  8501. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  8502. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  8503. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  8504. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  8505. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  8506. at java.lang.reflect.Method.invoke(Method.java:498)
  8507. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  8508. at java.security.AccessController.doPrivileged(Native Method)
  8509. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  8510. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  8511. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  8512. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  8513. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  8514. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  8515. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  8516. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  8517. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  8518. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  8519. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  8520. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  8521. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  8522. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  8523. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  8524. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  8525. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  8526. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  8527. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  8528. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  8529. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  8530. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8531. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8532. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8533. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8534. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  8535. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  8536. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  8537. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  8538. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  8539. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  8540. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  8541. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
  8542. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8543. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8544. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8545. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8546. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  8547. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
  8548. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
  8549. at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
  8550. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8551. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8552. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8553. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8554. at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
  8555. at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
  8556. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  8557. at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
  8558. at scala.Option.getOrElse(Option.scala:121)
  8559. at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
  8560. at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
  8561. at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
  8562. at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
  8563. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8564. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8565. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8566. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8567. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
  8568. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
  8569. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
  8570. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  8571. at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
  8572. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  8573. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
  8574. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
  8575. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
  8576. at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
  8577. at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
  8578. at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
  8579. at <init>(<console>:15)
  8580. at <init>(<console>:42)
  8581. at <init>(<console>:44)
  8582. at .<init>(<console>:48)
  8583. at .<clinit>(<console>)
  8584. at .$print$lzycompute(<console>:7)
  8585. at .$print(<console>:6)
  8586. at $print(<console>)
  8587. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  8588. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  8589. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  8590. at java.lang.reflect.Method.invoke(Method.java:498)
  8591. at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
  8592. at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
  8593. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
  8594. at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
  8595. at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
  8596. at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
  8597. at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
  8598. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
  8599. at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
  8600. at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
  8601. at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
  8602. at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
  8603. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply$mcV$sp(SparkILoop.scala:38)
  8604. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  8605. at org.apache.spark.repl.SparkILoop$$anonfun$initializeSpark$1.apply(SparkILoop.scala:37)
  8606. at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
  8607. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:37)
  8608. at org.apache.spark.repl.SparkILoop.loadFiles(SparkILoop.scala:105)
  8609. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:920)
  8610. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  8611. at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
  8612. at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
  8613. at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
  8614. at org.apache.spark.repl.Main$.doMain(Main.scala:68)
  8615. at org.apache.spark.repl.Main$.main(Main.scala:51)
  8616. at org.apache.spark.repl.Main.main(Main.scala)
  8617. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  8618. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  8619. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  8620. at java.lang.reflect.Method.invoke(Method.java:498)
  8621. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
  8622. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
  8623. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
  8624. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
  8625. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  8626. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  8627. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  8628. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  8629. ... 153 more
  8630. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  8631. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  8632. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  8633. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  8634. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  8635. at java.security.AccessController.doPrivileged(Native Method)
  8636. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  8637. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  8638. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  8639. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  8640. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  8641. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  8642. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  8643. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  8644. at java.security.AccessController.doPrivileged(Native Method)
  8645. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  8646. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  8647. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  8648. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  8649. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  8650. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  8651. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  8652. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  8653. at java.security.AccessController.doPrivileged(Native Method)
  8654. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  8655. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  8656. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  8657. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  8658. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  8659. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  8660. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  8661. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  8662. at java.security.AccessController.doPrivileged(Native Method)
  8663. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  8664. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  8665. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  8666. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  8667. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  8668. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  8669. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  8670. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  8671. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  8672. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  8673. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  8674. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  8675. at java.security.AccessController.doPrivileged(Native Method)
  8676. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  8677. ... 150 more
  8678. ------
  8679.  
  8680. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8681. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8682. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8683. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8684. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  8685. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  8686. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  8687. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  8688. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  8689. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  8690. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  8691. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  8692. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  8693. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  8694. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  8695. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  8696. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  8697. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  8698. ... 125 more
  8699. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  8700. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  8701. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  8702. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  8703. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  8704. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  8705. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  8706. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  8707. at java.security.AccessController.doPrivileged(Native Method)
  8708. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  8709. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  8710. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  8711. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  8712. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  8713. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  8714. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  8715. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  8716. ... 137 more
  8717. Caused by: org.apache.derby.iapi.error.StandardException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@53314f76, see the next exception for details.
  8718. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  8719. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  8720. ... 153 more
  8721. Caused by: org.apache.derby.iapi.error.StandardException: Another instance of Derby may have already booted the database /Users/james/spark-2.1.0-bin-hadoop2.7/metastore_db.
  8722. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  8723. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  8724. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  8725. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  8726. at java.security.AccessController.doPrivileged(Native Method)
  8727. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  8728. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  8729. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  8730. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  8731. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  8732. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  8733. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  8734. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  8735. at java.security.AccessController.doPrivileged(Native Method)
  8736. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  8737. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  8738. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  8739. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  8740. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  8741. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  8742. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  8743. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  8744. at java.security.AccessController.doPrivileged(Native Method)
  8745. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  8746. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  8747. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  8748. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  8749. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  8750. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  8751. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  8752. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  8753. at java.security.AccessController.doPrivileged(Native Method)
  8754. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  8755. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  8756. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  8757. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  8758. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  8759. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  8760. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  8761. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  8762. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  8763. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  8764. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  8765. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  8766. at java.security.AccessController.doPrivileged(Native Method)
  8767. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  8768. ... 150 more
  8769. <console>:14: error: not found: value spark
  8770. import spark.implicits._
  8771. ^
  8772. <console>:14: error: not found: value spark
  8773. import spark.sql
  8774. ^
  8775. Welcome to
  8776. ____ __
  8777. / __/__ ___ _____/ /__
  8778. _\ \/ _ \/ _ `/ __/ '_/
  8779. /___/ .__/\_,_/_/ /_/\_\ version 2.1.0
  8780. /_/
  8781.  
  8782. Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_77)
  8783. Type in expressions to have them evaluated.
  8784. Type :help for more information.
  8785.  
  8786. scala> :quit
  8787.  
  8788. Script done on Thu Mar 30 08:43:45 2017
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!