API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Aug 13th, 2017
122
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 511.36 KB | None | 0 0
raw download clone embed print report
  1. $ ~/spark/bin/pyspark
  2. Python 2.7.13 (default, Jun 26 2017, 10:20:05)
  3. [GCC 7.1.1 20170622 (Red Hat 7.1.1-3)] on linux2
  4. Type "help", "copyright", "credits" or "license" for more information.
  5. 17/08/13 10:14:46 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  6. 17/08/13 10:14:54 WARN ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
  7. Welcome to
  8. ____ __
  9. / __/__ ___ _____/ /__
  10. _\ \/ _ \/ _ `/ __/ '_/
  11. /__ / .__/\_,_/_/ /_/\_\ version 2.2.0
  12. /_/
  13.  
  14. Using Python version 2.7.13 (default, Jun 26 2017 10:20:05)
  15. SparkSession available as 'spark'.
  16. >>> from pyspark.mllib.linalg.distributed import RowMatrix
  17. >>> rdd = sc.parallelize([[1.0,2.0,1.0],[1.0,5.0,1.0],[1.0,2.0,1.0],[4.0,2.0,4.0]])
  18. >>> mat = RowMatrix(rdd)
  19. >>> sim = mat.columnSimilarities(0.1)
  20. >>> sim.entries.collect()
  21. 17/08/13 10:15:19 ERROR Schema: Failed initialising database.
  22. Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  23. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  24. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  25. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  26. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  27. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  28. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  29. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  30. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  31. at java.security.AccessController.doPrivileged(Native Method)
  32. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  33. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  34. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  35. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  36. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  37. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  38. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  39. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  40. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  41. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  42. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  43. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  44. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  45. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  46. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  47. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  48. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  49. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  50. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  51. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  52. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  53. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  54. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  55. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  56. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  57. at java.lang.reflect.Method.invoke(Method.java:498)
  58. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  59. at java.security.AccessController.doPrivileged(Native Method)
  60. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  61. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  62. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  63. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  64. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  65. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  66. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  67. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  68. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  69. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  70. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  71. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  72. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  73. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  74. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  75. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  76. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  77. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  78. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  79. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  80. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  81. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  82. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  83. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  84. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  85. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  86. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  87. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  88. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  89. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  90. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  91. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  92. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  93. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  94. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  95. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  96. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  97. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  98. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  99. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  100. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  101. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  102. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  103. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  104. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  105. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  106. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  107. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  108. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  109. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  110. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  111. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  112. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  113. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  114. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  115. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  116. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  117. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  118. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  119. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  120. at scala.Option.getOrElse(Option.scala:121)
  121. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  122. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  123. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  124. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  125. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  126. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  127. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  128. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  129. at java.lang.reflect.Method.invoke(Method.java:498)
  130. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  131. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  132. at py4j.Gateway.invoke(Gateway.java:280)
  133. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  134. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  135. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  136. at java.lang.Thread.run(Thread.java:748)
  137. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  138. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  139. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  140. ... 113 more
  141. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  142. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  143. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  144. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  145. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  146. at java.security.AccessController.doPrivileged(Native Method)
  147. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  148. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  149. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  150. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  151. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  152. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  153. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  154. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  155. at java.security.AccessController.doPrivileged(Native Method)
  156. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  157. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  158. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  159. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  160. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  161. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  162. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  163. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  164. at java.security.AccessController.doPrivileged(Native Method)
  165. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  166. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  167. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  168. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  169. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  170. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  171. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  172. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  173. at java.security.AccessController.doPrivileged(Native Method)
  174. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  175. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  176. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  177. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  178. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  179. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  180. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  181. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  182. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  183. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  184. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  185. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  186. at java.security.AccessController.doPrivileged(Native Method)
  187. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  188. ... 110 more
  189. ------
  190.  
  191. org.datanucleus.exceptions.NucleusDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  192. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  193. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  194. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  195. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  196. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  197. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  198. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  199. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  200. at java.security.AccessController.doPrivileged(Native Method)
  201. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  202. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  203. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  204. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  205. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  206. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  207. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  208. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  209. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  210. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  211. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  212. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  213. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  214. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  215. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  216. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  217. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  218. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  219. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  220. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  221. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  222. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  223. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  224. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  225. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  226. at java.lang.reflect.Method.invoke(Method.java:498)
  227. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  228. at java.security.AccessController.doPrivileged(Native Method)
  229. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  230. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  231. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  232. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  233. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  234. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  235. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  236. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  237. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  238. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  239. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  240. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  241. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  242. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  243. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  244. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  245. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  246. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  247. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  248. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  249. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  250. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  251. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  252. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  253. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  254. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  255. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  256. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  257. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  258. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  259. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  260. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  261. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  262. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  263. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  264. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  265. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  266. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  267. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  268. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  269. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  270. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  271. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  272. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  273. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  274. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  275. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  276. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  277. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  278. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  279. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  280. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  281. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  282. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  283. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  284. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  285. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  286. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  287. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  288. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  289. at scala.Option.getOrElse(Option.scala:121)
  290. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  291. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  292. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  293. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  294. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  295. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  296. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  297. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  298. at java.lang.reflect.Method.invoke(Method.java:498)
  299. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  300. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  301. at py4j.Gateway.invoke(Gateway.java:280)
  302. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  303. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  304. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  305. at java.lang.Thread.run(Thread.java:748)
  306. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  307. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  308. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  309. ... 113 more
  310. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  311. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  312. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  313. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  314. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  315. at java.security.AccessController.doPrivileged(Native Method)
  316. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  317. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  318. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  319. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  320. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  321. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  322. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  323. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  324. at java.security.AccessController.doPrivileged(Native Method)
  325. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  326. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  327. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  328. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  329. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  330. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  331. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  332. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  333. at java.security.AccessController.doPrivileged(Native Method)
  334. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  335. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  336. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  337. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  338. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  339. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  340. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  341. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  342. at java.security.AccessController.doPrivileged(Native Method)
  343. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  344. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  345. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  346. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  347. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  348. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  349. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  350. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  351. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  352. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  353. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  354. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  355. at java.security.AccessController.doPrivileged(Native Method)
  356. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  357. ... 110 more
  358. ------
  359.  
  360. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:516)
  361. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  362. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  363. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  364. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  365. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  366. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  367. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  368. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  369. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  370. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  371. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  372. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  373. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  374. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  375. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  376. at java.lang.reflect.Method.invoke(Method.java:498)
  377. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  378. at java.security.AccessController.doPrivileged(Native Method)
  379. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  380. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  381. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  382. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  383. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  384. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  385. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  386. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  387. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  388. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  389. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  390. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  391. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  392. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  393. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  394. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  395. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  396. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  397. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  398. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  399. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  400. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  401. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  402. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  403. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  404. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  405. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  406. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  407. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  408. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  409. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  410. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  411. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  412. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  413. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  414. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  415. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  416. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  417. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  418. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  419. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  420. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  421. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  422. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  423. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  424. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  425. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  426. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  427. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  428. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  429. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  430. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  431. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  432. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  433. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  434. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  435. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  436. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  437. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  438. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  439. at scala.Option.getOrElse(Option.scala:121)
  440. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  441. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  442. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  443. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  444. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  445. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  446. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  447. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  448. at java.lang.reflect.Method.invoke(Method.java:498)
  449. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  450. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  451. at py4j.Gateway.invoke(Gateway.java:280)
  452. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  453. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  454. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  455. at java.lang.Thread.run(Thread.java:748)
  456. Caused by: java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  457. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  458. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  459. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  460. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  461. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  462. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  463. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  464. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  465. at java.security.AccessController.doPrivileged(Native Method)
  466. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  467. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  468. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  469. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  470. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  471. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  472. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  473. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  474. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  475. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  476. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  477. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  478. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  479. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  480. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  481. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  482. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  483. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  484. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  485. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  486. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  487. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  488. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  489. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  490. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  491. at java.lang.reflect.Method.invoke(Method.java:498)
  492. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  493. at java.security.AccessController.doPrivileged(Native Method)
  494. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  495. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  496. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  497. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  498. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  499. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  500. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  501. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  502. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  503. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  504. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  505. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  506. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  507. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  508. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  509. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  510. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  511. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  512. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  513. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  514. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  515. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  516. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  517. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  518. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  519. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  520. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  521. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  522. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  523. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  524. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  525. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  526. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  527. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  528. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  529. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  530. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  531. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  532. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  533. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  534. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  535. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  536. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  537. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  538. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  539. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  540. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  541. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  542. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  543. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  544. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  545. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  546. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  547. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  548. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  549. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  550. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  551. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  552. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  553. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  554. at scala.Option.getOrElse(Option.scala:121)
  555. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  556. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  557. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  558. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  559. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  560. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  561. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  562. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  563. at java.lang.reflect.Method.invoke(Method.java:498)
  564. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  565. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  566. at py4j.Gateway.invoke(Gateway.java:280)
  567. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  568. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  569. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  570. at java.lang.Thread.run(Thread.java:748)
  571. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  572. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  573. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  574. ... 113 more
  575. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  576. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  577. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  578. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  579. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  580. at java.security.AccessController.doPrivileged(Native Method)
  581. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  582. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  583. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  584. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  585. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  586. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  587. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  588. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  589. at java.security.AccessController.doPrivileged(Native Method)
  590. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  591. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  592. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  593. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  594. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  595. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  596. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  597. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  598. at java.security.AccessController.doPrivileged(Native Method)
  599. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  600. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  601. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  602. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  603. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  604. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  605. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  606. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  607. at java.security.AccessController.doPrivileged(Native Method)
  608. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  609. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  610. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  611. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  612. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  613. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  614. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  615. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  616. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  617. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  618. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  619. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  620. at java.security.AccessController.doPrivileged(Native Method)
  621. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  622. ... 110 more
  623. ------
  624.  
  625. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  626. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  627. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  628. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  629. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  630. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  631. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  632. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  633. ... 95 more
  634. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  635. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  636. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  637. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  638. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  639. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  640. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  641. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  642. at java.security.AccessController.doPrivileged(Native Method)
  643. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  644. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  645. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  646. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  647. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  648. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  649. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  650. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  651. ... 97 more
  652. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  653. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  654. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  655. ... 113 more
  656. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  657. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  658. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  659. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  660. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  661. at java.security.AccessController.doPrivileged(Native Method)
  662. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  663. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  664. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  665. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  666. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  667. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  668. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  669. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  670. at java.security.AccessController.doPrivileged(Native Method)
  671. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  672. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  673. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  674. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  675. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  676. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  677. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  678. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  679. at java.security.AccessController.doPrivileged(Native Method)
  680. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  681. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  682. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  683. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  684. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  685. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  686. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  687. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  688. at java.security.AccessController.doPrivileged(Native Method)
  689. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  690. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  691. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  692. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  693. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  694. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  695. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  696. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  697. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  698. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  699. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  700. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  701. at java.security.AccessController.doPrivileged(Native Method)
  702. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  703. ... 110 more
  704. Nested Throwables StackTrace:
  705. java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  706. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  707. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  708. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  709. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  710. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  711. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  712. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  713. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  714. at java.security.AccessController.doPrivileged(Native Method)
  715. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  716. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  717. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  718. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  719. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  720. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  721. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  722. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  723. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  724. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  725. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  726. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  727. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  728. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  729. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  730. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  731. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  732. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  733. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  734. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  735. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  736. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  737. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  738. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  739. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  740. at java.lang.reflect.Method.invoke(Method.java:498)
  741. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  742. at java.security.AccessController.doPrivileged(Native Method)
  743. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  744. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  745. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  746. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  747. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  748. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  749. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  750. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  751. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  752. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  753. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  754. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  755. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  756. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  757. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  758. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  759. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  760. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  761. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  762. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  763. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  764. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  765. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  766. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  767. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  768. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  769. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  770. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  771. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  772. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  773. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  774. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  775. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  776. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  777. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  778. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  779. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  780. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  781. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  782. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  783. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  784. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  785. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  786. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  787. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  788. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  789. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  790. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  791. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  792. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  793. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  794. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  795. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  796. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  797. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  798. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  799. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  800. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  801. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  802. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  803. at scala.Option.getOrElse(Option.scala:121)
  804. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  805. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  806. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  807. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  808. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  809. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  810. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  811. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  812. at java.lang.reflect.Method.invoke(Method.java:498)
  813. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  814. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  815. at py4j.Gateway.invoke(Gateway.java:280)
  816. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  817. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  818. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  819. at java.lang.Thread.run(Thread.java:748)
  820. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  821. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  822. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  823. ... 113 more
  824. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  825. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  826. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  827. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  828. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  829. at java.security.AccessController.doPrivileged(Native Method)
  830. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  831. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  832. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  833. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  834. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  835. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  836. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  837. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  838. at java.security.AccessController.doPrivileged(Native Method)
  839. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  840. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  841. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  842. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  843. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  844. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  845. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  846. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  847. at java.security.AccessController.doPrivileged(Native Method)
  848. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  849. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  850. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  851. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  852. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  853. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  854. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  855. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  856. at java.security.AccessController.doPrivileged(Native Method)
  857. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  858. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  859. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  860. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  861. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  862. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  863. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  864. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  865. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  866. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  867. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  868. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  869. at java.security.AccessController.doPrivileged(Native Method)
  870. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  871. ... 110 more
  872. ------
  873.  
  874. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  875. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  876. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  877. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  878. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  879. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  880. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  881. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  882. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  883. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  884. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  885. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  886. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  887. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  888. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  889. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  890. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  891. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  892. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  893. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  894. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  895. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  896. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  897. at java.lang.reflect.Method.invoke(Method.java:498)
  898. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  899. at java.security.AccessController.doPrivileged(Native Method)
  900. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  901. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  902. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  903. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  904. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  905. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  906. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  907. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  908. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  909. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  910. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  911. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  912. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  913. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  914. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  915. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  916. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  917. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  918. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  919. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  920. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  921. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  922. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  923. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  924. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  925. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  926. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  927. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  928. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  929. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  930. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  931. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  932. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  933. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  934. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  935. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  936. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  937. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  938. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  939. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  940. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  941. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  942. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  943. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  944. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  945. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  946. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  947. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  948. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  949. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  950. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  951. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  952. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  953. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  954. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  955. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  956. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  957. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  958. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  959. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  960. at scala.Option.getOrElse(Option.scala:121)
  961. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  962. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  963. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  964. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  965. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  966. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  967. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  968. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  969. at java.lang.reflect.Method.invoke(Method.java:498)
  970. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  971. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  972. at py4j.Gateway.invoke(Gateway.java:280)
  973. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  974. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  975. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  976. at java.lang.Thread.run(Thread.java:748)
  977. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  978. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  979. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  980. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  981. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  982. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  983. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  984. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  985. at java.security.AccessController.doPrivileged(Native Method)
  986. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  987. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  988. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  989. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  990. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  991. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  992. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  993. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  994. ... 97 more
  995. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  996. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  997. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  998. ... 113 more
  999. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  1000. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1001. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1002. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  1003. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  1004. at java.security.AccessController.doPrivileged(Native Method)
  1005. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  1006. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  1007. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1008. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1009. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1010. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1011. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1012. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  1013. at java.security.AccessController.doPrivileged(Native Method)
  1014. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  1015. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  1016. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1017. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1018. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1019. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1020. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1021. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  1022. at java.security.AccessController.doPrivileged(Native Method)
  1023. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  1024. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  1025. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1026. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1027. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1028. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1029. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1030. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  1031. at java.security.AccessController.doPrivileged(Native Method)
  1032. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  1033. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  1034. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  1035. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1036. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1037. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  1038. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  1039. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  1040. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  1041. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  1042. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1043. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1044. at java.security.AccessController.doPrivileged(Native Method)
  1045. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  1046. ... 110 more
  1047. 17/08/13 10:15:19 WARN HiveMetaStore: Retrying creating default database after error: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  1048. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  1049. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1050. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1051. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  1052. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  1053. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  1054. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1055. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1056. at java.security.AccessController.doPrivileged(Native Method)
  1057. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  1058. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1059. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1060. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  1061. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  1062. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  1063. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  1064. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  1065. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  1066. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  1067. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  1068. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1069. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1070. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1071. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1072. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  1073. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  1074. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  1075. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  1076. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  1077. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  1078. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  1079. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1080. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1081. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1082. at java.lang.reflect.Method.invoke(Method.java:498)
  1083. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  1084. at java.security.AccessController.doPrivileged(Native Method)
  1085. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  1086. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  1087. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  1088. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  1089. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  1090. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  1091. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  1092. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  1093. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  1094. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  1095. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  1096. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  1097. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  1098. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  1099. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  1100. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  1101. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  1102. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  1103. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  1104. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  1105. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  1106. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1107. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1108. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1109. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1110. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  1111. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  1112. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  1113. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  1114. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  1115. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  1116. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  1117. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  1118. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  1119. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  1120. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  1121. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1122. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1123. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1124. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1125. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  1126. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  1127. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  1128. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  1129. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  1130. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  1131. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  1132. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  1133. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  1134. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  1135. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  1136. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  1137. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  1138. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  1139. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  1140. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  1141. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  1142. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  1143. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  1144. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  1145. at scala.Option.getOrElse(Option.scala:121)
  1146. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  1147. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  1148. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  1149. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  1150. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  1151. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1152. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1153. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1154. at java.lang.reflect.Method.invoke(Method.java:498)
  1155. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  1156. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  1157. at py4j.Gateway.invoke(Gateway.java:280)
  1158. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  1159. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  1160. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  1161. at java.lang.Thread.run(Thread.java:748)
  1162. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  1163. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1164. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  1165. ... 113 more
  1166. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  1167. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1168. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1169. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  1170. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  1171. at java.security.AccessController.doPrivileged(Native Method)
  1172. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  1173. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  1174. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1175. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1176. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1177. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1178. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1179. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  1180. at java.security.AccessController.doPrivileged(Native Method)
  1181. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  1182. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  1183. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1184. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1185. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1186. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1187. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1188. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  1189. at java.security.AccessController.doPrivileged(Native Method)
  1190. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  1191. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  1192. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1193. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1194. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1195. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1196. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1197. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  1198. at java.security.AccessController.doPrivileged(Native Method)
  1199. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  1200. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  1201. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  1202. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1203. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1204. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  1205. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  1206. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  1207. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  1208. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  1209. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1210. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1211. at java.security.AccessController.doPrivileged(Native Method)
  1212. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  1213. ... 110 more
  1214. ------
  1215.  
  1216. javax.jdo.JDOFatalDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  1217. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  1218. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1219. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1220. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  1221. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  1222. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  1223. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1224. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1225. at java.security.AccessController.doPrivileged(Native Method)
  1226. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  1227. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1228. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1229. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  1230. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  1231. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  1232. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  1233. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  1234. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  1235. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  1236. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  1237. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1238. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1239. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1240. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1241. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  1242. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  1243. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  1244. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  1245. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  1246. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  1247. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  1248. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1249. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1250. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1251. at java.lang.reflect.Method.invoke(Method.java:498)
  1252. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  1253. at java.security.AccessController.doPrivileged(Native Method)
  1254. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  1255. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  1256. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  1257. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  1258. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  1259. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  1260. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  1261. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  1262. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  1263. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  1264. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  1265. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  1266. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  1267. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  1268. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  1269. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  1270. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  1271. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  1272. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  1273. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  1274. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  1275. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1276. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1277. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1278. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1279. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  1280. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  1281. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  1282. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  1283. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  1284. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  1285. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  1286. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  1287. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  1288. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  1289. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  1290. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1291. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1292. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1293. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1294. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  1295. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  1296. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  1297. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  1298. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  1299. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  1300. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  1301. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  1302. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  1303. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  1304. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  1305. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  1306. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  1307. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  1308. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  1309. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  1310. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  1311. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  1312. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  1313. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  1314. at scala.Option.getOrElse(Option.scala:121)
  1315. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  1316. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  1317. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  1318. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  1319. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  1320. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1321. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1322. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1323. at java.lang.reflect.Method.invoke(Method.java:498)
  1324. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  1325. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  1326. at py4j.Gateway.invoke(Gateway.java:280)
  1327. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  1328. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  1329. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  1330. at java.lang.Thread.run(Thread.java:748)
  1331. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  1332. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1333. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  1334. ... 113 more
  1335. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  1336. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1337. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1338. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  1339. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  1340. at java.security.AccessController.doPrivileged(Native Method)
  1341. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  1342. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  1343. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1344. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1345. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1346. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1347. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1348. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  1349. at java.security.AccessController.doPrivileged(Native Method)
  1350. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  1351. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  1352. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1353. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1354. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1355. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1356. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1357. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  1358. at java.security.AccessController.doPrivileged(Native Method)
  1359. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  1360. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  1361. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1362. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1363. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1364. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1365. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1366. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  1367. at java.security.AccessController.doPrivileged(Native Method)
  1368. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  1369. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  1370. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  1371. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1372. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1373. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  1374. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  1375. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  1376. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  1377. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  1378. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1379. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1380. at java.security.AccessController.doPrivileged(Native Method)
  1381. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  1382. ... 110 more
  1383. ------
  1384.  
  1385. at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:436)
  1386. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:788)
  1387. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  1388. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  1389. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1390. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1391. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1392. at java.lang.reflect.Method.invoke(Method.java:498)
  1393. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  1394. at java.security.AccessController.doPrivileged(Native Method)
  1395. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  1396. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  1397. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  1398. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  1399. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  1400. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  1401. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  1402. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  1403. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  1404. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  1405. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  1406. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  1407. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  1408. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  1409. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  1410. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  1411. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  1412. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  1413. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  1414. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  1415. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  1416. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1417. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1418. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1419. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1420. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  1421. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  1422. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  1423. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  1424. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  1425. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  1426. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  1427. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  1428. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  1429. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  1430. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  1431. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1432. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1433. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1434. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1435. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  1436. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  1437. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  1438. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  1439. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  1440. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  1441. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  1442. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  1443. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  1444. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  1445. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  1446. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  1447. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  1448. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  1449. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  1450. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  1451. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  1452. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  1453. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  1454. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  1455. at scala.Option.getOrElse(Option.scala:121)
  1456. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  1457. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  1458. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  1459. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  1460. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  1461. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1462. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1463. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1464. at java.lang.reflect.Method.invoke(Method.java:498)
  1465. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  1466. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  1467. at py4j.Gateway.invoke(Gateway.java:280)
  1468. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  1469. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  1470. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  1471. at java.lang.Thread.run(Thread.java:748)
  1472. NestedThrowablesStackTrace:
  1473. java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  1474. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  1475. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1476. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1477. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  1478. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  1479. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  1480. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1481. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1482. at java.security.AccessController.doPrivileged(Native Method)
  1483. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  1484. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1485. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1486. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  1487. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  1488. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  1489. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  1490. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  1491. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  1492. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  1493. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  1494. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1495. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1496. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1497. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1498. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  1499. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  1500. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  1501. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  1502. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  1503. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  1504. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  1505. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1506. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1507. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1508. at java.lang.reflect.Method.invoke(Method.java:498)
  1509. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  1510. at java.security.AccessController.doPrivileged(Native Method)
  1511. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  1512. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  1513. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  1514. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  1515. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  1516. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  1517. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  1518. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  1519. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  1520. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  1521. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  1522. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  1523. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  1524. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  1525. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  1526. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  1527. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  1528. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  1529. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  1530. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  1531. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  1532. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1533. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1534. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1535. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1536. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  1537. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  1538. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  1539. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  1540. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  1541. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  1542. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  1543. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  1544. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  1545. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  1546. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  1547. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1548. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1549. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1550. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1551. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  1552. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  1553. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  1554. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  1555. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  1556. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  1557. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  1558. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  1559. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  1560. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  1561. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  1562. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  1563. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  1564. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  1565. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  1566. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  1567. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  1568. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  1569. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  1570. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  1571. at scala.Option.getOrElse(Option.scala:121)
  1572. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  1573. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  1574. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  1575. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  1576. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  1577. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1578. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1579. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1580. at java.lang.reflect.Method.invoke(Method.java:498)
  1581. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  1582. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  1583. at py4j.Gateway.invoke(Gateway.java:280)
  1584. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  1585. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  1586. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  1587. at java.lang.Thread.run(Thread.java:748)
  1588. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  1589. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1590. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  1591. ... 113 more
  1592. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  1593. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1594. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1595. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  1596. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  1597. at java.security.AccessController.doPrivileged(Native Method)
  1598. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  1599. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  1600. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1601. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1602. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1603. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1604. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1605. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  1606. at java.security.AccessController.doPrivileged(Native Method)
  1607. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  1608. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  1609. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1610. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1611. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1612. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1613. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1614. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  1615. at java.security.AccessController.doPrivileged(Native Method)
  1616. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  1617. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  1618. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1619. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1620. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1621. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1622. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1623. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  1624. at java.security.AccessController.doPrivileged(Native Method)
  1625. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  1626. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  1627. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  1628. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1629. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1630. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  1631. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  1632. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  1633. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  1634. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  1635. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1636. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1637. at java.security.AccessController.doPrivileged(Native Method)
  1638. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  1639. ... 110 more
  1640. ------
  1641.  
  1642. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1643. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1644. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1645. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1646. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  1647. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  1648. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  1649. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  1650. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  1651. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1652. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1653. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1654. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1655. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  1656. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  1657. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  1658. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  1659. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  1660. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  1661. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  1662. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1663. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1664. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1665. at java.lang.reflect.Method.invoke(Method.java:498)
  1666. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  1667. at java.security.AccessController.doPrivileged(Native Method)
  1668. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  1669. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  1670. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  1671. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  1672. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  1673. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  1674. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  1675. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  1676. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  1677. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  1678. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  1679. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  1680. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  1681. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  1682. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  1683. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  1684. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  1685. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  1686. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  1687. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  1688. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  1689. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1690. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1691. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1692. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1693. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  1694. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  1695. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  1696. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  1697. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  1698. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  1699. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  1700. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  1701. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  1702. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  1703. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  1704. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1705. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1706. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1707. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1708. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  1709. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  1710. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  1711. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  1712. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  1713. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  1714. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  1715. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  1716. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  1717. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  1718. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  1719. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  1720. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  1721. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  1722. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  1723. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  1724. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  1725. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  1726. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  1727. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  1728. at scala.Option.getOrElse(Option.scala:121)
  1729. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  1730. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  1731. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  1732. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  1733. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  1734. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1735. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1736. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1737. at java.lang.reflect.Method.invoke(Method.java:498)
  1738. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  1739. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  1740. at py4j.Gateway.invoke(Gateway.java:280)
  1741. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  1742. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  1743. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  1744. at java.lang.Thread.run(Thread.java:748)
  1745. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  1746. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1747. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1748. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  1749. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  1750. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  1751. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1752. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1753. at java.security.AccessController.doPrivileged(Native Method)
  1754. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  1755. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1756. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1757. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  1758. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  1759. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  1760. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  1761. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  1762. ... 97 more
  1763. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  1764. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1765. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  1766. ... 113 more
  1767. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  1768. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1769. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1770. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  1771. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  1772. at java.security.AccessController.doPrivileged(Native Method)
  1773. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  1774. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  1775. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1776. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1777. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1778. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1779. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1780. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  1781. at java.security.AccessController.doPrivileged(Native Method)
  1782. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  1783. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  1784. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1785. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1786. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1787. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1788. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1789. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  1790. at java.security.AccessController.doPrivileged(Native Method)
  1791. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  1792. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  1793. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1794. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1795. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1796. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1797. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1798. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  1799. at java.security.AccessController.doPrivileged(Native Method)
  1800. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  1801. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  1802. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  1803. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1804. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1805. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  1806. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  1807. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  1808. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  1809. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  1810. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1811. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1812. at java.security.AccessController.doPrivileged(Native Method)
  1813. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  1814. ... 110 more
  1815. 17/08/13 10:15:19 ERROR Schema: Failed initialising database.
  1816. Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  1817. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  1818. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1819. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1820. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  1821. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  1822. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  1823. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1824. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1825. at java.security.AccessController.doPrivileged(Native Method)
  1826. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  1827. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1828. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1829. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  1830. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  1831. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  1832. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  1833. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  1834. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  1835. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  1836. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  1837. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1838. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1839. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1840. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1841. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  1842. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  1843. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  1844. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  1845. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  1846. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  1847. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  1848. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1849. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1850. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1851. at java.lang.reflect.Method.invoke(Method.java:498)
  1852. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  1853. at java.security.AccessController.doPrivileged(Native Method)
  1854. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  1855. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  1856. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  1857. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  1858. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  1859. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  1860. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  1861. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  1862. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  1863. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  1864. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  1865. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  1866. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  1867. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  1868. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  1869. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  1870. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  1871. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  1872. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  1873. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  1874. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  1875. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1876. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1877. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1878. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1879. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  1880. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  1881. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  1882. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  1883. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  1884. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  1885. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  1886. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  1887. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  1888. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  1889. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  1890. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  1891. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  1892. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  1893. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  1894. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  1895. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  1896. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  1897. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  1898. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  1899. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  1900. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  1901. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  1902. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  1903. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  1904. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  1905. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  1906. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  1907. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  1908. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  1909. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  1910. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  1911. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  1912. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  1913. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  1914. at scala.Option.getOrElse(Option.scala:121)
  1915. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  1916. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  1917. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  1918. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  1919. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  1920. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  1921. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  1922. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  1923. at java.lang.reflect.Method.invoke(Method.java:498)
  1924. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  1925. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  1926. at py4j.Gateway.invoke(Gateway.java:280)
  1927. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  1928. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  1929. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  1930. at java.lang.Thread.run(Thread.java:748)
  1931. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  1932. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1933. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  1934. ... 113 more
  1935. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  1936. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1937. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  1938. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  1939. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  1940. at java.security.AccessController.doPrivileged(Native Method)
  1941. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  1942. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  1943. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1944. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1945. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1946. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1947. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1948. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  1949. at java.security.AccessController.doPrivileged(Native Method)
  1950. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  1951. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  1952. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1953. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1954. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1955. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1956. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1957. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  1958. at java.security.AccessController.doPrivileged(Native Method)
  1959. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  1960. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  1961. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1962. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1963. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  1964. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  1965. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  1966. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  1967. at java.security.AccessController.doPrivileged(Native Method)
  1968. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  1969. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  1970. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  1971. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  1972. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  1973. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  1974. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  1975. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  1976. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  1977. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  1978. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1979. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  1980. at java.security.AccessController.doPrivileged(Native Method)
  1981. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  1982. ... 110 more
  1983. ------
  1984.  
  1985. org.datanucleus.exceptions.NucleusDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  1986. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  1987. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1988. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  1989. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  1990. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  1991. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  1992. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1993. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  1994. at java.security.AccessController.doPrivileged(Native Method)
  1995. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  1996. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1997. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  1998. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  1999. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  2000. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  2001. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  2002. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  2003. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  2004. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  2005. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  2006. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2007. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2008. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2009. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2010. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  2011. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  2012. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  2013. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  2014. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  2015. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  2016. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  2017. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2018. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2019. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2020. at java.lang.reflect.Method.invoke(Method.java:498)
  2021. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  2022. at java.security.AccessController.doPrivileged(Native Method)
  2023. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  2024. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  2025. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  2026. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  2027. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  2028. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  2029. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  2030. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  2031. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  2032. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  2033. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  2034. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  2035. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  2036. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  2037. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  2038. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  2039. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  2040. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  2041. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  2042. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  2043. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  2044. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2045. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2046. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2047. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2048. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  2049. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  2050. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  2051. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  2052. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  2053. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  2054. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  2055. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  2056. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  2057. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  2058. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  2059. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2060. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2061. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2062. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2063. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  2064. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  2065. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  2066. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  2067. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  2068. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  2069. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  2070. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  2071. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  2072. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  2073. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  2074. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  2075. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  2076. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  2077. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  2078. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  2079. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  2080. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  2081. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  2082. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  2083. at scala.Option.getOrElse(Option.scala:121)
  2084. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  2085. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  2086. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  2087. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  2088. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  2089. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2090. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2091. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2092. at java.lang.reflect.Method.invoke(Method.java:498)
  2093. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  2094. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  2095. at py4j.Gateway.invoke(Gateway.java:280)
  2096. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  2097. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  2098. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  2099. at java.lang.Thread.run(Thread.java:748)
  2100. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  2101. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2102. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  2103. ... 113 more
  2104. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  2105. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2106. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2107. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  2108. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  2109. at java.security.AccessController.doPrivileged(Native Method)
  2110. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  2111. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  2112. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2113. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2114. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2115. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2116. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2117. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  2118. at java.security.AccessController.doPrivileged(Native Method)
  2119. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  2120. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  2121. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2122. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2123. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2124. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2125. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2126. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  2127. at java.security.AccessController.doPrivileged(Native Method)
  2128. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  2129. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  2130. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2131. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2132. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2133. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2134. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2135. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  2136. at java.security.AccessController.doPrivileged(Native Method)
  2137. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  2138. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  2139. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  2140. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2141. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2142. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  2143. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  2144. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  2145. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  2146. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  2147. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2148. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2149. at java.security.AccessController.doPrivileged(Native Method)
  2150. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  2151. ... 110 more
  2152. ------
  2153.  
  2154. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:516)
  2155. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  2156. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2157. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2158. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2159. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2160. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  2161. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  2162. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  2163. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  2164. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  2165. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  2166. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  2167. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2168. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2169. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2170. at java.lang.reflect.Method.invoke(Method.java:498)
  2171. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  2172. at java.security.AccessController.doPrivileged(Native Method)
  2173. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  2174. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  2175. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  2176. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  2177. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  2178. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  2179. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  2180. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  2181. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  2182. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  2183. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  2184. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  2185. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  2186. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  2187. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  2188. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  2189. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  2190. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  2191. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  2192. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  2193. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  2194. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2195. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2196. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2197. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2198. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  2199. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  2200. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  2201. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  2202. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  2203. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  2204. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  2205. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  2206. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  2207. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  2208. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  2209. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2210. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2211. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2212. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2213. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  2214. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  2215. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  2216. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  2217. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  2218. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  2219. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  2220. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  2221. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  2222. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  2223. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  2224. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  2225. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  2226. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  2227. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  2228. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  2229. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  2230. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  2231. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  2232. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  2233. at scala.Option.getOrElse(Option.scala:121)
  2234. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  2235. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  2236. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  2237. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  2238. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  2239. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2240. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2241. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2242. at java.lang.reflect.Method.invoke(Method.java:498)
  2243. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  2244. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  2245. at py4j.Gateway.invoke(Gateway.java:280)
  2246. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  2247. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  2248. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  2249. at java.lang.Thread.run(Thread.java:748)
  2250. Caused by: java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  2251. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  2252. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2253. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2254. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  2255. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  2256. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  2257. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2258. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2259. at java.security.AccessController.doPrivileged(Native Method)
  2260. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  2261. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2262. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2263. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  2264. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  2265. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  2266. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  2267. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  2268. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  2269. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  2270. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  2271. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2272. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2273. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2274. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2275. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  2276. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  2277. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  2278. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  2279. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  2280. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  2281. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  2282. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2283. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2284. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2285. at java.lang.reflect.Method.invoke(Method.java:498)
  2286. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  2287. at java.security.AccessController.doPrivileged(Native Method)
  2288. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  2289. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  2290. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  2291. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  2292. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  2293. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  2294. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  2295. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  2296. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  2297. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  2298. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  2299. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  2300. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  2301. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  2302. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  2303. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  2304. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  2305. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  2306. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  2307. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  2308. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  2309. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2310. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2311. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2312. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2313. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  2314. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  2315. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  2316. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  2317. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  2318. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  2319. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  2320. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  2321. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  2322. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  2323. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  2324. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2325. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2326. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2327. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2328. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  2329. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  2330. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  2331. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  2332. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  2333. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  2334. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  2335. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  2336. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  2337. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  2338. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  2339. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  2340. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  2341. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  2342. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  2343. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  2344. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  2345. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  2346. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  2347. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  2348. at scala.Option.getOrElse(Option.scala:121)
  2349. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  2350. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  2351. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  2352. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  2353. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  2354. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2355. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2356. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2357. at java.lang.reflect.Method.invoke(Method.java:498)
  2358. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  2359. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  2360. at py4j.Gateway.invoke(Gateway.java:280)
  2361. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  2362. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  2363. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  2364. at java.lang.Thread.run(Thread.java:748)
  2365. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  2366. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2367. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  2368. ... 113 more
  2369. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  2370. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2371. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2372. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  2373. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  2374. at java.security.AccessController.doPrivileged(Native Method)
  2375. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  2376. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  2377. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2378. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2379. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2380. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2381. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2382. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  2383. at java.security.AccessController.doPrivileged(Native Method)
  2384. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  2385. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  2386. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2387. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2388. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2389. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2390. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2391. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  2392. at java.security.AccessController.doPrivileged(Native Method)
  2393. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  2394. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  2395. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2396. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2397. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2398. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2399. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2400. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  2401. at java.security.AccessController.doPrivileged(Native Method)
  2402. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  2403. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  2404. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  2405. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2406. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2407. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  2408. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  2409. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  2410. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  2411. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  2412. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2413. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2414. at java.security.AccessController.doPrivileged(Native Method)
  2415. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  2416. ... 110 more
  2417. ------
  2418.  
  2419. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2420. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2421. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2422. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2423. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  2424. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  2425. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  2426. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  2427. ... 95 more
  2428. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  2429. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2430. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2431. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  2432. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  2433. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  2434. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2435. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2436. at java.security.AccessController.doPrivileged(Native Method)
  2437. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  2438. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2439. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2440. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  2441. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  2442. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  2443. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  2444. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  2445. ... 97 more
  2446. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  2447. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2448. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  2449. ... 113 more
  2450. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  2451. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2452. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2453. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  2454. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  2455. at java.security.AccessController.doPrivileged(Native Method)
  2456. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  2457. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  2458. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2459. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2460. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2461. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2462. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2463. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  2464. at java.security.AccessController.doPrivileged(Native Method)
  2465. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  2466. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  2467. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2468. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2469. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2470. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2471. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2472. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  2473. at java.security.AccessController.doPrivileged(Native Method)
  2474. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  2475. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  2476. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2477. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2478. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2479. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2480. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2481. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  2482. at java.security.AccessController.doPrivileged(Native Method)
  2483. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  2484. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  2485. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  2486. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2487. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2488. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  2489. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  2490. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  2491. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  2492. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  2493. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2494. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2495. at java.security.AccessController.doPrivileged(Native Method)
  2496. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  2497. ... 110 more
  2498. Nested Throwables StackTrace:
  2499. java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  2500. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  2501. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2502. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2503. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  2504. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  2505. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  2506. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2507. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2508. at java.security.AccessController.doPrivileged(Native Method)
  2509. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  2510. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2511. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2512. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  2513. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  2514. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  2515. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  2516. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  2517. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  2518. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  2519. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  2520. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2521. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2522. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2523. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2524. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  2525. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  2526. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  2527. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  2528. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  2529. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  2530. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  2531. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2532. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2533. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2534. at java.lang.reflect.Method.invoke(Method.java:498)
  2535. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  2536. at java.security.AccessController.doPrivileged(Native Method)
  2537. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  2538. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  2539. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  2540. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  2541. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  2542. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  2543. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  2544. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  2545. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  2546. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  2547. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  2548. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  2549. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  2550. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  2551. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  2552. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  2553. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  2554. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  2555. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  2556. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  2557. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  2558. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2559. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2560. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2561. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2562. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  2563. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  2564. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  2565. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  2566. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  2567. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  2568. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  2569. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  2570. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  2571. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  2572. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  2573. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2574. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2575. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2576. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2577. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  2578. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  2579. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  2580. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  2581. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  2582. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  2583. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  2584. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  2585. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  2586. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  2587. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  2588. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  2589. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  2590. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  2591. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  2592. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  2593. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  2594. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  2595. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  2596. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  2597. at scala.Option.getOrElse(Option.scala:121)
  2598. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  2599. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  2600. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  2601. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  2602. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  2603. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2604. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2605. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2606. at java.lang.reflect.Method.invoke(Method.java:498)
  2607. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  2608. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  2609. at py4j.Gateway.invoke(Gateway.java:280)
  2610. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  2611. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  2612. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  2613. at java.lang.Thread.run(Thread.java:748)
  2614. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  2615. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2616. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  2617. ... 113 more
  2618. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  2619. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2620. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2621. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  2622. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  2623. at java.security.AccessController.doPrivileged(Native Method)
  2624. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  2625. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  2626. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2627. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2628. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2629. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2630. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2631. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  2632. at java.security.AccessController.doPrivileged(Native Method)
  2633. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  2634. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  2635. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2636. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2637. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2638. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2639. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2640. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  2641. at java.security.AccessController.doPrivileged(Native Method)
  2642. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  2643. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  2644. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2645. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2646. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2647. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2648. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2649. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  2650. at java.security.AccessController.doPrivileged(Native Method)
  2651. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  2652. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  2653. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  2654. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2655. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2656. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  2657. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  2658. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  2659. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  2660. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  2661. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2662. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2663. at java.security.AccessController.doPrivileged(Native Method)
  2664. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  2665. ... 110 more
  2666. ------
  2667.  
  2668. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2669. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2670. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2671. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2672. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  2673. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  2674. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  2675. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  2676. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  2677. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2678. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2679. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2680. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2681. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  2682. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  2683. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  2684. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  2685. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  2686. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  2687. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  2688. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2689. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2690. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2691. at java.lang.reflect.Method.invoke(Method.java:498)
  2692. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  2693. at java.security.AccessController.doPrivileged(Native Method)
  2694. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  2695. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  2696. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  2697. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  2698. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  2699. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  2700. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  2701. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  2702. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  2703. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  2704. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  2705. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  2706. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  2707. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  2708. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  2709. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  2710. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  2711. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  2712. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  2713. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  2714. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  2715. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2716. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2717. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2718. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2719. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  2720. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  2721. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  2722. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  2723. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  2724. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  2725. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  2726. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  2727. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  2728. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  2729. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  2730. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2731. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2732. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2733. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2734. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  2735. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  2736. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  2737. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  2738. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  2739. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  2740. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  2741. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  2742. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  2743. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  2744. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  2745. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  2746. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  2747. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  2748. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  2749. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  2750. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  2751. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  2752. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  2753. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  2754. at scala.Option.getOrElse(Option.scala:121)
  2755. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  2756. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  2757. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  2758. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  2759. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  2760. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2761. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2762. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2763. at java.lang.reflect.Method.invoke(Method.java:498)
  2764. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  2765. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  2766. at py4j.Gateway.invoke(Gateway.java:280)
  2767. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  2768. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  2769. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  2770. at java.lang.Thread.run(Thread.java:748)
  2771. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  2772. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2773. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2774. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  2775. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  2776. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  2777. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2778. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2779. at java.security.AccessController.doPrivileged(Native Method)
  2780. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  2781. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2782. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2783. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  2784. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  2785. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  2786. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  2787. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  2788. ... 97 more
  2789. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  2790. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2791. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  2792. ... 113 more
  2793. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  2794. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2795. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  2796. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  2797. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  2798. at java.security.AccessController.doPrivileged(Native Method)
  2799. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  2800. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  2801. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2802. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2803. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2804. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2805. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2806. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  2807. at java.security.AccessController.doPrivileged(Native Method)
  2808. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  2809. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  2810. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2811. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2812. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2813. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2814. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2815. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  2816. at java.security.AccessController.doPrivileged(Native Method)
  2817. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  2818. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  2819. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2820. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2821. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  2822. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  2823. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  2824. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  2825. at java.security.AccessController.doPrivileged(Native Method)
  2826. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  2827. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  2828. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  2829. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  2830. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  2831. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  2832. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  2833. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  2834. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  2835. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  2836. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2837. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  2838. at java.security.AccessController.doPrivileged(Native Method)
  2839. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  2840. ... 110 more
  2841. 17/08/13 10:15:19 WARN Hive: Failed to access metastore. This class should not accessed in runtime.
  2842. org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  2843. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1236)
  2844. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  2845. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  2846. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  2847. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  2848. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2849. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2850. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2851. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2852. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  2853. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  2854. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  2855. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  2856. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  2857. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  2858. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  2859. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  2860. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  2861. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  2862. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  2863. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  2864. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  2865. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  2866. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  2867. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  2868. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  2869. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  2870. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  2871. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  2872. at scala.Option.getOrElse(Option.scala:121)
  2873. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  2874. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  2875. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  2876. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  2877. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  2878. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2879. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2880. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2881. at java.lang.reflect.Method.invoke(Method.java:498)
  2882. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  2883. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  2884. at py4j.Gateway.invoke(Gateway.java:280)
  2885. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  2886. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  2887. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  2888. at java.lang.Thread.run(Thread.java:748)
  2889. Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  2890. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
  2891. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  2892. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  2893. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  2894. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  2895. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  2896. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  2897. ... 45 more
  2898. Caused by: java.lang.reflect.InvocationTargetException
  2899. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2900. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2901. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2902. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2903. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  2904. ... 51 more
  2905. Caused by: javax.jdo.JDOFatalDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  2906. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  2907. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2908. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  2909. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  2910. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  2911. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  2912. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2913. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  2914. at java.security.AccessController.doPrivileged(Native Method)
  2915. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  2916. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2917. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  2918. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  2919. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  2920. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  2921. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  2922. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  2923. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  2924. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  2925. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  2926. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2927. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2928. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2929. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2930. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  2931. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  2932. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  2933. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  2934. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  2935. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  2936. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  2937. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2938. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2939. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2940. at java.lang.reflect.Method.invoke(Method.java:498)
  2941. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  2942. at java.security.AccessController.doPrivileged(Native Method)
  2943. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  2944. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  2945. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  2946. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  2947. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  2948. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  2949. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  2950. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  2951. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  2952. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  2953. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  2954. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  2955. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  2956. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  2957. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  2958. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  2959. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  2960. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  2961. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  2962. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  2963. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  2964. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2965. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2966. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2967. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2968. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  2969. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  2970. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  2971. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  2972. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  2973. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  2974. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  2975. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  2976. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  2977. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  2978. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  2979. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2980. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  2981. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  2982. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  2983. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  2984. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  2985. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  2986. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  2987. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  2988. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  2989. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  2990. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  2991. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  2992. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  2993. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  2994. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  2995. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  2996. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  2997. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  2998. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  2999. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  3000. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  3001. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  3002. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  3003. at scala.Option.getOrElse(Option.scala:121)
  3004. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  3005. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  3006. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  3007. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  3008. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  3009. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3010. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3011. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3012. at java.lang.reflect.Method.invoke(Method.java:498)
  3013. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  3014. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  3015. at py4j.Gateway.invoke(Gateway.java:280)
  3016. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  3017. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  3018. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  3019. at java.lang.Thread.run(Thread.java:748)
  3020. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  3021. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3022. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  3023. ... 113 more
  3024. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  3025. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3026. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3027. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  3028. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  3029. at java.security.AccessController.doPrivileged(Native Method)
  3030. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  3031. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  3032. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3033. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3034. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3035. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3036. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3037. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  3038. at java.security.AccessController.doPrivileged(Native Method)
  3039. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  3040. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  3041. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3042. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3043. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3044. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3045. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3046. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  3047. at java.security.AccessController.doPrivileged(Native Method)
  3048. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  3049. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  3050. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3051. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3052. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3053. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3054. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3055. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  3056. at java.security.AccessController.doPrivileged(Native Method)
  3057. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  3058. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  3059. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  3060. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3061. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3062. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  3063. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  3064. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  3065. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  3066. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  3067. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3068. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3069. at java.security.AccessController.doPrivileged(Native Method)
  3070. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  3071. ... 110 more
  3072. ------
  3073.  
  3074. NestedThrowables:
  3075. java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  3076. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  3077. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3078. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3079. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  3080. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  3081. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  3082. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3083. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3084. at java.security.AccessController.doPrivileged(Native Method)
  3085. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  3086. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3087. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3088. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  3089. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  3090. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  3091. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  3092. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  3093. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  3094. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  3095. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  3096. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3097. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3098. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3099. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3100. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  3101. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  3102. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  3103. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  3104. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  3105. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  3106. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  3107. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3108. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3109. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3110. at java.lang.reflect.Method.invoke(Method.java:498)
  3111. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  3112. at java.security.AccessController.doPrivileged(Native Method)
  3113. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  3114. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  3115. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  3116. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  3117. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  3118. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  3119. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  3120. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  3121. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  3122. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  3123. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  3124. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  3125. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  3126. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  3127. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  3128. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  3129. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  3130. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  3131. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  3132. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  3133. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  3134. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3135. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3136. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3137. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3138. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  3139. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  3140. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  3141. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  3142. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  3143. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  3144. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  3145. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  3146. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  3147. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  3148. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  3149. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3150. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3151. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3152. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3153. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  3154. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  3155. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  3156. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  3157. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  3158. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  3159. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  3160. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  3161. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  3162. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  3163. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  3164. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  3165. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  3166. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  3167. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  3168. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  3169. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  3170. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  3171. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  3172. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  3173. at scala.Option.getOrElse(Option.scala:121)
  3174. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  3175. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  3176. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  3177. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  3178. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  3179. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3180. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3181. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3182. at java.lang.reflect.Method.invoke(Method.java:498)
  3183. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  3184. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  3185. at py4j.Gateway.invoke(Gateway.java:280)
  3186. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  3187. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  3188. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  3189. at java.lang.Thread.run(Thread.java:748)
  3190. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  3191. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3192. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  3193. ... 113 more
  3194. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  3195. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3196. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3197. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  3198. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  3199. at java.security.AccessController.doPrivileged(Native Method)
  3200. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  3201. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  3202. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3203. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3204. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3205. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3206. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3207. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  3208. at java.security.AccessController.doPrivileged(Native Method)
  3209. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  3210. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  3211. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3212. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3213. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3214. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3215. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3216. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  3217. at java.security.AccessController.doPrivileged(Native Method)
  3218. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  3219. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  3220. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3221. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3222. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3223. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3224. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3225. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  3226. at java.security.AccessController.doPrivileged(Native Method)
  3227. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  3228. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  3229. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  3230. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3231. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3232. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  3233. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  3234. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  3235. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  3236. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  3237. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3238. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3239. at java.security.AccessController.doPrivileged(Native Method)
  3240. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  3241. ... 110 more
  3242. ------
  3243.  
  3244. at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:436)
  3245. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:788)
  3246. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  3247. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  3248. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3249. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3250. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3251. at java.lang.reflect.Method.invoke(Method.java:498)
  3252. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  3253. at java.security.AccessController.doPrivileged(Native Method)
  3254. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  3255. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  3256. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  3257. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  3258. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  3259. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  3260. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  3261. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  3262. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  3263. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  3264. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  3265. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  3266. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  3267. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  3268. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  3269. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  3270. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  3271. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  3272. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  3273. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  3274. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  3275. ... 56 more
  3276. Caused by: java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  3277. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  3278. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3279. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3280. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  3281. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  3282. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  3283. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3284. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3285. at java.security.AccessController.doPrivileged(Native Method)
  3286. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  3287. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3288. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3289. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  3290. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  3291. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  3292. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  3293. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  3294. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  3295. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  3296. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  3297. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3298. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3299. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3300. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3301. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  3302. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  3303. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  3304. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  3305. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  3306. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  3307. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  3308. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3309. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3310. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3311. at java.lang.reflect.Method.invoke(Method.java:498)
  3312. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  3313. at java.security.AccessController.doPrivileged(Native Method)
  3314. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  3315. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  3316. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  3317. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  3318. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  3319. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  3320. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  3321. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  3322. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  3323. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  3324. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  3325. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  3326. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  3327. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  3328. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  3329. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  3330. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  3331. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  3332. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  3333. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  3334. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  3335. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3336. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3337. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3338. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3339. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  3340. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  3341. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  3342. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  3343. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  3344. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  3345. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  3346. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  3347. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  3348. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  3349. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  3350. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3351. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3352. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3353. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3354. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  3355. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  3356. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  3357. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  3358. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  3359. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  3360. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  3361. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  3362. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  3363. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  3364. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  3365. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  3366. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  3367. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  3368. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  3369. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  3370. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  3371. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  3372. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  3373. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  3374. at scala.Option.getOrElse(Option.scala:121)
  3375. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  3376. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  3377. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  3378. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  3379. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  3380. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3381. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3382. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3383. at java.lang.reflect.Method.invoke(Method.java:498)
  3384. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  3385. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  3386. at py4j.Gateway.invoke(Gateway.java:280)
  3387. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  3388. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  3389. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  3390. at java.lang.Thread.run(Thread.java:748)
  3391. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  3392. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3393. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  3394. ... 113 more
  3395. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  3396. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3397. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3398. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  3399. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  3400. at java.security.AccessController.doPrivileged(Native Method)
  3401. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  3402. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  3403. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3404. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3405. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3406. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3407. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3408. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  3409. at java.security.AccessController.doPrivileged(Native Method)
  3410. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  3411. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  3412. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3413. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3414. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3415. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3416. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3417. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  3418. at java.security.AccessController.doPrivileged(Native Method)
  3419. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  3420. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  3421. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3422. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3423. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3424. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3425. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3426. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  3427. at java.security.AccessController.doPrivileged(Native Method)
  3428. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  3429. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  3430. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  3431. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3432. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3433. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  3434. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  3435. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  3436. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  3437. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  3438. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3439. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3440. at java.security.AccessController.doPrivileged(Native Method)
  3441. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  3442. ... 110 more
  3443. ------
  3444.  
  3445. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3446. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3447. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3448. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3449. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  3450. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  3451. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  3452. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  3453. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  3454. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3455. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3456. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3457. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3458. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  3459. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  3460. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  3461. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  3462. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  3463. ... 85 more
  3464. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  3465. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3466. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3467. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  3468. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  3469. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  3470. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3471. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3472. at java.security.AccessController.doPrivileged(Native Method)
  3473. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  3474. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3475. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3476. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  3477. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  3478. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  3479. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  3480. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  3481. ... 97 more
  3482. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  3483. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3484. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  3485. ... 113 more
  3486. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  3487. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3488. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3489. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  3490. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  3491. at java.security.AccessController.doPrivileged(Native Method)
  3492. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  3493. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  3494. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3495. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3496. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3497. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3498. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3499. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  3500. at java.security.AccessController.doPrivileged(Native Method)
  3501. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  3502. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  3503. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3504. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3505. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3506. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3507. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3508. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  3509. at java.security.AccessController.doPrivileged(Native Method)
  3510. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  3511. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  3512. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3513. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3514. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3515. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3516. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3517. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  3518. at java.security.AccessController.doPrivileged(Native Method)
  3519. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  3520. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  3521. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  3522. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3523. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3524. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  3525. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  3526. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  3527. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  3528. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  3529. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3530. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3531. at java.security.AccessController.doPrivileged(Native Method)
  3532. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  3533. ... 110 more
  3534. 17/08/13 10:15:19 ERROR Schema: Failed initialising database.
  3535. Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  3536. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  3537. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3538. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3539. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  3540. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  3541. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  3542. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3543. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3544. at java.security.AccessController.doPrivileged(Native Method)
  3545. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  3546. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3547. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3548. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  3549. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  3550. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  3551. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  3552. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  3553. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  3554. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  3555. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  3556. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3557. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3558. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3559. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3560. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  3561. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  3562. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  3563. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  3564. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  3565. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  3566. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  3567. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3568. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3569. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3570. at java.lang.reflect.Method.invoke(Method.java:498)
  3571. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  3572. at java.security.AccessController.doPrivileged(Native Method)
  3573. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  3574. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  3575. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  3576. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  3577. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  3578. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  3579. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  3580. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  3581. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  3582. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  3583. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  3584. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  3585. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  3586. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  3587. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  3588. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  3589. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  3590. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  3591. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  3592. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  3593. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  3594. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3595. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3596. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3597. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3598. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  3599. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  3600. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  3601. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  3602. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  3603. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  3604. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  3605. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  3606. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3607. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3608. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3609. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3610. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  3611. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  3612. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  3613. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  3614. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  3615. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  3616. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  3617. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  3618. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  3619. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  3620. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  3621. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  3622. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  3623. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  3624. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  3625. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  3626. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  3627. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  3628. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  3629. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  3630. at scala.Option.getOrElse(Option.scala:121)
  3631. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  3632. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  3633. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  3634. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  3635. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  3636. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3637. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3638. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3639. at java.lang.reflect.Method.invoke(Method.java:498)
  3640. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  3641. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  3642. at py4j.Gateway.invoke(Gateway.java:280)
  3643. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  3644. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  3645. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  3646. at java.lang.Thread.run(Thread.java:748)
  3647. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  3648. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3649. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  3650. ... 110 more
  3651. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  3652. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3653. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3654. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  3655. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  3656. at java.security.AccessController.doPrivileged(Native Method)
  3657. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  3658. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  3659. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3660. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3661. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3662. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3663. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3664. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  3665. at java.security.AccessController.doPrivileged(Native Method)
  3666. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  3667. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  3668. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3669. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3670. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3671. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3672. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3673. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  3674. at java.security.AccessController.doPrivileged(Native Method)
  3675. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  3676. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  3677. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3678. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3679. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3680. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3681. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3682. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  3683. at java.security.AccessController.doPrivileged(Native Method)
  3684. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  3685. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  3686. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  3687. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3688. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3689. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  3690. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  3691. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  3692. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  3693. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  3694. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3695. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3696. at java.security.AccessController.doPrivileged(Native Method)
  3697. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  3698. ... 107 more
  3699. ------
  3700.  
  3701. org.datanucleus.exceptions.NucleusDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  3702. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  3703. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3704. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3705. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  3706. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  3707. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  3708. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3709. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3710. at java.security.AccessController.doPrivileged(Native Method)
  3711. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  3712. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3713. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3714. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  3715. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  3716. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  3717. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  3718. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  3719. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  3720. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  3721. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  3722. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3723. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3724. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3725. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3726. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  3727. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  3728. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  3729. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  3730. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  3731. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  3732. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  3733. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3734. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3735. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3736. at java.lang.reflect.Method.invoke(Method.java:498)
  3737. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  3738. at java.security.AccessController.doPrivileged(Native Method)
  3739. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  3740. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  3741. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  3742. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  3743. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  3744. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  3745. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  3746. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  3747. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  3748. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  3749. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  3750. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  3751. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  3752. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  3753. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  3754. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  3755. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  3756. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  3757. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  3758. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  3759. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  3760. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3761. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3762. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3763. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3764. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  3765. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  3766. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  3767. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  3768. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  3769. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  3770. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  3771. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  3772. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3773. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3774. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3775. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3776. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  3777. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  3778. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  3779. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  3780. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  3781. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  3782. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  3783. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  3784. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  3785. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  3786. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  3787. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  3788. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  3789. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  3790. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  3791. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  3792. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  3793. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  3794. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  3795. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  3796. at scala.Option.getOrElse(Option.scala:121)
  3797. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  3798. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  3799. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  3800. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  3801. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  3802. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3803. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3804. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3805. at java.lang.reflect.Method.invoke(Method.java:498)
  3806. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  3807. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  3808. at py4j.Gateway.invoke(Gateway.java:280)
  3809. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  3810. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  3811. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  3812. at java.lang.Thread.run(Thread.java:748)
  3813. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  3814. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3815. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  3816. ... 110 more
  3817. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  3818. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3819. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  3820. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  3821. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  3822. at java.security.AccessController.doPrivileged(Native Method)
  3823. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  3824. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  3825. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3826. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3827. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3828. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3829. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3830. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  3831. at java.security.AccessController.doPrivileged(Native Method)
  3832. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  3833. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  3834. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3835. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3836. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3837. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3838. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3839. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  3840. at java.security.AccessController.doPrivileged(Native Method)
  3841. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  3842. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  3843. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3844. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3845. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  3846. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  3847. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  3848. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  3849. at java.security.AccessController.doPrivileged(Native Method)
  3850. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  3851. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  3852. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  3853. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  3854. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  3855. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  3856. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  3857. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  3858. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  3859. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  3860. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3861. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  3862. at java.security.AccessController.doPrivileged(Native Method)
  3863. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  3864. ... 107 more
  3865. ------
  3866.  
  3867. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:516)
  3868. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  3869. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3870. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3871. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3872. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3873. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  3874. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  3875. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  3876. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  3877. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  3878. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  3879. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  3880. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3881. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3882. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3883. at java.lang.reflect.Method.invoke(Method.java:498)
  3884. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  3885. at java.security.AccessController.doPrivileged(Native Method)
  3886. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  3887. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  3888. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  3889. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  3890. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  3891. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  3892. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  3893. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  3894. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  3895. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  3896. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  3897. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  3898. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  3899. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  3900. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  3901. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  3902. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  3903. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  3904. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  3905. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  3906. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  3907. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3908. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3909. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3910. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3911. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  3912. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  3913. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  3914. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  3915. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  3916. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  3917. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  3918. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  3919. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3920. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3921. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3922. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3923. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  3924. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  3925. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  3926. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  3927. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  3928. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  3929. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  3930. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  3931. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  3932. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  3933. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  3934. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  3935. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  3936. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  3937. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  3938. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  3939. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  3940. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  3941. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  3942. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  3943. at scala.Option.getOrElse(Option.scala:121)
  3944. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  3945. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  3946. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  3947. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  3948. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  3949. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3950. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3951. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3952. at java.lang.reflect.Method.invoke(Method.java:498)
  3953. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  3954. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  3955. at py4j.Gateway.invoke(Gateway.java:280)
  3956. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  3957. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  3958. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  3959. at java.lang.Thread.run(Thread.java:748)
  3960. Caused by: java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  3961. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  3962. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3963. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  3964. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  3965. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  3966. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  3967. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3968. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  3969. at java.security.AccessController.doPrivileged(Native Method)
  3970. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  3971. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3972. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  3973. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  3974. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  3975. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  3976. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  3977. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  3978. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  3979. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  3980. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  3981. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  3982. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  3983. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  3984. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  3985. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  3986. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  3987. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  3988. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  3989. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  3990. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  3991. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  3992. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  3993. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  3994. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  3995. at java.lang.reflect.Method.invoke(Method.java:498)
  3996. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  3997. at java.security.AccessController.doPrivileged(Native Method)
  3998. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  3999. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  4000. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  4001. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  4002. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  4003. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  4004. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  4005. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  4006. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  4007. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  4008. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  4009. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  4010. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  4011. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  4012. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  4013. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  4014. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  4015. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  4016. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  4017. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  4018. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  4019. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4020. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4021. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4022. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4023. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  4024. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  4025. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  4026. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  4027. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  4028. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  4029. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  4030. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  4031. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4032. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4033. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4034. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4035. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  4036. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  4037. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  4038. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  4039. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  4040. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  4041. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  4042. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  4043. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  4044. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  4045. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  4046. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  4047. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  4048. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  4049. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  4050. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  4051. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  4052. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  4053. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  4054. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  4055. at scala.Option.getOrElse(Option.scala:121)
  4056. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  4057. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  4058. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  4059. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  4060. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  4061. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4062. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4063. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4064. at java.lang.reflect.Method.invoke(Method.java:498)
  4065. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  4066. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  4067. at py4j.Gateway.invoke(Gateway.java:280)
  4068. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  4069. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  4070. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  4071. at java.lang.Thread.run(Thread.java:748)
  4072. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  4073. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4074. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  4075. ... 110 more
  4076. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  4077. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4078. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4079. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  4080. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  4081. at java.security.AccessController.doPrivileged(Native Method)
  4082. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  4083. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  4084. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4085. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4086. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4087. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4088. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4089. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  4090. at java.security.AccessController.doPrivileged(Native Method)
  4091. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  4092. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  4093. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4094. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4095. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4096. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4097. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4098. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  4099. at java.security.AccessController.doPrivileged(Native Method)
  4100. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  4101. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  4102. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4103. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4104. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4105. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4106. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4107. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  4108. at java.security.AccessController.doPrivileged(Native Method)
  4109. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  4110. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  4111. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  4112. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4113. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4114. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  4115. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  4116. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  4117. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  4118. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  4119. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4120. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4121. at java.security.AccessController.doPrivileged(Native Method)
  4122. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  4123. ... 107 more
  4124. ------
  4125.  
  4126. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4127. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4128. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4129. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4130. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  4131. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  4132. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  4133. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  4134. ... 92 more
  4135. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  4136. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4137. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4138. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  4139. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  4140. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  4141. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4142. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4143. at java.security.AccessController.doPrivileged(Native Method)
  4144. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  4145. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4146. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4147. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  4148. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  4149. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  4150. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  4151. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  4152. ... 94 more
  4153. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  4154. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4155. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  4156. ... 110 more
  4157. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  4158. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4159. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4160. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  4161. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  4162. at java.security.AccessController.doPrivileged(Native Method)
  4163. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  4164. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  4165. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4166. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4167. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4168. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4169. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4170. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  4171. at java.security.AccessController.doPrivileged(Native Method)
  4172. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  4173. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  4174. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4175. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4176. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4177. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4178. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4179. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  4180. at java.security.AccessController.doPrivileged(Native Method)
  4181. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  4182. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  4183. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4184. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4185. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4186. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4187. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4188. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  4189. at java.security.AccessController.doPrivileged(Native Method)
  4190. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  4191. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  4192. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  4193. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4194. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4195. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  4196. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  4197. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  4198. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  4199. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  4200. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4201. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4202. at java.security.AccessController.doPrivileged(Native Method)
  4203. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  4204. ... 107 more
  4205. Nested Throwables StackTrace:
  4206. java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  4207. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  4208. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4209. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4210. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  4211. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  4212. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  4213. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4214. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4215. at java.security.AccessController.doPrivileged(Native Method)
  4216. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  4217. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4218. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4219. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  4220. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  4221. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  4222. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  4223. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  4224. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  4225. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  4226. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  4227. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4228. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4229. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4230. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4231. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  4232. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  4233. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  4234. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  4235. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  4236. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  4237. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  4238. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4239. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4240. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4241. at java.lang.reflect.Method.invoke(Method.java:498)
  4242. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  4243. at java.security.AccessController.doPrivileged(Native Method)
  4244. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  4245. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  4246. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  4247. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  4248. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  4249. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  4250. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  4251. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  4252. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  4253. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  4254. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  4255. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  4256. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  4257. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  4258. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  4259. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  4260. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  4261. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  4262. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  4263. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  4264. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  4265. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4266. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4267. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4268. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4269. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  4270. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  4271. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  4272. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  4273. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  4274. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  4275. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  4276. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  4277. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4278. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4279. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4280. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4281. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  4282. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  4283. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  4284. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  4285. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  4286. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  4287. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  4288. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  4289. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  4290. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  4291. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  4292. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  4293. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  4294. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  4295. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  4296. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  4297. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  4298. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  4299. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  4300. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  4301. at scala.Option.getOrElse(Option.scala:121)
  4302. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  4303. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  4304. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  4305. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  4306. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  4307. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4308. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4309. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4310. at java.lang.reflect.Method.invoke(Method.java:498)
  4311. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  4312. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  4313. at py4j.Gateway.invoke(Gateway.java:280)
  4314. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  4315. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  4316. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  4317. at java.lang.Thread.run(Thread.java:748)
  4318. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  4319. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4320. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  4321. ... 110 more
  4322. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  4323. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4324. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4325. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  4326. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  4327. at java.security.AccessController.doPrivileged(Native Method)
  4328. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  4329. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  4330. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4331. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4332. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4333. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4334. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4335. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  4336. at java.security.AccessController.doPrivileged(Native Method)
  4337. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  4338. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  4339. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4340. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4341. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4342. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4343. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4344. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  4345. at java.security.AccessController.doPrivileged(Native Method)
  4346. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  4347. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  4348. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4349. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4350. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4351. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4352. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4353. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  4354. at java.security.AccessController.doPrivileged(Native Method)
  4355. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  4356. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  4357. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  4358. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4359. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4360. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  4361. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  4362. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  4363. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  4364. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  4365. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4366. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4367. at java.security.AccessController.doPrivileged(Native Method)
  4368. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  4369. ... 107 more
  4370. ------
  4371.  
  4372. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4373. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4374. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4375. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4376. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  4377. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  4378. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  4379. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  4380. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  4381. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4382. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4383. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4384. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4385. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  4386. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  4387. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  4388. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  4389. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  4390. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  4391. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  4392. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4393. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4394. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4395. at java.lang.reflect.Method.invoke(Method.java:498)
  4396. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  4397. at java.security.AccessController.doPrivileged(Native Method)
  4398. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  4399. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  4400. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  4401. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  4402. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  4403. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  4404. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  4405. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  4406. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  4407. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  4408. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  4409. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  4410. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  4411. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  4412. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  4413. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  4414. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  4415. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  4416. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  4417. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  4418. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  4419. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4420. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4421. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4422. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4423. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  4424. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  4425. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  4426. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  4427. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  4428. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  4429. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  4430. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  4431. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4432. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4433. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4434. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4435. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  4436. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  4437. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  4438. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  4439. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  4440. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  4441. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  4442. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  4443. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  4444. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  4445. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  4446. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  4447. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  4448. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  4449. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  4450. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  4451. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  4452. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  4453. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  4454. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  4455. at scala.Option.getOrElse(Option.scala:121)
  4456. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  4457. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  4458. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  4459. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  4460. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  4461. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4462. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4463. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4464. at java.lang.reflect.Method.invoke(Method.java:498)
  4465. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  4466. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  4467. at py4j.Gateway.invoke(Gateway.java:280)
  4468. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  4469. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  4470. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  4471. at java.lang.Thread.run(Thread.java:748)
  4472. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  4473. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4474. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4475. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  4476. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  4477. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  4478. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4479. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4480. at java.security.AccessController.doPrivileged(Native Method)
  4481. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  4482. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4483. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4484. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  4485. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  4486. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  4487. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  4488. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  4489. ... 94 more
  4490. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  4491. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4492. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  4493. ... 110 more
  4494. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  4495. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4496. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4497. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  4498. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  4499. at java.security.AccessController.doPrivileged(Native Method)
  4500. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  4501. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  4502. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4503. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4504. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4505. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4506. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4507. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  4508. at java.security.AccessController.doPrivileged(Native Method)
  4509. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  4510. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  4511. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4512. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4513. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4514. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4515. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4516. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  4517. at java.security.AccessController.doPrivileged(Native Method)
  4518. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  4519. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  4520. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4521. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4522. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4523. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4524. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4525. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  4526. at java.security.AccessController.doPrivileged(Native Method)
  4527. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  4528. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  4529. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  4530. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4531. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4532. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  4533. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  4534. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  4535. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  4536. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  4537. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4538. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4539. at java.security.AccessController.doPrivileged(Native Method)
  4540. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  4541. ... 107 more
  4542. 17/08/13 10:15:19 WARN HiveMetaStore: Retrying creating default database after error: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  4543. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  4544. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4545. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4546. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  4547. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  4548. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  4549. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4550. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4551. at java.security.AccessController.doPrivileged(Native Method)
  4552. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  4553. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4554. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4555. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  4556. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  4557. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  4558. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  4559. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  4560. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  4561. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  4562. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  4563. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4564. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4565. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4566. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4567. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  4568. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  4569. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  4570. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  4571. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  4572. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  4573. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  4574. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4575. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4576. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4577. at java.lang.reflect.Method.invoke(Method.java:498)
  4578. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  4579. at java.security.AccessController.doPrivileged(Native Method)
  4580. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  4581. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  4582. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  4583. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  4584. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  4585. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  4586. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  4587. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  4588. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  4589. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  4590. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  4591. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  4592. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  4593. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  4594. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  4595. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  4596. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  4597. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  4598. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  4599. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  4600. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  4601. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4602. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4603. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4604. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4605. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  4606. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  4607. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  4608. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  4609. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  4610. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  4611. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  4612. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  4613. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4614. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4615. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4616. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4617. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  4618. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  4619. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  4620. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  4621. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  4622. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  4623. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  4624. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  4625. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  4626. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  4627. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  4628. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  4629. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  4630. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  4631. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  4632. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  4633. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  4634. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  4635. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  4636. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  4637. at scala.Option.getOrElse(Option.scala:121)
  4638. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  4639. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  4640. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  4641. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  4642. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  4643. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4644. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4645. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4646. at java.lang.reflect.Method.invoke(Method.java:498)
  4647. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  4648. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  4649. at py4j.Gateway.invoke(Gateway.java:280)
  4650. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  4651. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  4652. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  4653. at java.lang.Thread.run(Thread.java:748)
  4654. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  4655. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4656. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  4657. ... 110 more
  4658. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  4659. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4660. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4661. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  4662. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  4663. at java.security.AccessController.doPrivileged(Native Method)
  4664. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  4665. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  4666. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4667. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4668. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4669. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4670. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4671. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  4672. at java.security.AccessController.doPrivileged(Native Method)
  4673. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  4674. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  4675. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4676. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4677. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4678. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4679. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4680. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  4681. at java.security.AccessController.doPrivileged(Native Method)
  4682. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  4683. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  4684. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4685. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4686. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4687. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4688. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4689. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  4690. at java.security.AccessController.doPrivileged(Native Method)
  4691. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  4692. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  4693. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  4694. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4695. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4696. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  4697. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  4698. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  4699. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  4700. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  4701. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4702. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4703. at java.security.AccessController.doPrivileged(Native Method)
  4704. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  4705. ... 107 more
  4706. ------
  4707.  
  4708. javax.jdo.JDOFatalDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  4709. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  4710. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4711. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4712. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  4713. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  4714. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  4715. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4716. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4717. at java.security.AccessController.doPrivileged(Native Method)
  4718. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  4719. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4720. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4721. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  4722. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  4723. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  4724. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  4725. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  4726. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  4727. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  4728. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  4729. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4730. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4731. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4732. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4733. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  4734. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  4735. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  4736. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  4737. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  4738. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  4739. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  4740. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4741. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4742. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4743. at java.lang.reflect.Method.invoke(Method.java:498)
  4744. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  4745. at java.security.AccessController.doPrivileged(Native Method)
  4746. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  4747. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  4748. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  4749. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  4750. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  4751. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  4752. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  4753. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  4754. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  4755. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  4756. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  4757. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  4758. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  4759. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  4760. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  4761. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  4762. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  4763. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  4764. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  4765. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  4766. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  4767. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4768. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4769. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4770. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4771. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  4772. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  4773. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  4774. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  4775. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  4776. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  4777. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  4778. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  4779. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4780. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4781. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4782. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4783. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  4784. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  4785. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  4786. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  4787. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  4788. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  4789. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  4790. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  4791. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  4792. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  4793. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  4794. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  4795. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  4796. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  4797. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  4798. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  4799. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  4800. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  4801. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  4802. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  4803. at scala.Option.getOrElse(Option.scala:121)
  4804. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  4805. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  4806. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  4807. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  4808. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  4809. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4810. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4811. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4812. at java.lang.reflect.Method.invoke(Method.java:498)
  4813. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  4814. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  4815. at py4j.Gateway.invoke(Gateway.java:280)
  4816. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  4817. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  4818. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  4819. at java.lang.Thread.run(Thread.java:748)
  4820. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  4821. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4822. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  4823. ... 110 more
  4824. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  4825. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4826. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  4827. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  4828. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  4829. at java.security.AccessController.doPrivileged(Native Method)
  4830. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  4831. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  4832. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4833. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4834. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4835. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4836. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4837. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  4838. at java.security.AccessController.doPrivileged(Native Method)
  4839. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  4840. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  4841. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4842. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4843. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4844. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4845. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4846. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  4847. at java.security.AccessController.doPrivileged(Native Method)
  4848. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  4849. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  4850. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4851. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4852. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  4853. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  4854. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  4855. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  4856. at java.security.AccessController.doPrivileged(Native Method)
  4857. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  4858. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  4859. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  4860. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  4861. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  4862. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  4863. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  4864. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  4865. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  4866. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  4867. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4868. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  4869. at java.security.AccessController.doPrivileged(Native Method)
  4870. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  4871. ... 107 more
  4872. ------
  4873.  
  4874. at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:436)
  4875. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:788)
  4876. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  4877. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  4878. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4879. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4880. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4881. at java.lang.reflect.Method.invoke(Method.java:498)
  4882. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  4883. at java.security.AccessController.doPrivileged(Native Method)
  4884. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  4885. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  4886. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  4887. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  4888. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  4889. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  4890. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  4891. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  4892. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  4893. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  4894. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  4895. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  4896. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  4897. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  4898. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  4899. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  4900. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  4901. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  4902. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  4903. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  4904. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  4905. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4906. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4907. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4908. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4909. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  4910. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  4911. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  4912. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  4913. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  4914. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  4915. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  4916. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  4917. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4918. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4919. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4920. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4921. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  4922. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  4923. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  4924. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  4925. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  4926. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  4927. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  4928. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  4929. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  4930. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  4931. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  4932. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  4933. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  4934. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  4935. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  4936. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  4937. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  4938. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  4939. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  4940. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  4941. at scala.Option.getOrElse(Option.scala:121)
  4942. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  4943. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  4944. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  4945. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  4946. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  4947. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4948. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4949. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4950. at java.lang.reflect.Method.invoke(Method.java:498)
  4951. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  4952. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  4953. at py4j.Gateway.invoke(Gateway.java:280)
  4954. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  4955. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  4956. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  4957. at java.lang.Thread.run(Thread.java:748)
  4958. NestedThrowablesStackTrace:
  4959. java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  4960. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  4961. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4962. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  4963. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  4964. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  4965. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  4966. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4967. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  4968. at java.security.AccessController.doPrivileged(Native Method)
  4969. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  4970. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4971. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  4972. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  4973. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  4974. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  4975. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  4976. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  4977. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  4978. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  4979. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  4980. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  4981. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  4982. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  4983. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  4984. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  4985. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  4986. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  4987. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  4988. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  4989. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  4990. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  4991. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  4992. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  4993. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  4994. at java.lang.reflect.Method.invoke(Method.java:498)
  4995. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  4996. at java.security.AccessController.doPrivileged(Native Method)
  4997. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  4998. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  4999. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  5000. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  5001. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  5002. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  5003. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  5004. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  5005. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  5006. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  5007. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  5008. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  5009. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  5010. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  5011. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  5012. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  5013. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  5014. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  5015. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  5016. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  5017. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  5018. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5019. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5020. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5021. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5022. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  5023. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  5024. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  5025. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  5026. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  5027. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  5028. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  5029. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  5030. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5031. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5032. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5033. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5034. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  5035. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  5036. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  5037. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  5038. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  5039. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  5040. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  5041. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  5042. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  5043. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  5044. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  5045. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  5046. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  5047. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  5048. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  5049. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  5050. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  5051. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  5052. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  5053. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  5054. at scala.Option.getOrElse(Option.scala:121)
  5055. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  5056. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  5057. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  5058. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  5059. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  5060. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5061. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5062. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5063. at java.lang.reflect.Method.invoke(Method.java:498)
  5064. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  5065. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  5066. at py4j.Gateway.invoke(Gateway.java:280)
  5067. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  5068. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  5069. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  5070. at java.lang.Thread.run(Thread.java:748)
  5071. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  5072. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5073. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  5074. ... 110 more
  5075. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  5076. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5077. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5078. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  5079. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  5080. at java.security.AccessController.doPrivileged(Native Method)
  5081. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  5082. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  5083. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5084. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5085. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5086. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5087. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5088. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  5089. at java.security.AccessController.doPrivileged(Native Method)
  5090. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  5091. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  5092. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5093. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5094. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5095. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5096. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5097. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  5098. at java.security.AccessController.doPrivileged(Native Method)
  5099. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  5100. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  5101. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5102. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5103. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5104. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5105. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5106. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  5107. at java.security.AccessController.doPrivileged(Native Method)
  5108. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  5109. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  5110. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  5111. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5112. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5113. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  5114. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  5115. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  5116. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  5117. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  5118. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5119. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5120. at java.security.AccessController.doPrivileged(Native Method)
  5121. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  5122. ... 107 more
  5123. ------
  5124.  
  5125. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5126. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5127. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5128. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5129. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  5130. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  5131. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  5132. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  5133. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  5134. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5135. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5136. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5137. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5138. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  5139. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  5140. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  5141. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  5142. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  5143. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  5144. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  5145. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5146. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5147. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5148. at java.lang.reflect.Method.invoke(Method.java:498)
  5149. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  5150. at java.security.AccessController.doPrivileged(Native Method)
  5151. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  5152. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  5153. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  5154. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  5155. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  5156. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  5157. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  5158. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  5159. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  5160. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  5161. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  5162. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  5163. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  5164. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  5165. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
  5166. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  5167. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  5168. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  5169. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  5170. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  5171. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  5172. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5173. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5174. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5175. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5176. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  5177. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  5178. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  5179. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  5180. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  5181. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  5182. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  5183. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  5184. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5185. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5186. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5187. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5188. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  5189. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  5190. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  5191. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  5192. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  5193. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  5194. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  5195. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  5196. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  5197. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  5198. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  5199. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  5200. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  5201. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  5202. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  5203. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  5204. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  5205. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  5206. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  5207. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  5208. at scala.Option.getOrElse(Option.scala:121)
  5209. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  5210. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  5211. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  5212. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  5213. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  5214. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5215. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5216. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5217. at java.lang.reflect.Method.invoke(Method.java:498)
  5218. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  5219. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  5220. at py4j.Gateway.invoke(Gateway.java:280)
  5221. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  5222. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  5223. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  5224. at java.lang.Thread.run(Thread.java:748)
  5225. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  5226. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5227. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5228. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  5229. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  5230. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  5231. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5232. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5233. at java.security.AccessController.doPrivileged(Native Method)
  5234. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  5235. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5236. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5237. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  5238. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  5239. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  5240. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  5241. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  5242. ... 94 more
  5243. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  5244. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5245. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  5246. ... 110 more
  5247. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  5248. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5249. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5250. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  5251. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  5252. at java.security.AccessController.doPrivileged(Native Method)
  5253. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  5254. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  5255. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5256. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5257. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5258. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5259. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5260. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  5261. at java.security.AccessController.doPrivileged(Native Method)
  5262. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  5263. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  5264. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5265. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5266. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5267. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5268. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5269. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  5270. at java.security.AccessController.doPrivileged(Native Method)
  5271. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  5272. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  5273. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5274. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5275. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5276. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5277. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5278. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  5279. at java.security.AccessController.doPrivileged(Native Method)
  5280. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  5281. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  5282. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  5283. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5284. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5285. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  5286. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  5287. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  5288. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  5289. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  5290. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5291. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5292. at java.security.AccessController.doPrivileged(Native Method)
  5293. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  5294. ... 107 more
  5295. 17/08/13 10:15:19 ERROR Schema: Failed initialising database.
  5296. Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  5297. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  5298. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5299. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5300. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  5301. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  5302. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  5303. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5304. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5305. at java.security.AccessController.doPrivileged(Native Method)
  5306. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  5307. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5308. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5309. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  5310. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  5311. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  5312. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  5313. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  5314. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  5315. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  5316. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  5317. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5318. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5319. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5320. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5321. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  5322. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  5323. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  5324. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  5325. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  5326. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  5327. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  5328. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5329. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5330. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5331. at java.lang.reflect.Method.invoke(Method.java:498)
  5332. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  5333. at java.security.AccessController.doPrivileged(Native Method)
  5334. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  5335. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  5336. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  5337. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  5338. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  5339. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  5340. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  5341. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  5342. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  5343. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  5344. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  5345. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  5346. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  5347. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  5348. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  5349. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  5350. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  5351. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  5352. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  5353. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  5354. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  5355. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5356. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5357. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5358. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5359. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  5360. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  5361. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  5362. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  5363. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  5364. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  5365. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  5366. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  5367. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5368. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5369. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5370. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5371. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  5372. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  5373. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  5374. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  5375. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  5376. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  5377. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  5378. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  5379. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  5380. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  5381. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  5382. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  5383. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  5384. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  5385. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  5386. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  5387. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  5388. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  5389. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  5390. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  5391. at scala.Option.getOrElse(Option.scala:121)
  5392. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  5393. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  5394. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  5395. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  5396. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  5397. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5398. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5399. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5400. at java.lang.reflect.Method.invoke(Method.java:498)
  5401. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  5402. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  5403. at py4j.Gateway.invoke(Gateway.java:280)
  5404. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  5405. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  5406. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  5407. at java.lang.Thread.run(Thread.java:748)
  5408. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  5409. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5410. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  5411. ... 110 more
  5412. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  5413. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5414. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5415. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  5416. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  5417. at java.security.AccessController.doPrivileged(Native Method)
  5418. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  5419. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  5420. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5421. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5422. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5423. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5424. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5425. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  5426. at java.security.AccessController.doPrivileged(Native Method)
  5427. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  5428. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  5429. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5430. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5431. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5432. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5433. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5434. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  5435. at java.security.AccessController.doPrivileged(Native Method)
  5436. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  5437. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  5438. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5439. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5440. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5441. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5442. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5443. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  5444. at java.security.AccessController.doPrivileged(Native Method)
  5445. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  5446. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  5447. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  5448. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5449. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5450. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  5451. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  5452. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  5453. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  5454. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  5455. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5456. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5457. at java.security.AccessController.doPrivileged(Native Method)
  5458. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  5459. ... 107 more
  5460. ------
  5461.  
  5462. org.datanucleus.exceptions.NucleusDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  5463. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  5464. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5465. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5466. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  5467. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  5468. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  5469. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5470. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5471. at java.security.AccessController.doPrivileged(Native Method)
  5472. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  5473. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5474. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5475. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  5476. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  5477. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  5478. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  5479. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  5480. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  5481. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  5482. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  5483. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5484. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5485. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5486. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5487. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  5488. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  5489. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  5490. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  5491. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  5492. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  5493. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  5494. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5495. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5496. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5497. at java.lang.reflect.Method.invoke(Method.java:498)
  5498. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  5499. at java.security.AccessController.doPrivileged(Native Method)
  5500. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  5501. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  5502. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  5503. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  5504. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  5505. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  5506. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  5507. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  5508. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  5509. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  5510. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  5511. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  5512. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  5513. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  5514. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  5515. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  5516. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  5517. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  5518. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  5519. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  5520. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  5521. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5522. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5523. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5524. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5525. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  5526. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  5527. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  5528. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  5529. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  5530. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  5531. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  5532. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  5533. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5534. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5535. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5536. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5537. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  5538. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  5539. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  5540. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  5541. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  5542. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  5543. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  5544. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  5545. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  5546. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  5547. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  5548. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  5549. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  5550. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  5551. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  5552. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  5553. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  5554. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  5555. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  5556. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  5557. at scala.Option.getOrElse(Option.scala:121)
  5558. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  5559. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  5560. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  5561. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  5562. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  5563. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5564. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5565. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5566. at java.lang.reflect.Method.invoke(Method.java:498)
  5567. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  5568. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  5569. at py4j.Gateway.invoke(Gateway.java:280)
  5570. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  5571. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  5572. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  5573. at java.lang.Thread.run(Thread.java:748)
  5574. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  5575. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5576. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  5577. ... 110 more
  5578. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  5579. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5580. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5581. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  5582. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  5583. at java.security.AccessController.doPrivileged(Native Method)
  5584. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  5585. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  5586. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5587. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5588. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5589. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5590. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5591. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  5592. at java.security.AccessController.doPrivileged(Native Method)
  5593. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  5594. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  5595. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5596. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5597. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5598. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5599. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5600. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  5601. at java.security.AccessController.doPrivileged(Native Method)
  5602. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  5603. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  5604. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5605. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5606. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5607. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5608. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5609. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  5610. at java.security.AccessController.doPrivileged(Native Method)
  5611. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  5612. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  5613. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  5614. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5615. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5616. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  5617. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  5618. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  5619. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  5620. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  5621. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5622. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5623. at java.security.AccessController.doPrivileged(Native Method)
  5624. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  5625. ... 107 more
  5626. ------
  5627.  
  5628. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:516)
  5629. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  5630. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5631. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5632. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5633. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5634. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  5635. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  5636. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  5637. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  5638. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  5639. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  5640. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  5641. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5642. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5643. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5644. at java.lang.reflect.Method.invoke(Method.java:498)
  5645. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  5646. at java.security.AccessController.doPrivileged(Native Method)
  5647. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  5648. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  5649. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  5650. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  5651. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  5652. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  5653. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  5654. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  5655. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  5656. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  5657. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  5658. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  5659. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  5660. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  5661. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  5662. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  5663. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  5664. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  5665. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  5666. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  5667. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  5668. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5669. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5670. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5671. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5672. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  5673. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  5674. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  5675. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  5676. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  5677. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  5678. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  5679. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  5680. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5681. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5682. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5683. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5684. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  5685. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  5686. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  5687. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  5688. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  5689. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  5690. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  5691. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  5692. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  5693. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  5694. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  5695. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  5696. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  5697. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  5698. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  5699. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  5700. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  5701. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  5702. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  5703. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  5704. at scala.Option.getOrElse(Option.scala:121)
  5705. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  5706. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  5707. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  5708. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  5709. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  5710. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5711. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5712. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5713. at java.lang.reflect.Method.invoke(Method.java:498)
  5714. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  5715. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  5716. at py4j.Gateway.invoke(Gateway.java:280)
  5717. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  5718. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  5719. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  5720. at java.lang.Thread.run(Thread.java:748)
  5721. Caused by: java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  5722. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  5723. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5724. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5725. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  5726. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  5727. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  5728. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5729. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5730. at java.security.AccessController.doPrivileged(Native Method)
  5731. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  5732. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5733. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5734. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  5735. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  5736. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  5737. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  5738. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  5739. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  5740. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  5741. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  5742. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5743. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5744. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5745. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5746. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  5747. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  5748. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  5749. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  5750. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  5751. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  5752. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  5753. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5754. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5755. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5756. at java.lang.reflect.Method.invoke(Method.java:498)
  5757. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  5758. at java.security.AccessController.doPrivileged(Native Method)
  5759. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  5760. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  5761. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  5762. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  5763. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  5764. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  5765. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  5766. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  5767. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  5768. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  5769. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  5770. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  5771. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  5772. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  5773. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  5774. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  5775. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  5776. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  5777. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  5778. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  5779. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  5780. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5781. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5782. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5783. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5784. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  5785. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  5786. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  5787. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  5788. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  5789. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  5790. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  5791. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  5792. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5793. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5794. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5795. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5796. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  5797. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  5798. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  5799. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  5800. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  5801. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  5802. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  5803. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  5804. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  5805. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  5806. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  5807. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  5808. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  5809. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  5810. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  5811. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  5812. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  5813. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  5814. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  5815. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  5816. at scala.Option.getOrElse(Option.scala:121)
  5817. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  5818. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  5819. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  5820. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  5821. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  5822. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  5823. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  5824. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  5825. at java.lang.reflect.Method.invoke(Method.java:498)
  5826. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  5827. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  5828. at py4j.Gateway.invoke(Gateway.java:280)
  5829. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  5830. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  5831. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  5832. at java.lang.Thread.run(Thread.java:748)
  5833. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  5834. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5835. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  5836. ... 110 more
  5837. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  5838. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5839. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5840. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  5841. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  5842. at java.security.AccessController.doPrivileged(Native Method)
  5843. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  5844. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  5845. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5846. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5847. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5848. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5849. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5850. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  5851. at java.security.AccessController.doPrivileged(Native Method)
  5852. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  5853. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  5854. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5855. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5856. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5857. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5858. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5859. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  5860. at java.security.AccessController.doPrivileged(Native Method)
  5861. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  5862. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  5863. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5864. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5865. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5866. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5867. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5868. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  5869. at java.security.AccessController.doPrivileged(Native Method)
  5870. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  5871. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  5872. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  5873. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5874. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5875. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  5876. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  5877. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  5878. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  5879. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  5880. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5881. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5882. at java.security.AccessController.doPrivileged(Native Method)
  5883. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  5884. ... 107 more
  5885. ------
  5886.  
  5887. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5888. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5889. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5890. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5891. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  5892. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  5893. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  5894. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  5895. ... 92 more
  5896. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  5897. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5898. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5899. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  5900. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  5901. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  5902. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5903. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5904. at java.security.AccessController.doPrivileged(Native Method)
  5905. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  5906. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5907. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5908. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  5909. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  5910. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  5911. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  5912. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  5913. ... 94 more
  5914. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  5915. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5916. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  5917. ... 110 more
  5918. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  5919. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5920. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  5921. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  5922. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  5923. at java.security.AccessController.doPrivileged(Native Method)
  5924. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  5925. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  5926. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5927. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5928. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5929. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5930. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5931. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  5932. at java.security.AccessController.doPrivileged(Native Method)
  5933. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  5934. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  5935. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5936. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5937. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5938. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5939. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5940. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  5941. at java.security.AccessController.doPrivileged(Native Method)
  5942. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  5943. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  5944. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5945. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5946. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  5947. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  5948. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  5949. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  5950. at java.security.AccessController.doPrivileged(Native Method)
  5951. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  5952. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  5953. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  5954. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  5955. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  5956. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  5957. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  5958. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  5959. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  5960. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  5961. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5962. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  5963. at java.security.AccessController.doPrivileged(Native Method)
  5964. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  5965. ... 107 more
  5966. Nested Throwables StackTrace:
  5967. java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
  5968. java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  5969. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5970. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  5971. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  5972. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  5973. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  5974. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5975. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  5976. at java.security.AccessController.doPrivileged(Native Method)
  5977. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  5978. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5979. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  5980. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  5981. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  5982. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  5983. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  5984. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  5985. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  5986. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  5987. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  5988. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  5989. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  5990. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  5991. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  5992. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  5993. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  5994. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  5995. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  5996. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  5997. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  5998. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  5999. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6000. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6001. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6002. at java.lang.reflect.Method.invoke(Method.java:498)
  6003. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  6004. at java.security.AccessController.doPrivileged(Native Method)
  6005. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  6006. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  6007. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  6008. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  6009. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  6010. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  6011. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  6012. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  6013. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  6014. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  6015. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  6016. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  6017. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  6018. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  6019. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  6020. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  6021. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  6022. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  6023. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  6024. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  6025. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  6026. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6027. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6028. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6029. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6030. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  6031. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  6032. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  6033. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  6034. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  6035. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  6036. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  6037. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  6038. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6039. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6040. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6041. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6042. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  6043. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  6044. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  6045. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  6046. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  6047. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  6048. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  6049. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  6050. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  6051. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  6052. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  6053. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  6054. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  6055. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  6056. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  6057. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  6058. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  6059. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  6060. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  6061. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  6062. at scala.Option.getOrElse(Option.scala:121)
  6063. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  6064. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  6065. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  6066. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  6067. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  6068. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6069. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6070. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6071. at java.lang.reflect.Method.invoke(Method.java:498)
  6072. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  6073. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  6074. at py4j.Gateway.invoke(Gateway.java:280)
  6075. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  6076. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  6077. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  6078. at java.lang.Thread.run(Thread.java:748)
  6079. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  6080. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6081. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  6082. ... 110 more
  6083. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  6084. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6085. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6086. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  6087. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  6088. at java.security.AccessController.doPrivileged(Native Method)
  6089. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  6090. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  6091. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6092. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6093. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6094. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6095. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6096. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  6097. at java.security.AccessController.doPrivileged(Native Method)
  6098. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  6099. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  6100. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6101. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6102. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6103. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6104. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6105. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  6106. at java.security.AccessController.doPrivileged(Native Method)
  6107. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  6108. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  6109. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6110. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6111. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6112. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6113. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6114. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  6115. at java.security.AccessController.doPrivileged(Native Method)
  6116. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  6117. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  6118. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  6119. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6120. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6121. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  6122. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  6123. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  6124. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  6125. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  6126. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  6127. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  6128. at java.security.AccessController.doPrivileged(Native Method)
  6129. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  6130. ... 107 more
  6131. ------
  6132.  
  6133. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6134. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6135. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6136. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6137. at com.jolbox.bonecp.PoolUtil.generateSQLException(PoolUtil.java:192)
  6138. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:422)
  6139. at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
  6140. at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
  6141. at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
  6142. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6143. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6144. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6145. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6146. at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
  6147. at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
  6148. at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
  6149. at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
  6150. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
  6151. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
  6152. at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
  6153. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6154. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6155. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6156. at java.lang.reflect.Method.invoke(Method.java:498)
  6157. at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
  6158. at java.security.AccessController.doPrivileged(Native Method)
  6159. at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
  6160. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
  6161. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
  6162. at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
  6163. at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365)
  6164. at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394)
  6165. at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291)
  6166. at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258)
  6167. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
  6168. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  6169. at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57)
  6170. at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66)
  6171. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593)
  6172. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571)
  6173. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624)
  6174. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
  6175. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  6176. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  6177. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  6178. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  6179. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  6180. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6181. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6182. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6183. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6184. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  6185. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  6186. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  6187. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  6188. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  6189. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  6190. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  6191. at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:191)
  6192. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  6193. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  6194. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  6195. at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  6196. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
  6197. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:362)
  6198. at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:266)
  6199. at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
  6200. at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
  6201. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:194)
  6202. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  6203. at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:194)
  6204. at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
  6205. at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:193)
  6206. at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:105)
  6207. at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:93)
  6208. at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
  6209. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
  6210. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
  6211. at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:35)
  6212. at org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:289)
  6213. at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$instantiateSessionState(SparkSession.scala:1050)
  6214. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  6215. at org.apache.spark.sql.SparkSession$$anonfun$sessionState$2.apply(SparkSession.scala:130)
  6216. at scala.Option.getOrElse(Option.scala:121)
  6217. at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:129)
  6218. at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:126)
  6219. at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:65)
  6220. at org.apache.spark.sql.SparkSession.createDataFrame(SparkSession.scala:283)
  6221. at org.apache.spark.mllib.api.python.PythonMLLibAPI.getMatrixEntries(PythonMLLibAPI.scala:1207)
  6222. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  6223. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  6224. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  6225. at java.lang.reflect.Method.invoke(Method.java:498)
  6226. at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  6227. at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  6228. at py4j.Gateway.invoke(Gateway.java:280)
  6229. at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  6230. at py4j.commands.CallCommand.execute(CallCommand.java:79)
  6231. at py4j.GatewayConnection.run(GatewayConnection.java:214)
  6232. at java.lang.Thread.run(Thread.java:748)
  6233. Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  6234. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  6235. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  6236. at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
  6237. at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
  6238. at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
  6239. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  6240. at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
  6241. at java.security.AccessController.doPrivileged(Native Method)
  6242. at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
  6243. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  6244. at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
  6245. at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
  6246. at java.sql.DriverManager.getConnection(DriverManager.java:664)
  6247. at java.sql.DriverManager.getConnection(DriverManager.java:208)
  6248. at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
  6249. at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
  6250. ... 94 more
  6251. Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@3234df5e, see the next exception for details.
  6252. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6253. at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
  6254. ... 110 more
  6255. Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/kiuchi/spias-elasticsearch/query/metastore_db.
  6256. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6257. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  6258. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source)
  6259. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source)
  6260. at java.security.AccessController.doPrivileged(Native Method)
  6261. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source)
  6262. at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
  6263. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6264. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6265. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6266. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6267. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6268. at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source)
  6269. at java.security.AccessController.doPrivileged(Native Method)
  6270. at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source)
  6271. at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
  6272. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6273. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6274. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6275. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6276. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6277. at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source)
  6278. at java.security.AccessController.doPrivileged(Native Method)
  6279. at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source)
  6280. at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
  6281. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6282. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6283. at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
  6284. at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
  6285. at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
  6286. at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
  6287. at java.security.AccessController.doPrivileged(Native Method)
  6288. at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
  6289. at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
  6290. at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
  6291. at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
  6292. at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
  6293. at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
  6294. at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
  6295. at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
  6296. at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
  6297. at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
  6298. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  6299. at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
  6300. at java.security.AccessController.doPrivileged(Native Method)
  6301. at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
  6302. ... 107 more
  6303. Traceback (most recent call last):
  6304. File "<stdin>", line 1, in <module>
  6305. File "/home/kiuchi/spark/python/pyspark/mllib/linalg/distributed.py", line 826, in entries
  6306. entries_df = callMLlibFunc("getMatrixEntries", self._java_matrix_wrapper._java_model)
  6307. File "/home/kiuchi/spark/python/pyspark/mllib/common.py", line 130, in callMLlibFunc
  6308. return callJavaFunc(sc, api, *args)
  6309. File "/home/kiuchi/spark/python/pyspark/mllib/common.py", line 123, in callJavaFunc
  6310. return _java2py(sc, func(*args))
  6311. File "/home/kiuchi/spark/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
  6312. File "/home/kiuchi/spark/python/pyspark/sql/utils.py", line 79, in deco
  6313. raise IllegalArgumentException(s.split(': ', 1)[1], stackTrace)
  6314. pyspark.sql.utils.IllegalArgumentException: u"Error while instantiating 'org.apache.spark.sql.hive.HiveSessionStateBuilder':"
  6315. >>>
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!