API tools faq
paste
Advertisement
Guest User

Untitled

a guest
May 12th, 2016
136
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 17.55 KB | None | 0 0
raw download clone embed print report
  1. 16/05/12 01:07:20 INFO SparkILoop: Created spark context..
  2. Spark context available as sc.
  3. 16/05/12 01:07:21 INFO HiveContext: Initializing execution hive, version 1.2.1
  4. 16/05/12 01:07:21 INFO ClientWrapper: Inspected Hadoop version: 2.7.1.2.4.0.0-169
  5. 16/05/12 01:07:21 INFO ClientWrapper: Loaded org.apache.hadoop.hive.shims.Hadoop23Shims for Hadoop version 2.7.1.2.4.0.0-169
  6. 16/05/12 01:07:21 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
  7. 16/05/12 01:07:21 INFO ObjectStore: ObjectStore, initialize called
  8. 16/05/12 01:07:21 INFO Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
  9. 16/05/12 01:07:21 INFO Persistence: Property datanucleus.cache.level2 unknown - will be ignored
  10. 16/05/12 01:07:23 INFO ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
  11. 16/05/12 01:07:24 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
  12. 16/05/12 01:07:24 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
  13. 16/05/12 01:07:25 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
  14. 16/05/12 01:07:25 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
  15. 16/05/12 01:07:25 INFO MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
  16. 16/05/12 01:07:25 INFO ObjectStore: Initialized ObjectStore
  17. 16/05/12 01:07:25 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0
  18. 16/05/12 01:07:25 WARN ObjectStore: Failed to get database default, returning NoSuchObjectException
  19. 16/05/12 01:07:26 INFO HiveMetaStore: Added admin role in metastore
  20. 16/05/12 01:07:26 INFO HiveMetaStore: Added public role in metastore
  21. 16/05/12 01:07:26 WARN Hive: Failed to access metastore. This class should not accessed in runtime.
  22. org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  23. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1236)
  24. at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
  25. at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
  26. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  27. at org.apache.spark.sql.hive.client.ClientWrapper.<init>(ClientWrapper.scala:194)
  28. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:238)
  29. at org.apache.spark.sql.hive.HiveContext.executionHive$lzycompute(HiveContext.scala:218)
  30. at org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala:208)
  31. at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:462)
  32. at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:461)
  33. at org.apache.spark.sql.UDFRegistration.<init>(UDFRegistration.scala:40)
  34. at org.apache.spark.sql.SQLContext.<init>(SQLContext.scala:330)
  35. at org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:90)
  36. at org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:101)
  37. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  38. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  39. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  40. at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
  41. at org.apache.spark.repl.SparkILoop.createSQLContext(SparkILoop.scala:1028)
  42. at $line4.$read$$iwC$$iwC.<init>(<console>:15)
  43. at $line4.$read$$iwC.<init>(<console>:24)
  44. at $line4.$read.<init>(<console>:26)
  45. at $line4.$read$.<init>(<console>:30)
  46. at $line4.$read$.<clinit>(<console>)
  47. at $line4.$eval$.<init>(<console>:7)
  48. at $line4.$eval$.<clinit>(<console>)
  49. at $line4.$eval.$print(<console>)
  50. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  51. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  52. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  53. at java.lang.reflect.Method.invoke(Method.java:497)
  54. at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)
  55. at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1346)
  56. at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)
  57. at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)
  58. at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)
  59. at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)
  60. at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)
  61. at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)
  62. at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:132)
  63. at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:124)
  64. at org.apache.spark.repl.SparkIMain.beQuietDuring(SparkIMain.scala:324)
  65. at org.apache.spark.repl.SparkILoopInit$class.initializeSpark(SparkILoopInit.scala:124)
  66. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:64)
  67. at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1$$anonfun$apply$mcZ$sp$5.apply$mcV$sp(SparkILoop.scala:974)
  68. at org.apache.spark.repl.SparkILoopInit$class.runThunks(SparkILoopInit.scala:159)
  69. at org.apache.spark.repl.SparkILoop.runThunks(SparkILoop.scala:64)
  70. at org.apache.spark.repl.SparkILoopInit$class.postInitialization(SparkILoopInit.scala:108)
  71. at org.apache.spark.repl.SparkILoop.postInitialization(SparkILoop.scala:64)
  72. at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:991)
  73. at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
  74. at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
  75. at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
  76. at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)
  77. at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)
  78. at org.apache.spark.repl.Main$.main(Main.scala:31)
  79. at org.apache.spark.repl.Main.main(Main.scala)
  80. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  81. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  82. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  83. at java.lang.reflect.Method.invoke(Method.java:497)
  84. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
  85. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
  86. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
  87. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
  88. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  89. Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  90. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
  91. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  92. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  93. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  94. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  95. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  96. at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
  97. ... 65 more
  98. Caused by: java.lang.reflect.InvocationTargetException
  99. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  100. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  101. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  102. at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
  103. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  104. ... 71 more
  105. Caused by: java.lang.NoSuchMethodError: org.apache.thrift.EncodingUtils.setBit(BIZ)B
  106. at org.apache.hadoop.hive.metastore.api.PrivilegeGrantInfo.setCreateTimeIsSet(PrivilegeGrantInfo.java:245)
  107. at org.apache.hadoop.hive.metastore.api.PrivilegeGrantInfo.<init>(PrivilegeGrantInfo.java:163)
  108. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultRoles_core(HiveMetaStore.java:675)
  109. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultRoles(HiveMetaStore.java:645)
  110. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:462)
  111. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  112. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  113. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  114. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  115. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  116. ... 76 more
  117. 16/05/12 01:07:26 INFO ObjectStore: ObjectStore, initialize called
  118. 16/05/12 01:07:26 INFO Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing
  119. 16/05/12 01:07:26 INFO MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
  120. 16/05/12 01:07:26 INFO ObjectStore: Initialized ObjectStore
  121. 16/05/12 01:07:26 INFO HiveMetaStore: Added admin role in metastore
  122. 16/05/12 01:07:26 INFO HiveMetaStore: Added public role in metastore
  123. java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  124. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
  125. at org.apache.spark.sql.hive.client.ClientWrapper.<init>(ClientWrapper.scala:194)
  126. at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:238)
  127. at org.apache.spark.sql.hive.HiveContext.executionHive$lzycompute(HiveContext.scala:218)
  128. at org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala:208)
  129. at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(HiveContext.scala:462)
  130. at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.scala:461)
  131. at org.apache.spark.sql.UDFRegistration.<init>(UDFRegistration.scala:40)
  132. at org.apache.spark.sql.SQLContext.<init>(SQLContext.scala:330)
  133. at org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:90)
  134. at org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:101)
  135. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  136. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  137. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  138. at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
  139. at org.apache.spark.repl.SparkILoop.createSQLContext(SparkILoop.scala:1028)
  140. at $iwC$$iwC.<init>(<console>:15)
  141. at $iwC.<init>(<console>:24)
  142. at <init>(<console>:26)
  143. at .<init>(<console>:30)
  144. at .<clinit>(<console>)
  145. at .<init>(<console>:7)
  146. at .<clinit>(<console>)
  147. at $print(<console>)
  148. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  149. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  150. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  151. at java.lang.reflect.Method.invoke(Method.java:497)
  152. at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)
  153. at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1346)
  154. at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)
  155. at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)
  156. at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)
  157. at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)
  158. at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)
  159. at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)
  160. at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:132)
  161. at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:124)
  162. at org.apache.spark.repl.SparkIMain.beQuietDuring(SparkIMain.scala:324)
  163. at org.apache.spark.repl.SparkILoopInit$class.initializeSpark(SparkILoopInit.scala:124)
  164. at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:64)
  165. at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1$$anonfun$apply$mcZ$sp$5.apply$mcV$sp(SparkILoop.scala:974)
  166. at org.apache.spark.repl.SparkILoopInit$class.runThunks(SparkILoopInit.scala:159)
  167. at org.apache.spark.repl.SparkILoop.runThunks(SparkILoop.scala:64)
  168. at org.apache.spark.repl.SparkILoopInit$class.postInitialization(SparkILoopInit.scala:108)
  169. at org.apache.spark.repl.SparkILoop.postInitialization(SparkILoop.scala:64)
  170. at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:991)
  171. at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
  172. at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
  173. at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
  174. at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)
  175. at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)
  176. at org.apache.spark.repl.Main$.main(Main.scala:31)
  177. at org.apache.spark.repl.Main.main(Main.scala)
  178. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  179. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  180. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  181. at java.lang.reflect.Method.invoke(Method.java:497)
  182. at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
  183. at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
  184. at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
  185. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
  186. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  187. Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  188. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
  189. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  190. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  191. at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  192. at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
  193. at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
  194. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
  195. ... 62 more
  196. Caused by: java.lang.reflect.InvocationTargetException
  197. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  198. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  199. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  200. at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
  201. at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
  202. ... 68 more
  203. Caused by: java.lang.NoSuchMethodError: org.apache.thrift.EncodingUtils.setBit(BIZ)B
  204. at org.apache.hadoop.hive.metastore.api.PrivilegeGrantInfo.setCreateTimeIsSet(PrivilegeGrantInfo.java:245)
  205. at org.apache.hadoop.hive.metastore.api.PrivilegeGrantInfo.<init>(PrivilegeGrantInfo.java:163)
  206. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultRoles_core(HiveMetaStore.java:675)
  207. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultRoles(HiveMetaStore.java:645)
  208. at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:462)
  209. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
  210. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
  211. at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  212. at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
  213. at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
  214. ... 73 more
  215.  
  216. <console>:16: error: not found: value sqlContext
  217. import sqlContext.implicits._
  218. ^
  219. <console>:16: error: not found: value sqlContext
  220. import sqlContext.sql
  221. ^
  222.  
  223. scala>
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!