API tools faq
paste
Guest User

Cannot set Hive metastore directory

a guest
Dec 31st, 2015
271
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Java 26.29 KB | None | 0 0
raw download clone embed print report
  1. scala> import org.apache.spark.sql.hive.HiveContext
  2. import org.apache.spark.sql.hive.HiveContext
  3.  
  4. scala> import java.nio.file.Files
  5. import java.nio.file.Files
  6.  
  7. scala> val hiveDir = Files.createTempDirectory("hive")
  8. hiveDir: java.nio.file.Path = /var/folders/gg/g3hk6fcj4rxc6lb1qsvxc_vdxxwf28/T/hive5050481206678469338
  9.  
  10. scala> val hiveContext = new HiveContext(sc)
  11. 15/12/31 12:05:27 INFO HiveContext: Initializing execution hive, version 0.13.1
  12. hiveContext: org.apache.spark.sql.hive.HiveContext = org.apache.spark.sql.hive.HiveContext@6f959640
  13.  
  14. scala> hiveContext.sql(s"SET hive.metastore.warehouse.dir=${hiveDir.toUri}")
  15. 15/12/31 12:05:34 INFO HiveContext: Initializing HiveMetastoreConnection version 0.13.1 using Spark classes.
  16. 15/12/31 12:05:34 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  17. 15/12/31 12:05:34 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
  18. 15/12/31 12:05:34 INFO ObjectStore: ObjectStore, initialize called
  19. 15/12/31 12:05:34 INFO Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
  20. 15/12/31 12:05:34 INFO Persistence: Property datanucleus.cache.level2 unknown - will be ignored
  21. 15/12/31 12:05:34 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies)
  22. 15/12/31 12:05:34 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies)
  23. 15/12/31 12:05:35 INFO ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
  24. 15/12/31 12:05:35 INFO MetaStoreDirectSql: MySQL check failed, assuming we are not on mysql: Lexical error at line 1, column 5.  Encountered: "@" (64), after : "".
  25. 15/12/31 12:05:35 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
  26. 15/12/31 12:05:35 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
  27. 15/12/31 12:05:36 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
  28. 15/12/31 12:05:36 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
  29. 15/12/31 12:05:36 INFO ObjectStore: Initialized ObjectStore
  30. 15/12/31 12:05:36 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 0.13.1aa
  31. 15/12/31 12:05:36 INFO HiveMetaStore: Added admin role in metastore
  32. 15/12/31 12:05:36 INFO HiveMetaStore: Added public role in metastore
  33. 15/12/31 12:05:36 INFO HiveMetaStore: No user is added in admin role, since config is empty
  34. 15/12/31 12:05:36 INFO SessionState: No Tez session required at this point. hive.execution.engine=mr.
  35. res0: org.apache.spark.sql.DataFrame = [: string]
  36.  
  37. scala> Seq("create database foo").foreach(hiveContext.sql)
  38. 15/12/31 12:05:42 INFO ParseDriver: Parsing command: create database foo
  39. 15/12/31 12:05:42 INFO ParseDriver: Parse Completed
  40. 15/12/31 12:05:42 INFO PerfLogger: <PERFLOG method=Driver.run from=org.apache.hadoop.hive.ql.Driver>
  41. 15/12/31 12:05:42 INFO PerfLogger: <PERFLOG method=TimeToSubmit from=org.apache.hadoop.hive.ql.Driver>
  42. 15/12/31 12:05:42 INFO Driver: Concurrency mode is disabled, not creating a lock manager
  43. 15/12/31 12:05:42 INFO PerfLogger: <PERFLOG method=compile from=org.apache.hadoop.hive.ql.Driver>
  44. 15/12/31 12:05:42 INFO PerfLogger: <PERFLOG method=parse from=org.apache.hadoop.hive.ql.Driver>
  45. 15/12/31 12:05:42 INFO ParseDriver: Parsing command: create database foo
  46. 15/12/31 12:05:42 INFO ParseDriver: Parse Completed
  47. 15/12/31 12:05:42 INFO PerfLogger: </PERFLOG method=parse start=1451588742628 end=1451588742962 duration=334 from=org.apache.hadoop.hive.ql.Driver>
  48. 15/12/31 12:05:42 INFO PerfLogger: <PERFLOG method=semanticAnalyze from=org.apache.hadoop.hive.ql.Driver>
  49. 15/12/31 12:05:42 INFO Driver: Semantic Analysis Completed
  50. 15/12/31 12:05:42 INFO PerfLogger: </PERFLOG method=semanticAnalyze start=1451588742962 end=1451588742994 duration=32 from=org.apache.hadoop.hive.ql.Driver>
  51. 15/12/31 12:05:42 INFO Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null)
  52. 15/12/31 12:05:42 INFO PerfLogger: </PERFLOG method=compile start=1451588742610 end=1451588742999 duration=389 from=org.apache.hadoop.hive.ql.Driver>
  53. 15/12/31 12:05:42 INFO PerfLogger: <PERFLOG method=Driver.execute from=org.apache.hadoop.hive.ql.Driver>
  54. 15/12/31 12:05:42 INFO Driver: Starting command: create database foo
  55. 15/12/31 12:05:43 INFO PerfLogger: </PERFLOG method=TimeToSubmit start=1451588742609 end=1451588743014 duration=405 from=org.apache.hadoop.hive.ql.Driver>
  56. 15/12/31 12:05:43 INFO PerfLogger: <PERFLOG method=runTasks from=org.apache.hadoop.hive.ql.Driver>
  57. 15/12/31 12:05:43 INFO PerfLogger: <PERFLOG method=task.DDL.Stage-0 from=org.apache.hadoop.hive.ql.Driver>
  58. 15/12/31 12:05:43 INFO HiveMetaStore: 0: create_database: Database(name:foo, description:null, locationUri:null, parameters:null, ownerName:aa8y, ownerType:USER)
  59. 15/12/31 12:05:43 INFO audit: ugi=aa8y  ip=unknown-ip-addr  cmd=create_database: Database(name:foo, description:null, locationUri:null, parameters:null, ownerName:aa8y, ownerType:USER)
  60.  
  61. 15/12/31 12:05:43 INFO HiveMetaStore: 0: get_database: foo
  62. 15/12/31 12:05:43 INFO audit: ugi=aa8y  ip=unknown-ip-addr  cmd=get_database: foo
  63. 15/12/31 12:05:43 ERROR RetryingHMSHandler: MetaException(message:Unable to create database path file:/user/hive/warehouse/foo.db, failed to create database foo)
  64.     at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database_core(HiveMetaStore.java:734)
  65.     at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database(HiveMetaStore.java:771)
  66.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  67.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  68.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  69.     at java.lang.reflect.Method.invoke(Method.java:497)
  70.     at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
  71.     at com.sun.proxy.$Proxy22.create_database(Unknown Source)
  72.     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createDatabase(HiveMetaStoreClient.java:534)
  73.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  74.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  75.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  76.     at java.lang.reflect.Method.invoke(Method.java:497)
  77.     at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
  78.     at com.sun.proxy.$Proxy23.createDatabase(Unknown Source)
  79.     at org.apache.hadoop.hive.ql.metadata.Hive.createDatabase(Hive.java:242)
  80.     at org.apache.hadoop.hive.ql.exec.DDLTask.createDatabase(DDLTask.java:3973)
  81.     at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:246)
  82.     at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
  83.     at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
  84.     at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1503)
  85.     at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1270)
  86.     at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1088)
  87.     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911)
  88.     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901)
  89.     at org.apache.spark.sql.hive.client.ClientWrapper$$anonfun$runHive$1.apply(ClientWrapper.scala:345)
  90.     at org.apache.spark.sql.hive.client.ClientWrapper$$anonfun$runHive$1.apply(ClientWrapper.scala:326)
  91.     at org.apache.spark.sql.hive.client.ClientWrapper.withHiveState(ClientWrapper.scala:155)
  92.     at org.apache.spark.sql.hive.client.ClientWrapper.runHive(ClientWrapper.scala:326)
  93.     at org.apache.spark.sql.hive.client.ClientWrapper.runSqlHive(ClientWrapper.scala:316)
  94.     at org.apache.spark.sql.hive.HiveContext.runSqlHive(HiveContext.scala:473)
  95.     at org.apache.spark.sql.hive.execution.HiveNativeCommand.run(HiveNativeCommand.scala:33)
  96.     at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:57)
  97.     at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:57)
  98.     at org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:68)
  99.     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88)
  100.     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88)
  101.     at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147)
  102.     at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:87)
  103.     at org.apache.spark.sql.SQLContext$QueryExecution.toRdd$lzycompute(SQLContext.scala:950)
  104.     at org.apache.spark.sql.SQLContext$QueryExecution.toRdd(SQLContext.scala:950)
  105.     at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:144)
  106.     at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:128)
  107.     at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:51)
  108.     at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:755)
  109.     at $line23.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$anonfun$1.apply(<console>:26)
  110.     at $line23.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$anonfun$1.apply(<console>:26)
  111.     at scala.collection.immutable.List.foreach(List.scala:318)
  112.     at $line23.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:26)
  113.     at $line23.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:31)
  114.     at $line23.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:33)
  115.     at $line23.$read$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:35)
  116.     at $line23.$read$$iwC$$iwC$$iwC$$iwC.<init>(<console>:37)
  117.     at $line23.$read$$iwC$$iwC$$iwC.<init>(<console>:39)
  118.     at $line23.$read$$iwC$$iwC.<init>(<console>:41)
  119.     at $line23.$read$$iwC.<init>(<console>:43)
  120.     at $line23.$read.<init>(<console>:45)
  121.     at $line23.$read$.<init>(<console>:49)
  122.     at $line23.$read$.<clinit>(<console>)
  123.     at $line23.$eval$.<init>(<console>:7)
  124.     at $line23.$eval$.<clinit>(<console>)
  125.     at $line23.$eval.$print(<console>)
  126.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  127.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  128.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  129.     at java.lang.reflect.Method.invoke(Method.java:497)
  130.     at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)
  131.     at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1338)
  132.     at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)
  133.     at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)
  134.     at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)
  135.     at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)
  136.     at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)
  137.     at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)
  138.     at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:657)
  139.     at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:665)
  140.     at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$loop(SparkILoop.scala:670)
  141.     at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:997)
  142.     at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
  143.     at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
  144.     at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
  145.     at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)
  146.     at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)
  147.     at org.apache.spark.repl.Main$.main(Main.scala:31)
  148.     at org.apache.spark.repl.Main.main(Main.scala)
  149.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  150.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  151.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  152.     at java.lang.reflect.Method.invoke(Method.java:497)
  153.     at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:665)
  154.     at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:170)
  155.     at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:193)
  156.     at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:112)
  157.     at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  158.  
  159. 15/12/31 12:05:43 ERROR DDLTask: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Unable to create database path file:/user/hive/warehouse/foo.db, failed to create database foo)
  160.     at org.apache.hadoop.hive.ql.metadata.Hive.createDatabase(Hive.java:248)
  161.     at org.apache.hadoop.hive.ql.exec.DDLTask.createDatabase(DDLTask.java:3973)
  162.     at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:246)
  163.     at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
  164.     at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
  165.     at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1503)
  166.     at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1270)
  167.     at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1088)
  168.     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911)
  169.     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901)
  170.     at org.apache.spark.sql.hive.client.ClientWrapper$$anonfun$runHive$1.apply(ClientWrapper.scala:345)
  171.     at org.apache.spark.sql.hive.client.ClientWrapper$$anonfun$runHive$1.apply(ClientWrapper.scala:326)
  172.     at org.apache.spark.sql.hive.client.ClientWrapper.withHiveState(ClientWrapper.scala:155)
  173.     at org.apache.spark.sql.hive.client.ClientWrapper.runHive(ClientWrapper.scala:326)
  174.     at org.apache.spark.sql.hive.client.ClientWrapper.runSqlHive(ClientWrapper.scala:316)
  175.     at org.apache.spark.sql.hive.HiveContext.runSqlHive(HiveContext.scala:473)
  176.     at org.apache.spark.sql.hive.execution.HiveNativeCommand.run(HiveNativeCommand.scala:33)
  177.     at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:57)
  178.     at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:57)
  179.     at org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:68)
  180.     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88)
  181.     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88)
  182.     at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147)
  183.     at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:87)
  184.     at org.apache.spark.sql.SQLContext$QueryExecution.toRdd$lzycompute(SQLContext.scala:950)
  185.     at org.apache.spark.sql.SQLContext$QueryExecution.toRdd(SQLContext.scala:950)
  186.     at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:144)
  187.     at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:128)
  188.     at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:51)
  189.     at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:755)
  190.     at $line23.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$anonfun$1.apply(<console>:26)
  191.     at $line23.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$anonfun$1.apply(<console>:26)
  192.     at scala.collection.immutable.List.foreach(List.scala:318)
  193.     at $line23.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:26)
  194.     at $line23.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:31)
  195.     at $line23.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:33)
  196.     at $line23.$read$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:35)
  197.     at $line23.$read$$iwC$$iwC$$iwC$$iwC.<init>(<console>:37)
  198.     at $line23.$read$$iwC$$iwC$$iwC.<init>(<console>:39)
  199.     at $line23.$read$$iwC$$iwC.<init>(<console>:41)
  200.     at $line23.$read$$iwC.<init>(<console>:43)
  201.     at $line23.$read.<init>(<console>:45)
  202.     at $line23.$read$.<init>(<console>:49)
  203.     at $line23.$read$.<clinit>(<console>)
  204.     at $line23.$eval$.<init>(<console>:7)
  205.     at $line23.$eval$.<clinit>(<console>)
  206.     at $line23.$eval.$print(<console>)
  207.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  208.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  209.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  210.     at java.lang.reflect.Method.invoke(Method.java:497)
  211.     at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)
  212.     at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1338)
  213.     at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)
  214.     at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)
  215.     at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)
  216.     at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)
  217.     at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)
  218.     at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)
  219.     at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:657)
  220.     at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:665)
  221.     at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$loop(SparkILoop.scala:670)
  222.     at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:997)
  223.     at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
  224.     at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
  225.     at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
  226.     at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)
  227.     at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)
  228.     at org.apache.spark.repl.Main$.main(Main.scala:31)
  229.     at org.apache.spark.repl.Main.main(Main.scala)
  230.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  231.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  232.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  233.     at java.lang.reflect.Method.invoke(Method.java:497)
  234.     at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:665)
  235.     at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:170)
  236.     at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:193)
  237.     at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:112)
  238.     at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  239. Caused by: MetaException(message:Unable to create database path file:/user/hive/warehouse/foo.db, failed to create database foo)
  240.     at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database_core(HiveMetaStore.java:734)
  241.     at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database(HiveMetaStore.java:771)
  242.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  243.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  244.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  245.     at java.lang.reflect.Method.invoke(Method.java:497)
  246.     at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
  247.     at com.sun.proxy.$Proxy22.create_database(Unknown Source)
  248.     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createDatabase(HiveMetaStoreClient.java:534)
  249.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  250.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  251.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  252.     at java.lang.reflect.Method.invoke(Method.java:497)
  253.     at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
  254.     at com.sun.proxy.$Proxy23.createDatabase(Unknown Source)
  255.     at org.apache.hadoop.hive.ql.metadata.Hive.createDatabase(Hive.java:242)
  256.     ... 78 more
  257.  
  258. 15/12/31 12:05:43 ERROR Driver: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Unable to create database path file:/user/hive/warehouse/foo.db, failed to create database foo)
  259. 15/12/31 12:05:43 INFO PerfLogger: </PERFLOG method=Driver.execute start=1451588742999 end=1451588743088 duration=89 from=org.apache.hadoop.hive.ql.Driver>
  260. 15/12/31 12:05:43 INFO PerfLogger: <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
  261. 15/12/31 12:05:43 INFO PerfLogger: </PERFLOG method=releaseLocks start=1451588743089 end=1451588743089 duration=0 from=org.apache.hadoop.hive.ql.Driver>
  262. 15/12/31 12:05:43 ERROR ClientWrapper:
  263. ======================
  264. HIVE FAILURE OUTPUT
  265. ======================
  266. SET hive.metastore.warehouse.dir=file:///var/folders/gg/g3hk6fcj4rxc6lb1qsvxc_vdxxwf28/T/hive5050481206678469338/
  267. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Unable to create database path file:/user/hive/warehouse/foo.db, failed to create database foo)
  268.  
  269. ======================
  270. END HIVE FAILURE OUTPUT
  271. ======================
  272.  
  273. org.apache.spark.sql.execution.QueryExecutionException: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Unable to create database path file:/user/hive/warehouse/foo.db, failed to create database foo)
  274.     at org.apache.spark.sql.hive.client.ClientWrapper$$anonfun$runHive$1.apply(ClientWrapper.scala:349)
  275.     at org.apache.spark.sql.hive.client.ClientWrapper$$anonfun$runHive$1.apply(ClientWrapper.scala:326)
  276.     at org.apache.spark.sql.hive.client.ClientWrapper.withHiveState(ClientWrapper.scala:155)
  277.     at org.apache.spark.sql.hive.client.ClientWrapper.runHive(ClientWrapper.scala:326)
  278.     at org.apache.spark.sql.hive.client.ClientWrapper.runSqlHive(ClientWrapper.scala:316)
  279.     at org.apache.spark.sql.hive.HiveContext.runSqlHive(HiveContext.scala:473)
  280.     at org.apache.spark.sql.hive.execution.HiveNativeCommand.run(HiveNativeCommand.scala:33)
  281.     at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:57)
  282.     at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:57)
  283.     at org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:68)
  284.     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88)
  285.     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88)
  286.     at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147)
  287.     at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:87)
  288.     at org.apache.spark.sql.SQLContext$QueryExecution.toRdd$lzycompute(SQLContext.scala:950)
  289.     at org.apache.spark.sql.SQLContext$QueryExecution.toRdd(SQLContext.scala:950)
  290.     at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:144)
  291.     at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:128)
  292.     at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:51)
  293.     at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:755)
  294.     at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$anonfun$1.apply(<console>:26)
  295.     at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$anonfun$1.apply(<console>:26)
  296.     at scala.collection.immutable.List.foreach(List.scala:318)
  297.     at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:26)
  298.     at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:31)
  299.     at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:33)
  300.     at $iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:35)
  301.     at $iwC$$iwC$$iwC$$iwC.<init>(<console>:37)
  302.     at $iwC$$iwC$$iwC.<init>(<console>:39)
  303.     at $iwC$$iwC.<init>(<console>:41)
  304.     at $iwC.<init>(<console>:43)
  305.     at <init>(<console>:45)
  306.     at .<init>(<console>:49)
  307.     at .<clinit>(<console>)
  308.     at .<init>(<console>:7)
  309.     at .<clinit>(<console>)
  310.     at $print(<console>)
  311.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  312.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  313.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  314.     at java.lang.reflect.Method.invoke(Method.java:497)
  315.     at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)
  316.     at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1338)
  317.     at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)
  318.     at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)
  319.     at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)
  320.     at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)
  321.     at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)
  322.     at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)
  323.     at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:657)
  324.     at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:665)
  325.     at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$loop(SparkILoop.scala:670)
  326.     at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:997)
  327.     at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
  328.     at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
  329.     at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
  330.     at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)
  331.     at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)
  332.     at org.apache.spark.repl.Main$.main(Main.scala:31)
  333.     at org.apache.spark.repl.Main.main(Main.scala)
  334.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  335.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  336.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  337.     at java.lang.reflect.Method.invoke(Method.java:497)
  338.     at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:665)
  339.     at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:170)
  340.     at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:193)
  341.     at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:112)
  342.     at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  343.  
  344.  
  345. scala>
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!