Advertisement
Guest User

Untitled

a guest
Feb 11th, 2016
54
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 7.20 KB | None | 0 0
  1. hive> create external table frb_test (recvTime string, fiwareServicePath string, entityId string, entityType string, `ORL.SOU.DH.SSTA10.T.HVAC.HeatLoad` string, `ORL.SOU.DH.SSTA10.T.HVAC.HeatLoad_md` array<struct<name:string,type:string,value:string>>) row format serde 'org.openx.data.jsonserde.JsonSerDe' location '/user/frb/test';
  2. OK
  3. Time taken: 0.286 seconds
  4.  
  5. $ hadoop fs -cat /user/frb/test/deleteme
  6. {"recvTime":"2016-02-09T18:03:48.986Z","fiwareServicePath":"orl_sou","entityId":"ORL.SOU.DH.SSTA10","entityType":"ETS", "ORL.SOU.DH.SSTA10.T.HVAC.HeatLoad":"10.673299789428711", "ORL.SOU.DH.SSTA10.T.HVAC.HeatLoad_md":[{"name":"dofTimestamp","type":"ms","value":"2016-02-08T23:00:00.000Z"},{"name":"tag","type":"text","value":"ORL.SOU.DH.SSTA10.T.HVAC.HeatLoad"},{"name":"description","type":"text","value":"Electrical heat load"},{"name":"quality","type":"0:GOOD, +0:ERROR","value":"10813440"},{"name":"max","type":"max","value":"null"},{"name":"min","type":"min","value":"null"},{"name":"lcl","type":"lcl","value":"null"},{"name":"ucl","type":"ucl","value":"null"}]}
  7.  
  8. hive> add jar /home/frb/json-serde-1.3.7-jar-with-dependencies.jar;
  9. hive> select `orl.sou.dh.ssta10.t.hvac.heatload` from frb_test; Total jobs = 1
  10. Launching Job 1 out of 1
  11. Number of reduce tasks is set to 0 since there's no reduce operator
  12. Starting Job = job_1455032234756_0008, Tracking URL = http://namenode.fiware.org:8088/proxy/application_1455032234756_0008/
  13. Kill Command = /opt/cloudera/parcels/CDH-5.3.2-1.cdh5.3.2.p0.10/lib/hadoop/bin/hadoop job -kill job_1455032234756_0008
  14. Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
  15. 2016-02-11 17:05:56,150 Stage-1 map = 0%, reduce = 0%
  16. 2016-02-11 17:06:23,653 Stage-1 map = 100%, reduce = 0%
  17. Ended Job = job_1455032234756_0008 with errors
  18. Error during job, obtaining debugging information...
  19. Examining task ID: task_1455032234756_0008_m_000000 (and more) from job job_1455032234756_0008
  20.  
  21. Task with the most failures(4):
  22. -----
  23. Task ID:
  24. task_1455032234756_0008_m_000000
  25.  
  26. URL:
  27. http://namenode.fiware.org:8088/taskdetails.jsp?jobid=job_1455032234756_0008&tipid=task_1455032234756_0008_m_000000
  28. -----
  29. Diagnostic Messages for this Task:
  30. Error: java.lang.RuntimeException: Error in configuring object
  31. at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
  32. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
  33. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
  34. at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:446)
  35. at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  36. at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
  37. at java.security.AccessController.doPrivileged(Native Method)
  38. at javax.security.auth.Subject.doAs(Subject.java:416)
  39. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
  40. at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
  41. Caused by: java.lang.reflect.InvocationTargetException
  42. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  43. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  44. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  45. at java.lang.reflect.Method.invoke(Method.java:616)
  46. at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
  47. ... 9 more
  48. Caused by: java.lang.RuntimeException: Error in configuring object
  49. at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
  50. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
  51. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
  52. at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
  53. ... 14 more
  54. Caused by: java.lang.reflect.InvocationTargetException
  55. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  56. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  57. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  58. at java.lang.reflect.Method.invoke(Method.java:616)
  59. at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
  60. ... 17 more
  61. Caused by: java.lang.RuntimeException: Map operator initialization failed
  62. at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:157)
  63. ... 22 more
  64. Caused by: java.lang.RuntimeException: cannot find field orl from [0:recvtime, 1:fiwareservicepath, 2:entityid, 3:entitytype, 4:orl.sou.dh.ssta10.t.hvac.heatload, 5:orl.sou.dh.ssta10.t.hvac.heatload_md]
  65. at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:415)
  66. at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:150)
  67. at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:79)
  68. at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:934)
  69. at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:960)
  70. at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65)
  71. at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
  72. at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
  73. at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
  74. at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189)
  75. at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
  76. at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:424)
  77. at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
  78. at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:136)
  79. ... 22 more
  80.  
  81.  
  82. FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
  83. MapReduce Jobs Launched:
  84. Stage-Stage-1: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
  85. Total MapReduce CPU Time Spent: 0 msec
  86.  
  87. hive> set hive.support.quoted.identifiers=column;
  88. hive> select `orl.sou.dh.ssta10.t.hvac.heatload` from frb_test;
  89. Total jobs = 1
  90. Launching Job 1 out of 1
  91. Number of reduce tasks is set to 0 since there's no reduce operator
  92. Starting Job = job_1455032234756_0009, Tracking URL = http://namenode.fiware.org:8088/proxy/application_1455032234756_0009/
  93. Kill Command = /opt/cloudera/parcels/CDH-5.3.2-1.cdh5.3.2.p0.10/lib/hadoop/bin/hadoop job -kill job_1455032234756_0009
  94. ...
  95. Caused by: java.lang.RuntimeException: cannot find field orl from [0:recvtime, 1:fiwareservicepath, 2:entityid, 3:entitytype, 4:orl.sou.dh.ssta10.t.hvac.heatload, 5:orl.sou.dh.ssta10.t.hvac.heatload_md]
  96. ...
  97. FAILED: Execution Error, return code 2 from
  98. org.apache.hadoop.hive.ql.exec.mr.MapRedTask
  99. MapReduce Jobs Launched:
  100. Stage-Stage-1: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
  101. Total MapReduce CPU Time Spent: 0 msec
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement