API tools faq
paste
Guest User

Untitled

a guest
Sep 30th, 2016
184
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 10.83 KB | None | 0 0
raw download clone embed print report
  1. 16/09/30 14:42:58 WARN streaming.StreamJob: -file option is deprecated, please use generic option -files instead.
  2. packageJobJar: [/root/LAB3/mapper.py, /root/LAB3/reducer.py] [/usr/hdp/2.5.0.0-1245/hadoop-mapreduce/hadoop-streaming-2.7.3.2.5.0.0-1245.jar] /tmp/streamjob2382160873678624642.jar tmpDir=null
  3. 16/09/30 14:42:59 INFO impl.TimelineClientImpl: Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
  4. 16/09/30 14:42:59 INFO client.RMProxy: Connecting to ResourceManager at sandbox.hortonworks.com/172.17.0.2:8050
  5. 16/09/30 14:42:59 INFO client.AHSProxy: Connecting to Application History server at sandbox.hortonworks.com/172.17.0.2:10200
  6. 16/09/30 14:42:59 INFO impl.TimelineClientImpl: Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
  7. 16/09/30 14:42:59 INFO client.RMProxy: Connecting to ResourceManager at sandbox.hortonworks.com/172.17.0.2:8050
  8. 16/09/30 14:42:59 INFO client.AHSProxy: Connecting to Application History server at sandbox.hortonworks.com/172.17.0.2:10200
  9. 16/09/30 14:43:00 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
  10. 16/09/30 14:43:00 INFO lzo.LzoCodec: Successfully loaded & initialized native-lzo library [hadoop-lzo rev 7a4b57bedce694048432dd5bf5b90a6c8ccdba80]
  11. 16/09/30 14:43:00 INFO mapred.FileInputFormat: Total input paths to process : 1
  12. 16/09/30 14:43:00 INFO mapreduce.JobSubmitter: number of splits:2
  13. 16/09/30 14:43:00 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1475243242823_0007
  14. 16/09/30 14:43:01 INFO impl.YarnClientImpl: Submitted application application_1475243242823_0007
  15. 16/09/30 14:43:01 INFO mapreduce.Job: The url to track the job: http://sandbox.hortonworks.com:8088/proxy/application_1475243242823_0007/
  16. 16/09/30 14:43:01 INFO mapreduce.Job: Running job: job_1475243242823_0007
  17. 16/09/30 14:43:08 INFO mapreduce.Job: Job job_1475243242823_0007 running in uber mode : false
  18. 16/09/30 14:43:08 INFO mapreduce.Job: map 0% reduce 0%
  19. 16/09/30 14:43:19 INFO mapreduce.Job: map 100% reduce 0%
  20. 16/09/30 14:43:23 INFO mapreduce.Job: Task Id : attempt_1475243242823_0007_r_000000_0, Status : FAILED
  21. Error: java.lang.RuntimeException: Error in configuring object
  22. at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
  23. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
  24. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  25. at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:409)
  26. at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
  27. at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
  28. at java.security.AccessController.doPrivileged(Native Method)
  29. at javax.security.auth.Subject.doAs(Subject.java:415)
  30. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
  31. at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
  32. Caused by: java.lang.reflect.InvocationTargetException
  33. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  34. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  35. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  36. at java.lang.reflect.Method.invoke(Method.java:606)
  37. at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
  38. ... 9 more
  39. Caused by: java.lang.RuntimeException: configuration exception
  40. at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222)
  41. at org.apache.hadoop.streaming.PipeReducer.configure(PipeReducer.java:67)
  42. ... 14 more
  43. Caused by: java.io.IOException: Cannot run program "/hadoop/yarn/local/usercache/root/appcache/application_1475243242823_0007/container_1475243242823_0007_01_000004/./reducer.py": error=2, No such file or directory
  44. at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
  45. at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209)
  46. ... 15 more
  47. Caused by: java.io.IOException: error=2, No such file or directory
  48. at java.lang.UNIXProcess.forkAndExec(Native Method)
  49. at java.lang.UNIXProcess.<init>(UNIXProcess.java:187)
  50. at java.lang.ProcessImpl.start(ProcessImpl.java:130)
  51. at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
  52. ... 16 more
  53.  
  54. Container killed by the ApplicationMaster.
  55. Container killed on request. Exit code is 143
  56. Container exited with a non-zero exit code 143
  57.  
  58. 16/09/30 14:43:27 INFO mapreduce.Job: Task Id : attempt_1475243242823_0007_r_000000_1, Status : FAILED
  59. Error: java.lang.RuntimeException: Error in configuring object
  60. at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
  61. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
  62. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  63. at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:409)
  64. at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
  65. at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
  66. at java.security.AccessController.doPrivileged(Native Method)
  67. at javax.security.auth.Subject.doAs(Subject.java:415)
  68. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
  69. at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
  70. Caused by: java.lang.reflect.InvocationTargetException
  71. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  72. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  73. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  74. at java.lang.reflect.Method.invoke(Method.java:606)
  75. at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
  76. ... 9 more
  77. Caused by: java.lang.RuntimeException: configuration exception
  78. at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222)
  79. at org.apache.hadoop.streaming.PipeReducer.configure(PipeReducer.java:67)
  80. ... 14 more
  81. Caused by: java.io.IOException: Cannot run program "/hadoop/yarn/local/usercache/root/appcache/application_1475243242823_0007/container_1475243242823_0007_01_000005/./reducer.py": error=2, No such file or directory
  82. at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
  83. at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209)
  84. ... 15 more
  85. Caused by: java.io.IOException: error=2, No such file or directory
  86. at java.lang.UNIXProcess.forkAndExec(Native Method)
  87. at java.lang.UNIXProcess.<init>(UNIXProcess.java:187)
  88. at java.lang.ProcessImpl.start(ProcessImpl.java:130)
  89. at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
  90. ... 16 more
  91.  
  92. Container killed by the ApplicationMaster.
  93. Container killed on request. Exit code is 143
  94. Container exited with a non-zero exit code 143
  95.  
  96. 16/09/30 14:43:31 INFO mapreduce.Job: Task Id : attempt_1475243242823_0007_r_000000_2, Status : FAILED
  97. Error: java.lang.RuntimeException: Error in configuring object
  98. at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
  99. at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
  100. at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
  101. at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:409)
  102. at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
  103. at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
  104. at java.security.AccessController.doPrivileged(Native Method)
  105. at javax.security.auth.Subject.doAs(Subject.java:415)
  106. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
  107. at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
  108. Caused by: java.lang.reflect.InvocationTargetException
  109. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  110. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  111. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  112. at java.lang.reflect.Method.invoke(Method.java:606)
  113. at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
  114. ... 9 more
  115. Caused by: java.lang.RuntimeException: configuration exception
  116. at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222)
  117. at org.apache.hadoop.streaming.PipeReducer.configure(PipeReducer.java:67)
  118. ... 14 more
  119. Caused by: java.io.IOException: Cannot run program "/hadoop/yarn/local/usercache/root/appcache/application_1475243242823_0007/container_1475243242823_0007_01_000006/./reducer.py": error=2, No such file or directory
  120. at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
  121. at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209)
  122. ... 15 more
  123. Caused by: java.io.IOException: error=2, No such file or directory
  124. at java.lang.UNIXProcess.forkAndExec(Native Method)
  125. at java.lang.UNIXProcess.<init>(UNIXProcess.java:187)
  126. at java.lang.ProcessImpl.start(ProcessImpl.java:130)
  127. at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
  128. ... 16 more
  129.  
  130. Container killed by the ApplicationMaster.
  131. Container killed on request. Exit code is 143
  132. Container exited with a non-zero exit code 143
  133.  
  134. 16/09/30 14:43:36 INFO mapreduce.Job: map 100% reduce 100%
  135. 16/09/30 14:43:37 INFO mapreduce.Job: Job job_1475243242823_0007 failed with state FAILED due to: Task failed task_1475243242823_0007_r_000000
  136. Job failed as tasks failed. failedMaps:0 failedReduces:1
  137.  
  138. 16/09/30 14:43:37 INFO mapreduce.Job: Counters: 37
  139. File System Counters
  140. FILE: Number of bytes read=0
  141. FILE: Number of bytes written=726173
  142. FILE: Number of read operations=0
  143. FILE: Number of large read operations=0
  144. FILE: Number of write operations=0
  145. HDFS: Number of bytes read=492917
  146. HDFS: Number of bytes written=0
  147. HDFS: Number of read operations=6
  148. HDFS: Number of large read operations=0
  149. HDFS: Number of write operations=0
  150. Job Counters
  151. Failed reduce tasks=4
  152. Launched map tasks=2
  153. Launched reduce tasks=4
  154. Data-local map tasks=2
  155. Total time spent by all maps in occupied slots (ms)=18930
  156. Total time spent by all reduces in occupied slots (ms)=8522
  157. Total time spent by all map tasks (ms)=18930
  158. Total time spent by all reduce tasks (ms)=8522
  159. Total vcore-milliseconds taken by all map tasks=18930
  160. Total vcore-milliseconds taken by all reduce tasks=8522
  161. Total megabyte-milliseconds taken by all map tasks=4732500
  162. Total megabyte-milliseconds taken by all reduce tasks=2130500
  163. Map-Reduce Framework
  164. Map input records=34558
  165. Map output records=34558
  166. Map output bytes=361613
  167. Map output materialized bytes=430741
  168. Input split bytes=232
  169. Combine input records=0
  170. Spilled Records=34558
  171. Failed Shuffles=0
  172. Merged Map outputs=0
  173. GC time elapsed (ms)=168
  174. CPU time spent (ms)=1760
  175. Physical memory (bytes) snapshot=379465728
  176. Virtual memory (bytes) snapshot=1698050048
  177. Total committed heap usage (bytes)=286261248
  178. File Input Format Counters
  179. Bytes Read=492685
  180. 16/09/30 14:43:37 ERROR streaming.StreamJob: Job not successful!
  181. Streaming Command Failed!
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!