API tools faq
paste
Advertisement
Guest User

lesson 3 job output

a guest
Sep 19th, 2014
228
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 7.65 KB | None | 0 0
raw download clone embed print report
  1. hadoop jar /usr/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop-streaming-2.0.0-mr1-cdh4.1.1.jar -mapper lesson3Path_map.py -reducer lesson3Path_reduce.py -file lesson3Path_map.py -file lesson3Path_reduce.py -input classassign/access_log -output classassign/lesson3Path_output1
  2. packageJobJar: [lesson3Path_map.py, lesson3Path_reduce.py, /tmp/hadoop-training/hadoop-unjar1540235236570458142/] [] /tmp/streamjob2697070147933820048.jar tmpDir=null
  3. 14/09/19 19:49:24 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
  4. 14/09/19 19:49:25 WARN snappy.LoadSnappy: Snappy native library is available
  5. 14/09/19 19:49:25 INFO snappy.LoadSnappy: Snappy native library loaded
  6. 14/09/19 19:49:25 INFO mapred.FileInputFormat: Total input paths to process : 1
  7. 14/09/19 19:49:25 INFO streaming.StreamJob: getLocalDirs(): [/var/lib/hadoop-hdfs/cache/training/mapred/local]
  8. 14/09/19 19:49:25 INFO streaming.StreamJob: Running job: job_201409191321_0005
  9. 14/09/19 19:49:25 INFO streaming.StreamJob: To kill this job, run:
  10. 14/09/19 19:49:25 INFO streaming.StreamJob: UNDEF/bin/hadoop job -Dmapred.job.tracker=0.0.0.0:8021 -kill job_201409191321_0005
  11. 14/09/19 19:49:30 INFO streaming.StreamJob: Tracking URL: http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201409191321_0005
  12. 14/09/19 19:49:31 INFO streaming.StreamJob: map 0% reduce 0%
  13. 14/09/19 19:49:44 INFO streaming.StreamJob: map 5% reduce 0%
  14. 14/09/19 19:49:47 INFO streaming.StreamJob: map 10% reduce 0%
  15. 14/09/19 19:49:50 INFO streaming.StreamJob: map 13% reduce 0%
  16. 14/09/19 19:49:53 INFO streaming.StreamJob: map 18% reduce 0%
  17. 14/09/19 19:49:56 INFO streaming.StreamJob: map 22% reduce 0%
  18. 14/09/19 19:49:59 INFO streaming.StreamJob: map 25% reduce 0%
  19. 14/09/19 19:50:11 INFO streaming.StreamJob: map 25% reduce 8%
  20. 14/09/19 19:50:14 INFO streaming.StreamJob: map 30% reduce 8%
  21. 14/09/19 19:50:17 INFO streaming.StreamJob: map 35% reduce 8%
  22. 14/09/19 19:50:20 INFO streaming.StreamJob: map 38% reduce 8%
  23. 14/09/19 19:50:23 INFO streaming.StreamJob: map 43% reduce 8%
  24. 14/09/19 19:50:26 INFO streaming.StreamJob: map 48% reduce 8%
  25. 14/09/19 19:50:29 INFO streaming.StreamJob: map 50% reduce 8%
  26. 14/09/19 19:50:35 INFO streaming.StreamJob: map 50% reduce 17%
  27. 14/09/19 19:50:42 INFO streaming.StreamJob: map 55% reduce 17%
  28. 14/09/19 19:50:46 INFO streaming.StreamJob: map 60% reduce 17%
  29. 14/09/19 19:50:49 INFO streaming.StreamJob: map 63% reduce 17%
  30. 14/09/19 19:50:52 INFO streaming.StreamJob: map 68% reduce 17%
  31. 14/09/19 19:50:55 INFO streaming.StreamJob: map 72% reduce 17%
  32. 14/09/19 19:50:58 INFO streaming.StreamJob: map 75% reduce 17%
  33. 14/09/19 19:51:05 INFO streaming.StreamJob: map 75% reduce 25%
  34. 14/09/19 19:51:13 INFO streaming.StreamJob: map 82% reduce 25%
  35. 14/09/19 19:51:17 INFO streaming.StreamJob: map 89% reduce 25%
  36. 14/09/19 19:51:20 INFO streaming.StreamJob: map 94% reduce 25%
  37. 14/09/19 19:51:23 INFO streaming.StreamJob: map 97% reduce 29%
  38. 14/09/19 19:51:26 INFO streaming.StreamJob: map 100% reduce 29%
  39. 14/09/19 19:51:30 INFO streaming.StreamJob: map 100% reduce 67%
  40. 14/09/19 19:51:39 INFO streaming.StreamJob: map 100% reduce 74%
  41. 14/09/19 19:51:42 INFO streaming.StreamJob: map 100% reduce 80%
  42. 14/09/19 19:51:45 INFO streaming.StreamJob: map 100% reduce 88%
  43. 14/09/19 19:51:48 INFO streaming.StreamJob: map 100% reduce 95%
  44. 14/09/19 19:51:53 INFO streaming.StreamJob: map 100% reduce 0%
  45. 14/09/19 19:52:03 INFO streaming.StreamJob: map 100% reduce 33%
  46. 14/09/19 19:52:06 INFO streaming.StreamJob: map 100% reduce 67%
  47. 14/09/19 19:52:12 INFO streaming.StreamJob: map 100% reduce 72%
  48. 14/09/19 19:52:15 INFO streaming.StreamJob: map 100% reduce 78%
  49. 14/09/19 19:52:18 INFO streaming.StreamJob: map 100% reduce 85%
  50. 14/09/19 19:52:21 INFO streaming.StreamJob: map 100% reduce 92%
  51. 14/09/19 19:52:24 INFO streaming.StreamJob: map 100% reduce 99%
  52. 14/09/19 19:52:28 INFO streaming.StreamJob: map 100% reduce 0%
  53. 14/09/19 19:52:35 INFO streaming.StreamJob: map 100% reduce 33%
  54. 14/09/19 19:52:38 INFO streaming.StreamJob: map 100% reduce 67%
  55. 14/09/19 19:52:44 INFO streaming.StreamJob: map 100% reduce 72%
  56. 14/09/19 19:52:47 INFO streaming.StreamJob: map 100% reduce 78%
  57. 14/09/19 19:52:50 INFO streaming.StreamJob: map 100% reduce 86%
  58. 14/09/19 19:52:53 INFO streaming.StreamJob: map 100% reduce 93%
  59. 14/09/19 19:52:56 INFO streaming.StreamJob: map 100% reduce 100%
  60. 14/09/19 19:53:00 INFO streaming.StreamJob: map 100% reduce 0%
  61. 14/09/19 19:53:07 INFO streaming.StreamJob: map 100% reduce 33%
  62. 14/09/19 19:53:10 INFO streaming.StreamJob: map 100% reduce 67%
  63. 14/09/19 19:53:19 INFO streaming.StreamJob: map 100% reduce 77%
  64. 14/09/19 19:53:22 INFO streaming.StreamJob: map 100% reduce 84%
  65. 14/09/19 19:53:25 INFO streaming.StreamJob: map 100% reduce 91%
  66. 14/09/19 19:53:28 INFO streaming.StreamJob: map 100% reduce 98%
  67. 14/09/19 19:53:33 INFO streaming.StreamJob: map 100% reduce 0%
  68. 14/09/19 19:53:39 INFO streaming.StreamJob: map 100% reduce 100%
  69. 14/09/19 19:53:39 INFO streaming.StreamJob: To kill this job, run:
  70. 14/09/19 19:53:39 INFO streaming.StreamJob: UNDEF/bin/hadoop job -Dmapred.job.tracker=0.0.0.0:8021 -kill job_201409191321_0005
  71. 14/09/19 19:53:44 INFO streaming.StreamJob: Tracking URL: http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201409191321_0005
  72. 14/09/19 19:53:44 ERROR streaming.StreamJob: Job not successful. Error: NA
  73. 14/09/19 19:53:44 INFO streaming.StreamJob: killJob...
  74. Streaming Command Failed!
  75.  
  76.  
  77.  
  78.  
  79.  
  80.  
  81.  
  82.  
  83.  
  84.  
  85.  
  86. Hadoop job_201409191321_0005 on 0
  87. User: training
  88. Job Name: streamjob2697070147933820048.jar
  89. Job File: hdfs://0.0.0.0:8020/var/lib/hadoop-hdfs/cache/mapred/mapred/staging/training/.staging/job_201409191321_0005/job.xml
  90. Submit Host: localhost.localdomain
  91. Submit Host Address: 127.0.0.1
  92. Job-ACLs: All users are allowed
  93. Job Setup: Successful
  94. Status: Failed
  95. Failure Info:NA
  96. Started at: Fri Sep 19 19:49:25 EDT 2014
  97. Failed at: Fri Sep 19 19:53:39 EDT 2014
  98. Failed in: 4mins, 13sec
  99. Job Cleanup: Successful
  100. Black-listed TaskTrackers: 1
  101. Kind % Complete Num Tasks Pending Running Complete Killed Failed/Killed
  102. Task Attempts
  103. map 100.00%
  104. 8 0 0 8 0 0 / 0
  105. reduce 100.00%
  106. 1 0 0 0 1 4 / 0
  107.  
  108.  
  109. Counter Map Reduce Total
  110. File System Counters FILE: Number of bytes read 0 0 220,686,513
  111. FILE: Number of bytes written 0 0 442,878,696
  112. FILE: Number of read operations 0 0 0
  113. FILE: Number of large read operations 0 0 0
  114. FILE: Number of write operations 0 0 0
  115. HDFS: Number of bytes read 0 0 504,971,083
  116. HDFS: Number of bytes written 0 0 0
  117. HDFS: Number of read operations 0 0 16
  118. HDFS: Number of large read operations 0 0 0
  119. HDFS: Number of write operations 0 0 0
  120. Job Counters Failed reduce tasks 0 0 1
  121. Launched map tasks 0 0 8
  122. Launched reduce tasks 0 0 4
  123. Data-local map tasks 0 0 8
  124. Total time spent by all maps in occupied slots (ms) 0 0 234,791
  125. Total time spent by all reduces in occupied slots (ms) 0 0 212,095
  126. Total time spent by all maps waiting after reserving slots (ms) 0 0 0
  127. Total time spent by all reduces waiting after reserving slots (ms) 0 0 0
  128. Map-Reduce Framework Map input records 0 0 4,477,843
  129. Map output records 0 0 4,477,843
  130. Map output bytes 0 0 211,723,089
  131. Input split bytes 0 0 872
  132. Combine input records 0 0 0
  133. Combine output records 0 0 0
  134. Spilled Records 0 0 8,955,686
  135. CPU time spent (ms) 0 0 68,080
  136. Physical memory (bytes) snapshot 0 0 1,544,773,632
  137. Virtual memory (bytes) snapshot 0 0 3,105,579,008
  138. Total committed heap usage (bytes) 0 0 1,284,014,080
  139. org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter BYTES_READ 504,941,532 0 504,941,532
  140. Map Completion Graph - close
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!