API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Mar 21st, 2017
171
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 12.25 KB | None | 0 0
raw download clone embed print report
  1. 17/03/21 00:56:21 INFO client.RMProxy: Connecting to ResourceManager at cluster-one-m/10.138.0.2:8032
  2. 17/03/21 00:56:22 INFO client.RMProxy: Connecting to ResourceManager at cluster-one-m/10.138.0.2:8032
  3. 17/03/21 00:56:22 INFO gcs.GoogleHadoopFileSystemBase: GHFS version: 1.6.0-hadoop2
  4. 17/03/21 00:56:23 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
  5. 17/03/21 00:56:25 WARN gcsio.GoogleCloudStorage: Repairing batch of 1 missing directories.
  6. 17/03/21 00:56:25 WARN gcsio.GoogleCloudStorage: Successfully repaired 1/1 implicit directories.
  7. 17/03/21 00:56:27 INFO mapred.FileInputFormat: Total input paths to process : 3035
  8. 17/03/21 00:56:27 INFO mapreduce.JobSubmitter: number of splits:3035
  9. 17/03/21 00:56:27 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1490054002611_0001
  10. 17/03/21 00:56:28 INFO impl.YarnClientImpl: Submitted application application_1490054002611_0001
  11. 17/03/21 00:56:28 INFO mapreduce.Job: The url to track the job: http://cluster-one-m:8088/proxy/application_1490054002611_0001/
  12. 17/03/21 00:56:28 INFO mapreduce.Job: Running job: job_1490054002611_0001
  13. 17/03/21 00:56:35 INFO mapreduce.Job: Job job_1490054002611_0001 running in uber mode : false
  14. 17/03/21 00:56:35 INFO mapreduce.Job: map 0% reduce 0%
  15. 17/03/21 00:57:06 INFO mapreduce.Job: map 1% reduce 0%
  16. 17/03/21 00:57:37 INFO mapreduce.Job: map 2% reduce 0%
  17. 17/03/21 00:58:08 INFO mapreduce.Job: map 3% reduce 0%
  18. 17/03/21 00:58:37 INFO mapreduce.Job: map 4% reduce 0%
  19. 17/03/21 00:59:08 INFO mapreduce.Job: map 5% reduce 0%
  20. 17/03/21 00:59:39 INFO mapreduce.Job: map 6% reduce 0%
  21. 17/03/21 01:00:08 INFO mapreduce.Job: map 7% reduce 0%
  22. 17/03/21 01:00:39 INFO mapreduce.Job: map 8% reduce 0%
  23. 17/03/21 01:01:09 INFO mapreduce.Job: map 9% reduce 0%
  24. 17/03/21 01:01:39 INFO mapreduce.Job: map 10% reduce 0%
  25. 17/03/21 01:02:09 INFO mapreduce.Job: map 11% reduce 0%
  26. 17/03/21 01:02:40 INFO mapreduce.Job: map 12% reduce 0%
  27. 17/03/21 01:03:12 INFO mapreduce.Job: map 13% reduce 0%
  28. 17/03/21 01:03:38 INFO mapreduce.Job: map 14% reduce 0%
  29. 17/03/21 01:04:07 INFO mapreduce.Job: map 15% reduce 0%
  30. 17/03/21 01:04:40 INFO mapreduce.Job: map 16% reduce 0%
  31. 17/03/21 01:05:10 INFO mapreduce.Job: map 17% reduce 0%
  32. 17/03/21 01:05:39 INFO mapreduce.Job: map 18% reduce 0%
  33. 17/03/21 01:06:07 INFO mapreduce.Job: map 19% reduce 0%
  34. 17/03/21 01:06:37 INFO mapreduce.Job: map 20% reduce 0%
  35. 17/03/21 01:07:06 INFO mapreduce.Job: map 20% reduce 1%
  36. 17/03/21 01:07:12 INFO mapreduce.Job: map 21% reduce 1%
  37. 17/03/21 01:07:53 INFO mapreduce.Job: map 22% reduce 1%
  38. 17/03/21 01:08:36 INFO mapreduce.Job: map 23% reduce 1%
  39. 17/03/21 01:09:19 INFO mapreduce.Job: map 24% reduce 1%
  40. 17/03/21 01:10:00 INFO mapreduce.Job: map 25% reduce 1%
  41. 17/03/21 01:10:40 INFO mapreduce.Job: map 26% reduce 1%
  42. 17/03/21 01:11:23 INFO mapreduce.Job: map 27% reduce 1%
  43. 17/03/21 01:12:05 INFO mapreduce.Job: map 28% reduce 1%
  44. 17/03/21 01:12:45 INFO mapreduce.Job: map 29% reduce 1%
  45. 17/03/21 01:13:27 INFO mapreduce.Job: map 30% reduce 1%
  46. 17/03/21 01:14:07 INFO mapreduce.Job: map 31% reduce 1%
  47. 17/03/21 01:14:46 INFO mapreduce.Job: map 32% reduce 1%
  48. 17/03/21 01:15:29 INFO mapreduce.Job: map 33% reduce 1%
  49. 17/03/21 01:16:12 INFO mapreduce.Job: map 34% reduce 1%
  50. 17/03/21 01:16:53 INFO mapreduce.Job: map 35% reduce 1%
  51. 17/03/21 01:17:34 INFO mapreduce.Job: map 36% reduce 1%
  52. 17/03/21 01:18:17 INFO mapreduce.Job: map 37% reduce 1%
  53. 17/03/21 01:18:59 INFO mapreduce.Job: map 38% reduce 1%
  54. 17/03/21 01:19:36 INFO mapreduce.Job: map 39% reduce 1%
  55. 17/03/21 01:20:18 INFO mapreduce.Job: map 40% reduce 1%
  56. 17/03/21 01:20:55 INFO mapreduce.Job: map 40% reduce 2%
  57. 17/03/21 01:21:01 INFO mapreduce.Job: map 41% reduce 2%
  58. 17/03/21 01:21:43 INFO mapreduce.Job: map 42% reduce 2%
  59. 17/03/21 01:22:22 INFO mapreduce.Job: map 43% reduce 2%
  60. 17/03/21 01:23:06 INFO mapreduce.Job: map 44% reduce 2%
  61. 17/03/21 01:23:47 INFO mapreduce.Job: map 45% reduce 2%
  62. 17/03/21 01:24:11 INFO mapreduce.Job: map 45% reduce 3%
  63. 17/03/21 01:24:28 INFO mapreduce.Job: map 46% reduce 3%
  64. 17/03/21 01:25:14 INFO mapreduce.Job: map 47% reduce 3%
  65. 17/03/21 01:25:53 INFO mapreduce.Job: map 48% reduce 3%
  66. 17/03/21 01:26:36 INFO mapreduce.Job: map 49% reduce 3%
  67. 17/03/21 01:27:20 INFO mapreduce.Job: map 50% reduce 3%
  68. 17/03/21 01:28:01 INFO mapreduce.Job: map 51% reduce 3%
  69. 17/03/21 01:28:43 INFO mapreduce.Job: map 52% reduce 3%
  70. 17/03/21 01:29:26 INFO mapreduce.Job: map 53% reduce 3%
  71. 17/03/21 01:30:07 INFO mapreduce.Job: map 54% reduce 3%
  72. 17/03/21 01:30:50 INFO mapreduce.Job: map 55% reduce 3%
  73. 17/03/21 01:31:29 INFO mapreduce.Job: map 56% reduce 3%
  74. 17/03/21 01:32:08 INFO mapreduce.Job: map 57% reduce 3%
  75. 17/03/21 01:32:51 INFO mapreduce.Job: map 58% reduce 3%
  76. 17/03/21 01:33:32 INFO mapreduce.Job: map 59% reduce 3%
  77. 17/03/21 01:34:13 INFO mapreduce.Job: map 60% reduce 3%
  78. 17/03/21 01:34:56 INFO mapreduce.Job: map 61% reduce 3%
  79. 17/03/21 01:35:37 INFO mapreduce.Job: map 62% reduce 3%
  80. 17/03/21 01:36:17 INFO mapreduce.Job: map 63% reduce 3%
  81. 17/03/21 01:36:41 INFO mapreduce.Job: map 63% reduce 4%
  82. 17/03/21 01:36:59 INFO mapreduce.Job: map 64% reduce 4%
  83. 17/03/21 01:37:39 INFO mapreduce.Job: map 65% reduce 4%
  84. 17/03/21 01:38:18 INFO mapreduce.Job: map 66% reduce 4%
  85. 17/03/21 01:39:01 INFO mapreduce.Job: map 67% reduce 4%
  86. 17/03/21 01:39:39 INFO mapreduce.Job: map 68% reduce 4%
  87. 17/03/21 01:40:18 INFO mapreduce.Job: map 69% reduce 4%
  88. 17/03/21 01:40:57 INFO mapreduce.Job: map 70% reduce 4%
  89. 17/03/21 01:41:36 INFO mapreduce.Job: map 71% reduce 4%
  90. 17/03/21 01:42:17 INFO mapreduce.Job: map 72% reduce 4%
  91. 17/03/21 01:42:56 INFO mapreduce.Job: map 73% reduce 4%
  92. 17/03/21 01:43:36 INFO mapreduce.Job: map 74% reduce 4%
  93. 17/03/21 01:44:16 INFO mapreduce.Job: map 75% reduce 4%
  94. 17/03/21 01:44:58 INFO mapreduce.Job: map 76% reduce 4%
  95. 17/03/21 01:45:40 INFO mapreduce.Job: map 77% reduce 4%
  96. 17/03/21 01:46:18 INFO mapreduce.Job: map 78% reduce 4%
  97. 17/03/21 01:46:57 INFO mapreduce.Job: map 79% reduce 4%
  98. 17/03/21 01:47:36 INFO mapreduce.Job: map 80% reduce 4%
  99. 17/03/21 01:48:18 INFO mapreduce.Job: map 81% reduce 4%
  100. 17/03/21 01:48:41 INFO mapreduce.Job: map 81% reduce 5%
  101. 17/03/21 01:49:01 INFO mapreduce.Job: map 82% reduce 5%
  102. 17/03/21 01:49:39 INFO mapreduce.Job: map 83% reduce 5%
  103. 17/03/21 01:50:20 INFO mapreduce.Job: map 84% reduce 5%
  104. 17/03/21 01:50:59 INFO mapreduce.Job: map 85% reduce 5%
  105. 17/03/21 01:51:39 INFO mapreduce.Job: map 86% reduce 5%
  106. 17/03/21 01:52:18 INFO mapreduce.Job: map 87% reduce 5%
  107. 17/03/21 01:52:55 INFO mapreduce.Job: map 88% reduce 5%
  108. 17/03/21 01:53:36 INFO mapreduce.Job: map 89% reduce 5%
  109. 17/03/21 01:54:17 INFO mapreduce.Job: map 90% reduce 5%
  110. 17/03/21 01:54:55 INFO mapreduce.Job: map 91% reduce 5%
  111. 17/03/21 01:55:36 INFO mapreduce.Job: map 92% reduce 5%
  112. 17/03/21 01:56:15 INFO mapreduce.Job: map 93% reduce 5%
  113. 17/03/21 01:56:54 INFO mapreduce.Job: map 94% reduce 5%
  114. 17/03/21 01:57:35 INFO mapreduce.Job: map 95% reduce 5%
  115. 17/03/21 01:58:14 INFO mapreduce.Job: map 96% reduce 5%
  116. 17/03/21 01:58:51 INFO mapreduce.Job: map 97% reduce 5%
  117. 17/03/21 01:59:32 INFO mapreduce.Job: map 98% reduce 5%
  118. 17/03/21 02:00:10 INFO mapreduce.Job: map 99% reduce 5%
  119. 17/03/21 02:00:30 INFO mapreduce.Job: map 99% reduce 6%
  120. 17/03/21 02:00:44 INFO mapreduce.Job: map 100% reduce 6%
  121. 17/03/21 02:01:06 INFO mapreduce.Job: map 100% reduce 8%
  122. 17/03/21 02:01:09 INFO mapreduce.Job: map 100% reduce 9%
  123. 17/03/21 02:01:10 INFO mapreduce.Job: map 100% reduce 12%
  124. 17/03/21 02:01:11 INFO mapreduce.Job: map 100% reduce 15%
  125. 17/03/21 02:01:12 INFO mapreduce.Job: map 100% reduce 17%
  126. 17/03/21 02:01:13 INFO mapreduce.Job: map 100% reduce 21%
  127. 17/03/21 02:01:14 INFO mapreduce.Job: map 100% reduce 22%
  128. 17/03/21 02:01:15 INFO mapreduce.Job: map 100% reduce 23%
  129. 17/03/21 02:01:16 INFO mapreduce.Job: map 100% reduce 24%
  130. 17/03/21 02:01:17 INFO mapreduce.Job: map 100% reduce 25%
  131. 17/03/21 02:01:18 INFO mapreduce.Job: map 100% reduce 27%
  132. 17/03/21 02:01:19 INFO mapreduce.Job: map 100% reduce 29%
  133. 17/03/21 02:01:21 INFO mapreduce.Job: map 100% reduce 31%
  134. 17/03/21 02:01:22 INFO mapreduce.Job: map 100% reduce 32%
  135. 17/03/21 02:01:24 INFO mapreduce.Job: map 100% reduce 33%
  136. 17/03/21 02:01:25 INFO mapreduce.Job: map 100% reduce 34%
  137. 17/03/21 02:01:28 INFO mapreduce.Job: map 100% reduce 36%
  138. 17/03/21 02:01:31 INFO mapreduce.Job: map 100% reduce 38%
  139. 17/03/21 02:01:34 INFO mapreduce.Job: map 100% reduce 46%
  140. 17/03/21 02:01:37 INFO mapreduce.Job: map 100% reduce 49%
  141. 17/03/21 02:01:38 INFO mapreduce.Job: map 100% reduce 50%
  142. 17/03/21 02:01:40 INFO mapreduce.Job: map 100% reduce 51%
  143. 17/03/21 02:01:41 INFO mapreduce.Job: map 100% reduce 52%
  144. 17/03/21 02:01:43 INFO mapreduce.Job: map 100% reduce 57%
  145. 17/03/21 02:01:46 INFO mapreduce.Job: map 100% reduce 60%
  146. 17/03/21 02:01:47 INFO mapreduce.Job: map 100% reduce 61%
  147. 17/03/21 02:01:49 INFO mapreduce.Job: map 100% reduce 64%
  148. 17/03/21 02:01:50 INFO mapreduce.Job: map 100% reduce 67%
  149. 17/03/21 02:01:52 INFO mapreduce.Job: map 100% reduce 68%
  150. 17/03/21 02:01:55 INFO mapreduce.Job: map 100% reduce 69%
  151. 17/03/21 02:01:56 INFO mapreduce.Job: map 100% reduce 70%
  152. 17/03/21 02:01:59 INFO mapreduce.Job: map 100% reduce 71%
  153. 17/03/21 02:02:00 INFO mapreduce.Job: map 100% reduce 78%
  154. 17/03/21 02:02:02 INFO mapreduce.Job: map 100% reduce 83%
  155. 17/03/21 02:02:03 INFO mapreduce.Job: map 100% reduce 85%
  156. 17/03/21 02:02:05 INFO mapreduce.Job: map 100% reduce 87%
  157. 17/03/21 02:02:06 INFO mapreduce.Job: map 100% reduce 89%
  158. 17/03/21 02:02:08 INFO mapreduce.Job: map 100% reduce 90%
  159. 17/03/21 02:02:09 INFO mapreduce.Job: map 100% reduce 91%
  160. 17/03/21 02:02:11 INFO mapreduce.Job: map 100% reduce 93%
  161. 17/03/21 02:02:12 INFO mapreduce.Job: map 100% reduce 95%
  162. 17/03/21 02:02:14 INFO mapreduce.Job: map 100% reduce 97%
  163. 17/03/21 02:02:15 INFO mapreduce.Job: map 100% reduce 98%
  164. 17/03/21 02:02:17 INFO mapreduce.Job: map 100% reduce 99%
  165. 17/03/21 02:02:20 INFO mapreduce.Job: map 100% reduce 100%
  166. 17/03/21 02:02:27 INFO mapreduce.Job: Job job_1490054002611_0001 completed successfully
  167. 17/03/21 02:02:27 INFO mapreduce.Job: Counters: 56
  168. File System Counters
  169. FILE: Number of bytes read=3420439301
  170. FILE: Number of bytes written=7221141341
  171. FILE: Number of read operations=0
  172. FILE: Number of large read operations=0
  173. FILE: Number of write operations=0
  174. GS: Number of bytes read=1141053926
  175. GS: Number of bytes written=234665158
  176. GS: Number of read operations=0
  177. GS: Number of large read operations=0
  178. GS: Number of write operations=0
  179. HDFS: Number of bytes read=388153
  180. HDFS: Number of bytes written=0
  181. HDFS: Number of read operations=3035
  182. HDFS: Number of large read operations=0
  183. HDFS: Number of write operations=0
  184. Job Counters
  185. Killed map tasks=1
  186. Killed reduce tasks=1
  187. Launched map tasks=3035
  188. Launched reduce tasks=13
  189. Rack-local map tasks=3035
  190. Total time spent by all maps in occupied slots (ms)=76372746
  191. Total time spent by all reduces in occupied slots (ms)=36245730
  192. Total time spent by all map tasks (ms)=25457582
  193. Total time spent by all reduce tasks (ms)=6040955
  194. Total vcore-milliseconds taken by all map tasks=25457582
  195. Total vcore-milliseconds taken by all reduce tasks=12081910
  196. Total megabyte-milliseconds taken by all map tasks=78205691904
  197. Total megabyte-milliseconds taken by all reduce tasks=37115627520
  198. Map-Reduce Framework
  199. Map input records=3035
  200. Map output records=210623260
  201. Map output bytes=2999192680
  202. Map output materialized bytes=3420657749
  203. Input split bytes=388153
  204. Combine input records=0
  205. Combine output records=0
  206. Reduce input groups=1217608
  207. Reduce shuffle bytes=3420657749
  208. Reduce input records=210623260
  209. Reduce output records=1217608
  210. Spilled Records=421246520
  211. Shuffled Maps =36420
  212. Failed Shuffles=0
  213. Merged Map outputs=36420
  214. GC time elapsed (ms)=438006
  215. CPU time spent (ms)=9288880
  216. Physical memory (bytes) snapshot=1129278623744
  217. Virtual memory (bytes) snapshot=13430944673792
  218. Total committed heap usage (bytes)=1000340455424
  219. Shuffle Errors
  220. BAD_ID=0
  221. CONNECTION=0
  222. IO_ERROR=0
  223. WRONG_LENGTH=0
  224. WRONG_MAP=0
  225. WRONG_REDUCE=0
  226. File Input Format Counters
  227. Bytes Read=1141053926
  228. File Output Format Counters
  229. Bytes Written=234665158
  230. Job output is complete
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!