SHARE
TWEET

Untitled

a guest Mar 21st, 2017 64 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. 17/03/21 00:56:21 INFO client.RMProxy: Connecting to ResourceManager at cluster-one-m/10.138.0.2:8032
  2. 17/03/21 00:56:22 INFO client.RMProxy: Connecting to ResourceManager at cluster-one-m/10.138.0.2:8032
  3. 17/03/21 00:56:22 INFO gcs.GoogleHadoopFileSystemBase: GHFS version: 1.6.0-hadoop2
  4. 17/03/21 00:56:23 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
  5. 17/03/21 00:56:25 WARN gcsio.GoogleCloudStorage: Repairing batch of 1 missing directories.
  6. 17/03/21 00:56:25 WARN gcsio.GoogleCloudStorage: Successfully repaired 1/1 implicit directories.
  7. 17/03/21 00:56:27 INFO mapred.FileInputFormat: Total input paths to process : 3035
  8. 17/03/21 00:56:27 INFO mapreduce.JobSubmitter: number of splits:3035
  9. 17/03/21 00:56:27 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1490054002611_0001
  10. 17/03/21 00:56:28 INFO impl.YarnClientImpl: Submitted application application_1490054002611_0001
  11. 17/03/21 00:56:28 INFO mapreduce.Job: The url to track the job: http://cluster-one-m:8088/proxy/application_1490054002611_0001/
  12. 17/03/21 00:56:28 INFO mapreduce.Job: Running job: job_1490054002611_0001
  13. 17/03/21 00:56:35 INFO mapreduce.Job: Job job_1490054002611_0001 running in uber mode : false
  14. 17/03/21 00:56:35 INFO mapreduce.Job:  map 0% reduce 0%
  15. 17/03/21 00:57:06 INFO mapreduce.Job:  map 1% reduce 0%
  16. 17/03/21 00:57:37 INFO mapreduce.Job:  map 2% reduce 0%
  17. 17/03/21 00:58:08 INFO mapreduce.Job:  map 3% reduce 0%
  18. 17/03/21 00:58:37 INFO mapreduce.Job:  map 4% reduce 0%
  19. 17/03/21 00:59:08 INFO mapreduce.Job:  map 5% reduce 0%
  20. 17/03/21 00:59:39 INFO mapreduce.Job:  map 6% reduce 0%
  21. 17/03/21 01:00:08 INFO mapreduce.Job:  map 7% reduce 0%
  22. 17/03/21 01:00:39 INFO mapreduce.Job:  map 8% reduce 0%
  23. 17/03/21 01:01:09 INFO mapreduce.Job:  map 9% reduce 0%
  24. 17/03/21 01:01:39 INFO mapreduce.Job:  map 10% reduce 0%
  25. 17/03/21 01:02:09 INFO mapreduce.Job:  map 11% reduce 0%
  26. 17/03/21 01:02:40 INFO mapreduce.Job:  map 12% reduce 0%
  27. 17/03/21 01:03:12 INFO mapreduce.Job:  map 13% reduce 0%
  28. 17/03/21 01:03:38 INFO mapreduce.Job:  map 14% reduce 0%
  29. 17/03/21 01:04:07 INFO mapreduce.Job:  map 15% reduce 0%
  30. 17/03/21 01:04:40 INFO mapreduce.Job:  map 16% reduce 0%
  31. 17/03/21 01:05:10 INFO mapreduce.Job:  map 17% reduce 0%
  32. 17/03/21 01:05:39 INFO mapreduce.Job:  map 18% reduce 0%
  33. 17/03/21 01:06:07 INFO mapreduce.Job:  map 19% reduce 0%
  34. 17/03/21 01:06:37 INFO mapreduce.Job:  map 20% reduce 0%
  35. 17/03/21 01:07:06 INFO mapreduce.Job:  map 20% reduce 1%
  36. 17/03/21 01:07:12 INFO mapreduce.Job:  map 21% reduce 1%
  37. 17/03/21 01:07:53 INFO mapreduce.Job:  map 22% reduce 1%
  38. 17/03/21 01:08:36 INFO mapreduce.Job:  map 23% reduce 1%
  39. 17/03/21 01:09:19 INFO mapreduce.Job:  map 24% reduce 1%
  40. 17/03/21 01:10:00 INFO mapreduce.Job:  map 25% reduce 1%
  41. 17/03/21 01:10:40 INFO mapreduce.Job:  map 26% reduce 1%
  42. 17/03/21 01:11:23 INFO mapreduce.Job:  map 27% reduce 1%
  43. 17/03/21 01:12:05 INFO mapreduce.Job:  map 28% reduce 1%
  44. 17/03/21 01:12:45 INFO mapreduce.Job:  map 29% reduce 1%
  45. 17/03/21 01:13:27 INFO mapreduce.Job:  map 30% reduce 1%
  46. 17/03/21 01:14:07 INFO mapreduce.Job:  map 31% reduce 1%
  47. 17/03/21 01:14:46 INFO mapreduce.Job:  map 32% reduce 1%
  48. 17/03/21 01:15:29 INFO mapreduce.Job:  map 33% reduce 1%
  49. 17/03/21 01:16:12 INFO mapreduce.Job:  map 34% reduce 1%
  50. 17/03/21 01:16:53 INFO mapreduce.Job:  map 35% reduce 1%
  51. 17/03/21 01:17:34 INFO mapreduce.Job:  map 36% reduce 1%
  52. 17/03/21 01:18:17 INFO mapreduce.Job:  map 37% reduce 1%
  53. 17/03/21 01:18:59 INFO mapreduce.Job:  map 38% reduce 1%
  54. 17/03/21 01:19:36 INFO mapreduce.Job:  map 39% reduce 1%
  55. 17/03/21 01:20:18 INFO mapreduce.Job:  map 40% reduce 1%
  56. 17/03/21 01:20:55 INFO mapreduce.Job:  map 40% reduce 2%
  57. 17/03/21 01:21:01 INFO mapreduce.Job:  map 41% reduce 2%
  58. 17/03/21 01:21:43 INFO mapreduce.Job:  map 42% reduce 2%
  59. 17/03/21 01:22:22 INFO mapreduce.Job:  map 43% reduce 2%
  60. 17/03/21 01:23:06 INFO mapreduce.Job:  map 44% reduce 2%
  61. 17/03/21 01:23:47 INFO mapreduce.Job:  map 45% reduce 2%
  62. 17/03/21 01:24:11 INFO mapreduce.Job:  map 45% reduce 3%
  63. 17/03/21 01:24:28 INFO mapreduce.Job:  map 46% reduce 3%
  64. 17/03/21 01:25:14 INFO mapreduce.Job:  map 47% reduce 3%
  65. 17/03/21 01:25:53 INFO mapreduce.Job:  map 48% reduce 3%
  66. 17/03/21 01:26:36 INFO mapreduce.Job:  map 49% reduce 3%
  67. 17/03/21 01:27:20 INFO mapreduce.Job:  map 50% reduce 3%
  68. 17/03/21 01:28:01 INFO mapreduce.Job:  map 51% reduce 3%
  69. 17/03/21 01:28:43 INFO mapreduce.Job:  map 52% reduce 3%
  70. 17/03/21 01:29:26 INFO mapreduce.Job:  map 53% reduce 3%
  71. 17/03/21 01:30:07 INFO mapreduce.Job:  map 54% reduce 3%
  72. 17/03/21 01:30:50 INFO mapreduce.Job:  map 55% reduce 3%
  73. 17/03/21 01:31:29 INFO mapreduce.Job:  map 56% reduce 3%
  74. 17/03/21 01:32:08 INFO mapreduce.Job:  map 57% reduce 3%
  75. 17/03/21 01:32:51 INFO mapreduce.Job:  map 58% reduce 3%
  76. 17/03/21 01:33:32 INFO mapreduce.Job:  map 59% reduce 3%
  77. 17/03/21 01:34:13 INFO mapreduce.Job:  map 60% reduce 3%
  78. 17/03/21 01:34:56 INFO mapreduce.Job:  map 61% reduce 3%
  79. 17/03/21 01:35:37 INFO mapreduce.Job:  map 62% reduce 3%
  80. 17/03/21 01:36:17 INFO mapreduce.Job:  map 63% reduce 3%
  81. 17/03/21 01:36:41 INFO mapreduce.Job:  map 63% reduce 4%
  82. 17/03/21 01:36:59 INFO mapreduce.Job:  map 64% reduce 4%
  83. 17/03/21 01:37:39 INFO mapreduce.Job:  map 65% reduce 4%
  84. 17/03/21 01:38:18 INFO mapreduce.Job:  map 66% reduce 4%
  85. 17/03/21 01:39:01 INFO mapreduce.Job:  map 67% reduce 4%
  86. 17/03/21 01:39:39 INFO mapreduce.Job:  map 68% reduce 4%
  87. 17/03/21 01:40:18 INFO mapreduce.Job:  map 69% reduce 4%
  88. 17/03/21 01:40:57 INFO mapreduce.Job:  map 70% reduce 4%
  89. 17/03/21 01:41:36 INFO mapreduce.Job:  map 71% reduce 4%
  90. 17/03/21 01:42:17 INFO mapreduce.Job:  map 72% reduce 4%
  91. 17/03/21 01:42:56 INFO mapreduce.Job:  map 73% reduce 4%
  92. 17/03/21 01:43:36 INFO mapreduce.Job:  map 74% reduce 4%
  93. 17/03/21 01:44:16 INFO mapreduce.Job:  map 75% reduce 4%
  94. 17/03/21 01:44:58 INFO mapreduce.Job:  map 76% reduce 4%
  95. 17/03/21 01:45:40 INFO mapreduce.Job:  map 77% reduce 4%
  96. 17/03/21 01:46:18 INFO mapreduce.Job:  map 78% reduce 4%
  97. 17/03/21 01:46:57 INFO mapreduce.Job:  map 79% reduce 4%
  98. 17/03/21 01:47:36 INFO mapreduce.Job:  map 80% reduce 4%
  99. 17/03/21 01:48:18 INFO mapreduce.Job:  map 81% reduce 4%
  100. 17/03/21 01:48:41 INFO mapreduce.Job:  map 81% reduce 5%
  101. 17/03/21 01:49:01 INFO mapreduce.Job:  map 82% reduce 5%
  102. 17/03/21 01:49:39 INFO mapreduce.Job:  map 83% reduce 5%
  103. 17/03/21 01:50:20 INFO mapreduce.Job:  map 84% reduce 5%
  104. 17/03/21 01:50:59 INFO mapreduce.Job:  map 85% reduce 5%
  105. 17/03/21 01:51:39 INFO mapreduce.Job:  map 86% reduce 5%
  106. 17/03/21 01:52:18 INFO mapreduce.Job:  map 87% reduce 5%
  107. 17/03/21 01:52:55 INFO mapreduce.Job:  map 88% reduce 5%
  108. 17/03/21 01:53:36 INFO mapreduce.Job:  map 89% reduce 5%
  109. 17/03/21 01:54:17 INFO mapreduce.Job:  map 90% reduce 5%
  110. 17/03/21 01:54:55 INFO mapreduce.Job:  map 91% reduce 5%
  111. 17/03/21 01:55:36 INFO mapreduce.Job:  map 92% reduce 5%
  112. 17/03/21 01:56:15 INFO mapreduce.Job:  map 93% reduce 5%
  113. 17/03/21 01:56:54 INFO mapreduce.Job:  map 94% reduce 5%
  114. 17/03/21 01:57:35 INFO mapreduce.Job:  map 95% reduce 5%
  115. 17/03/21 01:58:14 INFO mapreduce.Job:  map 96% reduce 5%
  116. 17/03/21 01:58:51 INFO mapreduce.Job:  map 97% reduce 5%
  117. 17/03/21 01:59:32 INFO mapreduce.Job:  map 98% reduce 5%
  118. 17/03/21 02:00:10 INFO mapreduce.Job:  map 99% reduce 5%
  119. 17/03/21 02:00:30 INFO mapreduce.Job:  map 99% reduce 6%
  120. 17/03/21 02:00:44 INFO mapreduce.Job:  map 100% reduce 6%
  121. 17/03/21 02:01:06 INFO mapreduce.Job:  map 100% reduce 8%
  122. 17/03/21 02:01:09 INFO mapreduce.Job:  map 100% reduce 9%
  123. 17/03/21 02:01:10 INFO mapreduce.Job:  map 100% reduce 12%
  124. 17/03/21 02:01:11 INFO mapreduce.Job:  map 100% reduce 15%
  125. 17/03/21 02:01:12 INFO mapreduce.Job:  map 100% reduce 17%
  126. 17/03/21 02:01:13 INFO mapreduce.Job:  map 100% reduce 21%
  127. 17/03/21 02:01:14 INFO mapreduce.Job:  map 100% reduce 22%
  128. 17/03/21 02:01:15 INFO mapreduce.Job:  map 100% reduce 23%
  129. 17/03/21 02:01:16 INFO mapreduce.Job:  map 100% reduce 24%
  130. 17/03/21 02:01:17 INFO mapreduce.Job:  map 100% reduce 25%
  131. 17/03/21 02:01:18 INFO mapreduce.Job:  map 100% reduce 27%
  132. 17/03/21 02:01:19 INFO mapreduce.Job:  map 100% reduce 29%
  133. 17/03/21 02:01:21 INFO mapreduce.Job:  map 100% reduce 31%
  134. 17/03/21 02:01:22 INFO mapreduce.Job:  map 100% reduce 32%
  135. 17/03/21 02:01:24 INFO mapreduce.Job:  map 100% reduce 33%
  136. 17/03/21 02:01:25 INFO mapreduce.Job:  map 100% reduce 34%
  137. 17/03/21 02:01:28 INFO mapreduce.Job:  map 100% reduce 36%
  138. 17/03/21 02:01:31 INFO mapreduce.Job:  map 100% reduce 38%
  139. 17/03/21 02:01:34 INFO mapreduce.Job:  map 100% reduce 46%
  140. 17/03/21 02:01:37 INFO mapreduce.Job:  map 100% reduce 49%
  141. 17/03/21 02:01:38 INFO mapreduce.Job:  map 100% reduce 50%
  142. 17/03/21 02:01:40 INFO mapreduce.Job:  map 100% reduce 51%
  143. 17/03/21 02:01:41 INFO mapreduce.Job:  map 100% reduce 52%
  144. 17/03/21 02:01:43 INFO mapreduce.Job:  map 100% reduce 57%
  145. 17/03/21 02:01:46 INFO mapreduce.Job:  map 100% reduce 60%
  146. 17/03/21 02:01:47 INFO mapreduce.Job:  map 100% reduce 61%
  147. 17/03/21 02:01:49 INFO mapreduce.Job:  map 100% reduce 64%
  148. 17/03/21 02:01:50 INFO mapreduce.Job:  map 100% reduce 67%
  149. 17/03/21 02:01:52 INFO mapreduce.Job:  map 100% reduce 68%
  150. 17/03/21 02:01:55 INFO mapreduce.Job:  map 100% reduce 69%
  151. 17/03/21 02:01:56 INFO mapreduce.Job:  map 100% reduce 70%
  152. 17/03/21 02:01:59 INFO mapreduce.Job:  map 100% reduce 71%
  153. 17/03/21 02:02:00 INFO mapreduce.Job:  map 100% reduce 78%
  154. 17/03/21 02:02:02 INFO mapreduce.Job:  map 100% reduce 83%
  155. 17/03/21 02:02:03 INFO mapreduce.Job:  map 100% reduce 85%
  156. 17/03/21 02:02:05 INFO mapreduce.Job:  map 100% reduce 87%
  157. 17/03/21 02:02:06 INFO mapreduce.Job:  map 100% reduce 89%
  158. 17/03/21 02:02:08 INFO mapreduce.Job:  map 100% reduce 90%
  159. 17/03/21 02:02:09 INFO mapreduce.Job:  map 100% reduce 91%
  160. 17/03/21 02:02:11 INFO mapreduce.Job:  map 100% reduce 93%
  161. 17/03/21 02:02:12 INFO mapreduce.Job:  map 100% reduce 95%
  162. 17/03/21 02:02:14 INFO mapreduce.Job:  map 100% reduce 97%
  163. 17/03/21 02:02:15 INFO mapreduce.Job:  map 100% reduce 98%
  164. 17/03/21 02:02:17 INFO mapreduce.Job:  map 100% reduce 99%
  165. 17/03/21 02:02:20 INFO mapreduce.Job:  map 100% reduce 100%
  166. 17/03/21 02:02:27 INFO mapreduce.Job: Job job_1490054002611_0001 completed successfully
  167. 17/03/21 02:02:27 INFO mapreduce.Job: Counters: 56
  168.     File System Counters
  169.         FILE: Number of bytes read=3420439301
  170.         FILE: Number of bytes written=7221141341
  171.         FILE: Number of read operations=0
  172.         FILE: Number of large read operations=0
  173.         FILE: Number of write operations=0
  174.         GS: Number of bytes read=1141053926
  175.         GS: Number of bytes written=234665158
  176.         GS: Number of read operations=0
  177.         GS: Number of large read operations=0
  178.         GS: Number of write operations=0
  179.         HDFS: Number of bytes read=388153
  180.         HDFS: Number of bytes written=0
  181.         HDFS: Number of read operations=3035
  182.         HDFS: Number of large read operations=0
  183.         HDFS: Number of write operations=0
  184.     Job Counters
  185.         Killed map tasks=1
  186.         Killed reduce tasks=1
  187.         Launched map tasks=3035
  188.         Launched reduce tasks=13
  189.         Rack-local map tasks=3035
  190.         Total time spent by all maps in occupied slots (ms)=76372746
  191.         Total time spent by all reduces in occupied slots (ms)=36245730
  192.         Total time spent by all map tasks (ms)=25457582
  193.         Total time spent by all reduce tasks (ms)=6040955
  194.         Total vcore-milliseconds taken by all map tasks=25457582
  195.         Total vcore-milliseconds taken by all reduce tasks=12081910
  196.         Total megabyte-milliseconds taken by all map tasks=78205691904
  197.         Total megabyte-milliseconds taken by all reduce tasks=37115627520
  198.     Map-Reduce Framework
  199.         Map input records=3035
  200.         Map output records=210623260
  201.         Map output bytes=2999192680
  202.         Map output materialized bytes=3420657749
  203.         Input split bytes=388153
  204.         Combine input records=0
  205.         Combine output records=0
  206.         Reduce input groups=1217608
  207.         Reduce shuffle bytes=3420657749
  208.         Reduce input records=210623260
  209.         Reduce output records=1217608
  210.         Spilled Records=421246520
  211.         Shuffled Maps =36420
  212.         Failed Shuffles=0
  213.         Merged Map outputs=36420
  214.         GC time elapsed (ms)=438006
  215.         CPU time spent (ms)=9288880
  216.         Physical memory (bytes) snapshot=1129278623744
  217.         Virtual memory (bytes) snapshot=13430944673792
  218.         Total committed heap usage (bytes)=1000340455424
  219.     Shuffle Errors
  220.         BAD_ID=0
  221.         CONNECTION=0
  222.         IO_ERROR=0
  223.         WRONG_LENGTH=0
  224.         WRONG_MAP=0
  225.         WRONG_REDUCE=0
  226.     File Input Format Counters
  227.         Bytes Read=1141053926
  228.     File Output Format Counters
  229.         Bytes Written=234665158
  230. Job output is complete
RAW Paste Data
Top