Guest User

output log

a guest
Jul 10th, 2018
369
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 17.99 KB | None | 0 0
  1. C:\Syncfusion\BigData\3.2.0.20\BigDataSDK\SDK\Hadoop\bin>hadoop jar C:\Users\suman\Desktop\domaingendercount.jar /Data/people.csv /Data/Output/
  2. 18/07/07 23:51:57 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
  3. 18/07/07 23:51:57 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
  4. 18/07/07 23:51:58 INFO mapred.FileInputFormat: Total input paths to process : 1
  5. 18/07/07 23:51:58 INFO mapreduce.JobSubmitter: number of splits:2
  6. 18/07/07 23:51:58 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1530507159389_0006
  7. 18/07/07 23:51:59 INFO impl.YarnClientImpl: Submitted application application_1530507159389_0006
  8. 18/07/07 23:51:59 INFO mapreduce.Job: The url to track the job: http://suman-PC:8088/proxy/application_1530507159389_0006/
  9. 18/07/07 23:51:59 INFO mapreduce.Job: Running job: job_1530507159389_0006
  10. 18/07/07 23:52:07 INFO mapreduce.Job: Job job_1530507159389_0006 running in uber mode : false
  11. 18/07/07 23:52:07 INFO mapreduce.Job: map 0% reduce 0%
  12. 18/07/07 23:52:16 INFO mapreduce.Job: Task Id : attempt_1530507159389_0006_m_000000_0, Status : FAILED
  13. Error: java.io.IOException: wrong value class: class org.apache.hadoop.io.IntWritable is not class org.apache.hadoop.io.Text
  14. at org.apache.hadoop.mapred.IFile$Writer.append(IFile.java:194)
  15. at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1350)
  16. at peoplemail.DomainGenderCount$ReduceClass.reduce(DomainGenderCount.java:52)
  17. at peoplemail.DomainGenderCount$ReduceClass.reduce(DomainGenderCount.java:1)
  18. at org.apache.hadoop.mapred.Task$OldCombinerRunner.combine(Task.java:1615)
  19. at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1637)
  20. at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1489)
  21. at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:460)
  22. at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  23. at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
  24. at java.security.AccessController.doPrivileged(Native Method)
  25. at javax.security.auth.Subject.doAs(Subject.java:415)
  26. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
  27. at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  28.  
  29. 18/07/07 23:52:18 INFO mapreduce.Job: Task Id : attempt_1530507159389_0006_m_000001_0, Status : FAILED
  30. Error: java.lang.ArrayIndexOutOfBoundsException: 4
  31. at peoplemail.DomainGenderCount$MapClass.map(DomainGenderCount.java:34)
  32. at peoplemail.DomainGenderCount$MapClass.map(DomainGenderCount.java:1)
  33. at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  34. at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
  35. at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  36. at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
  37. at java.security.AccessController.doPrivileged(Native Method)
  38. at javax.security.auth.Subject.doAs(Subject.java:415)
  39. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
  40. at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  41.  
  42. 18/07/07 23:52:26 INFO mapreduce.Job: map 50% reduce 0%
  43. 18/07/07 23:52:26 INFO mapreduce.Job: Task Id : attempt_1530507159389_0006_m_000000_1, Status : FAILED
  44. Error: java.io.IOException: wrong value class: class org.apache.hadoop.io.IntWritable is not class org.apache.hadoop.io.Text
  45. at org.apache.hadoop.mapred.IFile$Writer.append(IFile.java:194)
  46. at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1350)
  47. at peoplemail.DomainGenderCount$ReduceClass.reduce(DomainGenderCount.java:52)
  48. at peoplemail.DomainGenderCount$ReduceClass.reduce(DomainGenderCount.java:1)
  49. at org.apache.hadoop.mapred.Task$OldCombinerRunner.combine(Task.java:1615)
  50. at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1637)
  51. at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1489)
  52. at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:460)
  53. at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  54. at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
  55. at java.security.AccessController.doPrivileged(Native Method)
  56. at javax.security.auth.Subject.doAs(Subject.java:415)
  57. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
  58. at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  59.  
  60. 18/07/07 23:52:27 INFO mapreduce.Job: map 0% reduce 0%
  61. 18/07/07 23:52:28 INFO mapreduce.Job: map 50% reduce 0%
  62. 18/07/07 23:52:28 INFO mapreduce.Job: Task Id : attempt_1530507159389_0006_m_000001_1, Status : FAILED
  63. Error: java.lang.ArrayIndexOutOfBoundsException: 4
  64. at peoplemail.DomainGenderCount$MapClass.map(DomainGenderCount.java:34)
  65. at peoplemail.DomainGenderCount$MapClass.map(DomainGenderCount.java:1)
  66. at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  67. at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
  68. at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  69. at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
  70. at java.security.AccessController.doPrivileged(Native Method)
  71. at javax.security.auth.Subject.doAs(Subject.java:415)
  72. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
  73. at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  74.  
  75. 18/07/07 23:52:29 INFO mapreduce.Job: map 0% reduce 0%
  76. 18/07/07 23:52:37 INFO mapreduce.Job: Task Id : attempt_1530507159389_0006_m_000000_2, Status : FAILED
  77. Error: java.io.IOException: wrong value class: class org.apache.hadoop.io.IntWritable is not class org.apache.hadoop.io.Text
  78. at org.apache.hadoop.mapred.IFile$Writer.append(IFile.java:194)
  79. at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1350)
  80. at peoplemail.DomainGenderCount$ReduceClass.reduce(DomainGenderCount.java:52)
  81. at peoplemail.DomainGenderCount$ReduceClass.reduce(DomainGenderCount.java:1)
  82. at org.apache.hadoop.mapred.Task$OldCombinerRunner.combine(Task.java:1615)
  83. at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1637)
  84. at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1489)
  85. at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:460)
  86. at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  87. at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
  88. at java.security.AccessController.doPrivileged(Native Method)
  89. at javax.security.auth.Subject.doAs(Subject.java:415)
  90. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
  91. at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  92.  
  93. 18/07/07 23:52:39 INFO mapreduce.Job: Task Id : attempt_1530507159389_0006_m_000001_2, Status : FAILED
  94. Error: java.lang.ArrayIndexOutOfBoundsException: 4
  95. at peoplemail.DomainGenderCount$MapClass.map(DomainGenderCount.java:34)
  96. at peoplemail.DomainGenderCount$MapClass.map(DomainGenderCount.java:1)
  97. at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  98. at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
  99. at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  100. at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
  101. at java.security.AccessController.doPrivileged(Native Method)
  102. at javax.security.auth.Subject.doAs(Subject.java:415)
  103. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
  104. at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  105.  
  106. 18/07/07 23:52:47 INFO mapreduce.Job: map 100% reduce 100%
  107. 18/07/07 23:52:48 INFO mapreduce.Job: Job job_1530507159389_0006 failed with state FAILED due to: Task failed task_1530507159389_0006_m_000000
  108. Job failed as tasks failed. failedMaps:1 failedReduces:0
  109.  
  110. 18/07/07 23:52:48 INFO mapreduce.Job: Counters: 17
  111. Job Counters
  112. Failed map tasks=7
  113. Killed map tasks=1
  114. Killed reduce tasks=1
  115. Launched map tasks=8
  116. Other local map tasks=6
  117. Rack-local map tasks=2
  118. Total time spent by all maps in occupied slots (ms)=243436
  119. Total time spent by all reduces in occupied slots (ms)=0
  120. Total time spent by all map tasks (ms)=60859
  121. Total time spent by all reduce tasks (ms)=0
  122. Total vcore-milliseconds taken by all map tasks=60859
  123. Total vcore-milliseconds taken by all reduce tasks=0
  124. Total megabyte-milliseconds taken by all map tasks=62319616
  125. Total megabyte-milliseconds taken by all reduce tasks=0
  126. Map-Reduce Framework
  127. CPU time spent (ms)=0
  128. Physical memory (bytes) snapshot=0
  129. Virtual memory (bytes) snapshot=0
  130. Exception in thread "main" java.io.IOException: Job failed!
  131. at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:865)
  132. at peoplemail.DomainGenderCount.run(DomainGenderCount.java:81)
  133. at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
  134. at peoplemail.DomainGenderCount.main(DomainGenderCount.java:61)
  135. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  136. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  137. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  138. at java.lang.reflect.Method.invoke(Method.java:606)
  139. at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
  140. at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
  141.  
  142. C:\Syncfusion\BigData\3.2.0.20\BigDataSDK\SDK\Hadoop\bin>hadoop jar C:\Users\suman\Desktop\domaingendercount.jar /Data/people.csv /Data/Output/
  143. 18/07/07 23:54:09 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
  144. 18/07/07 23:54:09 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
  145. 18/07/07 23:54:10 INFO mapred.FileInputFormat: Total input paths to process : 1
  146. 18/07/07 23:54:10 INFO mapreduce.JobSubmitter: number of splits:2
  147. 18/07/07 23:54:10 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1530507159389_0007
  148. 18/07/07 23:54:11 INFO impl.YarnClientImpl: Submitted application application_1530507159389_0007
  149. 18/07/07 23:54:11 INFO mapreduce.Job: The url to track the job: http://suman-PC:8088/proxy/application_1530507159389_0007/
  150. 18/07/07 23:54:11 INFO mapreduce.Job: Running job: job_1530507159389_0007
  151. 18/07/07 23:54:21 INFO mapreduce.Job: Job job_1530507159389_0007 running in uber mode : false
  152. 18/07/07 23:54:21 INFO mapreduce.Job: map 0% reduce 0%
  153. 18/07/07 23:54:31 INFO mapreduce.Job: Task Id : attempt_1530507159389_0007_m_000000_0, Status : FAILED
  154. Error: java.io.IOException: wrong value class: class org.apache.hadoop.io.IntWritable is not class org.apache.hadoop.io.Text
  155. at org.apache.hadoop.mapred.IFile$Writer.append(IFile.java:194)
  156. at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1350)
  157. at peoplemail.DomainGenderCount$ReduceClass.reduce(DomainGenderCount.java:52)
  158. at peoplemail.DomainGenderCount$ReduceClass.reduce(DomainGenderCount.java:1)
  159. at org.apache.hadoop.mapred.Task$OldCombinerRunner.combine(Task.java:1615)
  160. at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1637)
  161. at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1489)
  162. at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:460)
  163. at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  164. at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
  165. at java.security.AccessController.doPrivileged(Native Method)
  166. at javax.security.auth.Subject.doAs(Subject.java:415)
  167. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
  168. at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  169.  
  170. 18/07/07 23:54:33 INFO mapreduce.Job: Task Id : attempt_1530507159389_0007_m_000001_0, Status : FAILED
  171. Error: java.lang.ArrayIndexOutOfBoundsException: 5
  172. at peoplemail.DomainGenderCount$MapClass.map(DomainGenderCount.java:34)
  173. at peoplemail.DomainGenderCount$MapClass.map(DomainGenderCount.java:1)
  174. at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  175. at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
  176. at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  177. at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
  178. at java.security.AccessController.doPrivileged(Native Method)
  179. at javax.security.auth.Subject.doAs(Subject.java:415)
  180. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
  181. at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  182.  
  183. 18/07/07 23:54:40 INFO mapreduce.Job: Task Id : attempt_1530507159389_0007_m_000000_1, Status : FAILED
  184. Error: java.io.IOException: wrong value class: class org.apache.hadoop.io.IntWritable is not class org.apache.hadoop.io.Text
  185. at org.apache.hadoop.mapred.IFile$Writer.append(IFile.java:194)
  186. at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1350)
  187. at peoplemail.DomainGenderCount$ReduceClass.reduce(DomainGenderCount.java:52)
  188. at peoplemail.DomainGenderCount$ReduceClass.reduce(DomainGenderCount.java:1)
  189. at org.apache.hadoop.mapred.Task$OldCombinerRunner.combine(Task.java:1615)
  190. at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1637)
  191. at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1489)
  192. at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:460)
  193. at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  194. at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
  195. at java.security.AccessController.doPrivileged(Native Method)
  196. at javax.security.auth.Subject.doAs(Subject.java:415)
  197. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
  198. at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  199.  
  200. 18/07/07 23:54:41 INFO mapreduce.Job: Task Id : attempt_1530507159389_0007_m_000001_1, Status : FAILED
  201. Error: java.lang.ArrayIndexOutOfBoundsException: 5
  202. at peoplemail.DomainGenderCount$MapClass.map(DomainGenderCount.java:34)
  203. at peoplemail.DomainGenderCount$MapClass.map(DomainGenderCount.java:1)
  204. at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  205. at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
  206. at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  207. at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
  208. at java.security.AccessController.doPrivileged(Native Method)
  209. at javax.security.auth.Subject.doAs(Subject.java:415)
  210. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
  211. at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  212.  
  213. 18/07/07 23:54:49 INFO mapreduce.Job: Task Id : attempt_1530507159389_0007_m_000000_2, Status : FAILED
  214. Error: java.io.IOException: wrong value class: class org.apache.hadoop.io.IntWritable is not class org.apache.hadoop.io.Text
  215. at org.apache.hadoop.mapred.IFile$Writer.append(IFile.java:194)
  216. at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1350)
  217. at peoplemail.DomainGenderCount$ReduceClass.reduce(DomainGenderCount.java:52)
  218. at peoplemail.DomainGenderCount$ReduceClass.reduce(DomainGenderCount.java:1)
  219. at org.apache.hadoop.mapred.Task$OldCombinerRunner.combine(Task.java:1615)
  220. at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1637)
  221. at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1489)
  222. at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:460)
  223. at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  224. at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
  225. at java.security.AccessController.doPrivileged(Native Method)
  226. at javax.security.auth.Subject.doAs(Subject.java:415)
  227. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
  228. at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  229.  
  230. 18/07/07 23:54:50 INFO mapreduce.Job: Task Id : attempt_1530507159389_0007_m_000001_2, Status : FAILED
  231. Error: java.lang.ArrayIndexOutOfBoundsException: 5
  232. at peoplemail.DomainGenderCount$MapClass.map(DomainGenderCount.java:34)
  233. at peoplemail.DomainGenderCount$MapClass.map(DomainGenderCount.java:1)
  234. at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  235. at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
  236. at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  237. at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
  238. at java.security.AccessController.doPrivileged(Native Method)
  239. at javax.security.auth.Subject.doAs(Subject.java:415)
  240. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
  241. at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
  242.  
  243. 18/07/07 23:54:57 INFO mapreduce.Job: map 50% reduce 0%
  244. 18/07/07 23:54:58 INFO mapreduce.Job: map 100% reduce 100%
  245. 18/07/07 23:54:59 INFO mapreduce.Job: Job job_1530507159389_0007 failed with state FAILED due to: Task failed task_1530507159389_0007_m_000000
  246. Job failed as tasks failed. failedMaps:1 failedReduces:0
  247.  
  248. 18/07/07 23:54:59 INFO mapreduce.Job: Counters: 17
  249. Job Counters
  250. Failed map tasks=7
  251. Killed map tasks=1
  252. Killed reduce tasks=1
  253. Launched map tasks=8
  254. Other local map tasks=6
  255. Rack-local map tasks=2
  256. Total time spent by all maps in occupied slots (ms)=213464
  257. Total time spent by all reduces in occupied slots (ms)=0
  258. Total time spent by all map tasks (ms)=53366
  259. Total time spent by all reduce tasks (ms)=0
  260. Total vcore-milliseconds taken by all map tasks=53366
  261. Total vcore-milliseconds taken by all reduce tasks=0
  262. Total megabyte-milliseconds taken by all map tasks=54646784
  263. Total megabyte-milliseconds taken by all reduce tasks=0
  264. Map-Reduce Framework
  265. CPU time spent (ms)=0
  266. Physical memory (bytes) snapshot=0
  267. Virtual memory (bytes) snapshot=0
  268. Exception in thread "main" java.io.IOException: Job failed!
  269. at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:865)
  270. at peoplemail.DomainGenderCount.run(DomainGenderCount.java:81)
  271. at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
  272. at peoplemail.DomainGenderCount.main(DomainGenderCount.java:61)
  273. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  274. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  275. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  276. at java.lang.reflect.Method.invoke(Method.java:606)
  277. at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
  278. at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Add Comment
Please, Sign In to add comment