API tools faq
paste
Guest User

TaskTracker

a guest
May 21st, 2011
661
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 79.10 KB | None | 0 0
raw download clone embed print report
  1. Tasktracker log:
  2.  
  3. 2011-05-21 14:51:59,116 INFO org.apache.hadoop.mapred.TaskTracker: STARTUP_MSG:
  4. /************************************************************
  5. STARTUP_MSG: Starting TaskTracker
  6. STARTUP_MSG: host = ap100/127.0.0.1
  7. STARTUP_MSG: args = []
  8. STARTUP_MSG: version = 0.20.203.0
  9. STARTUP_MSG: build = http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security-203 -r 1099333; compiled by 'oom' on Wed May 4 07:57:50 PDT 2011
  10. ************************************************************/
  11. 2011-05-21 14:51:59,759 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
  12. 2011-05-21 14:51:59,934 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
  13. 2011-05-21 14:51:59,944 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
  14. 2011-05-21 14:51:59,945 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics system started
  15. 2011-05-21 14:52:01,231 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered.
  16. 2011-05-21 14:52:01,239 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
  17. 2011-05-21 14:52:06,654 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
  18. 2011-05-21 14:52:06,843 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
  19. 2011-05-21 14:52:06,916 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
  20. 2011-05-21 14:52:06,924 INFO org.apache.hadoop.mapred.TaskTracker: Starting tasktracker with owner as hadoop
  21. 2011-05-21 14:52:07,084 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source jvm registered.
  22. 2011-05-21 14:52:07,085 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source TaskTrackerMetrics registered.
  23. 2011-05-21 14:52:07,139 INFO org.apache.hadoop.ipc.Server: Starting SocketReader
  24. 2011-05-21 14:52:07,143 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcDetailedActivityForPort49008 registered.
  25. 2011-05-21 14:52:07,143 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcActivityForPort49008 registered.
  26. 2011-05-21 14:52:07,147 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
  27. 2011-05-21 14:52:07,149 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 49008: starting
  28. 2011-05-21 14:52:07,150 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 49008: starting
  29. 2011-05-21 14:52:07,150 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 49008: starting
  30. 2011-05-21 14:52:07,150 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 49008: starting
  31. 2011-05-21 14:52:07,154 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 49008: starting
  32. 2011-05-21 14:52:07,154 INFO org.apache.hadoop.mapred.TaskTracker: TaskTracker up at: localhost.localdomain/127.0.0.1:49008
  33. 2011-05-21 14:52:07,154 INFO org.apache.hadoop.mapred.TaskTracker: Starting tracker tracker_localhost.localdomain:localhost.localdomain/127.0.0.1:49008
  34. 2011-05-21 14:52:56,586 INFO org.apache.hadoop.mapred.TaskTracker: Starting thread: Map-events fetcher for all reduce tasks on tracker_localhost.localdomain:localhost.localdomain/127.0.0.1:49008
  35. 2011-05-21 14:52:56,588 INFO org.apache.hadoop.mapred.TaskTracker: Using MemoryCalculatorPlugin : org.apache.hadoop.util.LinuxMemoryCalculatorPlugin@f9c40
  36. 2011-05-21 14:52:56,622 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0
  37. 2011-05-21 14:52:56,623 WARN org.apache.hadoop.mapred.TaskTracker: TaskTracker's totalMemoryAllottedForTasks is -1. TaskMemoryManager is disabled.
  38. 2011-05-21 14:52:56,647 INFO org.apache.hadoop.mapred.IndexCache: IndexCache created with max memory = 10485760
  39. 2011-05-21 14:52:56,675 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ShuffleServerMetrics registered.
  40. 2011-05-21 14:52:56,678 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 50060
  41. 2011-05-21 14:52:56,679 INFO org.apache.hadoop.http.HttpServer: listener.getLocalPort() returned 50060 webServer.getConnectors()[0].getLocalPort() returned 50060
  42. 2011-05-21 14:52:56,679 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50060
  43. 2011-05-21 14:52:56,679 INFO org.mortbay.log: jetty-6.1.26
  44. 2011-05-21 14:52:57,082 INFO org.mortbay.log: Started [email protected]:50060
  45. 2011-05-21 14:52:57,082 INFO org.apache.hadoop.mapred.TaskTracker: FILE_CACHE_SIZE for mapOutputServlet set to : 2000
  46. 2011-05-21 14:52:57,091 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105191244_0002 for user-log deletion with retainTimeStamp:1306072376635
  47. 2011-05-21 14:52:57,092 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105201436_0001 for user-log deletion with retainTimeStamp:1306072376635
  48. 2011-05-21 14:52:57,092 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105201430_0001 for user-log deletion with retainTimeStamp:1306072376635
  49. 2011-05-21 14:52:57,092 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105171858_0001 for user-log deletion with retainTimeStamp:1306072376635
  50. 2011-05-21 14:52:57,092 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105191520_0001 for user-log deletion with retainTimeStamp:1306072376635
  51. 2011-05-21 14:52:57,092 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105201316_0001 for user-log deletion with retainTimeStamp:1306072376635
  52. 2011-05-21 14:52:57,092 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105191739_0001 for user-log deletion with retainTimeStamp:1306072376635
  53. 2011-05-21 14:52:57,092 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105201343_0001 for user-log deletion with retainTimeStamp:1306072376635
  54. 2011-05-21 14:52:57,092 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105191425_0001 for user-log deletion with retainTimeStamp:1306072376635
  55. 2011-05-21 14:52:57,093 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105191441_0001 for user-log deletion with retainTimeStamp:1306072376635
  56. 2011-05-21 14:52:57,093 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105191244_0001 for user-log deletion with retainTimeStamp:1306072376635
  57. 2011-05-21 14:52:57,093 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105201500_0001 for user-log deletion with retainTimeStamp:1306072376635
  58. 2011-05-21 14:52:57,093 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105191907_0001 for user-log deletion with retainTimeStamp:1306072376635
  59. 2011-05-21 14:52:57,093 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105191244_0003 for user-log deletion with retainTimeStamp:1306072376635
  60. 2011-05-21 14:53:48,280 ERROR org.apache.hadoop.mapred.TaskTracker: Caught exception: java.io.IOException: Call to master/192.168.0.2:54311 failed on local exception: java.io.EOFException
  61. at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
  62. at org.apache.hadoop.ipc.Client.call(Client.java:1033)
  63. at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
  64. at org.apache.hadoop.mapred.$Proxy5.heartbeat(Unknown Source)
  65. at org.apache.hadoop.mapred.TaskTracker.transmitHeartBeat(TaskTracker.java:1667)
  66. at org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:1473)
  67. at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:2233)
  68. at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3432)
  69. Caused by: java.io.EOFException
  70. at java.io.DataInputStream.readInt(DataInputStream.java:375)
  71. at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:767)
  72. at org.apache.hadoop.ipc.Client$Connection.run(Client.java:712)
  73.  
  74. 2011-05-21 14:53:48,280 INFO org.apache.hadoop.mapred.TaskTracker: Resending 'status' to 'master' with reponseId '15
  75. 2011-05-21 14:53:49,282 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 0 time(s).
  76. 2011-05-21 14:53:50,283 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 1 time(s).
  77. 2011-05-21 14:53:51,284 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 2 time(s).
  78. 2011-05-21 14:53:52,285 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 3 time(s).
  79. 2011-05-21 14:53:53,286 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 4 time(s).
  80. 2011-05-21 14:53:54,286 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 5 time(s).
  81. 2011-05-21 14:53:55,287 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 6 time(s).
  82. 2011-05-21 14:53:56,288 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 7 time(s).
  83. 2011-05-21 14:53:57,289 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 8 time(s).
  84. 2011-05-21 14:53:58,290 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 9 time(s).
  85. 2011-05-21 14:53:58,291 ERROR org.apache.hadoop.mapred.TaskTracker: Caught exception: java.net.ConnectException: Call to master/192.168.0.2:54311 failed on connection exception: java.net.ConnectException: Connection refused
  86. at org.apache.hadoop.ipc.Client.wrapException(Client.java:1057)
  87. at org.apache.hadoop.ipc.Client.call(Client.java:1033)
  88. at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
  89. at org.apache.hadoop.mapred.$Proxy5.heartbeat(Unknown Source)
  90. at org.apache.hadoop.mapred.TaskTracker.transmitHeartBeat(TaskTracker.java:1667)
  91. at org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:1473)
  92. at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:2233)
  93. at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3432)
  94. Caused by: java.net.ConnectException: Connection refused
  95. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  96. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  97. at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
  98. at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:406)
  99. at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:414)
  100. at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:527)
  101. at org.apache.hadoop.ipc.Client$Connection.access$1800(Client.java:187)
  102. at org.apache.hadoop.ipc.Client.getConnection(Client.java:1164)
  103. at org.apache.hadoop.ipc.Client.call(Client.java:1010)
  104. ... 6 more
  105.  
  106. 2011-05-21 14:53:58,291 INFO org.apache.hadoop.mapred.TaskTracker: Resending 'status' to 'master' with reponseId '15
  107. 2011-05-21 14:53:59,293 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 0 time(s).
  108. 2011-05-21 14:54:00,294 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 1 time(s).
  109. 2011-05-21 14:54:01,294 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 2 time(s).
  110. 2011-05-21 14:54:02,295 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 3 time(s).
  111. 2011-05-21 14:54:03,296 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 4 time(s).
  112. 2011-05-21 14:54:04,297 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 5 time(s).
  113. 2011-05-21 14:54:05,298 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 6 time(s).
  114. 2011-05-21 14:54:06,299 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 7 time(s).
  115. 2011-05-21 14:54:07,300 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 8 time(s).
  116. 2011-05-21 14:54:08,301 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 9 time(s).
  117. 2011-05-21 14:54:08,302 ERROR org.apache.hadoop.mapred.TaskTracker: Caught exception: java.net.ConnectException: Call to master/192.168.0.2:54311 failed on connection exception: java.net.ConnectException: Connection refused
  118. at org.apache.hadoop.ipc.Client.wrapException(Client.java:1057)
  119. at org.apache.hadoop.ipc.Client.call(Client.java:1033)
  120. at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
  121. at org.apache.hadoop.mapred.$Proxy5.heartbeat(Unknown Source)
  122. at org.apache.hadoop.mapred.TaskTracker.transmitHeartBeat(TaskTracker.java:1667)
  123. at org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:1473)
  124. at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:2233)
  125. at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3432)
  126. Caused by: java.net.ConnectException: Connection refused
  127. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  128. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  129. at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
  130. at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:406)
  131. at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:414)
  132. at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:527)
  133. at org.apache.hadoop.ipc.Client$Connection.access$1800(Client.java:187)
  134. at org.apache.hadoop.ipc.Client.getConnection(Client.java:1164)
  135. at org.apache.hadoop.ipc.Client.call(Client.java:1010)
  136. ... 6 more
  137.  
  138. 2011-05-21 14:54:08,302 INFO org.apache.hadoop.mapred.TaskTracker: Resending 'status' to 'master' with reponseId '15
  139. 2011-05-21 14:54:09,303 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 0 time(s).
  140. 2011-05-21 14:54:14,625 INFO org.apache.hadoop.mapred.TaskTracker: Recieved RenitTrackerAction from JobTracker
  141. 2011-05-21 14:54:14,652 INFO org.apache.hadoop.mapred.TaskTracker: Shutting down: Map-events fetcher for all reduce tasks on tracker_localhost.localdomain:localhost.localdomain/127.0.0.1:49008
  142. 2011-05-21 14:54:14,653 INFO org.apache.hadoop.ipc.Server: Stopping server on 49008
  143. 2011-05-21 14:54:14,654 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 49008: exiting
  144. 2011-05-21 14:54:14,654 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 49008: exiting
  145. 2011-05-21 14:54:14,654 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 49008: exiting
  146. 2011-05-21 14:54:14,655 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 49008
  147. 2011-05-21 14:54:14,655 INFO org.apache.hadoop.ipc.metrics.RpcInstrumentation: shut down
  148. 2011-05-21 14:54:14,655 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 49008: exiting
  149. 2011-05-21 14:54:14,656 WARN org.apache.hadoop.mapred.TaskTracker: Reinitializing local state
  150. 2011-05-21 14:54:14,656 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
  151. 2011-05-21 14:54:14,656 INFO org.apache.hadoop.mapred.TaskTracker: Starting tasktracker with owner as hadoop
  152. 2011-05-21 14:54:14,660 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name jvm already exists!
  153. 2011-05-21 14:54:14,661 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name TaskTrackerMetrics already exists!
  154. 2011-05-21 14:54:14,662 INFO org.apache.hadoop.ipc.Server: Starting SocketReader
  155. 2011-05-21 14:54:14,662 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcDetailedActivityForPort38271 registered.
  156. 2011-05-21 14:54:14,663 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcActivityForPort38271 registered.
  157. 2011-05-21 14:54:14,663 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
  158. 2011-05-21 14:54:14,665 INFO org.apache.hadoop.mapred.TaskTracker: TaskTracker up at: localhost.localdomain/127.0.0.1:38271
  159. 2011-05-21 14:54:14,665 INFO org.apache.hadoop.mapred.TaskTracker: Starting tracker tracker_localhost.localdomain:localhost.localdomain/127.0.0.1:38271
  160. 2011-05-21 14:54:14,674 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 38271: starting
  161. 2011-05-21 14:54:14,675 INFO org.apache.hadoop.mapred.TaskTracker: Using MemoryCalculatorPlugin : org.apache.hadoop.util.LinuxMemoryCalculatorPlugin@1eb5666
  162. 2011-05-21 14:54:14,675 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 38271: starting
  163. 2011-05-21 14:54:14,675 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 38271: starting
  164. 2011-05-21 14:54:14,675 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 38271: starting
  165. 2011-05-21 14:54:14,675 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 38271: starting
  166. 2011-05-21 14:54:14,678 INFO org.apache.hadoop.mapred.TaskTracker: Starting thread: Map-events fetcher for all reduce tasks on tracker_localhost.localdomain:localhost.localdomain/127.0.0.1:38271
  167. 2011-05-21 14:54:14,677 WARN org.apache.hadoop.mapred.TaskTracker: TaskTracker's totalMemoryAllottedForTasks is -1. TaskMemoryManager is disabled.
  168. 2011-05-21 14:54:14,679 INFO org.apache.hadoop.mapred.IndexCache: IndexCache created with max memory = 10485760
  169. 2011-05-21 14:55:35,864 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000008_0 task's state:UNASSIGNED
  170. 2011-05-21 14:55:35,868 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000008_0 which needs 1 slots
  171. 2011-05-21 14:55:35,868 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105211454_0001_m_000008_0 which needs 1 slots
  172. 2011-05-21 14:55:36,226 INFO org.apache.hadoop.mapred.JobLocalizer: Initializing user hadoop on this TT.
  173. 2011-05-21 14:55:36,477 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_-1435574959
  174. 2011-05-21 14:55:36,477 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_-1435574959 spawned.
  175. 2011-05-21 14:55:36,481 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000008_0/taskjvm.sh
  176. 2011-05-21 14:55:37,367 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_-1435574959 given task: attempt_201105211454_0001_m_000008_0
  177. 2011-05-21 14:55:41,066 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000008_0 0.0% setup
  178. 2011-05-21 14:55:41,068 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000008_0 is done.
  179. 2011-05-21 14:55:41,069 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000008_0 was -1
  180. 2011-05-21 14:55:41,070 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
  181. 2011-05-21 14:55:41,155 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_-1435574959 exited with exit code 0. Number of tasks it ran: 1
  182. 2011-05-21 14:55:41,188 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library
  183. 2011-05-21 14:55:41,233 INFO org.apache.hadoop.io.nativeio.NativeIO: Initialized cache for UID to User mapping with a cache timeout of 14400 seconds.
  184. 2011-05-21 14:55:41,234 INFO org.apache.hadoop.io.nativeio.NativeIO: Got UserName hadoop for UID 1001 from the native implementation
  185. 2011-05-21 14:55:41,891 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000000_0 task's state:UNASSIGNED
  186. 2011-05-21 14:55:41,891 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000001_0 task's state:UNASSIGNED
  187. 2011-05-21 14:55:41,891 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000000_0 which needs 1 slots
  188. 2011-05-21 14:55:41,891 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105211454_0001_m_000000_0 which needs 1 slots
  189. 2011-05-21 14:55:41,891 INFO org.apache.hadoop.mapred.TaskTracker: Received KillTaskAction for task: attempt_201105211454_0001_m_000008_0
  190. 2011-05-21 14:55:41,892 INFO org.apache.hadoop.mapred.TaskTracker: About to purge task: attempt_201105211454_0001_m_000008_0
  191. 2011-05-21 14:55:41,892 INFO org.apache.hadoop.mapred.IndexCache: Map ID attempt_201105211454_0001_m_000008_0 not found in cache
  192. 2011-05-21 14:55:41,898 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000001_0 which needs 1 slots
  193. 2011-05-21 14:55:41,898 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 1 and trying to launch attempt_201105211454_0001_m_000001_0 which needs 1 slots
  194. 2011-05-21 14:55:41,903 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_147283024
  195. 2011-05-21 14:55:41,904 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_147283024 spawned.
  196. 2011-05-21 14:55:41,905 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_1732511660
  197. 2011-05-21 14:55:41,906 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_1732511660 spawned.
  198. 2011-05-21 14:55:41,907 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000000_0/taskjvm.sh
  199. 2011-05-21 14:55:41,925 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000001_0/taskjvm.sh
  200. 2011-05-21 14:55:43,470 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_147283024 given task: attempt_201105211454_0001_m_000000_0
  201. 2011-05-21 14:55:43,492 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_1732511660 given task: attempt_201105211454_0001_m_000001_0
  202. 2011-05-21 14:55:50,619 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000001_0 1.0%
  203. 2011-05-21 14:55:50,621 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000000_0 1.0%
  204. 2011-05-21 14:55:50,626 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000001_0 1.0%
  205. 2011-05-21 14:55:50,633 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000000_0 1.0%
  206. 2011-05-21 14:55:50,635 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000001_0 is done.
  207. 2011-05-21 14:55:50,635 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000001_0 was 740452
  208. 2011-05-21 14:55:50,636 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 1
  209. 2011-05-21 14:55:50,639 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000000_0 is done.
  210. 2011-05-21 14:55:50,639 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000000_0 was 807006
  211. 2011-05-21 14:55:50,639 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
  212. 2011-05-21 14:55:50,756 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_147283024 exited with exit code 0. Number of tasks it ran: 1
  213. 2011-05-21 14:55:50,841 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_1732511660 exited with exit code 0. Number of tasks it ran: 1
  214. 2011-05-21 14:55:50,955 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000004_0 task's state:UNASSIGNED
  215. 2011-05-21 14:55:50,957 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000005_0 task's state:UNASSIGNED
  216. 2011-05-21 14:55:50,957 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_r_000000_0 task's state:UNASSIGNED
  217. 2011-05-21 14:55:50,957 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_r_000000_0 which needs 1 slots
  218. 2011-05-21 14:55:50,958 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105211454_0001_r_000000_0 which needs 1 slots
  219. 2011-05-21 14:55:50,958 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000004_0 which needs 1 slots
  220. 2011-05-21 14:55:50,958 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105211454_0001_m_000004_0 which needs 1 slots
  221. 2011-05-21 14:55:50,968 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000005_0 which needs 1 slots
  222. 2011-05-21 14:55:50,968 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 1 and trying to launch attempt_201105211454_0001_m_000005_0 which needs 1 slots
  223. 2011-05-21 14:55:50,973 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_r_-1435574959
  224. 2011-05-21 14:55:50,975 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_534546110
  225. 2011-05-21 14:55:50,976 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_r_-1435574959 spawned.
  226. 2011-05-21 14:55:50,976 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_534546110 spawned.
  227. 2011-05-21 14:55:50,983 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_-887848243
  228. 2011-05-21 14:55:50,983 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_-887848243 spawned.
  229. 2011-05-21 14:55:50,988 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000004_0/taskjvm.sh
  230. 2011-05-21 14:55:50,991 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000005_0/taskjvm.sh
  231. 2011-05-21 14:55:50,996 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_r_000000_0/taskjvm.sh
  232. 2011-05-21 14:55:52,929 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_r_-1435574959 given task: attempt_201105211454_0001_r_000000_0
  233. 2011-05-21 14:55:53,611 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_-887848243 given task: attempt_201105211454_0001_m_000005_0
  234. 2011-05-21 14:55:53,656 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_534546110 given task: attempt_201105211454_0001_m_000004_0
  235. 2011-05-21 14:55:58,027 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000004_0 1.0%
  236. 2011-05-21 14:55:58,029 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000004_0 is done.
  237. 2011-05-21 14:55:58,029 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000004_0 was 217654
  238. 2011-05-21 14:55:58,029 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 1
  239. 2011-05-21 14:55:58,065 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000005_0 1.0%
  240. 2011-05-21 14:55:58,068 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000005_0 is done.
  241. 2011-05-21 14:55:58,068 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000005_0 was 252620
  242. 2011-05-21 14:55:58,069 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
  243. 2011-05-21 14:55:58,198 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_-887848243 exited with exit code 0. Number of tasks it ran: 1
  244. 2011-05-21 14:55:58,261 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_534546110 exited with exit code 0. Number of tasks it ran: 1
  245. 2011-05-21 14:56:00,416 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.0% reduce > copy >
  246. 2011-05-21 14:56:01,294 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50184, bytes: 807006, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000000_0, duration: 71903989
  247. 2011-05-21 14:56:01,339 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50184, bytes: 740452, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000001_0, duration: 28054855
  248. 2011-05-21 14:56:01,373 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000003_0,0) failed :
  249. org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_0/output/file.out.index in any of the configured local directories
  250. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  251. at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  252. at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  253. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  254. at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  255. at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  256. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  257. at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  258. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  259. at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  260. at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  261. at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  262. at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  263. at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  264. at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  265. at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  266. at org.mortbay.jetty.Server.handle(Server.java:326)
  267. at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  268. at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  269. at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  270. at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  271. at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  272. at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  273. at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  274.  
  275. 2011-05-21 14:56:01,374 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000003_0. Ignored.
  276. 2011-05-21 14:56:01,403 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50185, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000003_0, duration: 30652922
  277. 2011-05-21 14:56:01,403 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_0/output/file.out.index in any of the configured local directories
  278. 2011-05-21 14:56:06,416 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000003_0,0) failed :
  279. org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_0/output/file.out.index in any of the configured local directories
  280. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  281. at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  282. at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  283. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  284. at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  285. at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  286. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  287. at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  288. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  289. at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  290. at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  291. at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  292. at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  293. at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  294. at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  295. at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  296. at org.mortbay.jetty.Server.handle(Server.java:326)
  297. at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  298. at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  299. at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  300. at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  301. at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  302. at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  303. at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  304.  
  305. 2011-05-21 14:56:06,417 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000003_0. Ignored.
  306. 2011-05-21 14:56:06,422 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  307. 2011-05-21 14:56:06,425 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50186, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000003_0, duration: 9686895
  308. 2011-05-21 14:56:06,426 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_0/output/file.out.index in any of the configured local directories
  309. 2011-05-21 14:56:11,449 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000003_0,0) failed :
  310. org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_0/output/file.out.index in any of the configured local directories
  311. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  312. at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  313. at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  314. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  315. at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  316. at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  317. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  318. at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  319. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  320. at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  321. at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  322. at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  323. at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  324. at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  325. at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  326. at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  327. at org.mortbay.jetty.Server.handle(Server.java:326)
  328. at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  329. at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  330. at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  331. at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  332. at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  333. at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  334. at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  335.  
  336. 2011-05-21 14:56:11,450 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000003_0. Ignored.
  337. 2011-05-21 14:56:11,454 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50187, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000003_0, duration: 5148548
  338. 2011-05-21 14:56:11,454 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_0/output/file.out.index in any of the configured local directories
  339. 2011-05-21 14:56:12,428 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  340. 2011-05-21 14:56:14,998 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000003_1 task's state:UNASSIGNED
  341. 2011-05-21 14:56:14,998 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000003_1 which needs 1 slots
  342. 2011-05-21 14:56:14,998 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105211454_0001_m_000003_1 which needs 1 slots
  343. 2011-05-21 14:56:15,001 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_-1887686110
  344. 2011-05-21 14:56:15,001 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_-1887686110 spawned.
  345. 2011-05-21 14:56:15,004 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_1/taskjvm.sh
  346. 2011-05-21 14:56:15,433 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  347. 2011-05-21 14:56:15,862 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_-1887686110 given task: attempt_201105211454_0001_m_000003_1
  348. 2011-05-21 14:56:16,502 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000003_0,0) failed :
  349. org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_0/output/file.out.index in any of the configured local directories
  350. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  351. at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  352. at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  353. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  354. at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  355. at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  356. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  357. at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  358. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  359. at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  360. at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  361. at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  362. at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  363. at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  364. at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  365. at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  366. at org.mortbay.jetty.Server.handle(Server.java:326)
  367. at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  368. at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  369. at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  370. at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  371. at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  372. at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  373. at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  374.  
  375. 2011-05-21 14:56:16,503 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000003_0. Ignored.
  376. 2011-05-21 14:56:16,506 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50191, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000003_0, duration: 3102166
  377. 2011-05-21 14:56:16,506 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_0/output/file.out.index in any of the configured local directories
  378. 2011-05-21 14:56:19,521 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000003_1 1.0%
  379. 2011-05-21 14:56:19,523 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000003_1 is done.
  380. 2011-05-21 14:56:19,523 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000003_1 was 267013
  381. 2011-05-21 14:56:19,524 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
  382. 2011-05-21 14:56:19,618 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_-1887686110 exited with exit code 0. Number of tasks it ran: 1
  383. 2011-05-21 14:56:21,437 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  384. 2011-05-21 14:56:21,518 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000002_0,0) failed :
  385. org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_0/output/file.out.index in any of the configured local directories
  386. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  387. at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  388. at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  389. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  390. at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  391. at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  392. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  393. at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  394. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  395. at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  396. at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  397. at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  398. at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  399. at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  400. at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  401. at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  402. at org.mortbay.jetty.Server.handle(Server.java:326)
  403. at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  404. at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  405. at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  406. at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  407. at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  408. at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  409. at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  410.  
  411. 2011-05-21 14:56:21,518 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000002_0. Ignored.
  412. 2011-05-21 14:56:21,528 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50194, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000002_0, duration: 10930024
  413. 2011-05-21 14:56:21,528 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_0/output/file.out.index in any of the configured local directories
  414. 2011-05-21 14:56:26,536 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000002_0,0) failed :
  415. org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_0/output/file.out.index in any of the configured local directories
  416. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  417. at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  418. at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  419. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  420. at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  421. at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  422. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  423. at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  424. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  425. at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  426. at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  427. at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  428. at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  429. at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  430. at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  431. at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  432. at org.mortbay.jetty.Server.handle(Server.java:326)
  433. at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  434. at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  435. at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  436. at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  437. at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  438. at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  439. at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  440.  
  441. 2011-05-21 14:56:26,537 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000002_0. Ignored.
  442. 2011-05-21 14:56:26,539 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50195, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000002_0, duration: 3771412
  443. 2011-05-21 14:56:26,539 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_0/output/file.out.index in any of the configured local directories
  444. 2011-05-21 14:56:27,442 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  445. 2011-05-21 14:56:30,445 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  446. 2011-05-21 14:56:31,549 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000002_0,0) failed :
  447. org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_0/output/file.out.index in any of the configured local directories
  448. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  449. at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  450. at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  451. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  452. at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  453. at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  454. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  455. at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  456. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  457. at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  458. at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  459. at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  460. at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  461. at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  462. at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  463. at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  464. at org.mortbay.jetty.Server.handle(Server.java:326)
  465. at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  466. at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  467. at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  468. at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  469. at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  470. at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  471. at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  472.  
  473. 2011-05-21 14:56:31,549 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000002_0. Ignored.
  474. 2011-05-21 14:56:31,552 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50196, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000002_0, duration: 4560347
  475. 2011-05-21 14:56:31,553 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_0/output/file.out.index in any of the configured local directories
  476. 2011-05-21 14:56:36,451 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  477. 2011-05-21 14:56:36,560 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000002_0,0) failed :
  478. org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_0/output/file.out.index in any of the configured local directories
  479. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  480. at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  481. at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  482. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  483. at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  484. at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  485. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  486. at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  487. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  488. at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  489. at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  490. at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  491. at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  492. at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  493. at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  494. at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  495. at org.mortbay.jetty.Server.handle(Server.java:326)
  496. at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  497. at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  498. at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  499. at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  500. at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  501. at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  502. at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  503.  
  504. 2011-05-21 14:56:36,560 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000002_0. Ignored.
  505. 2011-05-21 14:56:36,562 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50197, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000002_0, duration: 3483028
  506. 2011-05-21 14:56:36,563 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_0/output/file.out.index in any of the configured local directories
  507. 2011-05-21 14:56:39,031 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000002_1 task's state:UNASSIGNED
  508. 2011-05-21 14:56:39,031 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000002_1 which needs 1 slots
  509. 2011-05-21 14:56:39,031 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105211454_0001_m_000002_1 which needs 1 slots
  510. 2011-05-21 14:56:39,034 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_2057831501
  511. 2011-05-21 14:56:39,035 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_2057831501 spawned.
  512. 2011-05-21 14:56:39,037 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_1/taskjvm.sh
  513. 2011-05-21 14:56:39,897 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_2057831501 given task: attempt_201105211454_0001_m_000002_1
  514. 2011-05-21 14:56:41,574 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000006_0,0) failed :
  515. org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  516. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  517. at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  518. at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  519. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  520. at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  521. at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  522. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  523. at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  524. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  525. at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  526. at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  527. at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  528. at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  529. at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  530. at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  531. at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  532. at org.mortbay.jetty.Server.handle(Server.java:326)
  533. at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  534. at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  535. at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  536. at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  537. at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  538. at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  539. at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  540.  
  541. 2011-05-21 14:56:41,575 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000006_0. Ignored.
  542. 2011-05-21 14:56:41,577 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50203, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000006_0, duration: 3580230
  543. 2011-05-21 14:56:41,578 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  544. 2011-05-21 14:56:42,457 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  545. 2011-05-21 14:56:43,506 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000002_1 1.0%
  546. 2011-05-21 14:56:43,508 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000002_1 is done.
  547. 2011-05-21 14:56:43,508 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000002_1 was 466814
  548. 2011-05-21 14:56:43,508 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
  549. 2011-05-21 14:56:43,601 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_2057831501 exited with exit code 0. Number of tasks it ran: 1
  550. 2011-05-21 14:56:45,461 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  551. 2011-05-21 14:56:46,583 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000006_0,0) failed :
  552. org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  553. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  554. at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  555. at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  556. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  557. at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  558. at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  559. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  560. at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  561. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  562. at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  563. at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  564. at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  565. at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  566. at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  567. at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  568. at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  569. at org.mortbay.jetty.Server.handle(Server.java:326)
  570. at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  571. at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  572. at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  573. at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  574. at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  575. at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  576. at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  577.  
  578. 2011-05-21 14:56:46,584 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000006_0. Ignored.
  579. 2011-05-21 14:56:46,587 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50204, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000006_0, duration: 4018860
  580. 2011-05-21 14:56:46,587 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  581. 2011-05-21 14:56:51,465 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  582. 2011-05-21 14:56:51,594 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000006_0,0) failed :
  583. org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  584. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  585. at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  586. at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  587. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  588. at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  589. at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  590. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  591. at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  592. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  593. at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  594. at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  595. at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  596. at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  597. at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  598. at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  599. at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  600. at org.mortbay.jetty.Server.handle(Server.java:326)
  601. at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  602. at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  603. at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  604. at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  605. at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  606. at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  607. at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  608.  
  609. 2011-05-21 14:56:51,594 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000006_0. Ignored.
  610. 2011-05-21 14:56:51,600 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50205, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000006_0, duration: 6509296
  611. 2011-05-21 14:56:51,600 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  612. 2011-05-21 14:56:56,615 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000006_0,0) failed :
  613. org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  614. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  615. at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  616. at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  617. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  618. at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  619. at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  620. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  621. at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  622. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  623. at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  624. at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  625. at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  626. at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  627. at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  628. at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  629. at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  630. at org.mortbay.jetty.Server.handle(Server.java:326)
  631. at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  632. at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  633. at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  634. at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  635. at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  636. at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  637. at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  638.  
  639. 2011-05-21 14:56:56,616 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000006_0. Ignored.
  640. 2011-05-21 14:56:56,618 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50206, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000006_0, duration: 3181551
  641. 2011-05-21 14:56:56,618 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  642. 2011-05-21 14:56:57,471 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  643. 2011-05-21 14:57:00,061 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000006_1 task's state:UNASSIGNED
  644. 2011-05-21 14:57:00,061 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000006_1 which needs 1 slots
  645. 2011-05-21 14:57:00,062 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105211454_0001_m_000006_1 which needs 1 slots
  646. 2011-05-21 14:57:00,066 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_-1200608404
  647. 2011-05-21 14:57:00,066 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_-1200608404 spawned.
  648. 2011-05-21 14:57:00,069 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_1/taskjvm.sh
  649. 2011-05-21 14:57:00,474 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  650. 2011-05-21 14:57:00,930 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_-1200608404 given task: attempt_201105211454_0001_m_000006_1
  651. 2011-05-21 14:57:01,627 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000006_0,0) failed :
  652. org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  653. at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  654. at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  655. at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  656. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  657. at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  658. at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  659. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  660. at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  661. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  662. at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  663. at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  664. at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  665. at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  666. at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  667. at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  668. at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  669. at org.mortbay.jetty.Server.handle(Server.java:326)
  670. at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  671. at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  672. at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  673. at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  674. at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  675. at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  676. at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  677.  
  678. 2011-05-21 14:57:01,635 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000006_0. Ignored.
  679. 2011-05-21 14:57:01,642 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50211, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000006_0, duration: 15388871
  680. 2011-05-21 14:57:01,643 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  681. 2011-05-21 14:57:04,542 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000006_1 1.0%
  682. 2011-05-21 14:57:04,544 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000006_1 is done.
  683. 2011-05-21 14:57:04,545 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000006_1 was 163388
  684. 2011-05-21 14:57:04,545 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
  685. 2011-05-21 14:57:04,638 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_-1200608404 exited with exit code 0. Number of tasks it ran: 1
  686. 2011-05-21 14:57:06,479 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  687. 2011-05-21 14:57:06,660 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50213, bytes: 217654, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000004_0, duration: 12219858
  688. 2011-05-21 14:57:06,678 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50213, bytes: 252620, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000005_0, duration: 11314179
  689. 2011-05-21 14:57:06,723 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50213, bytes: 267013, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000003_1, duration: 18333850
  690. 2011-05-21 14:57:06,754 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50214, bytes: 466814, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000002_1, duration: 18786111
  691. 2011-05-21 14:57:11,769 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50215, bytes: 163388, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000006_1, duration: 7220652
  692. 2011-05-21 14:57:11,777 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  693. 2011-05-21 14:57:12,325 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  694. 2011-05-21 14:57:12,491 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.6685357% reduce > reduce
  695. 2011-05-21 14:57:13,366 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_r_000000_0 is in commit-pending, task state:COMMIT_PENDING
  696. 2011-05-21 14:57:13,367 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.6685357% reduce > reduce
  697. 2011-05-21 14:57:15,086 INFO org.apache.hadoop.mapred.TaskTracker: Received commit task action for attempt_201105211454_0001_r_000000_0
  698. 2011-05-21 14:57:15,494 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 1.0% reduce > reduce
  699. 2011-05-21 14:57:15,497 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 1.0% reduce > reduce
  700. 2011-05-21 14:57:15,498 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_r_000000_0 is done.
  701. 2011-05-21 14:57:15,499 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_r_000000_0 was -1
  702. 2011-05-21 14:57:15,499 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
  703. 2011-05-21 14:57:15,564 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_r_-1435574959 exited with exit code 0. Number of tasks it ran: 1
  704. 2011-05-21 14:57:18,102 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000007_0 task's state:UNASSIGNED
  705. 2011-05-21 14:57:18,102 INFO org.apache.hadoop.mapred.TaskTracker: Received KillTaskAction for task: attempt_201105211454_0001_r_000000_0
  706. 2011-05-21 14:57:18,104 INFO org.apache.hadoop.mapred.TaskTracker: About to purge task: attempt_201105211454_0001_r_000000_0
  707. 2011-05-21 14:57:18,102 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000007_0 which needs 1 slots
  708. 2011-05-21 14:57:18,104 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105211454_0001_m_000007_0 which needs 1 slots
  709. 2011-05-21 14:57:18,108 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_1594890384
  710. 2011-05-21 14:57:18,109 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_1594890384 spawned.
  711. 2011-05-21 14:57:18,111 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000007_0/taskjvm.sh
  712. 2011-05-21 14:57:18,967 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_1594890384 given task: attempt_201105211454_0001_m_000007_0
  713. 2011-05-21 14:57:19,588 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000007_0 0.0%
  714. 2011-05-21 14:57:22,578 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000007_0 0.0% cleanup
  715. 2011-05-21 14:57:22,580 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000007_0 is done.
  716. 2011-05-21 14:57:22,580 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000007_0 was -1
  717. 2011-05-21 14:57:22,581 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
  718. 2011-05-21 14:57:22,653 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_1594890384 exited with exit code 0. Number of tasks it ran: 1
  719. 2011-05-21 14:57:24,205 INFO org.apache.hadoop.mapred.TaskTracker: Received 'KillJobAction' for job: job_201105211454_0001
  720. 2011-05-21 14:57:24,205 INFO org.apache.hadoop.mapred.IndexCache: Map ID attempt_201105211454_0001_m_000007_0 not found in cache
  721. 2011-05-21 14:57:24,207 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105211454_0001 for user-log deletion with retainTimeStamp:1306072644207
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!