Pastebin launched a little side project called VERYVIRAL.com, check it out ;-) Want more features on Pastebin? Sign Up, it's FREE!
Guest

TaskTracker

By: a guest on May 21st, 2011  |  syntax: None  |  size: 79.10 KB  |  views: 431  |  expires: Never
download  |  raw  |  embed  |  report abuse  |  print
Text below is selected. Please press Ctrl+C to copy to your clipboard. (⌘+C on Mac)
  1. Tasktracker log:
  2.  
  3.         2011-05-21 14:51:59,116 INFO org.apache.hadoop.mapred.TaskTracker: STARTUP_MSG:
  4.         /************************************************************
  5.         STARTUP_MSG: Starting TaskTracker
  6.         STARTUP_MSG:   host = ap100/127.0.0.1
  7.         STARTUP_MSG:   args = []
  8.         STARTUP_MSG:   version = 0.20.203.0
  9.         STARTUP_MSG:   build = http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security-203 -r 1099333; compiled by 'oom' on Wed May  4 07:57:50 PDT 2011
  10.         ************************************************************/
  11.         2011-05-21 14:51:59,759 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
  12.         2011-05-21 14:51:59,934 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
  13.         2011-05-21 14:51:59,944 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
  14.         2011-05-21 14:51:59,945 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics system started
  15.         2011-05-21 14:52:01,231 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered.
  16.         2011-05-21 14:52:01,239 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
  17.         2011-05-21 14:52:06,654 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
  18.         2011-05-21 14:52:06,843 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
  19.         2011-05-21 14:52:06,916 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
  20.         2011-05-21 14:52:06,924 INFO org.apache.hadoop.mapred.TaskTracker: Starting tasktracker with owner as hadoop
  21.         2011-05-21 14:52:07,084 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source jvm registered.
  22.         2011-05-21 14:52:07,085 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source TaskTrackerMetrics registered.
  23.         2011-05-21 14:52:07,139 INFO org.apache.hadoop.ipc.Server: Starting SocketReader
  24.         2011-05-21 14:52:07,143 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcDetailedActivityForPort49008 registered.
  25.         2011-05-21 14:52:07,143 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcActivityForPort49008 registered.
  26.         2011-05-21 14:52:07,147 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
  27.         2011-05-21 14:52:07,149 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 49008: starting
  28.         2011-05-21 14:52:07,150 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 49008: starting
  29.         2011-05-21 14:52:07,150 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 49008: starting
  30.         2011-05-21 14:52:07,150 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 49008: starting
  31.         2011-05-21 14:52:07,154 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 49008: starting
  32.         2011-05-21 14:52:07,154 INFO org.apache.hadoop.mapred.TaskTracker: TaskTracker up at: localhost.localdomain/127.0.0.1:49008
  33.         2011-05-21 14:52:07,154 INFO org.apache.hadoop.mapred.TaskTracker: Starting tracker tracker_localhost.localdomain:localhost.localdomain/127.0.0.1:49008
  34.         2011-05-21 14:52:56,586 INFO org.apache.hadoop.mapred.TaskTracker: Starting thread: Map-events fetcher for all reduce tasks on tracker_localhost.localdomain:localhost.localdomain/127.0.0.1:49008
  35.         2011-05-21 14:52:56,588 INFO org.apache.hadoop.mapred.TaskTracker:  Using MemoryCalculatorPlugin : org.apache.hadoop.util.LinuxMemoryCalculatorPlugin@f9c40
  36.         2011-05-21 14:52:56,622 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0
  37.         2011-05-21 14:52:56,623 WARN org.apache.hadoop.mapred.TaskTracker: TaskTracker's totalMemoryAllottedForTasks is -1. TaskMemoryManager is disabled.
  38.         2011-05-21 14:52:56,647 INFO org.apache.hadoop.mapred.IndexCache: IndexCache created with max memory = 10485760
  39.         2011-05-21 14:52:56,675 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ShuffleServerMetrics registered.
  40.         2011-05-21 14:52:56,678 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 50060
  41.         2011-05-21 14:52:56,679 INFO org.apache.hadoop.http.HttpServer: listener.getLocalPort() returned 50060 webServer.getConnectors()[0].getLocalPort() returned 50060
  42.         2011-05-21 14:52:56,679 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50060
  43.         2011-05-21 14:52:56,679 INFO org.mortbay.log: jetty-6.1.26
  44.         2011-05-21 14:52:57,082 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:50060
  45.         2011-05-21 14:52:57,082 INFO org.apache.hadoop.mapred.TaskTracker: FILE_CACHE_SIZE for mapOutputServlet set to : 2000
  46.         2011-05-21 14:52:57,091 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105191244_0002 for user-log deletion with retainTimeStamp:1306072376635
  47.         2011-05-21 14:52:57,092 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105201436_0001 for user-log deletion with retainTimeStamp:1306072376635
  48.         2011-05-21 14:52:57,092 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105201430_0001 for user-log deletion with retainTimeStamp:1306072376635
  49.         2011-05-21 14:52:57,092 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105171858_0001 for user-log deletion with retainTimeStamp:1306072376635
  50.         2011-05-21 14:52:57,092 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105191520_0001 for user-log deletion with retainTimeStamp:1306072376635
  51.         2011-05-21 14:52:57,092 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105201316_0001 for user-log deletion with retainTimeStamp:1306072376635
  52.         2011-05-21 14:52:57,092 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105191739_0001 for user-log deletion with retainTimeStamp:1306072376635
  53.         2011-05-21 14:52:57,092 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105201343_0001 for user-log deletion with retainTimeStamp:1306072376635
  54.         2011-05-21 14:52:57,092 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105191425_0001 for user-log deletion with retainTimeStamp:1306072376635
  55.         2011-05-21 14:52:57,093 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105191441_0001 for user-log deletion with retainTimeStamp:1306072376635
  56.         2011-05-21 14:52:57,093 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105191244_0001 for user-log deletion with retainTimeStamp:1306072376635
  57.         2011-05-21 14:52:57,093 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105201500_0001 for user-log deletion with retainTimeStamp:1306072376635
  58.         2011-05-21 14:52:57,093 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105191907_0001 for user-log deletion with retainTimeStamp:1306072376635
  59.         2011-05-21 14:52:57,093 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105191244_0003 for user-log deletion with retainTimeStamp:1306072376635
  60.         2011-05-21 14:53:48,280 ERROR org.apache.hadoop.mapred.TaskTracker: Caught exception: java.io.IOException: Call to master/192.168.0.2:54311 failed on local exception: java.io.EOFException
  61.                 at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
  62.                 at org.apache.hadoop.ipc.Client.call(Client.java:1033)
  63.                 at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
  64.                 at org.apache.hadoop.mapred.$Proxy5.heartbeat(Unknown Source)
  65.                 at org.apache.hadoop.mapred.TaskTracker.transmitHeartBeat(TaskTracker.java:1667)
  66.                 at org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:1473)
  67.                 at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:2233)
  68.                 at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3432)
  69.         Caused by: java.io.EOFException
  70.                 at java.io.DataInputStream.readInt(DataInputStream.java:375)
  71.                 at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:767)
  72.                 at org.apache.hadoop.ipc.Client$Connection.run(Client.java:712)
  73.  
  74.         2011-05-21 14:53:48,280 INFO org.apache.hadoop.mapred.TaskTracker: Resending 'status' to 'master' with reponseId '15
  75.         2011-05-21 14:53:49,282 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 0 time(s).
  76.         2011-05-21 14:53:50,283 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 1 time(s).
  77.         2011-05-21 14:53:51,284 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 2 time(s).
  78.         2011-05-21 14:53:52,285 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 3 time(s).
  79.         2011-05-21 14:53:53,286 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 4 time(s).
  80.         2011-05-21 14:53:54,286 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 5 time(s).
  81.         2011-05-21 14:53:55,287 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 6 time(s).
  82.         2011-05-21 14:53:56,288 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 7 time(s).
  83.         2011-05-21 14:53:57,289 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 8 time(s).
  84.         2011-05-21 14:53:58,290 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 9 time(s).
  85.         2011-05-21 14:53:58,291 ERROR org.apache.hadoop.mapred.TaskTracker: Caught exception: java.net.ConnectException: Call to master/192.168.0.2:54311 failed on connection exception: java.net.ConnectException: Connection refused
  86.                 at org.apache.hadoop.ipc.Client.wrapException(Client.java:1057)
  87.                 at org.apache.hadoop.ipc.Client.call(Client.java:1033)
  88.                 at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
  89.                 at org.apache.hadoop.mapred.$Proxy5.heartbeat(Unknown Source)
  90.                 at org.apache.hadoop.mapred.TaskTracker.transmitHeartBeat(TaskTracker.java:1667)
  91.                 at org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:1473)
  92.                 at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:2233)
  93.                 at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3432)
  94.         Caused by: java.net.ConnectException: Connection refused
  95.                 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  96.                 at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  97.                 at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
  98.                 at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:406)
  99.                 at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:414)
  100.                 at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:527)
  101.                 at org.apache.hadoop.ipc.Client$Connection.access$1800(Client.java:187)
  102.                 at org.apache.hadoop.ipc.Client.getConnection(Client.java:1164)
  103.                 at org.apache.hadoop.ipc.Client.call(Client.java:1010)
  104.                 ... 6 more
  105.  
  106.         2011-05-21 14:53:58,291 INFO org.apache.hadoop.mapred.TaskTracker: Resending 'status' to 'master' with reponseId '15
  107.         2011-05-21 14:53:59,293 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 0 time(s).
  108.         2011-05-21 14:54:00,294 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 1 time(s).
  109.         2011-05-21 14:54:01,294 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 2 time(s).
  110.         2011-05-21 14:54:02,295 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 3 time(s).
  111.         2011-05-21 14:54:03,296 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 4 time(s).
  112.         2011-05-21 14:54:04,297 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 5 time(s).
  113.         2011-05-21 14:54:05,298 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 6 time(s).
  114.         2011-05-21 14:54:06,299 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 7 time(s).
  115.         2011-05-21 14:54:07,300 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 8 time(s).
  116.         2011-05-21 14:54:08,301 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 9 time(s).
  117.         2011-05-21 14:54:08,302 ERROR org.apache.hadoop.mapred.TaskTracker: Caught exception: java.net.ConnectException: Call to master/192.168.0.2:54311 failed on connection exception: java.net.ConnectException: Connection refused
  118.                 at org.apache.hadoop.ipc.Client.wrapException(Client.java:1057)
  119.                 at org.apache.hadoop.ipc.Client.call(Client.java:1033)
  120.                 at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
  121.                 at org.apache.hadoop.mapred.$Proxy5.heartbeat(Unknown Source)
  122.                 at org.apache.hadoop.mapred.TaskTracker.transmitHeartBeat(TaskTracker.java:1667)
  123.                 at org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:1473)
  124.                 at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:2233)
  125.                 at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3432)
  126.         Caused by: java.net.ConnectException: Connection refused
  127.                 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  128.                 at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  129.                 at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
  130.                 at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:406)
  131.                 at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:414)
  132.                 at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:527)
  133.                 at org.apache.hadoop.ipc.Client$Connection.access$1800(Client.java:187)
  134.                 at org.apache.hadoop.ipc.Client.getConnection(Client.java:1164)
  135.                 at org.apache.hadoop.ipc.Client.call(Client.java:1010)
  136.                 ... 6 more
  137.  
  138.         2011-05-21 14:54:08,302 INFO org.apache.hadoop.mapred.TaskTracker: Resending 'status' to 'master' with reponseId '15
  139.         2011-05-21 14:54:09,303 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.0.2:54311. Already tried 0 time(s).
  140.         2011-05-21 14:54:14,625 INFO org.apache.hadoop.mapred.TaskTracker: Recieved RenitTrackerAction from JobTracker
  141.         2011-05-21 14:54:14,652 INFO org.apache.hadoop.mapred.TaskTracker: Shutting down: Map-events fetcher for all reduce tasks on tracker_localhost.localdomain:localhost.localdomain/127.0.0.1:49008
  142.         2011-05-21 14:54:14,653 INFO org.apache.hadoop.ipc.Server: Stopping server on 49008
  143.         2011-05-21 14:54:14,654 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 49008: exiting
  144.         2011-05-21 14:54:14,654 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 49008: exiting
  145.         2011-05-21 14:54:14,654 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 49008: exiting
  146.         2011-05-21 14:54:14,655 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 49008
  147.         2011-05-21 14:54:14,655 INFO org.apache.hadoop.ipc.metrics.RpcInstrumentation: shut down
  148.         2011-05-21 14:54:14,655 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 49008: exiting
  149.         2011-05-21 14:54:14,656 WARN org.apache.hadoop.mapred.TaskTracker: Reinitializing local state
  150.         2011-05-21 14:54:14,656 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
  151.         2011-05-21 14:54:14,656 INFO org.apache.hadoop.mapred.TaskTracker: Starting tasktracker with owner as hadoop
  152.         2011-05-21 14:54:14,660 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name jvm already exists!
  153.         2011-05-21 14:54:14,661 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name TaskTrackerMetrics already exists!
  154.         2011-05-21 14:54:14,662 INFO org.apache.hadoop.ipc.Server: Starting SocketReader
  155.         2011-05-21 14:54:14,662 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcDetailedActivityForPort38271 registered.
  156.         2011-05-21 14:54:14,663 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcActivityForPort38271 registered.
  157.         2011-05-21 14:54:14,663 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
  158.         2011-05-21 14:54:14,665 INFO org.apache.hadoop.mapred.TaskTracker: TaskTracker up at: localhost.localdomain/127.0.0.1:38271
  159.         2011-05-21 14:54:14,665 INFO org.apache.hadoop.mapred.TaskTracker: Starting tracker tracker_localhost.localdomain:localhost.localdomain/127.0.0.1:38271
  160.         2011-05-21 14:54:14,674 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 38271: starting
  161.         2011-05-21 14:54:14,675 INFO org.apache.hadoop.mapred.TaskTracker:  Using MemoryCalculatorPlugin : org.apache.hadoop.util.LinuxMemoryCalculatorPlugin@1eb5666
  162.         2011-05-21 14:54:14,675 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 38271: starting
  163.         2011-05-21 14:54:14,675 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 38271: starting
  164.         2011-05-21 14:54:14,675 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 38271: starting
  165.         2011-05-21 14:54:14,675 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 38271: starting
  166.         2011-05-21 14:54:14,678 INFO org.apache.hadoop.mapred.TaskTracker: Starting thread: Map-events fetcher for all reduce tasks on tracker_localhost.localdomain:localhost.localdomain/127.0.0.1:38271
  167.         2011-05-21 14:54:14,677 WARN org.apache.hadoop.mapred.TaskTracker: TaskTracker's totalMemoryAllottedForTasks is -1. TaskMemoryManager is disabled.
  168.         2011-05-21 14:54:14,679 INFO org.apache.hadoop.mapred.IndexCache: IndexCache created with max memory = 10485760
  169.         2011-05-21 14:55:35,864 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000008_0 task's state:UNASSIGNED
  170.         2011-05-21 14:55:35,868 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000008_0 which needs 1 slots
  171.         2011-05-21 14:55:35,868 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105211454_0001_m_000008_0 which needs 1 slots
  172.         2011-05-21 14:55:36,226 INFO org.apache.hadoop.mapred.JobLocalizer: Initializing user hadoop on this TT.
  173.         2011-05-21 14:55:36,477 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_-1435574959
  174.         2011-05-21 14:55:36,477 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_-1435574959 spawned.
  175.         2011-05-21 14:55:36,481 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000008_0/taskjvm.sh
  176.         2011-05-21 14:55:37,367 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_-1435574959 given task: attempt_201105211454_0001_m_000008_0
  177.         2011-05-21 14:55:41,066 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000008_0 0.0% setup
  178.         2011-05-21 14:55:41,068 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000008_0 is done.
  179.         2011-05-21 14:55:41,069 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000008_0  was -1
  180.         2011-05-21 14:55:41,070 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
  181.         2011-05-21 14:55:41,155 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_-1435574959 exited with exit code 0. Number of tasks it ran: 1
  182.         2011-05-21 14:55:41,188 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library
  183.         2011-05-21 14:55:41,233 INFO org.apache.hadoop.io.nativeio.NativeIO: Initialized cache for UID to User mapping with a cache timeout of 14400 seconds.
  184.         2011-05-21 14:55:41,234 INFO org.apache.hadoop.io.nativeio.NativeIO: Got UserName hadoop for UID 1001 from the native implementation
  185.         2011-05-21 14:55:41,891 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000000_0 task's state:UNASSIGNED
  186.         2011-05-21 14:55:41,891 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000001_0 task's state:UNASSIGNED
  187.         2011-05-21 14:55:41,891 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000000_0 which needs 1 slots
  188.         2011-05-21 14:55:41,891 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105211454_0001_m_000000_0 which needs 1 slots
  189.         2011-05-21 14:55:41,891 INFO org.apache.hadoop.mapred.TaskTracker: Received KillTaskAction for task: attempt_201105211454_0001_m_000008_0
  190.         2011-05-21 14:55:41,892 INFO org.apache.hadoop.mapred.TaskTracker: About to purge task: attempt_201105211454_0001_m_000008_0
  191.         2011-05-21 14:55:41,892 INFO org.apache.hadoop.mapred.IndexCache: Map ID attempt_201105211454_0001_m_000008_0 not found in cache
  192.         2011-05-21 14:55:41,898 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000001_0 which needs 1 slots
  193.         2011-05-21 14:55:41,898 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 1 and trying to launch attempt_201105211454_0001_m_000001_0 which needs 1 slots
  194.         2011-05-21 14:55:41,903 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_147283024
  195.         2011-05-21 14:55:41,904 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_147283024 spawned.
  196.         2011-05-21 14:55:41,905 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_1732511660
  197.         2011-05-21 14:55:41,906 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_1732511660 spawned.
  198.         2011-05-21 14:55:41,907 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000000_0/taskjvm.sh
  199.         2011-05-21 14:55:41,925 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000001_0/taskjvm.sh
  200.         2011-05-21 14:55:43,470 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_147283024 given task: attempt_201105211454_0001_m_000000_0
  201.         2011-05-21 14:55:43,492 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_1732511660 given task: attempt_201105211454_0001_m_000001_0
  202.         2011-05-21 14:55:50,619 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000001_0 1.0%
  203.         2011-05-21 14:55:50,621 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000000_0 1.0%
  204.         2011-05-21 14:55:50,626 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000001_0 1.0%
  205.         2011-05-21 14:55:50,633 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000000_0 1.0%
  206.         2011-05-21 14:55:50,635 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000001_0 is done.
  207.         2011-05-21 14:55:50,635 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000001_0  was 740452
  208.         2011-05-21 14:55:50,636 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 1
  209.         2011-05-21 14:55:50,639 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000000_0 is done.
  210.         2011-05-21 14:55:50,639 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000000_0  was 807006
  211.         2011-05-21 14:55:50,639 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
  212.         2011-05-21 14:55:50,756 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_147283024 exited with exit code 0. Number of tasks it ran: 1
  213.         2011-05-21 14:55:50,841 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_1732511660 exited with exit code 0. Number of tasks it ran: 1
  214.         2011-05-21 14:55:50,955 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000004_0 task's state:UNASSIGNED
  215.         2011-05-21 14:55:50,957 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000005_0 task's state:UNASSIGNED
  216.         2011-05-21 14:55:50,957 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_r_000000_0 task's state:UNASSIGNED
  217.         2011-05-21 14:55:50,957 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_r_000000_0 which needs 1 slots
  218.         2011-05-21 14:55:50,958 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105211454_0001_r_000000_0 which needs 1 slots
  219.         2011-05-21 14:55:50,958 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000004_0 which needs 1 slots
  220.         2011-05-21 14:55:50,958 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105211454_0001_m_000004_0 which needs 1 slots
  221.         2011-05-21 14:55:50,968 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000005_0 which needs 1 slots
  222.         2011-05-21 14:55:50,968 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 1 and trying to launch attempt_201105211454_0001_m_000005_0 which needs 1 slots
  223.         2011-05-21 14:55:50,973 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_r_-1435574959
  224.         2011-05-21 14:55:50,975 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_534546110
  225.         2011-05-21 14:55:50,976 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_r_-1435574959 spawned.
  226.         2011-05-21 14:55:50,976 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_534546110 spawned.
  227.         2011-05-21 14:55:50,983 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_-887848243
  228.         2011-05-21 14:55:50,983 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_-887848243 spawned.
  229.         2011-05-21 14:55:50,988 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000004_0/taskjvm.sh
  230.         2011-05-21 14:55:50,991 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000005_0/taskjvm.sh
  231.         2011-05-21 14:55:50,996 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_r_000000_0/taskjvm.sh
  232.         2011-05-21 14:55:52,929 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_r_-1435574959 given task: attempt_201105211454_0001_r_000000_0
  233.         2011-05-21 14:55:53,611 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_-887848243 given task: attempt_201105211454_0001_m_000005_0
  234.         2011-05-21 14:55:53,656 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_534546110 given task: attempt_201105211454_0001_m_000004_0
  235.         2011-05-21 14:55:58,027 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000004_0 1.0%
  236.         2011-05-21 14:55:58,029 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000004_0 is done.
  237.         2011-05-21 14:55:58,029 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000004_0  was 217654
  238.         2011-05-21 14:55:58,029 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 1
  239.         2011-05-21 14:55:58,065 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000005_0 1.0%
  240.         2011-05-21 14:55:58,068 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000005_0 is done.
  241.         2011-05-21 14:55:58,068 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000005_0  was 252620
  242.         2011-05-21 14:55:58,069 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
  243.         2011-05-21 14:55:58,198 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_-887848243 exited with exit code 0. Number of tasks it ran: 1
  244.         2011-05-21 14:55:58,261 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_534546110 exited with exit code 0. Number of tasks it ran: 1
  245.         2011-05-21 14:56:00,416 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.0% reduce > copy >
  246.         2011-05-21 14:56:01,294 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50184, bytes: 807006, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000000_0, duration: 71903989
  247.         2011-05-21 14:56:01,339 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50184, bytes: 740452, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000001_0, duration: 28054855
  248.         2011-05-21 14:56:01,373 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000003_0,0) failed :
  249.         org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_0/output/file.out.index in any of the configured local directories
  250.                 at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  251.                 at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  252.                 at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  253.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  254.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  255.                 at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  256.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  257.                 at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  258.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  259.                 at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  260.                 at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  261.                 at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  262.                 at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  263.                 at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  264.                 at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  265.                 at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  266.                 at org.mortbay.jetty.Server.handle(Server.java:326)
  267.                 at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  268.                 at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  269.                 at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  270.                 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  271.                 at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  272.                 at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  273.                 at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  274.  
  275.         2011-05-21 14:56:01,374 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000003_0. Ignored.
  276.         2011-05-21 14:56:01,403 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50185, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000003_0, duration: 30652922
  277.         2011-05-21 14:56:01,403 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_0/output/file.out.index in any of the configured local directories
  278.         2011-05-21 14:56:06,416 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000003_0,0) failed :
  279.         org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_0/output/file.out.index in any of the configured local directories
  280.                 at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  281.                 at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  282.                 at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  283.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  284.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  285.                 at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  286.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  287.                 at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  288.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  289.                 at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  290.                 at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  291.                 at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  292.                 at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  293.                 at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  294.                 at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  295.                 at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  296.                 at org.mortbay.jetty.Server.handle(Server.java:326)
  297.                 at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  298.                 at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  299.                 at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  300.                 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  301.                 at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  302.                 at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  303.                 at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  304.  
  305.         2011-05-21 14:56:06,417 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000003_0. Ignored.
  306.         2011-05-21 14:56:06,422 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  307.         2011-05-21 14:56:06,425 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50186, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000003_0, duration: 9686895
  308.         2011-05-21 14:56:06,426 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_0/output/file.out.index in any of the configured local directories
  309.         2011-05-21 14:56:11,449 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000003_0,0) failed :
  310.         org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_0/output/file.out.index in any of the configured local directories
  311.                 at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  312.                 at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  313.                 at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  314.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  315.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  316.                 at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  317.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  318.                 at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  319.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  320.                 at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  321.                 at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  322.                 at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  323.                 at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  324.                 at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  325.                 at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  326.                 at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  327.                 at org.mortbay.jetty.Server.handle(Server.java:326)
  328.                 at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  329.                 at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  330.                 at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  331.                 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  332.                 at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  333.                 at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  334.                 at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  335.  
  336.         2011-05-21 14:56:11,450 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000003_0. Ignored.
  337.         2011-05-21 14:56:11,454 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50187, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000003_0, duration: 5148548
  338.         2011-05-21 14:56:11,454 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_0/output/file.out.index in any of the configured local directories
  339.         2011-05-21 14:56:12,428 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  340.         2011-05-21 14:56:14,998 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000003_1 task's state:UNASSIGNED
  341.         2011-05-21 14:56:14,998 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000003_1 which needs 1 slots
  342.         2011-05-21 14:56:14,998 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105211454_0001_m_000003_1 which needs 1 slots
  343.         2011-05-21 14:56:15,001 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_-1887686110
  344.         2011-05-21 14:56:15,001 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_-1887686110 spawned.
  345.         2011-05-21 14:56:15,004 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_1/taskjvm.sh
  346.         2011-05-21 14:56:15,433 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  347.         2011-05-21 14:56:15,862 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_-1887686110 given task: attempt_201105211454_0001_m_000003_1
  348.         2011-05-21 14:56:16,502 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000003_0,0) failed :
  349.         org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_0/output/file.out.index in any of the configured local directories
  350.                 at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  351.                 at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  352.                 at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  353.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  354.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  355.                 at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  356.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  357.                 at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  358.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  359.                 at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  360.                 at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  361.                 at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  362.                 at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  363.                 at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  364.                 at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  365.                 at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  366.                 at org.mortbay.jetty.Server.handle(Server.java:326)
  367.                 at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  368.                 at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  369.                 at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  370.                 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  371.                 at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  372.                 at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  373.                 at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  374.  
  375.         2011-05-21 14:56:16,503 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000003_0. Ignored.
  376.         2011-05-21 14:56:16,506 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50191, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000003_0, duration: 3102166
  377.         2011-05-21 14:56:16,506 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000003_0/output/file.out.index in any of the configured local directories
  378.         2011-05-21 14:56:19,521 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000003_1 1.0%
  379.         2011-05-21 14:56:19,523 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000003_1 is done.
  380.         2011-05-21 14:56:19,523 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000003_1  was 267013
  381.         2011-05-21 14:56:19,524 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
  382.         2011-05-21 14:56:19,618 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_-1887686110 exited with exit code 0. Number of tasks it ran: 1
  383.         2011-05-21 14:56:21,437 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  384.         2011-05-21 14:56:21,518 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000002_0,0) failed :
  385.         org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_0/output/file.out.index in any of the configured local directories
  386.                 at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  387.                 at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  388.                 at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  389.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  390.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  391.                 at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  392.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  393.                 at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  394.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  395.                 at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  396.                 at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  397.                 at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  398.                 at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  399.                 at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  400.                 at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  401.                 at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  402.                 at org.mortbay.jetty.Server.handle(Server.java:326)
  403.                 at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  404.                 at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  405.                 at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  406.                 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  407.                 at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  408.                 at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  409.                 at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  410.  
  411.         2011-05-21 14:56:21,518 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000002_0. Ignored.
  412.         2011-05-21 14:56:21,528 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50194, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000002_0, duration: 10930024
  413.         2011-05-21 14:56:21,528 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_0/output/file.out.index in any of the configured local directories
  414.         2011-05-21 14:56:26,536 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000002_0,0) failed :
  415.         org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_0/output/file.out.index in any of the configured local directories
  416.                 at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  417.                 at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  418.                 at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  419.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  420.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  421.                 at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  422.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  423.                 at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  424.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  425.                 at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  426.                 at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  427.                 at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  428.                 at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  429.                 at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  430.                 at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  431.                 at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  432.                 at org.mortbay.jetty.Server.handle(Server.java:326)
  433.                 at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  434.                 at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  435.                 at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  436.                 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  437.                 at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  438.                 at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  439.                 at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  440.  
  441.         2011-05-21 14:56:26,537 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000002_0. Ignored.
  442.         2011-05-21 14:56:26,539 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50195, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000002_0, duration: 3771412
  443.         2011-05-21 14:56:26,539 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_0/output/file.out.index in any of the configured local directories
  444.         2011-05-21 14:56:27,442 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  445.         2011-05-21 14:56:30,445 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  446.         2011-05-21 14:56:31,549 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000002_0,0) failed :
  447.         org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_0/output/file.out.index in any of the configured local directories
  448.                 at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  449.                 at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  450.                 at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  451.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  452.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  453.                 at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  454.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  455.                 at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  456.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  457.                 at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  458.                 at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  459.                 at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  460.                 at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  461.                 at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  462.                 at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  463.                 at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  464.                 at org.mortbay.jetty.Server.handle(Server.java:326)
  465.                 at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  466.                 at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  467.                 at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  468.                 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  469.                 at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  470.                 at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  471.                 at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  472.  
  473.         2011-05-21 14:56:31,549 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000002_0. Ignored.
  474.         2011-05-21 14:56:31,552 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50196, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000002_0, duration: 4560347
  475.         2011-05-21 14:56:31,553 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_0/output/file.out.index in any of the configured local directories
  476.         2011-05-21 14:56:36,451 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  477.         2011-05-21 14:56:36,560 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000002_0,0) failed :
  478.         org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_0/output/file.out.index in any of the configured local directories
  479.                 at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  480.                 at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  481.                 at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  482.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  483.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  484.                 at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  485.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  486.                 at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  487.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  488.                 at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  489.                 at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  490.                 at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  491.                 at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  492.                 at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  493.                 at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  494.                 at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  495.                 at org.mortbay.jetty.Server.handle(Server.java:326)
  496.                 at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  497.                 at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  498.                 at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  499.                 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  500.                 at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  501.                 at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  502.                 at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  503.  
  504.         2011-05-21 14:56:36,560 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000002_0. Ignored.
  505.         2011-05-21 14:56:36,562 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50197, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000002_0, duration: 3483028
  506.         2011-05-21 14:56:36,563 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_0/output/file.out.index in any of the configured local directories
  507.         2011-05-21 14:56:39,031 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000002_1 task's state:UNASSIGNED
  508.         2011-05-21 14:56:39,031 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000002_1 which needs 1 slots
  509.         2011-05-21 14:56:39,031 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105211454_0001_m_000002_1 which needs 1 slots
  510.         2011-05-21 14:56:39,034 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_2057831501
  511.         2011-05-21 14:56:39,035 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_2057831501 spawned.
  512.         2011-05-21 14:56:39,037 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000002_1/taskjvm.sh
  513.         2011-05-21 14:56:39,897 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_2057831501 given task: attempt_201105211454_0001_m_000002_1
  514.         2011-05-21 14:56:41,574 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000006_0,0) failed :
  515.         org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  516.                 at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  517.                 at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  518.                 at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  519.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  520.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  521.                 at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  522.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  523.                 at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  524.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  525.                 at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  526.                 at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  527.                 at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  528.                 at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  529.                 at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  530.                 at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  531.                 at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  532.                 at org.mortbay.jetty.Server.handle(Server.java:326)
  533.                 at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  534.                 at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  535.                 at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  536.                 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  537.                 at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  538.                 at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  539.                 at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  540.  
  541.         2011-05-21 14:56:41,575 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000006_0. Ignored.
  542.         2011-05-21 14:56:41,577 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50203, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000006_0, duration: 3580230
  543.         2011-05-21 14:56:41,578 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  544.         2011-05-21 14:56:42,457 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  545.         2011-05-21 14:56:43,506 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000002_1 1.0%
  546.         2011-05-21 14:56:43,508 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000002_1 is done.
  547.         2011-05-21 14:56:43,508 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000002_1  was 466814
  548.         2011-05-21 14:56:43,508 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
  549.         2011-05-21 14:56:43,601 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_2057831501 exited with exit code 0. Number of tasks it ran: 1
  550.         2011-05-21 14:56:45,461 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  551.         2011-05-21 14:56:46,583 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000006_0,0) failed :
  552.         org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  553.                 at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  554.                 at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  555.                 at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  556.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  557.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  558.                 at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  559.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  560.                 at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  561.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  562.                 at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  563.                 at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  564.                 at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  565.                 at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  566.                 at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  567.                 at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  568.                 at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  569.                 at org.mortbay.jetty.Server.handle(Server.java:326)
  570.                 at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  571.                 at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  572.                 at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  573.                 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  574.                 at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  575.                 at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  576.                 at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  577.  
  578.         2011-05-21 14:56:46,584 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000006_0. Ignored.
  579.         2011-05-21 14:56:46,587 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50204, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000006_0, duration: 4018860
  580.         2011-05-21 14:56:46,587 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  581.         2011-05-21 14:56:51,465 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  582.         2011-05-21 14:56:51,594 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000006_0,0) failed :
  583.         org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  584.                 at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  585.                 at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  586.                 at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  587.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  588.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  589.                 at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  590.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  591.                 at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  592.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  593.                 at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  594.                 at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  595.                 at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  596.                 at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  597.                 at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  598.                 at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  599.                 at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  600.                 at org.mortbay.jetty.Server.handle(Server.java:326)
  601.                 at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  602.                 at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  603.                 at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  604.                 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  605.                 at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  606.                 at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  607.                 at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  608.  
  609.         2011-05-21 14:56:51,594 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000006_0. Ignored.
  610.         2011-05-21 14:56:51,600 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50205, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000006_0, duration: 6509296
  611.         2011-05-21 14:56:51,600 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  612.         2011-05-21 14:56:56,615 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000006_0,0) failed :
  613.         org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  614.                 at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  615.                 at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  616.                 at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  617.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  618.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  619.                 at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  620.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  621.                 at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  622.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  623.                 at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  624.                 at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  625.                 at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  626.                 at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  627.                 at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  628.                 at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  629.                 at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  630.                 at org.mortbay.jetty.Server.handle(Server.java:326)
  631.                 at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  632.                 at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  633.                 at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  634.                 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  635.                 at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  636.                 at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  637.                 at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  638.  
  639.         2011-05-21 14:56:56,616 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000006_0. Ignored.
  640.         2011-05-21 14:56:56,618 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50206, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000006_0, duration: 3181551
  641.         2011-05-21 14:56:56,618 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  642.         2011-05-21 14:56:57,471 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  643.         2011-05-21 14:57:00,061 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000006_1 task's state:UNASSIGNED
  644.         2011-05-21 14:57:00,061 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000006_1 which needs 1 slots
  645.         2011-05-21 14:57:00,062 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105211454_0001_m_000006_1 which needs 1 slots
  646.         2011-05-21 14:57:00,066 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_-1200608404
  647.         2011-05-21 14:57:00,066 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_-1200608404 spawned.
  648.         2011-05-21 14:57:00,069 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_1/taskjvm.sh
  649.         2011-05-21 14:57:00,474 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  650.         2011-05-21 14:57:00,930 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_-1200608404 given task: attempt_201105211454_0001_m_000006_1
  651.         2011-05-21 14:57:01,627 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201105211454_0001_m_000006_0,0) failed :
  652.         org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  653.                 at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
  654.                 at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
  655.                 at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3543)
  656.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  657.                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
  658.                 at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  659.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  660.                 at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
  661.                 at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  662.                 at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  663.                 at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  664.                 at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  665.                 at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  666.                 at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  667.                 at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
  668.                 at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  669.                 at org.mortbay.jetty.Server.handle(Server.java:326)
  670.                 at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  671.                 at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
  672.                 at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
  673.                 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
  674.                 at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  675.                 at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  676.                 at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  677.  
  678.         2011-05-21 14:57:01,635 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child with bad map output: attempt_201105211454_0001_m_000006_0. Ignored.
  679.         2011-05-21 14:57:01,642 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50211, bytes: 0, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000006_0, duration: 15388871
  680.         2011-05-21 14:57:01,643 WARN org.mortbay.log: /mapOutput: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000006_0/output/file.out.index in any of the configured local directories
  681.         2011-05-21 14:57:04,542 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000006_1 1.0%
  682.         2011-05-21 14:57:04,544 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000006_1 is done.
  683.         2011-05-21 14:57:04,545 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000006_1  was 163388
  684.         2011-05-21 14:57:04,545 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
  685.         2011-05-21 14:57:04,638 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_-1200608404 exited with exit code 0. Number of tasks it ran: 1
  686.         2011-05-21 14:57:06,479 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  687.         2011-05-21 14:57:06,660 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50213, bytes: 217654, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000004_0, duration: 12219858
  688.         2011-05-21 14:57:06,678 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50213, bytes: 252620, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000005_0, duration: 11314179
  689.         2011-05-21 14:57:06,723 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50213, bytes: 267013, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000003_1, duration: 18333850
  690.         2011-05-21 14:57:06,754 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50214, bytes: 466814, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000002_1, duration: 18786111
  691.         2011-05-21 14:57:11,769 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:50215, bytes: 163388, op: MAPRED_SHUFFLE, cliID: attempt_201105211454_0001_m_000006_1, duration: 7220652
  692.         2011-05-21 14:57:11,777 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  693.         2011-05-21 14:57:12,325 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.095238104% reduce > copy (2 of 7 at 0.21 MB/s) >
  694.         2011-05-21 14:57:12,491 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.6685357% reduce > reduce
  695.         2011-05-21 14:57:13,366 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_r_000000_0 is in commit-pending, task state:COMMIT_PENDING
  696.         2011-05-21 14:57:13,367 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 0.6685357% reduce > reduce
  697.         2011-05-21 14:57:15,086 INFO org.apache.hadoop.mapred.TaskTracker: Received commit task action for attempt_201105211454_0001_r_000000_0
  698.         2011-05-21 14:57:15,494 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 1.0% reduce > reduce
  699.         2011-05-21 14:57:15,497 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_r_000000_0 1.0% reduce > reduce
  700.         2011-05-21 14:57:15,498 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_r_000000_0 is done.
  701.         2011-05-21 14:57:15,499 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_r_000000_0  was -1
  702.         2011-05-21 14:57:15,499 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
  703.         2011-05-21 14:57:15,564 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_r_-1435574959 exited with exit code 0. Number of tasks it ran: 1
  704.         2011-05-21 14:57:18,102 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105211454_0001_m_000007_0 task's state:UNASSIGNED
  705.         2011-05-21 14:57:18,102 INFO org.apache.hadoop.mapred.TaskTracker: Received KillTaskAction for task: attempt_201105211454_0001_r_000000_0
  706.         2011-05-21 14:57:18,104 INFO org.apache.hadoop.mapred.TaskTracker: About to purge task: attempt_201105211454_0001_r_000000_0
  707.         2011-05-21 14:57:18,102 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105211454_0001_m_000007_0 which needs 1 slots
  708.         2011-05-21 14:57:18,104 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105211454_0001_m_000007_0 which needs 1 slots
  709.         2011-05-21 14:57:18,108 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105211454_0001_m_1594890384
  710.         2011-05-21 14:57:18,109 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105211454_0001_m_1594890384 spawned.
  711.         2011-05-21 14:57:18,111 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /usr/local/hadoop/temp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105211454_0001/attempt_201105211454_0001_m_000007_0/taskjvm.sh
  712.         2011-05-21 14:57:18,967 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105211454_0001_m_1594890384 given task: attempt_201105211454_0001_m_000007_0
  713.         2011-05-21 14:57:19,588 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000007_0 0.0%
  714.         2011-05-21 14:57:22,578 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105211454_0001_m_000007_0 0.0% cleanup
  715.         2011-05-21 14:57:22,580 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105211454_0001_m_000007_0 is done.
  716.         2011-05-21 14:57:22,580 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105211454_0001_m_000007_0  was -1
  717.         2011-05-21 14:57:22,581 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
  718.         2011-05-21 14:57:22,653 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105211454_0001_m_1594890384 exited with exit code 0. Number of tasks it ran: 1
  719.         2011-05-21 14:57:24,205 INFO org.apache.hadoop.mapred.TaskTracker: Received 'KillJobAction' for job: job_201105211454_0001
  720.         2011-05-21 14:57:24,205 INFO org.apache.hadoop.mapred.IndexCache: Map ID attempt_201105211454_0001_m_000007_0 not found in cache
  721.         2011-05-21 14:57:24,207 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201105211454_0001 for user-log deletion with retainTimeStamp:1306072644207