Don't like ads? PRO users don't see any ads ;-)
Public Pastes
Guest

Failed_Mapper

By: a guest on Aug 13th, 2012  |  syntax: None  |  size: 79.94 KB  |  hits: 64  |  expires: Never
download  |  raw  |  embed  |  report abuse  |  print
Text below is selected. Please press Ctrl+C to copy to your clipboard. (⌘+C on Mac)
  1. LogType:syslog
  2. LogLength:81061
  3. Log Contents:
  4. 2012-08-07 01:44:39,914 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.map.speculative;  Ignoring.
  5. 2012-08-07 01:44:39,934 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
  6. 2012-08-07 01:44:39,934 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.reduce.speculative;  Ignoring.
  7. 2012-08-07 01:44:39,935 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.reduce.speculative;  Ignoring.
  8. 2012-08-07 01:44:40,038 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
  9. 2012-08-07 01:44:40,039 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.map.speculative;  Ignoring.
  10. 2012-08-07 01:44:41,527 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
  11. 2012-08-07 01:44:41,953 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
  12. 2012-08-07 01:44:41,953 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started
  13. 2012-08-07 01:44:42,803 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.map.speculative;  Ignoring.
  14. 2012-08-07 01:44:42,813 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
  15. 2012-08-07 01:44:42,814 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.reduce.speculative;  Ignoring.
  16. 2012-08-07 01:44:42,814 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.reduce.speculative;  Ignoring.
  17. 2012-08-07 01:44:42,851 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
  18. 2012-08-07 01:44:42,851 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.map.speculative;  Ignoring.
  19. 2012-08-07 01:44:43,218 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
  20. 2012-08-07 01:44:44,662 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.map.speculative;  Ignoring.
  21. 2012-08-07 01:44:44,689 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
  22. 2012-08-07 01:44:44,689 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.reduce.speculative;  Ignoring.
  23. 2012-08-07 01:44:44,690 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.reduce.speculative;  Ignoring.
  24. 2012-08-07 01:44:44,745 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
  25. 2012-08-07 01:44:44,745 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.map.speculative;  Ignoring.
  26. 2012-08-07 01:44:44,783 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /disk/yarn/local/usercache/root/appcache/application_1344324770253_0001
  27. 2012-08-07 01:44:44,923 INFO [main] org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library
  28. 2012-08-07 01:44:45,203 WARN [main] org.apache.hadoop.conf.Configuration: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
  29. 2012-08-07 01:44:45,204 WARN [main] org.apache.hadoop.conf.Configuration: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap
  30. 2012-08-07 01:44:45,204 WARN [main] org.apache.hadoop.conf.Configuration: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
  31. 2012-08-07 01:44:45,204 WARN [main] org.apache.hadoop.conf.Configuration: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
  32. 2012-08-07 01:44:45,205 WARN [main] org.apache.hadoop.conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
  33. 2012-08-07 01:44:45,205 WARN [main] org.apache.hadoop.conf.Configuration: job.local.dir is deprecated. Instead, use mapreduce.job.local.dir
  34. 2012-08-07 01:44:45,220 WARN [main] org.apache.hadoop.conf.Configuration: mapred.cache.localFiles is deprecated. Instead, use mapreduce.job.cache.local.files
  35. 2012-08-07 01:44:45,220 WARN [main] org.apache.hadoop.conf.Configuration: mapred.job.id is deprecated. Instead, use mapreduce.job.id
  36. 2012-08-07 01:44:46,150 WARN [main] org.apache.hadoop.conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id
  37. 2012-08-07 01:44:48,830 INFO [main] org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.3-cdh4.0.0--1, built on 06/04/2012 23:16 GMT
  38. 2012-08-07 01:44:48,830 INFO [main] org.apache.zookeeper.ZooKeeper: Client environment:host.name=ihub-dn-g2
  39. 2012-08-07 01:44:48,830 INFO [main] org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.6.0_31
  40. 2012-08-07 01:44:48,830 INFO [main] org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc.
  41. 2012-08-07 01:44:48,830 INFO [main] org.apache.zookeeper.ZooKeeper: Client environment:java.home=/usr/java/jdk1.6.0_31/jre
  42. 2012-08-07 01:44:48,830 INFO [main] org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=/disk/yarn/local/usercache/root/appcache/application_1344324770253_0001/container_1344324770253_0001_01_000086:/etc/hadoop/conf:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop/hadoop-annotations-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-auth-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.0.0-tests.jar:/usr/lib/hadoop/hadoop-common.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar:/usr/lib/hadoop/lib/hadoop-common-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop/lib/commons-io-2.1.jar:/usr/lib/hadoop/lib/jsr305-1.3.9.jar:/usr/lib/hadoop/lib/stax-api-1.0.1.jar:/usr/lib/hadoop/lib/hadoop-annotations-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/jersey-json-1.8.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/json-simple-1.1.jar:/usr/lib/hadoop/lib/jersey-core-1.8.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/kfs-0.3.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/core-3.1.1.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop/lib/oro-2.0.8.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/lib/commons-math-2.1.jar:/usr/lib/hadoop/lib/commons-logging-api-1.1.jar:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/lib/log4j-1.2.15.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/snappy-java-1.0.3.2.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/hadoop-auth-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/hbase-0.92.1-cdh4.0.0-security.jar:/usr/lib/hadoop/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop/lib/zookeeper-3.4.3-cdh4.0.0.jar:/usr/lib/hadoop/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop/lib/commons-lang-2.5.jar:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop/lib/avro-1.5.4.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/jets3t-0.6.1.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/jline-0.9.94.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/lib/jetty-6.1.26.cloudera.1.jar:/usr/lib/hadoop/lib/jersey-server-1.8.jar:/usr/lib/hadoop/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.1.jar:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.0.0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.3.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.15.jar:/usr/lib/hadoop-hdfs/lib/snappy-java-1.0.3.2.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-hdfs/lib/zookeeper-3.4.3-cdh4.0.0.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-hdfs/lib/avro-1.5.4.jar:/usr/lib/hadoop-hdfs/lib/paranamer-2.3.jar:/usr/lib/hadoop-hdfs/lib/jline-0.9.94.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.0.0-cdh4.0.0-tests.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar:/usr/lib/hadoop-mapreduce/hadoop-archives.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp.jar:/usr/lib/hadoop-mapreduce/hadoop-extras-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-mapreduce/hadoop-extras.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle.jar:/usr/lib/hadoop-mapreduce/hadoop-archives-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-mapreduce/lib/commons-io-2.1.jar:/usr/lib/hadoop-mapreduce/lib/jersey-core-1.8.jar:/usr/lib/hadoop-mapreduce/lib/netty-3.2.3.Final.jar:/usr/lib/hadoop-mapreduce/lib/commons-daemon-1.0.3.jar:/usr/lib/hadoop-mapreduce/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/lib/jersey-guice-1.8.jar:/usr/lib/hadoop-mapreduce/lib/guice-3.0.jar:/usr/lib/hadoop-mapreduce/lib/log4j-1.2.15.jar:/usr/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar:/usr/lib/hadoop-mapreduce/lib/snappy-java-1.0.3.2.jar:/usr/lib/hadoop-mapreduce/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-mapreduce/lib/zookeeper-3.4.3-cdh4.0.0.jar:/usr/lib/hadoop-mapreduce/lib/junit-4.8.2.jar:/usr/lib/hadoop-mapreduce/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-mapreduce/lib/jdiff-1.0.9.jar:/usr/lib/hadoop-mapreduce/lib/avro-1.5.4.jar:/usr/lib/hadoop-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-mapreduce/lib/jline-0.9.94.jar:/usr/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/lib/jersey-server-1.8.jar:/usr/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-site.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-site-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api-2.0.0-cdh4.0.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/lib/commons-io-2.1.jar:/usr/lib/hadoop-yarn/lib/jersey-core-1.8.jar:/usr/lib/hadoop-yarn/lib/netty-3.2.3.Final.jar:/usr/lib/hadoop-yarn/lib/commons-daemon-1.0.3.jar:/usr/lib/hadoop-yarn/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-yarn/lib/jersey-guice-1.8.jar:/usr/lib/hadoop-yarn/lib/guice-3.0.jar:/usr/lib/hadoop-yarn/lib/log4j-1.2.15.jar:/usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/lib/hadoop-yarn/lib/snappy-java-1.0.3.2.jar:/usr/lib/hadoop-yarn/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.3-cdh4.0.0.jar:/usr/lib/hadoop-yarn/lib/junit-4.8.2.jar:/usr/lib/hadoop-yarn/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-yarn/lib/jdiff-1.0.9.jar:/usr/lib/hadoop-yarn/lib/avro-1.5.4.jar:/usr/lib/hadoop-yarn/lib/paranamer-2.3.jar:/usr/lib/hadoop-yarn/lib/asm-3.2.jar:/usr/lib/hadoop-yarn/lib/jline-0.9.94.jar:/usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-yarn/lib/jersey-server-1.8.jar:/usr/lib/hadoop-yarn/lib/javax.inject-1.jar:job.jar:/disk/yarn/local/usercache/root/appcache/application_1344324770253_0001/container_1344324770253_0001_01_000086/hadoop-mapreduce-client-core-2.0.0-cdh4.0.1.jar:/disk/yarn/local/usercache/root/appcache/application_1344324770253_0001/container_1344324770253_0001_01_000086/zookeeper-3.4.3-cdh4.0.0.jar:/disk/yarn/local/usercache/root/appcache/application_1344324770253_0001/container_1344324770253_0001_01_000086/job.jar:/disk/yarn/local/usercache/root/appcache/application_1344324770253_0001/container_1344324770253_0001_01_000086/hadoop-common-2.0.0-cdh4.0.0.jar:/disk/yarn/local/usercache/root/appcache/application_1344324770253_0001/container_1344324770253_0001_01_000086/hbase-0.92.1-cdh4.0.0-security.jar:/disk/yarn/local/usercache/root/appcache/application_1344324770253_0001/container_1344324770253_0001_01_000086/guava-11.0.2.jar
  43. 2012-08-07 01:44:48,830 INFO [main] org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/jdk1.6.0_31/jre/lib/amd64/server:/usr/java/jdk1.6.0_31/jre/lib/amd64:/usr/java/jdk1.6.0_31/jre/../lib/amd64:/disk/yarn/local/usercache/root/appcache/application_1344324770253_0001/container_1344324770253_0001_01_000086:/usr/lib/hadoop/lib/native:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
  44. 2012-08-07 01:44:48,830 INFO [main] org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/disk/yarn/local/usercache/root/appcache/application_1344324770253_0001/container_1344324770253_0001_01_000086/tmp
  45. 2012-08-07 01:44:48,830 INFO [main] org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=
  46. 2012-08-07 01:44:48,830 INFO [main] org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux
  47. 2012-08-07 01:44:48,830 INFO [main] org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64
  48. 2012-08-07 01:44:48,830 INFO [main] org.apache.zookeeper.ZooKeeper: Client environment:os.version=2.6.32-71.el6.x86_64
  49. 2012-08-07 01:44:48,830 INFO [main] org.apache.zookeeper.ZooKeeper: Client environment:user.name=yarn
  50. 2012-08-07 01:44:48,830 INFO [main] org.apache.zookeeper.ZooKeeper: Client environment:user.home=/var/lib/hadoop-yarn
  51. 2012-08-07 01:44:48,830 INFO [main] org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/disk/yarn/local/usercache/root/appcache/application_1344324770253_0001/container_1344324770253_0001_01_000086
  52. 2012-08-07 01:44:48,831 INFO [main] org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=ihub-an-l1:2181,ihub-nn-a2:2181,ihub-an-g1:2181 sessionTimeout=180000 watcher=hconnection
  53. 2012-08-07 01:44:48,936 INFO [main-SendThread()] org.apache.zookeeper.ClientCnxn: Opening socket connection to server /172.31.192.101:2181
  54. 2012-08-07 01:44:48,944 INFO [main] org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier of this process is 31563@ihub-dn-g2
  55. 2012-08-07 01:44:48,963 WARN [main-SendThread(ihub-an-g1:2181)] org.apache.zookeeper.client.ZooKeeperSaslClient: SecurityException: java.lang.SecurityException: Unable to locate a login configuration occurred when trying to find JAAS configuration.
  56. 2012-08-07 01:44:48,963 INFO [main-SendThread(ihub-an-g1:2181)] org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
  57. 2012-08-07 01:44:48,965 INFO [main-SendThread(ihub-an-g1:2181)] org.apache.zookeeper.ClientCnxn: Socket connection established to ihub-an-g1/172.31.192.101:2181, initiating session
  58. 2012-08-07 01:44:48,992 INFO [main-SendThread(ihub-an-g1:2181)] org.apache.zookeeper.ClientCnxn: Session establishment complete on server ihub-an-g1/172.31.192.101:2181, sessionid = 0x138fea6f5e60024, negotiated timeout = 40000
  59. 2012-08-07 01:44:49,426 WARN [main] org.apache.hadoop.conf.Configuration: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
  60. 2012-08-07 01:44:49,690 INFO [main] org.apache.hadoop.hbase.mapreduce.TableOutputFormat: Created table instance for transactions_str_as_dbl
  61. 2012-08-07 01:44:49,753 INFO [main] org.apache.hadoop.mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@6762ba99
  62. 2012-08-07 01:44:50,201 WARN [main] org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy native library is available
  63. 2012-08-07 01:44:50,201 INFO [main] org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy native library loaded
  64. 2012-08-07 01:45:51,913 WARN [main] org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Failed all from region=transactions_str_as_dbl,21115942901751992183067233,1344327626788.7c6d22923691fd784f6957e612b17fec., hostname=ihub-dn-l3, port=60020
  65. java.util.concurrent.ExecutionException: java.net.SocketTimeoutException: Call to ihub-dn-l3/172.31.192.153:60020 failed on socket timeout exception: java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/172.31.192.102:58479 remote=ihub-dn-l3/172.31.192.153:60020]
  66.         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
  67.         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
  68.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1557)
  69.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1409)
  70.         at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:943)
  71.         at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
  72.         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)
  73.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:121)
  74.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:82)
  75.         at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:598)
  76.         at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
  77.         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
  78.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:508)
  79.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:1)
  80.         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
  81.         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:725)
  82.         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
  83.         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
  84.         at java.security.AccessController.doPrivileged(Native Method)
  85.         at javax.security.auth.Subject.doAs(Subject.java:396)
  86.         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
  87.         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
  88. Caused by: java.net.SocketTimeoutException: Call to ihub-dn-l3/172.31.192.153:60020 failed on socket timeout exception: java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/172.31.192.102:58479 remote=ihub-dn-l3/172.31.192.153:60020]
  89.         at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:949)
  90.         at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:922)
  91.         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
  92.         at $Proxy13.multi(Unknown Source)
  93.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1386)
  94.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1384)
  95.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1365)
  96.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1383)
  97.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1381)
  98.         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  99.         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  100.         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  101.         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  102.         at java.lang.Thread.run(Thread.java:662)
  103. Caused by: java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/172.31.192.102:58479 remote=ihub-dn-l3/172.31.192.153:60020]
  104.         at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164)
  105.         at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:154)
  106.         at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:127)
  107.         at java.io.FilterInputStream.read(FilterInputStream.java:116)
  108.         at java.io.FilterInputStream.read(FilterInputStream.java:116)
  109.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection$PingInputStream.read(HBaseClient.java:311)
  110.         at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  111.         at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
  112.         at java.io.DataInputStream.readInt(DataInputStream.java:370)
  113.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:571)
  114.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:505)
  115. 2012-08-07 01:47:38,519 WARN [main] org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Failed all from region=transactions_str_as_dbl,19947532900599510187591305,1344327206886.7fdf4c98e2aca8df2898f474080f1530., hostname=ihub-dn-a3, port=60020
  116. java.util.concurrent.ExecutionException: java.net.SocketTimeoutException: Call to ihub-dn-a3/172.31.192.53:60020 failed on socket timeout exception: java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/172.31.192.102:38809 remote=ihub-dn-a3/172.31.192.53:60020]
  117.         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
  118.         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
  119.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1557)
  120.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1409)
  121.         at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:943)
  122.         at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
  123.         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)
  124.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:121)
  125.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:82)
  126.         at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:598)
  127.         at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
  128.         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
  129.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:508)
  130.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:1)
  131.         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
  132.         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:725)
  133.         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
  134.         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
  135.         at java.security.AccessController.doPrivileged(Native Method)
  136.         at javax.security.auth.Subject.doAs(Subject.java:396)
  137.         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
  138.         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
  139. Caused by: java.net.SocketTimeoutException: Call to ihub-dn-a3/172.31.192.53:60020 failed on socket timeout exception: java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/172.31.192.102:38809 remote=ihub-dn-a3/172.31.192.53:60020]
  140.         at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:949)
  141.         at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:922)
  142.         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
  143.         at $Proxy13.multi(Unknown Source)
  144.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1386)
  145.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1384)
  146.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1365)
  147.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1383)
  148.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1381)
  149.         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  150.         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  151.         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  152.         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  153.         at java.lang.Thread.run(Thread.java:662)
  154. Caused by: java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/172.31.192.102:38809 remote=ihub-dn-a3/172.31.192.53:60020]
  155.         at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164)
  156.         at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:154)
  157.         at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:127)
  158.         at java.io.FilterInputStream.read(FilterInputStream.java:116)
  159.         at java.io.FilterInputStream.read(FilterInputStream.java:116)
  160.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection$PingInputStream.read(HBaseClient.java:311)
  161.         at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  162.         at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
  163.         at java.io.DataInputStream.readInt(DataInputStream.java:370)
  164.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:571)
  165.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:505)
  166. 2012-08-07 01:48:42,004 WARN [main] org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Failed all from region=transactions_str_as_dbl,20361622900733202186851443,1344327228426.5c5f16ab25928051f1660ee0d2b52636., hostname=ihub-dn-l3, port=60020
  167. java.util.concurrent.ExecutionException: java.net.SocketTimeoutException: Call to ihub-dn-l3/172.31.192.153:60020 failed on socket timeout exception: java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/172.31.192.102:50265 remote=ihub-dn-l3/172.31.192.153:60020]
  168.         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
  169.         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
  170.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1557)
  171.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1409)
  172.         at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:943)
  173.         at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
  174.         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)
  175.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:121)
  176.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:82)
  177.         at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:598)
  178.         at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
  179.         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
  180.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:508)
  181.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:1)
  182.         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
  183.         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:725)
  184.         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
  185.         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
  186.         at java.security.AccessController.doPrivileged(Native Method)
  187.         at javax.security.auth.Subject.doAs(Subject.java:396)
  188.         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
  189.         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
  190. Caused by: java.net.SocketTimeoutException: Call to ihub-dn-l3/172.31.192.153:60020 failed on socket timeout exception: java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/172.31.192.102:50265 remote=ihub-dn-l3/172.31.192.153:60020]
  191.         at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:949)
  192.         at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:922)
  193.         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
  194.         at $Proxy13.multi(Unknown Source)
  195.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1386)
  196.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1384)
  197.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1365)
  198.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1383)
  199.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1381)
  200.         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  201.         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  202.         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  203.         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  204.         at java.lang.Thread.run(Thread.java:662)
  205. Caused by: java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/172.31.192.102:50265 remote=ihub-dn-l3/172.31.192.153:60020]
  206.         at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164)
  207.         at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:154)
  208.         at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:127)
  209.         at java.io.FilterInputStream.read(FilterInputStream.java:116)
  210.         at java.io.FilterInputStream.read(FilterInputStream.java:116)
  211.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection$PingInputStream.read(HBaseClient.java:311)
  212.         at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  213.         at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
  214.         at java.io.DataInputStream.readInt(DataInputStream.java:370)
  215.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:571)
  216.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:505)
  217. 2012-08-07 02:01:53,300 WARN [main] org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Failed all from region=transactions_str_as_dbl,17929912900560606187890916,1344358522026.74def75a6e7b1f9d7003988f4cb9b4b4., hostname=ihub-dn-l4, port=60020
  218. java.util.concurrent.ExecutionException: org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server ihub-dn-l4,60020,1344317192178 not running, aborting
  219.         at org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:2896)
  220.         at org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3038)
  221.         at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
  222.         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  223.         at java.lang.reflect.Method.invoke(Method.java:597)
  224.         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
  225.         at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1336)
  226.  
  227.         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
  228.         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
  229.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1557)
  230.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1409)
  231.         at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:943)
  232.         at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
  233.         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)
  234.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:121)
  235.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:82)
  236.         at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:598)
  237.         at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
  238.         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
  239.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:508)
  240.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:1)
  241.         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
  242.         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:725)
  243.         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
  244.         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
  245.         at java.security.AccessController.doPrivileged(Native Method)
  246.         at javax.security.auth.Subject.doAs(Subject.java:396)
  247.         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
  248.         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
  249. Caused by: org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server ihub-dn-l4,60020,1344317192178 not running, aborting
  250.         at org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:2896)
  251.         at org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3038)
  252.         at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
  253.         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  254.         at java.lang.reflect.Method.invoke(Method.java:597)
  255.         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
  256.         at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1336)
  257.  
  258.         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  259.         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
  260.         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
  261.         at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
  262.         at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:96)
  263.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.translateException(HConnectionManager.java:1652)
  264.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1367)
  265.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1383)
  266.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1381)
  267.         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  268.         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  269.         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  270.         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  271.         at java.lang.Thread.run(Thread.java:662)
  272. 2012-08-07 02:02:33,367 WARN [main] org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Encountered problems when prefetch META table:
  273. org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=10, exceptions:
  274. Tue Aug 07 02:01:54 PDT 2012, org.apache.hadoop.hbase.client.HTable$4@54ede19e, java.io.IOException: Call to ihub-dn-l4/172.31.192.154:60020 failed on local exception: java.io.EOFException
  275. Tue Aug 07 02:01:55 PDT 2012, org.apache.hadoop.hbase.client.HTable$4@54ede19e, java.net.ConnectException: Connection refused
  276. Tue Aug 07 02:01:56 PDT 2012, org.apache.hadoop.hbase.client.HTable$4@54ede19e, java.net.ConnectException: Connection refused
  277. Tue Aug 07 02:01:57 PDT 2012, org.apache.hadoop.hbase.client.HTable$4@54ede19e, java.net.ConnectException: Connection refused
  278. Tue Aug 07 02:01:59 PDT 2012, org.apache.hadoop.hbase.client.HTable$4@54ede19e, java.net.ConnectException: Connection refused
  279. Tue Aug 07 02:02:01 PDT 2012, org.apache.hadoop.hbase.client.HTable$4@54ede19e, java.net.ConnectException: Connection refused
  280. Tue Aug 07 02:02:05 PDT 2012, org.apache.hadoop.hbase.client.HTable$4@54ede19e, java.net.ConnectException: Connection refused
  281. Tue Aug 07 02:02:09 PDT 2012, org.apache.hadoop.hbase.client.HTable$4@54ede19e, java.net.ConnectException: Connection refused
  282. Tue Aug 07 02:02:17 PDT 2012, org.apache.hadoop.hbase.client.HTable$4@54ede19e, java.net.ConnectException: Connection refused
  283. Tue Aug 07 02:02:33 PDT 2012, org.apache.hadoop.hbase.client.HTable$4@54ede19e, java.net.ConnectException: Connection refused
  284.  
  285.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1345)
  286.         at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:655)
  287.         at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:155)
  288.         at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:52)
  289.         at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
  290.         at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:127)
  291.         at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:360)
  292.         at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:127)
  293.         at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:103)
  294.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:876)
  295.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:930)
  296.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818)
  297.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1524)
  298.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1409)
  299.         at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:943)
  300.         at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
  301.         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)
  302.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:121)
  303.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:82)
  304.         at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:598)
  305.         at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
  306.         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
  307.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:508)
  308.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:1)
  309.         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
  310.         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:725)
  311.         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
  312.         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
  313.         at java.security.AccessController.doPrivileged(Native Method)
  314.         at javax.security.auth.Subject.doAs(Subject.java:396)
  315.         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
  316.         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
  317. 2012-08-07 02:02:57,754 WARN [main] org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Failed all from region=transactions_str_as_dbl,17929912900560606187890916,1344358522026.74def75a6e7b1f9d7003988f4cb9b4b4., hostname=ihub-dn-l4, port=60020
  318. java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused
  319.         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
  320.         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
  321.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1557)
  322.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1409)
  323.         at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:943)
  324.         at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
  325.         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)
  326.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:121)
  327.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:82)
  328.         at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:598)
  329.         at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
  330.         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
  331.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:508)
  332.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:1)
  333.         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
  334.         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:725)
  335.         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
  336.         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
  337.         at java.security.AccessController.doPrivileged(Native Method)
  338.         at javax.security.auth.Subject.doAs(Subject.java:396)
  339.         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
  340.         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
  341. Caused by: java.net.ConnectException: Connection refused
  342.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  343.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  344.         at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
  345.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
  346.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
  347.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328)
  348.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362)
  349.         at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1045)
  350.         at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897)
  351.         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
  352.         at $Proxy13.multi(Unknown Source)
  353.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1386)
  354.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1384)
  355.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1365)
  356.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1383)
  357.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1381)
  358.         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  359.         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  360.         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  361.         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  362.         at java.lang.Thread.run(Thread.java:662)
  363. 2012-08-07 02:02:58,856 WARN [main] org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Failed all from region=transactions_str_as_dbl,17929912900560606187890916,1344358522026.74def75a6e7b1f9d7003988f4cb9b4b4., hostname=ihub-dn-l4, port=60020
  364. java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused
  365.         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
  366.         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
  367.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1557)
  368.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1409)
  369.         at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:943)
  370.         at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
  371.         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)
  372.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:121)
  373.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:82)
  374.         at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:598)
  375.         at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
  376.         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
  377.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:508)
  378.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:1)
  379.         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
  380.         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:725)
  381.         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
  382.         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
  383.         at java.security.AccessController.doPrivileged(Native Method)
  384.         at javax.security.auth.Subject.doAs(Subject.java:396)
  385.         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
  386.         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
  387. Caused by: java.net.ConnectException: Connection refused
  388.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  389.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  390.         at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
  391.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
  392.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
  393.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328)
  394.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362)
  395.         at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1045)
  396.         at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897)
  397.         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
  398.         at $Proxy13.multi(Unknown Source)
  399.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1386)
  400.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1384)
  401.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1365)
  402.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1383)
  403.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1381)
  404.         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  405.         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  406.         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  407.         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  408.         at java.lang.Thread.run(Thread.java:662)
  409. 2012-08-07 02:03:00,969 WARN [main] org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Failed all from region=transactions_str_as_dbl,17929912900560606187890916,1344358522026.74def75a6e7b1f9d7003988f4cb9b4b4., hostname=ihub-dn-l4, port=60020
  410. java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused
  411.         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
  412.         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
  413.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1557)
  414.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1409)
  415.         at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:943)
  416.         at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
  417.         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)
  418.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:121)
  419.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:82)
  420.         at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:598)
  421.         at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
  422.         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
  423.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:508)
  424.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:1)
  425.         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
  426.         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:725)
  427.         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
  428.         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
  429.         at java.security.AccessController.doPrivileged(Native Method)
  430.         at javax.security.auth.Subject.doAs(Subject.java:396)
  431.         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
  432.         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
  433. Caused by: java.net.ConnectException: Connection refused
  434.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  435.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  436.         at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
  437.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
  438.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
  439.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328)
  440.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362)
  441.         at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1045)
  442.         at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897)
  443.         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
  444.         at $Proxy13.multi(Unknown Source)
  445.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1386)
  446.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1384)
  447.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1365)
  448.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1383)
  449.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1381)
  450.         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  451.         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  452.         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  453.         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  454.         at java.lang.Thread.run(Thread.java:662)
  455. 2012-08-07 02:03:02,987 WARN [main] org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Failed all from region=transactions_str_as_dbl,17929912900560606187890916,1344358522026.74def75a6e7b1f9d7003988f4cb9b4b4., hostname=ihub-dn-l4, port=60020
  456. java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused
  457.         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
  458.         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
  459.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1557)
  460.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1409)
  461.         at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:943)
  462.         at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
  463.         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)
  464.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:121)
  465.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:82)
  466.         at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:598)
  467.         at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
  468.         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
  469.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:508)
  470.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:1)
  471.         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
  472.         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:725)
  473.         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
  474.         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
  475.         at java.security.AccessController.doPrivileged(Native Method)
  476.         at javax.security.auth.Subject.doAs(Subject.java:396)
  477.         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
  478.         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
  479. Caused by: java.net.ConnectException: Connection refused
  480.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  481.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  482.         at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
  483.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
  484.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
  485.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328)
  486.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362)
  487.         at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1045)
  488.         at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897)
  489.         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
  490.         at $Proxy13.multi(Unknown Source)
  491.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1386)
  492.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1384)
  493.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1365)
  494.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1383)
  495.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1381)
  496.         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  497.         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  498.         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  499.         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  500.         at java.lang.Thread.run(Thread.java:662)
  501. 2012-08-07 02:03:07,011 WARN [main] org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Failed all from region=transactions_str_as_dbl,17929912900560606187890916,1344358522026.74def75a6e7b1f9d7003988f4cb9b4b4., hostname=ihub-dn-l4, port=60020
  502. java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused
  503.         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
  504.         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
  505.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1557)
  506.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1409)
  507.         at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:943)
  508.         at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
  509.         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)
  510.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:121)
  511.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:82)
  512.         at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:598)
  513.         at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
  514.         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
  515.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:508)
  516.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:1)
  517.         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
  518.         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:725)
  519.         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
  520.         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
  521.         at java.security.AccessController.doPrivileged(Native Method)
  522.         at javax.security.auth.Subject.doAs(Subject.java:396)
  523.         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
  524.         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
  525. Caused by: java.net.ConnectException: Connection refused
  526.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  527.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  528.         at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
  529.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
  530.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
  531.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328)
  532.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362)
  533.         at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1045)
  534.         at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897)
  535.         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
  536.         at $Proxy13.multi(Unknown Source)
  537.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1386)
  538.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1384)
  539.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1365)
  540.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1383)
  541.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1381)
  542.         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  543.         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  544.         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  545.         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  546.         at java.lang.Thread.run(Thread.java:662)
  547. 2012-08-07 02:03:11,127 WARN [main] org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Failed all from region=transactions_str_as_dbl,17929912900560606187890916,1344358522026.74def75a6e7b1f9d7003988f4cb9b4b4., hostname=ihub-dn-l4, port=60020
  548. java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused
  549.         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
  550.         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
  551.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1557)
  552.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1409)
  553.         at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:943)
  554.         at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
  555.         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)
  556.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:121)
  557.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:82)
  558.         at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:598)
  559.         at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
  560.         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
  561.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:508)
  562.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:1)
  563.         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
  564.         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:725)
  565.         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
  566.         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
  567.         at java.security.AccessController.doPrivileged(Native Method)
  568.         at javax.security.auth.Subject.doAs(Subject.java:396)
  569.         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
  570.         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
  571. Caused by: java.net.ConnectException: Connection refused
  572.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  573.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  574.         at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
  575.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
  576.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
  577.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328)
  578.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362)
  579.         at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1045)
  580.         at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897)
  581.         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
  582.         at $Proxy13.multi(Unknown Source)
  583.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1386)
  584.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1384)
  585.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1365)
  586.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1383)
  587.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1381)
  588.         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  589.         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  590.         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  591.         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  592.         at java.lang.Thread.run(Thread.java:662)
  593. 2012-08-07 02:03:19,154 WARN [main] org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Failed all from region=transactions_str_as_dbl,17929912900560606187890916,1344358522026.74def75a6e7b1f9d7003988f4cb9b4b4., hostname=ihub-dn-l4, port=60020
  594. java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused
  595.         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
  596.         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
  597.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1557)
  598.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1409)
  599.         at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:943)
  600.         at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
  601.         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)
  602.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:121)
  603.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:82)
  604.         at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:598)
  605.         at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
  606.         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
  607.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:508)
  608.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:1)
  609.         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
  610.         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:725)
  611.         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
  612.         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
  613.         at java.security.AccessController.doPrivileged(Native Method)
  614.         at javax.security.auth.Subject.doAs(Subject.java:396)
  615.         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
  616.         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
  617. Caused by: java.net.ConnectException: Connection refused
  618.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  619.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  620.         at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
  621.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
  622.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
  623.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328)
  624.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362)
  625.         at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1045)
  626.         at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897)
  627.         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
  628.         at $Proxy13.multi(Unknown Source)
  629.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1386)
  630.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1384)
  631.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1365)
  632.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1383)
  633.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1381)
  634.         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  635.         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  636.         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  637.         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  638.         at java.lang.Thread.run(Thread.java:662)
  639. 2012-08-07 02:03:35,177 WARN [main] org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Failed all from region=transactions_str_as_dbl,17929912900560606187890916,1344358522026.74def75a6e7b1f9d7003988f4cb9b4b4., hostname=ihub-dn-l4, port=60020
  640. java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused
  641.         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
  642.         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
  643.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1557)
  644.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1409)
  645.         at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:943)
  646.         at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
  647.         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)
  648.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:121)
  649.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:82)
  650.         at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:598)
  651.         at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
  652.         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
  653.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:508)
  654.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:1)
  655.         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
  656.         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:725)
  657.         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
  658.         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
  659.         at java.security.AccessController.doPrivileged(Native Method)
  660.         at javax.security.auth.Subject.doAs(Subject.java:396)
  661.         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
  662.         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
  663. Caused by: java.net.ConnectException: Connection refused
  664.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  665.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  666.         at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
  667.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
  668.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
  669.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328)
  670.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362)
  671.         at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1045)
  672.         at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897)
  673.         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
  674.         at $Proxy13.multi(Unknown Source)
  675.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1386)
  676.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1384)
  677.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1365)
  678.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1383)
  679.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1381)
  680.         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  681.         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  682.         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  683.         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  684.         at java.lang.Thread.run(Thread.java:662)
  685. 2012-08-07 02:04:07,208 WARN [main] org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Failed all from region=transactions_str_as_dbl,17929912900560606187890916,1344358522026.74def75a6e7b1f9d7003988f4cb9b4b4., hostname=ihub-dn-l4, port=60020
  686. java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused
  687.         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
  688.         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
  689.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1557)
  690.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1409)
  691.         at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:943)
  692.         at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
  693.         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)
  694.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:121)
  695.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:82)
  696.         at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:598)
  697.         at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
  698.         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
  699.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:508)
  700.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:1)
  701.         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
  702.         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:725)
  703.         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
  704.         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
  705.         at java.security.AccessController.doPrivileged(Native Method)
  706.         at javax.security.auth.Subject.doAs(Subject.java:396)
  707.         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
  708.         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
  709. Caused by: java.net.ConnectException: Connection refused
  710.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  711.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
  712.         at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
  713.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
  714.         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
  715.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328)
  716.         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362)
  717.         at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1045)
  718.         at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897)
  719.         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
  720.         at $Proxy13.multi(Unknown Source)
  721.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1386)
  722.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1384)
  723.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1365)
  724.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1383)
  725.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1381)
  726.         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  727.         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  728.         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  729.         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  730.         at java.lang.Thread.run(Thread.java:662)
  731. 2012-08-07 02:04:07,211 ERROR [main] org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:root (auth:SIMPLE) cause:org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 69 actions: servers with issues: ihub-dn-l4:60020,
  732. 2012-08-07 02:04:07,211 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 69 actions: servers with issues: ihub-dn-l4:60020,
  733.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1641)
  734.         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1409)
  735.         at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:943)
  736.         at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
  737.         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)
  738.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:121)
  739.         at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:82)
  740.         at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:598)
  741.         at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
  742.         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
  743.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:508)
  744.         at com.intuit.ihub.hbase.poc.ImportData4$TsvImporter.map(ImportData4.java:1)
  745.         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
  746.         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:725)
  747.         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
  748.         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
  749.         at java.security.AccessController.doPrivileged(Native Method)
  750.         at javax.security.auth.Subject.doAs(Subject.java:396)
  751.         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
  752.         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
  753.  
  754. 2012-08-07 02:04:07,238 INFO [main] org.apache.hadoop.mapred.Task: Runnning cleanup for the task
  755. 2012-08-07 02:04:07,392 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...
  756. 2012-08-07 02:04:07,393 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.
  757. 2012-08-07 02:04:07,393 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete.
create a new version of this paste RAW Paste Data