SHARE
TWEET

Untitled

a guest Jul 17th, 2017 42 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. 2011-01-26 17:47:34,325 INFO org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Reconnecting to zookeeper
  2. 2011-01-26 17:47:34,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 17:14 GMT
  3. 2011-01-26 17:47:34,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:host.name=mtag3.prod.imageshack.com
  4. 2011-01-26 17:47:34,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.6.0_23
  5. 2011-01-26 17:47:34,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc.
  6. 2011-01-26 17:47:34,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.home=/usr/java/jdk1.6.0_23/jre
  7. 2011-01-26 17:47:34,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=/usr/lib/hbase/bin/../conf:/usr/java/jdk1.6.0_23/lib/tools.jar:/usr/lib/hbase/bin/..:/usr/lib/hbase/bin/../hbase-0.89.20100830.jar:/usr/lib/hbase/bin/../lib/activation-1.1.jar:/usr/lib/hbase/bin/../lib/ant-1.6.5.jar:/usr/lib/hbase/bin/../lib/ant-1.7.1.jar:/usr/lib/hbase/bin/../lib/ant-launcher-1.7.1.jar:/usr/lib/hbase/bin/../lib/asm-3.1.jar:/usr/lib/hbase/bin/../lib/avro-1.3.2.jar:/usr/lib/hbase/bin/../lib/commons-cli-1.2.jar:/usr/lib/hbase/bin/../lib/commons-codec-1.2.jar:/usr/lib/hbase/bin/../lib/commons-el-1.0.jar:/usr/lib/hbase/bin/../lib/commons-httpclient-3.1.jar:/usr/lib/hbase/bin/../lib/commons-lang-2.5.jar:/usr/lib/hbase/bin/../lib/commons-logging-1.1.1.jar:/usr/lib/hbase/bin/../lib/commons-net-1.4.1.jar:/usr/lib/hbase/bin/../lib/core-3.1.1.jar:/usr/lib/hbase/bin/../lib/guava-r05.jar:/usr/lib/hbase/bin/../lib/hadoop-0.20.1-dev-core.jar:/usr/lib/hbase/bin/../lib/hadoop-gpl-compression-0.1.0-dev.jar:/usr/lib/hbase/bin/../lib/hbase-0.89.20100830.jar:/usr/lib/hbase/bin/../lib/hsqldb-1.8.0.10.jar:/usr/lib/hbase/bin/../lib/jackson-core-asl-1.5.2.jar:/usr/lib/hbase/bin/../lib/jackson-mapper-asl-1.5.2.jar:/usr/lib/hbase/bin/../lib/jasper-compiler-5.5.23.jar:/usr/lib/hbase/bin/../lib/jasper-runtime-5.5.23.jar:/usr/lib/hbase/bin/../lib/jaxb-api-2.1.jar:/usr/lib/hbase/bin/../lib/jaxb-impl-2.1.12.jar:/usr/lib/hbase/bin/../lib/jersey-core-1.1.5.1.jar:/usr/lib/hbase/bin/../lib/jersey-json-1.1.5.1.jar:/usr/lib/hbase/bin/../lib/jersey-server-1.1.5.1.jar:/usr/lib/hbase/bin/../lib/jets3t-0.7.1.jar:/usr/lib/hbase/bin/../lib/jettison-1.1.jar:/usr/lib/hbase/bin/../lib/jetty-6.1.24.jar:/usr/lib/hbase/bin/../lib/jetty-util-6.1.24.jar:/usr/lib/hbase/bin/../lib/jline-0.9.94.jar:/usr/lib/hbase/bin/../lib/jruby-complete-1.4.0.jar:/usr/lib/hbase/bin/../lib/json-simple-1.1.jar:/usr/lib/hbase/bin/../lib/jsp-2.1-6.1.14.jar:/usr/lib/hbase/bin/../lib/jsp-api-2.0.jar:/usr/lib/hbase/bin/../lib/jsp-api-2.1-6.1.14.jar:/usr/lib/hbase/bin/../lib/jsr311-api-1.1.1.jar:/usr/lib/hbase/bin/../lib/kfs-0.3.jar:/usr/lib/hbase/bin/../lib/log4j-1.2.15.jar:/usr/lib/hbase/bin/../lib/oro-2.0.8.jar:/usr/lib/hbase/bin/../lib/paranamer-2.2.jar:/usr/lib/hbase/bin/../lib/paranamer-ant-2.2.jar:/usr/lib/hbase/bin/../lib/paranamer-generator-2.2.jar:/usr/lib/hbase/bin/../lib/protobuf-java-2.3.0.jar:/usr/lib/hbase/bin/../lib/qdox-1.10.1.jar:/usr/lib/hbase/bin/../lib/servlet-api-2.4.jar:/usr/lib/hbase/bin/../lib/servlet-api-2.5-20081211.jar:/usr/lib/hbase/bin/../lib/servlet-api-2.5-6.1.14.jar:/usr/lib/hbase/bin/../lib/slf4j-api-1.5.8.jar:/usr/lib/hbase/bin/../lib/slf4j-log4j12-1.5.8.jar:/usr/lib/hbase/bin/../lib/stax-api-1.0.1.jar:/usr/lib/hbase/bin/../lib/stax-api-1.0-2.jar:/usr/lib/hbase/bin/../lib/thrift-0.2.0.jar:/usr/lib/hbase/bin/../lib/xmlenc-0.52.jar:/usr/lib/hbase/bin/../lib/zookeeper-3.3.1.jar
  8. 2011-01-26 17:47:34,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/lib/hbase/bin/../lib/native/Linux-amd64-64
  9. 2011-01-26 17:47:34,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
  10. 2011-01-26 17:47:34,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
  11. 2011-01-26 17:47:34,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux
  12. 2011-01-26 17:47:34,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64
  13. 2011-01-26 17:47:34,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=2.6.33.3-85.fc13.x86_64
  14. 2011-01-26 17:47:34,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=root
  15. 2011-01-26 17:47:34,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/root
  16. 2011-01-26 17:47:34,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/root
  17. 2011-01-26 17:47:34,352 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=mtag3:2181,mtag2:2181,mtag1:2181 sessionTimeout=60000 watcher=org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper@250d593e
  18. 2011-01-26 17:47:34,379 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server mtag1/10.103.7.1:2181
  19. 2011-01-26 17:47:34,384 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to mtag1/10.103.7.1:2181, initiating session
  20. 2011-01-26 17:47:34,386 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: <mtag1,mtag2,mtag3:/hbase,mtag3.prod.imageshack.com,60020,1296092854299>Connected to zookeeper again
  21. 2011-01-26 17:47:34,424 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server mtag1/10.103.7.1:2181, sessionid = 0x2ce0ab2710008f, negotiated timeout = 60000
  22. 2011-01-26 17:47:34,432 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Got ZooKeeper event, state: SyncConnected, type: None, path: null
  23. 2011-01-26 17:47:34,459 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: <mtag1,mtag2,mtag3:/hbase,mtag3.prod.imageshack.com,60020,1296092854299>Set watcher on master address ZNode /hbase/master
  24. 2011-01-26 17:47:34,521 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: <mtag1,mtag2,mtag3:/hbase,mtag3.prod.imageshack.com,60020,1296092854299>Read ZNode /hbase/master got 10.103.7.2:60000
  25. 2011-01-26 17:47:34,521 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Telling master at 10.103.7.2:60000 that we are up
  26. 2011-01-26 17:47:34,638 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Installed shutdown hook thread: Shutdownhook:regionserver60020
  27. 2011-01-26 17:47:34,663 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: sending initial server load: requests=0, regions=0, usedHeap=24, maxHeap=895
  28. 2011-01-26 17:47:34,680 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: <mtag1,mtag2,mtag3:/hbase,mtag3.prod.imageshack.com,60020,1296092854299>Created ZNode /hbase/rs/mtag3.prod.imageshack.com,60020,1296092854663 with data 10.103.7.3:60020
  29. 2011-01-26 17:47:34,695 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Config from master: hbase.regionserver.address=10.103.7.3
  30. 2011-01-26 17:47:34,695 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Config from master: fs.default.name=hdfs://mtag1:9000/hbase
  31. 2011-01-26 17:47:34,695 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Config from master: hbase.rootdir=hdfs://mtag1:9000/hbase
  32. 2011-01-26 17:47:34,695 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Master passed us address to use. Was=10.103.7.3:60020, Now=10.103.7.3
  33. 2011-01-26 17:47:34,829 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Log dir hdfs://mtag1:9000/hbase/.logs/mtag3.prod.imageshack.com,60020,1296092854663
  34. 2011-01-26 17:47:34,879 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: HLog configuration: blocksize=134217728, rollsize=127506840, enabled=true, flushlogentries=100, optionallogflushinternal=10000ms
  35. 2011-01-26 17:47:34,982 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed init
  36. java.io.IOException: cannot get log writer
  37.     at org.apache.hadoop.hbase.regionserver.wal.HLog.createWriter(HLog.java:553)
  38.     at org.apache.hadoop.hbase.regionserver.wal.HLog.rollWriter(HLog.java:453)
  39.     at org.apache.hadoop.hbase.regionserver.wal.HLog.<init>(HLog.java:355)
  40.     at org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateHLog(HRegionServer.java:1024)
  41.     at org.apache.hadoop.hbase.regionserver.HRegionServer.setupHLog(HRegionServer.java:1017)
  42.     at org.apache.hadoop.hbase.regionserver.HRegionServer.init(HRegionServer.java:825)
  43.     at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:521)
  44.     at java.lang.Thread.run(Thread.java:662)
  45. Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.protocol.ClientProtocol.abandonFile(java.lang.String, java.lang.String)
  46.     at java.lang.Class.getMethod(Class.java:1605)
  47.     at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:506)
  48.     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:968)
  49.     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:964)
  50.     at java.security.AccessController.doPrivileged(Native Method)
  51.     at javax.security.auth.Subject.doAs(Subject.java:396)
  52.     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:962)
  53.  
  54.     at org.apache.hadoop.ipc.Client.call(Client.java:826)
  55.     at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:222)
  56.     at $Proxy1.abandonFile(Unknown Source)
  57.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  58.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  59.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  60.     at java.lang.reflect.Method.invoke(Method.java:597)
  61.     at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
  62.     at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
  63.     at $Proxy1.abandonFile(Unknown Source)
  64.     at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:673)
  65.     at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:208)
  66.     at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:198)
  67.     at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:526)
  68.     at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:894)
  69.     at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:394)
  70.     at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.init(SequenceFileLogWriter.java:59)
  71.     at org.apache.hadoop.hbase.regionserver.wal.HLog.createWriter(HLog.java:550)
  72.     ... 7 more
  73. 2011-01-26 17:47:34,987 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: Aborting region server serverName=mtag3.prod.imageshack.com,60020,1296092854663, load=(requests=0, regions=0, usedHeap=0, maxHeap=0): Unhandled exception
  74. java.io.IOException: cannot get log writer
  75.     at org.apache.hadoop.hbase.regionserver.wal.HLog.createWriter(HLog.java:553)
  76.     at org.apache.hadoop.hbase.regionserver.wal.HLog.rollWriter(HLog.java:453)
  77.     at org.apache.hadoop.hbase.regionserver.wal.HLog.<init>(HLog.java:355)
  78.     at org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateHLog(HRegionServer.java:1024)
  79.     at org.apache.hadoop.hbase.regionserver.HRegionServer.setupHLog(HRegionServer.java:1017)
  80.     at org.apache.hadoop.hbase.regionserver.HRegionServer.init(HRegionServer.java:825)
  81.     at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:521)
  82.     at java.lang.Thread.run(Thread.java:662)
  83. Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.protocol.ClientProtocol.abandonFile(java.lang.String, java.lang.String)
  84.     at java.lang.Class.getMethod(Class.java:1605)
  85.     at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:506)
  86.     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:968)
  87.     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:964)
  88.     at java.security.AccessController.doPrivileged(Native Method)
  89.     at javax.security.auth.Subject.doAs(Subject.java:396)
  90.     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:962)
  91.  
  92.     at org.apache.hadoop.ipc.Client.call(Client.java:826)
  93.     at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:222)
  94.     at $Proxy1.abandonFile(Unknown Source)
  95.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  96.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  97.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  98.     at java.lang.reflect.Method.invoke(Method.java:597)
  99.     at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
  100.     at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
  101.     at $Proxy1.abandonFile(Unknown Source)
  102.     at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:673)
  103.     at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:208)
  104.     at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:198)
  105.     at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:526)
  106.     at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:894)
  107.     at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:394)
  108.     at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.init(SequenceFileLogWriter.java:59)
  109.     at org.apache.hadoop.hbase.regionserver.wal.HLog.createWriter(HLog.java:550)
  110.     ... 7 more
  111. 2011-01-26 17:47:34,987 INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 60020
  112. 2011-01-26 17:47:34,993 INFO org.apache.hadoop.hbase.regionserver.StoreFile: Allocating LruBlockCache with maximum size 3.4g
  113. 2011-01-26 17:47:35,009 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: aborting server at: mtag3.prod.imageshack.com,60020,1296092854663
  114. 2011-01-26 17:47:35,121 INFO org.apache.zookeeper.ZooKeeper: Session: 0x2ce0ab2710008f closed
  115. 2011-01-26 17:47:35,121 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: <mtag1,mtag2,mtag3:/hbase,mtag3.prod.imageshack.com,60020,1296092854299>Closed connection with ZooKeeper; /hbase/root-region-server
  116. 2011-01-26 17:47:35,121 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: regionserver60020 exiting
  117. 2011-01-26 17:47:35,122 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=Thread[Thread-9,5,main]
  118. 2011-01-26 17:47:35,122 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Starting fs shutdown hook thread.
  119. 2011-01-26 17:47:35,123 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown hook finished.
  120. Wed Jan 26 17:48:43 PST 2011 Starting regionserver on mtag3.prod.imageshack.com
  121. ulimit -n 128000
  122. 2011-01-26 17:48:43,616 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: vmInputArguments=[-Xmx5000m, -XX:+UseConcMarkSweepGC, -XX:+CMSIncrementalMode, -XX:MaxDirectMemorySize=6G, -XX:+UseConcMarkSweepGC, -XX:+CMSIncrementalMode, -XX:MaxDirectMemorySize=6G, -Dhbase.log.dir=/usr/lib/hbase/bin/../logs, -Dhbase.log.file=hbase-root-regionserver-mtag3.prod.imageshack.com.log, -Dhbase.home.dir=/usr/lib/hbase/bin/.., -Dhbase.id.str=root, -Dhbase.root.logger=INFO,DRFA, -Djava.library.path=/usr/lib/hbase/bin/../lib/native/Linux-amd64-64]
  123. 2011-01-26 17:48:43,627 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: My address is mtag3.prod.imageshack.com:60020
  124. 2011-01-26 17:48:43,660 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  125. 2011-01-26 17:48:43,661 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  126. 2011-01-26 17:48:43,662 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  127. 2011-01-26 17:48:43,665 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  128. 2011-01-26 17:48:43,667 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  129. 2011-01-26 17:48:43,667 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  130. 2011-01-26 17:48:43,667 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  131. 2011-01-26 17:48:43,666 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  132. 2011-01-26 17:48:43,665 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  133. 2011-01-26 17:48:43,668 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  134. 2011-01-26 17:48:43,681 INFO org.apache.hadoop.hbase.ipc.HBaseRpcMetrics: Initializing RPC Metrics with hostName=HRegionServer, port=60020
  135. 2011-01-26 17:48:43,737 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: globalMemStoreLimit=1.9g, globalMemStoreLimitLowMark=1.7g, maxHeap=4.9g
  136. 2011-01-26 17:48:43,739 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Runs every 10000000ms
  137. 2011-01-26 17:48:43,748 INFO org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Reconnecting to zookeeper
  138. 2011-01-26 17:48:43,757 INFO org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 17:14 GMT
  139. 2011-01-26 17:48:43,757 INFO org.apache.zookeeper.ZooKeeper: Client environment:host.name=mtag3.prod.imageshack.com
  140. 2011-01-26 17:48:43,757 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.6.0_23
  141. 2011-01-26 17:48:43,757 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc.
  142. 2011-01-26 17:48:43,757 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.home=/usr/java/jdk1.6.0_23/jre
  143. 2011-01-26 17:48:43,757 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=/usr/lib/hbase/bin/../conf:/usr/java/jdk1.6.0_23/lib/tools.jar:/usr/lib/hbase/bin/..:/usr/lib/hbase/bin/../hbase-0.89.20100830.jar:/usr/lib/hbase/bin/../lib/activation-1.1.jar:/usr/lib/hbase/bin/../lib/ant-1.6.5.jar:/usr/lib/hbase/bin/../lib/ant-1.7.1.jar:/usr/lib/hbase/bin/../lib/ant-launcher-1.7.1.jar:/usr/lib/hbase/bin/../lib/asm-3.1.jar:/usr/lib/hbase/bin/../lib/avro-1.3.2.jar:/usr/lib/hbase/bin/../lib/commons-cli-1.2.jar:/usr/lib/hbase/bin/../lib/commons-codec-1.2.jar:/usr/lib/hbase/bin/../lib/commons-el-1.0.jar:/usr/lib/hbase/bin/../lib/commons-httpclient-3.1.jar:/usr/lib/hbase/bin/../lib/commons-lang-2.5.jar:/usr/lib/hbase/bin/../lib/commons-logging-1.1.1.jar:/usr/lib/hbase/bin/../lib/commons-net-1.4.1.jar:/usr/lib/hbase/bin/../lib/core-3.1.1.jar:/usr/lib/hbase/bin/../lib/guava-r05.jar:/usr/lib/hbase/bin/../lib/hadoop-0.20.1-dev-core.jar:/usr/lib/hbase/bin/../lib/hadoop-gpl-compression-0.1.0-dev.jar:/usr/lib/hbase/bin/../lib/hbase-0.89.20100830.jar:/usr/lib/hbase/bin/../lib/hsqldb-1.8.0.10.jar:/usr/lib/hbase/bin/../lib/jackson-core-asl-1.5.2.jar:/usr/lib/hbase/bin/../lib/jackson-mapper-asl-1.5.2.jar:/usr/lib/hbase/bin/../lib/jasper-compiler-5.5.23.jar:/usr/lib/hbase/bin/../lib/jasper-runtime-5.5.23.jar:/usr/lib/hbase/bin/../lib/jaxb-api-2.1.jar:/usr/lib/hbase/bin/../lib/jaxb-impl-2.1.12.jar:/usr/lib/hbase/bin/../lib/jersey-core-1.1.5.1.jar:/usr/lib/hbase/bin/../lib/jersey-json-1.1.5.1.jar:/usr/lib/hbase/bin/../lib/jersey-server-1.1.5.1.jar:/usr/lib/hbase/bin/../lib/jets3t-0.7.1.jar:/usr/lib/hbase/bin/../lib/jettison-1.1.jar:/usr/lib/hbase/bin/../lib/jetty-6.1.24.jar:/usr/lib/hbase/bin/../lib/jetty-util-6.1.24.jar:/usr/lib/hbase/bin/../lib/jline-0.9.94.jar:/usr/lib/hbase/bin/../lib/jruby-complete-1.4.0.jar:/usr/lib/hbase/bin/../lib/json-simple-1.1.jar:/usr/lib/hbase/bin/../lib/jsp-2.1-6.1.14.jar:/usr/lib/hbase/bin/../lib/jsp-api-2.0.jar:/usr/lib/hbase/bin/../lib/jsp-api-2.1-6.1.14.jar:/usr/lib/hbase/bin/../lib/jsr311-api-1.1.1.jar:/usr/lib/hbase/bin/../lib/kfs-0.3.jar:/usr/lib/hbase/bin/../lib/log4j-1.2.15.jar:/usr/lib/hbase/bin/../lib/oro-2.0.8.jar:/usr/lib/hbase/bin/../lib/paranamer-2.2.jar:/usr/lib/hbase/bin/../lib/paranamer-ant-2.2.jar:/usr/lib/hbase/bin/../lib/paranamer-generator-2.2.jar:/usr/lib/hbase/bin/../lib/protobuf-java-2.3.0.jar:/usr/lib/hbase/bin/../lib/qdox-1.10.1.jar:/usr/lib/hbase/bin/../lib/servlet-api-2.4.jar:/usr/lib/hbase/bin/../lib/servlet-api-2.5-20081211.jar:/usr/lib/hbase/bin/../lib/servlet-api-2.5-6.1.14.jar:/usr/lib/hbase/bin/../lib/slf4j-api-1.5.8.jar:/usr/lib/hbase/bin/../lib/slf4j-log4j12-1.5.8.jar:/usr/lib/hbase/bin/../lib/stax-api-1.0.1.jar:/usr/lib/hbase/bin/../lib/stax-api-1.0-2.jar:/usr/lib/hbase/bin/../lib/thrift-0.2.0.jar:/usr/lib/hbase/bin/../lib/xmlenc-0.52.jar:/usr/lib/hbase/bin/../lib/zookeeper-3.3.1.jar
  144. 2011-01-26 17:48:43,757 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/lib/hbase/bin/../lib/native/Linux-amd64-64
  145. 2011-01-26 17:48:43,757 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
  146. 2011-01-26 17:48:43,757 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
  147. 2011-01-26 17:48:43,757 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux
  148. 2011-01-26 17:48:43,758 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64
  149. 2011-01-26 17:48:43,758 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=2.6.33.3-85.fc13.x86_64
  150. 2011-01-26 17:48:43,758 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=root
  151. 2011-01-26 17:48:43,758 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/root
  152. 2011-01-26 17:48:43,758 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/root
  153. 2011-01-26 17:48:43,759 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=mtag3:2181,mtag2:2181,mtag1:2181 sessionTimeout=60000 watcher=org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper@250d593e
  154. 2011-01-26 17:48:43,775 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: <mtag1,mtag2,mtag3:/hbase,mtag3.prod.imageshack.com,60020,1296092923732>Connected to zookeeper again
  155. 2011-01-26 17:48:43,775 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server mtag3/10.103.7.3:2181
  156. 2011-01-26 17:48:43,781 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to mtag3/10.103.7.3:2181, initiating session
  157. 2011-01-26 17:48:43,799 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server mtag3/10.103.7.3:2181, sessionid = 0x22ce25ea165008a, negotiated timeout = 60000
  158. 2011-01-26 17:48:43,803 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Got ZooKeeper event, state: SyncConnected, type: None, path: null
  159. 2011-01-26 17:48:43,806 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: <mtag1,mtag2,mtag3:/hbase,mtag3.prod.imageshack.com,60020,1296092923732>Set watcher on master address ZNode /hbase/master
  160. 2011-01-26 17:48:43,890 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: <mtag1,mtag2,mtag3:/hbase,mtag3.prod.imageshack.com,60020,1296092923732>Read ZNode /hbase/master got 10.103.7.2:60000
  161. 2011-01-26 17:48:43,890 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Telling master at 10.103.7.2:60000 that we are up
  162. 2011-01-26 17:48:44,023 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Installed shutdown hook thread: Shutdownhook:regionserver60020
  163. 2011-01-26 17:48:44,028 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: sending initial server load: requests=0, regions=0, usedHeap=24, maxHeap=895
  164. 2011-01-26 17:48:44,047 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: <mtag1,mtag2,mtag3:/hbase,mtag3.prod.imageshack.com,60020,1296092923732>Created ZNode /hbase/rs/mtag3.prod.imageshack.com,60020,1296092924029 with data 10.103.7.3:60020
  165. 2011-01-26 17:48:44,069 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Config from master: hbase.regionserver.address=10.103.7.3
  166. 2011-01-26 17:48:44,070 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Config from master: fs.default.name=hdfs://mtag1:9000/hbase
  167. 2011-01-26 17:48:44,070 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Config from master: hbase.rootdir=hdfs://mtag1:9000/hbase
  168. 2011-01-26 17:48:44,070 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Master passed us address to use. Was=10.103.7.3:60020, Now=10.103.7.3
  169. 2011-01-26 17:48:44,236 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Log dir hdfs://mtag1:9000/hbase/.logs/mtag3.prod.imageshack.com,60020,1296092924029
  170. 2011-01-26 17:48:44,280 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: HLog configuration: blocksize=134217728, rollsize=127506840, enabled=true, flushlogentries=100, optionallogflushinternal=10000ms
  171. 2011-01-26 17:48:44,357 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed init
  172. java.io.IOException: cannot get log writer
  173.     at org.apache.hadoop.hbase.regionserver.wal.HLog.createWriter(HLog.java:553)
  174.     at org.apache.hadoop.hbase.regionserver.wal.HLog.rollWriter(HLog.java:453)
  175.     at org.apache.hadoop.hbase.regionserver.wal.HLog.<init>(HLog.java:355)
  176.     at org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateHLog(HRegionServer.java:1024)
  177.     at org.apache.hadoop.hbase.regionserver.HRegionServer.setupHLog(HRegionServer.java:1017)
  178.     at org.apache.hadoop.hbase.regionserver.HRegionServer.init(HRegionServer.java:825)
  179.     at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:521)
  180.     at java.lang.Thread.run(Thread.java:662)
  181. Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.protocol.ClientProtocol.abandonFile(java.lang.String, java.lang.String)
  182.     at java.lang.Class.getMethod(Class.java:1605)
  183.     at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:506)
  184.     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:968)
  185.     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:964)
  186.     at java.security.AccessController.doPrivileged(Native Method)
  187.     at javax.security.auth.Subject.doAs(Subject.java:396)
  188.     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:962)
  189.  
  190.     at org.apache.hadoop.ipc.Client.call(Client.java:826)
  191.     at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:222)
  192.     at $Proxy1.abandonFile(Unknown Source)
  193.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  194.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  195.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  196.     at java.lang.reflect.Method.invoke(Method.java:597)
  197.     at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
  198.     at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
  199.     at $Proxy1.abandonFile(Unknown Source)
  200.     at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:673)
  201.     at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:208)
  202.     at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:198)
  203.     at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:526)
  204.     at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:894)
  205.     at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:394)
  206.     at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.init(SequenceFileLogWriter.java:59)
  207.     at org.apache.hadoop.hbase.regionserver.wal.HLog.createWriter(HLog.java:550)
  208.     ... 7 more
  209. 2011-01-26 17:48:44,362 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: Aborting region server serverName=mtag3.prod.imageshack.com,60020,1296092924029, load=(requests=0, regions=0, usedHeap=0, maxHeap=0): Unhandled exception
  210. java.io.IOException: cannot get log writer
  211.     at org.apache.hadoop.hbase.regionserver.wal.HLog.createWriter(HLog.java:553)
  212.     at org.apache.hadoop.hbase.regionserver.wal.HLog.rollWriter(HLog.java:453)
  213.     at org.apache.hadoop.hbase.regionserver.wal.HLog.<init>(HLog.java:355)
  214.     at org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateHLog(HRegionServer.java:1024)
  215.     at org.apache.hadoop.hbase.regionserver.HRegionServer.setupHLog(HRegionServer.java:1017)
  216.     at org.apache.hadoop.hbase.regionserver.HRegionServer.init(HRegionServer.java:825)
  217.     at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:521)
  218.     at java.lang.Thread.run(Thread.java:662)
  219. Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.protocol.ClientProtocol.abandonFile(java.lang.String, java.lang.String)
  220.     at java.lang.Class.getMethod(Class.java:1605)
  221.     at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:506)
  222.     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:968)
  223.     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:964)
  224.     at java.security.AccessController.doPrivileged(Native Method)
  225.     at javax.security.auth.Subject.doAs(Subject.java:396)
  226.     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:962)
  227.  
  228.     at org.apache.hadoop.ipc.Client.call(Client.java:826)
  229.     at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:222)
  230.     at $Proxy1.abandonFile(Unknown Source)
  231.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  232.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  233.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  234.     at java.lang.reflect.Method.invoke(Method.java:597)
  235.     at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
  236.     at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
  237.     at $Proxy1.abandonFile(Unknown Source)
  238.     at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:673)
  239.     at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:208)
  240.     at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:198)
  241.     at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:526)
  242.     at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:894)
  243.     at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:394)
  244.     at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.init(SequenceFileLogWriter.java:59)
  245.     at org.apache.hadoop.hbase.regionserver.wal.HLog.createWriter(HLog.java:550)
  246.     ... 7 more
  247. 2011-01-26 17:48:44,362 INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 60020
  248. 2011-01-26 17:48:44,370 INFO org.apache.hadoop.hbase.regionserver.StoreFile: Allocating LruBlockCache with maximum size 3.4g
  249. 2011-01-26 17:48:44,390 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: aborting server at: mtag3.prod.imageshack.com,60020,1296092924029
  250. 2011-01-26 17:48:44,499 INFO org.apache.zookeeper.ZooKeeper: Session: 0x22ce25ea165008a closed
  251. 2011-01-26 17:48:44,499 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: <mtag1,mtag2,mtag3:/hbase,mtag3.prod.imageshack.com,60020,1296092923732>Closed connection with ZooKeeper; /hbase/root-region-server
  252. 2011-01-26 17:48:44,500 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: regionserver60020 exiting
  253. 2011-01-26 17:48:44,501 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=Thread[Thread-9,5,main]
  254. 2011-01-26 17:48:44,501 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Starting fs shutdown hook thread.
  255. 2011-01-26 17:48:44,502 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown hook finished.
  256. Wed Jan 26 17:50:47 PST 2011 Starting regionserver on mtag3.prod.imageshack.com
  257. ulimit -n 128000
  258. 2011-01-26 17:50:48,153 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: vmInputArguments=[-Xmx5000m, -XX:+UseConcMarkSweepGC, -XX:+CMSIncrementalMode, -XX:MaxDirectMemorySize=6G, -XX:+UseConcMarkSweepGC, -XX:+CMSIncrementalMode, -XX:MaxDirectMemorySize=6G, -Dhbase.log.dir=/usr/lib/hbase/bin/../logs, -Dhbase.log.file=hbase-root-regionserver-mtag3.prod.imageshack.com.log, -Dhbase.home.dir=/usr/lib/hbase/bin/.., -Dhbase.id.str=root, -Dhbase.root.logger=INFO,DRFA, -Djava.library.path=/usr/lib/hbase/bin/../lib/native/Linux-amd64-64]
  259. 2011-01-26 17:50:48,161 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: My address is mtag3.prod.imageshack.com:60020
  260. 2011-01-26 17:50:48,187 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  261. 2011-01-26 17:50:48,188 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  262. 2011-01-26 17:50:48,189 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  263. 2011-01-26 17:50:48,189 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  264. 2011-01-26 17:50:48,190 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  265. 2011-01-26 17:50:48,191 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  266. 2011-01-26 17:50:48,192 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  267. 2011-01-26 17:50:48,192 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  268. 2011-01-26 17:50:48,193 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  269. 2011-01-26 17:50:48,195 INFO org.apache.hadoop.ipc.HBaseServer: Starting SocketReader
  270. 2011-01-26 17:50:48,203 INFO org.apache.hadoop.hbase.ipc.HBaseRpcMetrics: Initializing RPC Metrics with hostName=HRegionServer, port=60020
  271. 2011-01-26 17:50:48,251 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: globalMemStoreLimit=1.9g, globalMemStoreLimitLowMark=1.7g, maxHeap=4.9g
  272. 2011-01-26 17:50:48,252 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Runs every 10000000ms
  273. 2011-01-26 17:50:48,260 INFO org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Reconnecting to zookeeper
  274. 2011-01-26 17:50:48,271 INFO org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 17:14 GMT
  275. 2011-01-26 17:50:48,271 INFO org.apache.zookeeper.ZooKeeper: Client environment:host.name=mtag3.prod.imageshack.com
  276. 2011-01-26 17:50:48,271 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.6.0_23
  277. 2011-01-26 17:50:48,271 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc.
  278. 2011-01-26 17:50:48,271 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.home=/usr/java/jdk1.6.0_23/jre
  279. 2011-01-26 17:50:48,272 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=/usr/lib/hbase/bin/../conf:/usr/java/jdk1.6.0_23/lib/tools.jar:/usr/lib/hbase/bin/..:/usr/lib/hbase/bin/../hbase-0.89.20100830.jar:/usr/lib/hbase/bin/../lib/activation-1.1.jar:/usr/lib/hbase/bin/../lib/ant-1.6.5.jar:/usr/lib/hbase/bin/../lib/ant-1.7.1.jar:/usr/lib/hbase/bin/../lib/ant-launcher-1.7.1.jar:/usr/lib/hbase/bin/../lib/asm-3.1.jar:/usr/lib/hbase/bin/../lib/avro-1.3.2.jar:/usr/lib/hbase/bin/../lib/commons-cli-1.2.jar:/usr/lib/hbase/bin/../lib/commons-codec-1.2.jar:/usr/lib/hbase/bin/../lib/commons-el-1.0.jar:/usr/lib/hbase/bin/../lib/commons-httpclient-3.1.jar:/usr/lib/hbase/bin/../lib/commons-lang-2.5.jar:/usr/lib/hbase/bin/../lib/commons-logging-1.1.1.jar:/usr/lib/hbase/bin/../lib/commons-net-1.4.1.jar:/usr/lib/hbase/bin/../lib/core-3.1.1.jar:/usr/lib/hbase/bin/../lib/guava-r05.jar:/usr/lib/hbase/bin/../lib/hadoop-0.20.1-dev-core.jar:/usr/lib/hbase/bin/../lib/hadoop-gpl-compression-0.1.0-dev.jar:/usr/lib/hbase/bin/../lib/hbase-0.89.20100830.jar:/usr/lib/hbase/bin/../lib/hsqldb-1.8.0.10.jar:/usr/lib/hbase/bin/../lib/jackson-core-asl-1.5.2.jar:/usr/lib/hbase/bin/../lib/jackson-mapper-asl-1.5.2.jar:/usr/lib/hbase/bin/../lib/jasper-compiler-5.5.23.jar:/usr/lib/hbase/bin/../lib/jasper-runtime-5.5.23.jar:/usr/lib/hbase/bin/../lib/jaxb-api-2.1.jar:/usr/lib/hbase/bin/../lib/jaxb-impl-2.1.12.jar:/usr/lib/hbase/bin/../lib/jersey-core-1.1.5.1.jar:/usr/lib/hbase/bin/../lib/jersey-json-1.1.5.1.jar:/usr/lib/hbase/bin/../lib/jersey-server-1.1.5.1.jar:/usr/lib/hbase/bin/../lib/jets3t-0.7.1.jar:/usr/lib/hbase/bin/../lib/jettison-1.1.jar:/usr/lib/hbase/bin/../lib/jetty-6.1.24.jar:/usr/lib/hbase/bin/../lib/jetty-util-6.1.24.jar:/usr/lib/hbase/bin/../lib/jline-0.9.94.jar:/usr/lib/hbase/bin/../lib/jruby-complete-1.4.0.jar:/usr/lib/hbase/bin/../lib/json-simple-1.1.jar:/usr/lib/hbase/bin/../lib/jsp-2.1-6.1.14.jar:/usr/lib/hbase/bin/../lib/jsp-api-2.0.jar:/usr/lib/hbase/bin/../lib/jsp-api-2.1-6.1.14.jar:/usr/lib/hbase/bin/../lib/jsr311-api-1.1.1.jar:/usr/lib/hbase/bin/../lib/kfs-0.3.jar:/usr/lib/hbase/bin/../lib/log4j-1.2.15.jar:/usr/lib/hbase/bin/../lib/oro-2.0.8.jar:/usr/lib/hbase/bin/../lib/paranamer-2.2.jar:/usr/lib/hbase/bin/../lib/paranamer-ant-2.2.jar:/usr/lib/hbase/bin/../lib/paranamer-generator-2.2.jar:/usr/lib/hbase/bin/../lib/protobuf-java-2.3.0.jar:/usr/lib/hbase/bin/../lib/qdox-1.10.1.jar:/usr/lib/hbase/bin/../lib/servlet-api-2.4.jar:/usr/lib/hbase/bin/../lib/servlet-api-2.5-20081211.jar:/usr/lib/hbase/bin/../lib/servlet-api-2.5-6.1.14.jar:/usr/lib/hbase/bin/../lib/slf4j-api-1.5.8.jar:/usr/lib/hbase/bin/../lib/slf4j-log4j12-1.5.8.jar:/usr/lib/hbase/bin/../lib/stax-api-1.0.1.jar:/usr/lib/hbase/bin/../lib/stax-api-1.0-2.jar:/usr/lib/hbase/bin/../lib/thrift-0.2.0.jar:/usr/lib/hbase/bin/../lib/xmlenc-0.52.jar:/usr/lib/hbase/bin/../lib/zookeeper-3.3.1.jar
  280. 2011-01-26 17:50:48,272 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/lib/hbase/bin/../lib/native/Linux-amd64-64
  281. 2011-01-26 17:50:48,272 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
  282. 2011-01-26 17:50:48,272 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
  283. 2011-01-26 17:50:48,272 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux
  284. 2011-01-26 17:50:48,272 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64
  285. 2011-01-26 17:50:48,272 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=2.6.33.3-85.fc13.x86_64
  286. 2011-01-26 17:50:48,272 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=root
  287. 2011-01-26 17:50:48,272 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/root
  288. 2011-01-26 17:50:48,272 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/usr/lib/hbase-0.89.20100830/logs
  289. 2011-01-26 17:50:48,273 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=mtag3:2181,mtag2:2181,mtag1:2181 sessionTimeout=60000 watcher=org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper@250d593e
  290. 2011-01-26 17:50:48,291 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: <mtag1,mtag2,mtag3:/hbase,mtag3.prod.imageshack.com,60020,1296093048246>Connected to zookeeper again
  291. 2011-01-26 17:50:48,291 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server mtag2/10.103.7.2:2181
  292. 2011-01-26 17:50:48,297 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to mtag2/10.103.7.2:2181, initiating session
  293. 2011-01-26 17:50:48,321 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server mtag2/10.103.7.2:2181, sessionid = 0x12cf6fe3b710099, negotiated timeout = 60000
  294. 2011-01-26 17:50:48,328 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: <mtag1,mtag2,mtag3:/hbase,mtag3.prod.imageshack.com,60020,1296093048246>Set watcher on master address ZNode /hbase/master
  295. 2011-01-26 17:50:48,337 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Got ZooKeeper event, state: SyncConnected, type: None, path: null
  296. 2011-01-26 17:50:48,379 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: <mtag1,mtag2,mtag3:/hbase,mtag3.prod.imageshack.com,60020,1296093048246>Read ZNode /hbase/master got 10.103.7.2:60000
  297. 2011-01-26 17:50:48,379 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Telling master at 10.103.7.2:60000 that we are up
  298. 2011-01-26 17:50:48,458 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Installed shutdown hook thread: Shutdownhook:regionserver60020
  299. 2011-01-26 17:50:48,481 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: sending initial server load: requests=0, regions=0, usedHeap=24, maxHeap=895
  300. 2011-01-26 17:50:48,500 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: <mtag1,mtag2,mtag3:/hbase,mtag3.prod.imageshack.com,60020,1296093048246>Created ZNode /hbase/rs/mtag3.prod.imageshack.com,60020,1296093048481 with data 10.103.7.3:60020
  301. 2011-01-26 17:50:48,522 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Config from master: hbase.regionserver.address=10.103.7.3
  302. 2011-01-26 17:50:48,523 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Config from master: fs.default.name=hdfs://mtag1:9000/hbase
  303. 2011-01-26 17:50:48,523 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Config from master: hbase.rootdir=hdfs://mtag1:9000/hbase
  304. 2011-01-26 17:50:48,523 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Master passed us address to use. Was=10.103.7.3:60020, Now=10.103.7.3
  305. 2011-01-26 17:50:48,664 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Log dir hdfs://mtag1:9000/hbase/.logs/mtag3.prod.imageshack.com,60020,1296093048481
  306. 2011-01-26 17:50:48,732 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: HLog configuration: blocksize=134217728, rollsize=127506840, enabled=true, flushlogentries=100, optionallogflushinternal=10000ms
  307. 2011-01-26 17:50:48,802 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed init
  308. java.io.IOException: cannot get log writer
  309.     at org.apache.hadoop.hbase.regionserver.wal.HLog.createWriter(HLog.java:553)
  310.     at org.apache.hadoop.hbase.regionserver.wal.HLog.rollWriter(HLog.java:453)
  311.     at org.apache.hadoop.hbase.regionserver.wal.HLog.<init>(HLog.java:355)
  312.     at org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateHLog(HRegionServer.java:1024)
  313.     at org.apache.hadoop.hbase.regionserver.HRegionServer.setupHLog(HRegionServer.java:1017)
  314.     at org.apache.hadoop.hbase.regionserver.HRegionServer.init(HRegionServer.java:825)
  315.     at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:521)
  316.     at java.lang.Thread.run(Thread.java:662)
  317. Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.protocol.ClientProtocol.abandonFile(java.lang.String, java.lang.String)
  318.     at java.lang.Class.getMethod(Class.java:1605)
  319.     at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:506)
  320.     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:968)
  321.     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:964)
  322.     at java.security.AccessController.doPrivileged(Native Method)
  323.     at javax.security.auth.Subject.doAs(Subject.java:396)
  324.     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:962)
  325.  
  326.     at org.apache.hadoop.ipc.Client.call(Client.java:826)
  327.     at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:222)
  328.     at $Proxy1.abandonFile(Unknown Source)
  329.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  330.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  331.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  332.     at java.lang.reflect.Method.invoke(Method.java:597)
  333.     at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
  334.     at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
  335.     at $Proxy1.abandonFile(Unknown Source)
  336.     at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:673)
  337.     at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:208)
  338.     at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:198)
  339.     at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:526)
  340.     at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:894)
  341.     at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:394)
  342.     at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.init(SequenceFileLogWriter.java:59)
  343.     at org.apache.hadoop.hbase.regionserver.wal.HLog.createWriter(HLog.java:550)
  344.     ... 7 more
  345. 2011-01-26 17:50:48,808 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: Aborting region server serverName=mtag3.prod.imageshack.com,60020,1296093048481, load=(requests=0, regions=0, usedHeap=0, maxHeap=0): Unhandled exception
  346. java.io.IOException: cannot get log writer
  347.     at org.apache.hadoop.hbase.regionserver.wal.HLog.createWriter(HLog.java:553)
  348.     at org.apache.hadoop.hbase.regionserver.wal.HLog.rollWriter(HLog.java:453)
  349.     at org.apache.hadoop.hbase.regionserver.wal.HLog.<init>(HLog.java:355)
  350.     at org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateHLog(HRegionServer.java:1024)
  351.     at org.apache.hadoop.hbase.regionserver.HRegionServer.setupHLog(HRegionServer.java:1017)
  352.     at org.apache.hadoop.hbase.regionserver.HRegionServer.init(HRegionServer.java:825)
  353.     at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:521)
  354.     at java.lang.Thread.run(Thread.java:662)
  355. Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.protocol.ClientProtocol.abandonFile(java.lang.String, java.lang.String)
  356.     at java.lang.Class.getMethod(Class.java:1605)
  357.     at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:506)
  358.     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:968)
  359.     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:964)
  360.     at java.security.AccessController.doPrivileged(Native Method)
  361.     at javax.security.auth.Subject.doAs(Subject.java:396)
  362.     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:962)
  363.  
  364.     at org.apache.hadoop.ipc.Client.call(Client.java:826)
  365.     at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:222)
  366.     at $Proxy1.abandonFile(Unknown Source)
  367.     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  368.     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  369.     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  370.     at java.lang.reflect.Method.invoke(Method.java:597)
  371.     at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
  372.     at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
  373.     at $Proxy1.abandonFile(Unknown Source)
  374.     at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:673)
  375.     at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:208)
  376.     at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:198)
  377.     at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:526)
  378.     at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:894)
  379.     at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:394)
  380.     at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.init(SequenceFileLogWriter.java:59)
  381.     at org.apache.hadoop.hbase.regionserver.wal.HLog.createWriter(HLog.java:550)
  382.     ... 7 more
  383. 2011-01-26 17:50:48,809 INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 60020
  384. 2011-01-26 17:50:48,815 INFO org.apache.hadoop.hbase.regionserver.StoreFile: Allocating LruBlockCache with maximum size 3.4g
  385. 2011-01-26 17:50:48,825 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: aborting server at: mtag3.prod.imageshack.com,60020,1296093048481
  386. 2011-01-26 17:50:48,939 INFO org.apache.zookeeper.ZooKeeper: Session: 0x12cf6fe3b710099 closed
  387. 2011-01-26 17:50:48,940 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: <mtag1,mtag2,mtag3:/hbase,mtag3.prod.imageshack.com,60020,1296093048246>Closed connection with ZooKeeper; /hbase/root-region-server
  388. 2011-01-26 17:50:48,940 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: regionserver60020 exiting
  389. 2011-01-26 17:50:48,941 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=Thread[Thread-9,5,main]
  390. 2011-01-26 17:50:48,941 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Starting fs shutdown hook thread.
  391. 2011-01-26 17:50:48,942 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown hook finished.
  392. 17:53:25 10.103.7.3 root@mtag3:/usr/lib/hbase/logs $
RAW Paste Data
Top