API tools faq
paste
Guest User

Untitled

a guest
Mar 4th, 2015
370
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 43.87 KB | None | 0 0
raw download clone embed print report
  1. 2015-03-03 19:44:18,318 DEBUG [LruStats #0] hfile.LruBlockCache: Total=5.02 MB, free=4.68 GB, max=4.69 GB, blocks=13, accesses=660, hits=633, hitRatio=95.91%, , cachingAccesses=646, cachingHits=620, cachingHitsRatio=95.98%, evictions=0, evicted=13, evictedPerRun=Infinity
  2. 2015-03-03 19:45:05,154 INFO [regionserver60020-SendThread(maxiq1.augmentiq.in:2181)] zookeeper.ClientCnxn: Client session timed out, have not heard from server in 26679ms for sessionid 0x24bdd9181c50000, closing socket connection and attempting reconnect
  3. 2015-03-03 19:45:05,220 INFO [regionserver60020-SendThread(maxiq1.augmentiq.in:2181)] zookeeper.ClientCnxn: Client session timed out, have not heard from server in 26679ms for sessionid 0x24bdd9181c50002, closing socket connection and attempting reconnect
  4. 2015-03-03 19:45:05,216 INFO [regionserver60020-SendThread(maxiq1.augmentiq.in:2181)] zookeeper.ClientCnxn: Client session timed out, have not heard from server in 26679ms for sessionid 0x24bdd9181c50001, closing socket connection and attempting reconnect
  5. 2015-03-03 19:45:43,756 INFO [regionserver60020-SendThread(maxiq2.augmentiq.in:2181)] zookeeper.ClientCnxn: Opening socket connection to server maxiq2.augmentiq.in/192.168.99.226:2181. Will not attempt to authenticate using SASL (unknown error)
  6. 2015-03-03 19:45:43,854 INFO [regionserver60020-SendThread(maxiq2.augmentiq.in:2181)] zookeeper.ClientCnxn: Opening socket connection to server maxiq2.augmentiq.in/192.168.99.226:2181. Will not attempt to authenticate using SASL (unknown error)
  7. 2015-03-03 19:45:43,957 INFO [regionserver60020-SendThread(maxiq2.augmentiq.in:2181)] zookeeper.ClientCnxn: Socket connection established to maxiq2.augmentiq.in/192.168.99.226:2181, initiating session
  8. 2015-03-03 19:45:43,958 INFO [regionserver60020-SendThread(maxiq2.augmentiq.in:2181)] zookeeper.ClientCnxn: Unable to reconnect to ZooKeeper service, session 0x24bdd9181c50001 has expired, closing socket connection
  9. 2015-03-03 19:45:43,962 INFO [regionserver60020-SendThread(maxiq2.augmentiq.in:2181)] zookeeper.ClientCnxn: Socket connection established to maxiq2.augmentiq.in/192.168.99.226:2181, initiating session
  10. 2015-03-03 19:45:43,963 INFO [regionserver60020-SendThread(maxiq2.augmentiq.in:2181)] zookeeper.ClientCnxn: Unable to reconnect to ZooKeeper service, session 0x24bdd9181c50000 has expired, closing socket connection
  11. 2015-03-03 19:45:43,971 INFO [regionserver60020-SendThread(maxiq2.augmentiq.in:2181)] zookeeper.ClientCnxn: Opening socket connection to server maxiq2.augmentiq.in/192.168.99.226:2181. Will not attempt to authenticate using SASL (unknown error)
  12. 2015-03-03 19:45:44,540 FATAL [regionserver60020-EventThread] regionserver.HRegionServer: ABORTING region server maxiq1.augmentiq.in,60020,1425352157901: regionserver:60020-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000, quorum=maxiq2.augmentiq.in:2181,maxiq1.augmentiq.in:2181,maxiq3.augmentiq.in:2181, baseZNode=/hbase regionserver:60020-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000 received expired from ZooKeeper, aborting
  13. org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired
  14. at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:401)
  15. at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:319)
  16. at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
  17. at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
  18. 2015-03-03 19:45:44,540 WARN [regionserver60020-EventThread] client.ConnectionManager$HConnectionImplementation: This client just lost it's session with ZooKeeper, closing it. It will be recreated next time someone needs it
  19. org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired
  20. at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:401)
  21. at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:319)
  22. at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
  23. at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
  24. 2015-03-03 19:45:45,857 INFO [regionserver60020-EventThread] client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x24bdd9181c50001
  25. 2015-03-03 19:45:45,857 INFO [regionserver60020-EventThread] zookeeper.ClientCnxn: EventThread shut down
  26. 2015-03-03 19:45:45,264 INFO [regionserver60020-SendThread(maxiq2.augmentiq.in:2181)] zookeeper.ClientCnxn: Socket connection established to maxiq2.augmentiq.in/192.168.99.226:2181, initiating session
  27. 2015-03-03 19:45:45,858 FATAL [regionserver60020-EventThread] regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint]
  28. 2015-03-03 19:45:45,859 INFO [regionserver60020-SendThread(maxiq2.augmentiq.in:2181)] zookeeper.ClientCnxn: Unable to reconnect to ZooKeeper service, session 0x24bdd9181c50002 has expired, closing socket connection
  29. 2015-03-03 19:45:45,859 WARN [regionserver60020-EventThread] client.ConnectionManager$HConnectionImplementation: This client just lost it's session with ZooKeeper, closing it. It will be recreated next time someone needs it
  30. org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired
  31. at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:401)
  32. at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:319)
  33. at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
  34. at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
  35. 2015-03-03 19:45:45,859 INFO [regionserver60020-EventThread] client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x24bdd9181c50002
  36. 2015-03-03 19:45:45,859 INFO [regionserver60020-EventThread] zookeeper.ClientCnxn: EventThread shut down
  37. 2015-03-03 19:45:48,249 INFO [regionserver60020-EventThread] regionserver.HRegionServer: STOPPED: regionserver:60020-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000, quorum=maxiq2.augmentiq.in:2181,maxiq1.augmentiq.in:2181,maxiq3.augmentiq.in:2181, baseZNode=/hbase regionserver:60020-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000-0x24bdd9181c50000 received expired from ZooKeeper, aborting
  38. 2015-03-03 19:45:48,249 INFO [regionserver60020-EventThread] zookeeper.ClientCnxn: EventThread shut down
  39. 2015-03-03 19:45:48,620 FATAL [regionserver60020] regionserver.HRegionServer: ABORTING region server maxiq1.augmentiq.in,60020,1425352157901: org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected; currently processing maxiq1.augmentiq.in,60020,1425352157901 as dead server
  40. at org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:341)
  41. at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:254)
  42. at org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:1343)
  43. at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:5087)
  44. at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
  45. at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
  46. at org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:73)
  47. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  48. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  49. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  50. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  51. at java.lang.Thread.run(Thread.java:744)
  52.  
  53. org.apache.hadoop.hbase.YouAreDeadException: org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected; currently processing maxiq1.augmentiq.in,60020,1425352157901 as dead server
  54. at org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:341)
  55. at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:254)
  56. at org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:1343)
  57. at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:5087)
  58. at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
  59. at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
  60. at org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:73)
  61. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  62. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  63. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  64. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  65. at java.lang.Thread.run(Thread.java:744)
  66.  
  67. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  68. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
  69. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  70. at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
  71. at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
  72. at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
  73. at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:300)
  74. at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:1061)
  75. at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:902)
  76. at java.lang.Thread.run(Thread.java:744)
  77. Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.YouAreDeadException): org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected; currently processing maxiq1.augmentiq.in,60020,1425352157901 as dead server
  78. at org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:341)
  79. at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:254)
  80. at org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:1343)
  81. at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:5087)
  82. at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
  83. at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
  84. at org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:73)
  85. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  86. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  87. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  88. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  89. at java.lang.Thread.run(Thread.java:744)
  90.  
  91. at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1498)
  92. at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1684)
  93. at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1737)
  94. at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerReport(RegionServerStatusProtos.java:5414)
  95. at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:1059)
  96. ... 2 more
  97. 2015-03-03 19:45:48,673 INFO [RS_OPEN_META-maxiq1:60020-0-MetaLogRoller] regionserver.LogRoller: LogRoller exiting.
  98. 2015-03-03 19:45:48,698 FATAL [regionserver60020] regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint]
  99. 2015-03-03 19:45:48,780 INFO [regionserver60020] regionserver.HRegionServer: STOPPED: org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected; currently processing maxiq1.augmentiq.in,60020,1425352157901 as dead server
  100. at org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:341)
  101. at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:254)
  102. at org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:1343)
  103. at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:5087)
  104. at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
  105. at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
  106. at org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:73)
  107. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  108. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  109. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  110. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  111. at java.lang.Thread.run(Thread.java:744)
  112.  
  113. 2015-03-03 19:45:48,780 INFO [regionserver60020] ipc.RpcServer: Stopping server on 60020
  114. 2015-03-03 19:45:48,780 INFO [RpcServer.listener,port=60020] ipc.RpcServer: RpcServer.listener,port=60020: stopping
  115. 2015-03-03 19:45:48,780 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped
  116. 2015-03-03 19:45:48,780 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping
  117. 2015-03-03 19:45:48,781 INFO [regionserver60020] regionserver.SplitLogWorker: Sending interrupt to stop the worker thread
  118. 2015-03-03 19:45:48,781 INFO [regionserver60020] regionserver.HRegionServer: Stopping infoServer
  119. 2015-03-03 19:45:48,781 INFO [SplitLogWorker-maxiq1.augmentiq.in,60020,1425352157901] regionserver.SplitLogWorker: SplitLogWorker interrupted while waiting for task, exiting: java.lang.InterruptedException
  120. 2015-03-03 19:45:48,781 INFO [SplitLogWorker-maxiq1.augmentiq.in,60020,1425352157901] regionserver.SplitLogWorker: SplitLogWorker maxiq1.augmentiq.in,60020,1425352157901 exiting
  121. 2015-03-03 19:45:49,040 INFO [regionserver60020] mortbay.log: Stopped [email protected]:60030
  122. 2015-03-03 19:45:49,141 INFO [Thread-20] regionserver.MemStoreFlusher: Thread-20 exiting
  123. 2015-03-03 19:45:49,141 INFO [regionserver60020.logRoller] regionserver.LogRoller: LogRoller exiting.
  124. 2015-03-03 19:45:49,141 INFO [regionserver60020.compactionChecker] regionserver.HRegionServer$CompactionChecker: regionserver60020.compactionChecker exiting
  125. 2015-03-03 19:45:49,141 INFO [regionserver60020] snapshot.RegionServerSnapshotManager: Stopping RegionServerSnapshotManager abruptly.
  126. 2015-03-03 19:45:49,141 INFO [regionserver60020.nonceCleaner] regionserver.ServerNonceManager$1: regionserver60020.nonceCleaner exiting
  127. 2015-03-03 19:45:49,142 DEBUG [RS_CLOSE_REGION-maxiq1:60020-1] handler.CloseRegionHandler: Processing close of errored_records,148236,1424541971630.87249e8dfe07ef8f7f19e1ea2795ffd2.
  128. 2015-03-03 19:45:49,142 DEBUG [RS_CLOSE_REGION-maxiq1:60020-2] handler.CloseRegionHandler: Processing close of app_graph_nodes,,1424438676903.fa5aaa2ee9c6c6210c631990927432c2.
  129. 2015-03-03 19:45:49,142 DEBUG [RS_CLOSE_REGION-maxiq1:60020-0] handler.CloseRegionHandler: Processing close of ambarismoketest,,1424888492704.4d4a3777dc204e49ee2f35f7d31b49ad.
  130. 2015-03-03 19:45:49,142 DEBUG [RS_CLOSE_REGION-maxiq1:60020-1] regionserver.HRegion: Closing errored_records,148236,1424541971630.87249e8dfe07ef8f7f19e1ea2795ffd2.: disabling compactions & flushes
  131. 2015-03-03 19:45:49,142 DEBUG [RS_CLOSE_REGION-maxiq1:60020-1] regionserver.HRegion: Updates disabled for region errored_records,148236,1424541971630.87249e8dfe07ef8f7f19e1ea2795ffd2.
  132. 2015-03-03 19:45:49,142 INFO [regionserver60020] regionserver.HRegionServer: aborting server maxiq1.augmentiq.in,60020,1425352157901
  133. 2015-03-03 19:45:49,142 DEBUG [regionserver60020] catalog.CatalogTracker: Stopping catalog tracker org.apache.hadoop.hbase.catalog.CatalogTracker@218d79e1
  134. 2015-03-03 19:45:49,142 DEBUG [RS_CLOSE_REGION-maxiq1:60020-2] regionserver.HRegion: Closing app_graph_nodes,,1424438676903.fa5aaa2ee9c6c6210c631990927432c2.: disabling compactions & flushes
  135. 2015-03-03 19:45:49,142 DEBUG [RS_CLOSE_REGION-maxiq1:60020-0] regionserver.HRegion: Closing ambarismoketest,,1424888492704.4d4a3777dc204e49ee2f35f7d31b49ad.: disabling compactions & flushes
  136. 2015-03-03 19:45:49,142 DEBUG [RS_CLOSE_REGION-maxiq1:60020-2] regionserver.HRegion: Updates disabled for region app_graph_nodes,,1424438676903.fa5aaa2ee9c6c6210c631990927432c2.
  137. 2015-03-03 19:45:49,142 INFO [regionserver60020] regionserver.CompactSplitThread: Waiting for Split Thread to finish...
  138. 2015-03-03 19:45:49,142 DEBUG [RS_CLOSE_REGION-maxiq1:60020-0] regionserver.HRegion: Updates disabled for region ambarismoketest,,1424888492704.4d4a3777dc204e49ee2f35f7d31b49ad.
  139. 2015-03-03 19:45:49,142 INFO [regionserver60020] regionserver.CompactSplitThread: Waiting for Merge Thread to finish...
  140. 2015-03-03 19:45:49,142 INFO [regionserver60020] regionserver.CompactSplitThread: Waiting for Large Compaction Thread to finish...
  141. 2015-03-03 19:45:49,143 INFO [regionserver60020] regionserver.CompactSplitThread: Waiting for Small Compaction Thread to finish...
  142. 2015-03-03 19:45:49,144 INFO [regionserver60020] regionserver.HRegionServer: Waiting on 9 regions to close
  143. 2015-03-03 19:45:49,144 DEBUG [regionserver60020] regionserver.HRegionServer: {87249e8dfe07ef8f7f19e1ea2795ffd2=errored_records,148236,1424541971630.87249e8dfe07ef8f7f19e1ea2795ffd2., fa5aaa2ee9c6c6210c631990927432c2=app_graph_nodes,,1424438676903.fa5aaa2ee9c6c6210c631990927432c2., 4d4a3777dc204e49ee2f35f7d31b49ad=ambarismoketest,,1424888492704.4d4a3777dc204e49ee2f35f7d31b49ad., 26ea1c5efb36c1aa02a3753063d210b1=errored_records,655365,1424598918123.26ea1c5efb36c1aa02a3753063d210b1., ac6901a4921bcd32f97b99b0b153dbb1=errored_records,2490192,1424543554571.ac6901a4921bcd32f97b99b0b153dbb1., 5b5730241a3550a89b261947095d8ec9=errored_records,7747337,1424727829872.5b5730241a3550a89b261947095d8ec9., 1588230740=hbase:meta,,1.1588230740, 8c1ef43db1c1a15e0217dc9c4b37f135=hbase:namespace,,1424327546112.8c1ef43db1c1a15e0217dc9c4b37f135., 771c73a257cb204a44d675fc184bcae4=hiveQuery,,1424438677793.771c73a257cb204a44d675fc184bcae4.}
  144. 2015-03-03 19:45:49,144 DEBUG [RS_CLOSE_META-maxiq1:60020-0] handler.CloseRegionHandler: Processing close of hbase:meta,,1.1588230740
  145. 2015-03-03 19:45:49,144 DEBUG [RS_CLOSE_META-maxiq1:60020-0] regionserver.HRegion: Closing hbase:meta,,1.1588230740: disabling compactions & flushes
  146. 2015-03-03 19:45:49,144 DEBUG [RS_CLOSE_META-maxiq1:60020-0] regionserver.HRegion: Updates disabled for region hbase:meta,,1.1588230740
  147. 2015-03-03 19:45:49,145 INFO [StoreCloserThread-errored_records,148236,1424541971630.87249e8dfe07ef8f7f19e1ea2795ffd2.-1] regionserver.HStore: Closed data-column
  148. 2015-03-03 19:45:49,145 INFO [RS_CLOSE_REGION-maxiq1:60020-1] regionserver.HRegion: Closed errored_records,148236,1424541971630.87249e8dfe07ef8f7f19e1ea2795ffd2.
  149. 2015-03-03 19:45:49,145 INFO [StoreCloserThread-app_graph_nodes,,1424438676903.fa5aaa2ee9c6c6210c631990927432c2.-1] regionserver.HStore: Closed data-column
  150. 2015-03-03 19:45:49,145 INFO [RS_CLOSE_REGION-maxiq1:60020-2] regionserver.HRegion: Closed app_graph_nodes,,1424438676903.fa5aaa2ee9c6c6210c631990927432c2.
  151. 2015-03-03 19:45:49,145 DEBUG [RS_CLOSE_REGION-maxiq1:60020-2] handler.CloseRegionHandler: Closed app_graph_nodes,,1424438676903.fa5aaa2ee9c6c6210c631990927432c2.
  152. 2015-03-03 19:45:49,145 DEBUG [RS_CLOSE_REGION-maxiq1:60020-2] handler.CloseRegionHandler: Processing close of errored_records,655365,1424598918123.26ea1c5efb36c1aa02a3753063d210b1.
  153. 2015-03-03 19:45:49,145 DEBUG [RS_CLOSE_REGION-maxiq1:60020-1] handler.CloseRegionHandler: Closed errored_records,148236,1424541971630.87249e8dfe07ef8f7f19e1ea2795ffd2.
  154. 2015-03-03 19:45:49,145 DEBUG [RS_CLOSE_REGION-maxiq1:60020-1] handler.CloseRegionHandler: Processing close of errored_records,2490192,1424543554571.ac6901a4921bcd32f97b99b0b153dbb1.
  155. 2015-03-03 19:45:49,146 DEBUG [RS_CLOSE_REGION-maxiq1:60020-1] regionserver.HRegion: Closing errored_records,2490192,1424543554571.ac6901a4921bcd32f97b99b0b153dbb1.: disabling compactions & flushes
  156. 2015-03-03 19:45:49,146 DEBUG [RS_CLOSE_REGION-maxiq1:60020-2] regionserver.HRegion: Closing errored_records,655365,1424598918123.26ea1c5efb36c1aa02a3753063d210b1.: disabling compactions & flushes
  157. 2015-03-03 19:45:49,146 DEBUG [RS_CLOSE_REGION-maxiq1:60020-1] regionserver.HRegion: Updates disabled for region errored_records,2490192,1424543554571.ac6901a4921bcd32f97b99b0b153dbb1.
  158. 2015-03-03 19:45:49,146 INFO [StoreCloserThread-ambarismoketest,,1424888492704.4d4a3777dc204e49ee2f35f7d31b49ad.-1] regionserver.HStore: Closed family
  159. 2015-03-03 19:45:49,146 DEBUG [RS_CLOSE_REGION-maxiq1:60020-2] regionserver.HRegion: Updates disabled for region errored_records,655365,1424598918123.26ea1c5efb36c1aa02a3753063d210b1.
  160. 2015-03-03 19:45:49,146 INFO [RS_CLOSE_REGION-maxiq1:60020-0] regionserver.HRegion: Closed ambarismoketest,,1424888492704.4d4a3777dc204e49ee2f35f7d31b49ad.
  161. 2015-03-03 19:45:49,146 DEBUG [RS_CLOSE_REGION-maxiq1:60020-0] handler.CloseRegionHandler: Closed ambarismoketest,,1424888492704.4d4a3777dc204e49ee2f35f7d31b49ad.
  162. 2015-03-03 19:45:49,146 DEBUG [RS_CLOSE_REGION-maxiq1:60020-0] handler.CloseRegionHandler: Processing close of errored_records,7747337,1424727829872.5b5730241a3550a89b261947095d8ec9.
  163. 2015-03-03 19:45:49,146 DEBUG [RS_CLOSE_REGION-maxiq1:60020-0] regionserver.HRegion: Closing errored_records,7747337,1424727829872.5b5730241a3550a89b261947095d8ec9.: disabling compactions & flushes
  164. 2015-03-03 19:45:49,146 DEBUG [RS_CLOSE_REGION-maxiq1:60020-0] regionserver.HRegion: Updates disabled for region errored_records,7747337,1424727829872.5b5730241a3550a89b261947095d8ec9.
  165. 2015-03-03 19:45:49,147 INFO [StoreCloserThread-hbase:meta,,1.1588230740-1] regionserver.HStore: Closed info
  166. 2015-03-03 19:45:49,147 INFO [RS_CLOSE_META-maxiq1:60020-0] regionserver.HRegion: Closed hbase:meta,,1.1588230740
  167. 2015-03-03 19:45:49,147 DEBUG [RS_CLOSE_META-maxiq1:60020-0] handler.CloseRegionHandler: Closed hbase:meta,,1.1588230740
  168. 2015-03-03 19:45:49,150 INFO [StoreCloserThread-errored_records,2490192,1424543554571.ac6901a4921bcd32f97b99b0b153dbb1.-1] regionserver.HStore: Closed data-column
  169. 2015-03-03 19:45:49,150 INFO [StoreCloserThread-errored_records,655365,1424598918123.26ea1c5efb36c1aa02a3753063d210b1.-1] regionserver.HStore: Closed data-column
  170. 2015-03-03 19:45:49,150 INFO [StoreCloserThread-errored_records,7747337,1424727829872.5b5730241a3550a89b261947095d8ec9.-1] regionserver.HStore: Closed data-column
  171. 2015-03-03 19:45:49,150 INFO [RS_CLOSE_REGION-maxiq1:60020-2] regionserver.HRegion: Closed errored_records,655365,1424598918123.26ea1c5efb36c1aa02a3753063d210b1.
  172. 2015-03-03 19:45:49,150 DEBUG [RS_CLOSE_REGION-maxiq1:60020-2] handler.CloseRegionHandler: Closed errored_records,655365,1424598918123.26ea1c5efb36c1aa02a3753063d210b1.
  173. 2015-03-03 19:45:49,150 INFO [RS_CLOSE_REGION-maxiq1:60020-1] regionserver.HRegion: Closed errored_records,2490192,1424543554571.ac6901a4921bcd32f97b99b0b153dbb1.
  174. 2015-03-03 19:45:49,150 DEBUG [RS_CLOSE_REGION-maxiq1:60020-1] handler.CloseRegionHandler: Closed errored_records,2490192,1424543554571.ac6901a4921bcd32f97b99b0b153dbb1.
  175. 2015-03-03 19:45:49,150 DEBUG [RS_CLOSE_REGION-maxiq1:60020-2] handler.CloseRegionHandler: Processing close of hbase:namespace,,1424327546112.8c1ef43db1c1a15e0217dc9c4b37f135.
  176. 2015-03-03 19:45:49,150 DEBUG [RS_CLOSE_REGION-maxiq1:60020-1] handler.CloseRegionHandler: Processing close of hiveQuery,,1424438677793.771c73a257cb204a44d675fc184bcae4.
  177. 2015-03-03 19:45:49,150 INFO [RS_CLOSE_REGION-maxiq1:60020-0] regionserver.HRegion: Closed errored_records,7747337,1424727829872.5b5730241a3550a89b261947095d8ec9.
  178. 2015-03-03 19:45:49,150 DEBUG [RS_CLOSE_REGION-maxiq1:60020-0] handler.CloseRegionHandler: Closed errored_records,7747337,1424727829872.5b5730241a3550a89b261947095d8ec9.
  179. 2015-03-03 19:45:49,150 DEBUG [RS_CLOSE_REGION-maxiq1:60020-2] regionserver.HRegion: Closing hbase:namespace,,1424327546112.8c1ef43db1c1a15e0217dc9c4b37f135.: disabling compactions & flushes
  180. 2015-03-03 19:45:49,150 DEBUG [RS_CLOSE_REGION-maxiq1:60020-1] regionserver.HRegion: Closing hiveQuery,,1424438677793.771c73a257cb204a44d675fc184bcae4.: disabling compactions & flushes
  181. 2015-03-03 19:45:49,150 DEBUG [RS_CLOSE_REGION-maxiq1:60020-2] regionserver.HRegion: Updates disabled for region hbase:namespace,,1424327546112.8c1ef43db1c1a15e0217dc9c4b37f135.
  182. 2015-03-03 19:45:49,150 DEBUG [RS_CLOSE_REGION-maxiq1:60020-1] regionserver.HRegion: Updates disabled for region hiveQuery,,1424438677793.771c73a257cb204a44d675fc184bcae4.
  183. 2015-03-03 19:45:49,151 INFO [StoreCloserThread-hiveQuery,,1424438677793.771c73a257cb204a44d675fc184bcae4.-1] regionserver.HStore: Closed data-column
  184. 2015-03-03 19:45:49,151 INFO [RS_CLOSE_REGION-maxiq1:60020-1] regionserver.HRegion: Closed hiveQuery,,1424438677793.771c73a257cb204a44d675fc184bcae4.
  185. 2015-03-03 19:45:49,152 DEBUG [RS_CLOSE_REGION-maxiq1:60020-1] handler.CloseRegionHandler: Closed hiveQuery,,1424438677793.771c73a257cb204a44d675fc184bcae4.
  186. 2015-03-03 19:45:49,152 INFO [StoreCloserThread-hbase:namespace,,1424327546112.8c1ef43db1c1a15e0217dc9c4b37f135.-1] regionserver.HStore: Closed info
  187. 2015-03-03 19:45:49,152 INFO [RS_CLOSE_REGION-maxiq1:60020-2] regionserver.HRegion: Closed hbase:namespace,,1424327546112.8c1ef43db1c1a15e0217dc9c4b37f135.
  188. 2015-03-03 19:45:49,152 DEBUG [RS_CLOSE_REGION-maxiq1:60020-2] handler.CloseRegionHandler: Closed hbase:namespace,,1424327546112.8c1ef43db1c1a15e0217dc9c4b37f135.
  189. 2015-03-03 19:45:49,344 INFO [regionserver60020] regionserver.HRegionServer: stopping server maxiq1.augmentiq.in,60020,1425352157901; all regions closed.
  190. 2015-03-03 19:45:49,344 DEBUG [RS_OPEN_META-maxiq1:60020-0-WAL.AsyncNotifier] wal.FSHLog: RS_OPEN_META-maxiq1:60020-0-WAL.AsyncNotifier interrupted while waiting for notification from AsyncSyncer thread
  191. 2015-03-03 19:45:49,344 INFO [RS_OPEN_META-maxiq1:60020-0-WAL.AsyncNotifier] wal.FSHLog: RS_OPEN_META-maxiq1:60020-0-WAL.AsyncNotifier exiting
  192. 2015-03-03 19:45:49,344 DEBUG [RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer0] wal.FSHLog: RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer0 interrupted while waiting for notification from AsyncWriter thread
  193. 2015-03-03 19:45:49,344 INFO [RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer0] wal.FSHLog: RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer0 exiting
  194. 2015-03-03 19:45:49,344 DEBUG [RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer1] wal.FSHLog: RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer1 interrupted while waiting for notification from AsyncWriter thread
  195. 2015-03-03 19:45:49,345 INFO [RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer1] wal.FSHLog: RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer1 exiting
  196. 2015-03-03 19:45:49,345 DEBUG [RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer2] wal.FSHLog: RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer2 interrupted while waiting for notification from AsyncWriter thread
  197. 2015-03-03 19:45:49,345 INFO [RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer2] wal.FSHLog: RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer2 exiting
  198. 2015-03-03 19:45:49,345 DEBUG [RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer3] wal.FSHLog: RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer3 interrupted while waiting for notification from AsyncWriter thread
  199. 2015-03-03 19:45:49,345 INFO [RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer3] wal.FSHLog: RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer3 exiting
  200. 2015-03-03 19:45:49,345 DEBUG [RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer4] wal.FSHLog: RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer4 interrupted while waiting for notification from AsyncWriter thread
  201. 2015-03-03 19:45:49,345 INFO [RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer4] wal.FSHLog: RS_OPEN_META-maxiq1:60020-0-WAL.AsyncSyncer4 exiting
  202. 2015-03-03 19:45:49,345 DEBUG [RS_OPEN_META-maxiq1:60020-0-WAL.AsyncWriter] wal.FSHLog: RS_OPEN_META-maxiq1:60020-0-WAL.AsyncWriter interrupted while waiting for newer writes added to local buffer
  203. 2015-03-03 19:45:49,345 INFO [RS_OPEN_META-maxiq1:60020-0-WAL.AsyncWriter] wal.FSHLog: RS_OPEN_META-maxiq1:60020-0-WAL.AsyncWriter exiting
  204. 2015-03-03 19:45:49,345 DEBUG [regionserver60020] wal.FSHLog: Closing WAL writer in hdfs://maxiq1.augmentiq.in:8020/apps/hbase/data/WALs/maxiq1.augmentiq.in,60020,1425352157901
  205. 2015-03-03 19:45:52,250 ERROR [regionserver60020] regionserver.HRegionServer: Metalog close and delete failed
  206. org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No lease on /apps/hbase/data/WALs/maxiq1.augmentiq.in,60020,1425352157901/maxiq1.augmentiq.in%2C60020%2C1425352157901.1425388768560.meta: File does not exist. [Lease. Holder: DFSClient_hb_rs_maxiq1.augmentiq.in,60020,1425352157901_-1017915147_33, pendingcreates: 2]
  207. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2946)
  208. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:3016)
  209. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:2996)
  210. at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:647)
  211. at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:484)
  212. at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
  213. at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
  214. at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
  215. at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
  216. at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
  217. at java.security.AccessController.doPrivileged(Native Method)
  218. at javax.security.auth.Subject.doAs(Subject.java:415)
  219. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
  220. at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
  221.  
  222. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  223. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
  224. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  225. at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
  226. at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:97)
  227. at org.apache.hadoop.hbase.RemoteExceptionHandler.checkThrowable(RemoteExceptionHandler.java:49)
  228. at org.apache.hadoop.hbase.regionserver.HRegionServer.closeWAL(HRegionServer.java:1177)
  229. at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:998)
  230. at java.lang.Thread.run(Thread.java:744)
  231. 2015-03-03 19:45:53,414 INFO [regionserver60020.leaseChecker] regionserver.Leases: regionserver60020.leaseChecker closing leases
  232. 2015-03-03 19:46:12,026 INFO [regionserver60020.leaseChecker] regionserver.Leases: regionserver60020.leaseChecker closed leases
  233. 2015-03-03 19:45:52,939 INFO [regionserver60020.periodicFlusher] regionserver.HRegionServer$PeriodicMemstoreFlusher: regionserver60020.periodicFlusher exiting
  234. 2015-03-03 19:46:12,033 DEBUG [regionserver60020-WAL.AsyncNotifier] wal.FSHLog: regionserver60020-WAL.AsyncNotifier interrupted while waiting for notification from AsyncSyncer thread
  235. 2015-03-03 19:46:12,033 INFO [regionserver60020-WAL.AsyncNotifier] wal.FSHLog: regionserver60020-WAL.AsyncNotifier exiting
  236. 2015-03-03 19:46:12,034 DEBUG [regionserver60020-WAL.AsyncSyncer0] wal.FSHLog: regionserver60020-WAL.AsyncSyncer0 interrupted while waiting for notification from AsyncWriter thread
  237. 2015-03-03 19:46:12,034 INFO [regionserver60020-WAL.AsyncSyncer0] wal.FSHLog: regionserver60020-WAL.AsyncSyncer0 exiting
  238. 2015-03-03 19:46:12,034 DEBUG [regionserver60020-WAL.AsyncSyncer1] wal.FSHLog: regionserver60020-WAL.AsyncSyncer1 interrupted while waiting for notification from AsyncWriter thread
  239. 2015-03-03 19:46:12,034 INFO [regionserver60020-WAL.AsyncSyncer1] wal.FSHLog: regionserver60020-WAL.AsyncSyncer1 exiting
  240. 2015-03-03 19:46:12,035 DEBUG [regionserver60020-WAL.AsyncSyncer2] wal.FSHLog: regionserver60020-WAL.AsyncSyncer2 interrupted while waiting for notification from AsyncWriter thread
  241. 2015-03-03 19:46:12,035 INFO [regionserver60020-WAL.AsyncSyncer2] wal.FSHLog: regionserver60020-WAL.AsyncSyncer2 exiting
  242. 2015-03-03 19:46:12,035 DEBUG [regionserver60020-WAL.AsyncSyncer3] wal.FSHLog: regionserver60020-WAL.AsyncSyncer3 interrupted while waiting for notification from AsyncWriter thread
  243. 2015-03-03 19:46:12,035 INFO [regionserver60020-WAL.AsyncSyncer3] wal.FSHLog: regionserver60020-WAL.AsyncSyncer3 exiting
  244. 2015-03-03 19:46:12,035 DEBUG [regionserver60020-WAL.AsyncSyncer4] wal.FSHLog: regionserver60020-WAL.AsyncSyncer4 interrupted while waiting for notification from AsyncWriter thread
  245. 2015-03-03 19:46:12,035 INFO [regionserver60020-WAL.AsyncSyncer4] wal.FSHLog: regionserver60020-WAL.AsyncSyncer4 exiting
  246. 2015-03-03 19:46:12,035 DEBUG [regionserver60020-WAL.AsyncWriter] wal.FSHLog: regionserver60020-WAL.AsyncWriter interrupted while waiting for newer writes added to local buffer
  247. 2015-03-03 19:46:12,035 INFO [regionserver60020-WAL.AsyncWriter] wal.FSHLog: regionserver60020-WAL.AsyncWriter exiting
  248. 2015-03-03 19:46:12,035 DEBUG [regionserver60020] wal.FSHLog: Closing WAL writer in hdfs://maxiq1.augmentiq.in:8020/apps/hbase/data/WALs/maxiq1.augmentiq.in,60020,1425352157901
  249. 2015-03-03 19:46:35,329 ERROR [regionserver60020] regionserver.HRegionServer: Close and delete failed
  250. org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No lease on /apps/hbase/data/WALs/maxiq1.augmentiq.in,60020,1425352157901/maxiq1.augmentiq.in%2C60020%2C1425352157901.1425391774142: File does not exist. [Lease. Holder: DFSClient_hb_rs_maxiq1.augmentiq.in,60020,1425352157901_-1017915147_33, pendingcreates: 2]
  251. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2946)
  252. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:3016)
  253. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:2996)
  254. at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:647)
  255. at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:484)
  256. at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
  257. at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
  258. at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
  259. at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
  260. at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
  261. at java.security.AccessController.doPrivileged(Native Method)
  262. at javax.security.auth.Subject.doAs(Subject.java:415)
  263. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
  264. at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
  265.  
  266. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  267. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
  268. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  269. at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
  270. at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:97)
  271. at org.apache.hadoop.hbase.RemoteExceptionHandler.checkThrowable(RemoteExceptionHandler.java:49)
  272. at org.apache.hadoop.hbase.regionserver.HRegionServer.closeWAL(HRegionServer.java:1188)
  273. at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:998)
  274. at java.lang.Thread.run(Thread.java:744)
  275. 2015-03-03 19:46:35,430 INFO [regionserver60020] regionserver.Leases: regionserver60020 closing leases
  276. 2015-03-03 19:46:35,430 INFO [regionserver60020] regionserver.Leases: regionserver60020 closed leases
  277. 2015-03-03 19:46:35,430 WARN [regionserver60020] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=maxiq2.augmentiq.in:2181,maxiq1.augmentiq.in:2181,maxiq3.augmentiq.in:2181, exception=org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/replication/rs/maxiq1.augmentiq.in,60020,1425352157901
  278. 2015-03-03 19:46:35,430 INFO [regionserver60020] util.RetryCounter: Sleeping 1000ms before retry #0...
  279. 2015-03-03 19:46:36,431 WARN [regionserver60020] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=maxiq2.augmentiq.in:2181,maxiq1.augmentiq.in:2181,maxiq3.augmentiq.in:2181, exception=org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/replication/rs/maxiq1.augmentiq.in,60020,1425352157901
  280. 2015-03-03 19:46:36,431 INFO [regionserver60020] util.RetryCounter: Sleeping 2000ms before retry #1...
  281. 2015-03-03 19:46:38,431 WARN [regionserver60020] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=maxiq2.augmentiq.in:2181,maxiq1.augmentiq.in:2181,maxiq3.augmentiq.in:2181, exception=org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/replication/rs/maxiq1.augmentiq.in,60020,1425352157901
  282. 2015-03-03 19:46:38,431 INFO [regionserver60020] util.RetryCounter: Sleeping 4000ms before retry #2...
  283. 2015-03-03 19:46:42,431 WARN [regionserver60020] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=maxiq2.augmentiq.in:2181,maxiq1.augmentiq.in:2181,maxiq3.augmentiq.in:2181, exception=org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/replication/rs/maxiq1.augmentiq.in,60020,1425352157901
  284. 2015-03-03 19:46:42,431 INFO [regionserver60020] util.RetryCounter: Sleeping 8000ms before retry #3...
  285. 2015-03-03 19:46:50,431 WARN [regionserver60020] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=maxiq2.augmentiq.in:2181,maxiq1.augmentiq.in:2181,maxiq3.augmentiq.in:2181, exception=org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/replication/rs/maxiq1.augmentiq.in,60020,1425352157901
  286. 2015-03-03 19:46:50,432 ERROR [regionserver60020] zookeeper.RecoverableZooKeeper: ZooKeeper getChildren failed after 4 attempts
  287. 2015-03-03 19:46:50,432 WARN [regionserver60020] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=maxiq2.augmentiq.in:2181,maxiq1.augmentiq.in:2181,maxiq3.augmentiq.in:2181, exception=org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/rs/maxiq1.augmentiq.in,60020,1425352157901
  288. 2015-03-03 19:46:50,432 INFO [regionserver60020] util.RetryCounter: Sleeping 1000ms before retry #0...
  289. 2015-03-03 19:46:51,432 WARN [regionserver60020] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=maxiq2.augmentiq.in:2181,maxiq1.augmentiq.in:2181,maxiq3.augmentiq.in:2181, exception=org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/rs/maxiq1.augmentiq.in,60020,1425352157901
  290. 2015-03-03 19:46:51,432 INFO [regionserver60020] util.RetryCounter: Sleeping 2000ms before retry #1...
  291. 2015-03-03 19:46:53,432 WARN [regionserver60020] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=maxiq2.augmentiq.in:2181,maxiq1.augmentiq.in:2181,maxiq3.augmentiq.in:2181, exception=org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/rs/maxiq1.augmentiq.in,60020,1425352157901
  292. 2015-03-03 19:46:53,432 INFO [regionserver60020] util.RetryCounter: Sleeping 4000ms before retry #2...
  293. 2015-03-03 19:46:57,433 WARN [regionserver60020] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=maxiq2.augmentiq.in:2181,maxiq1.augmentiq.in:2181,maxiq3.augmentiq.in:2181, exception=org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/rs/maxiq1.augmentiq.in,60020,1425352157901
  294. 2015-03-03 19:47:01,807 INFO [regionserver60020] util.RetryCounter: Sleeping 8000ms before retry #3...
  295. 2015-03-03 19:47:09,807 WARN [regionserver60020] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=maxiq2.augmentiq.in:2181,maxiq1.augmentiq.in:2181,maxiq3.augmentiq.in:2181, exception=org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/rs/maxiq1.augmentiq.in,60020,1425352157901
  296. 2015-03-03 19:47:09,807 ERROR [regionserver60020] zookeeper.RecoverableZooKeeper: ZooKeeper delete failed after 4 attempts
  297. 2015-03-03 19:47:09,808 WARN [regionserver60020] regionserver.HRegionServer: Failed deleting my ephemeral node
  298. org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/rs/maxiq1.augmentiq.in,60020,1425352157901
  299. at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
  300. at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
  301. at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
  302. at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:156)
  303. at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1270)
  304. at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1259)
  305. at org.apache.hadoop.hbase.regionserver.HRegionServer.deleteMyEphemeralNode(HRegionServer.java:1286)
  306. at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1016)
  307. at java.lang.Thread.run(Thread.java:744)
  308. 2015-03-03 19:47:09,917 INFO [regionserver60020] regionserver.HRegionServer: stopping server maxiq1.augmentiq.in,60020,1425352157901; zookeeper connection closed.
  309. 2015-03-03 19:47:09,917 INFO [regionserver60020] regionserver.HRegionServer: regionserver60020 exiting
  310. 2015-03-03 19:47:09,917 ERROR [main] regionserver.HRegionServerCommandLine: Region server exiting
  311. java.lang.RuntimeException: HRegionServer Aborted
  312. at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:66)
  313. at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:85)
  314. at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
  315. at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
  316. at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2403)
  317. 2015-03-03 19:47:09,927 INFO [Thread-11] regionserver.ShutdownHook: Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@1355c642
  318. 2015-03-03 19:47:09,927 INFO [Thread-11] regionserver.HRegionServer: STOPPED: Shutdown hook
  319. 2015-03-03 19:47:09,928 INFO [Thread-11] regionserver.ShutdownHook: Starting fs shutdown hook thread.
  320. 2015-03-03 19:47:09,929 INFO [Thread-11] regionserver.ShutdownHook: Shutdown hook finished.
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!