API tools faq
paste
Guest User

Untitled

a guest
May 14th, 2010
191
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 101.65 KB | None | 0 0
raw download clone embed print report
  1. Before shutdown command:
  2. 2010-05-14 13:46:48
  3. Full thread dump Java HotSpot(TM) 64-Bit Server VM (14.3-b01 mixed mode):
  4.  
  5. "Attach Listener" daemon prio=10 tid=0x00002aaeb2ac8000 nid=0x766d waiting on condition [0x0000000000000000]
  6. java.lang.Thread.State: RUNNABLE
  7.  
  8. "regionserver/193.0.23.14:60020.worker.regionCloser.inrdb_reg_line,,1273476414312" daemon prio=10 tid=0x00002aaeb43f1800 nid=0x758a waiting for monitor entry [0x00000000487fd000]
  9. java.lang.Thread.State: BLOCKED (on object monitor)
  10. at org.apache.hadoop.hbase.regionserver.StoreScanner.updateReaders(StoreScanner.java:251)
  11. - waiting to lock <0x00002aaac855b5e8> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  12. at org.apache.hadoop.hbase.regionserver.Store.notifyChangedReadersObservers(Store.java:621)
  13. at org.apache.hadoop.hbase.regionserver.Store.updateStorefiles(Store.java:607)
  14. at org.apache.hadoop.hbase.regionserver.Store.access$200(Store.java:88)
  15. at org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.commit(Store.java:1609)
  16. at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1028)
  17. at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:521)
  18. - locked <0x00002aaac6b288f8> (a java.lang.Object)
  19. at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:456)
  20. at org.apache.hadoop.hbase.regionserver.HRegionServer$RegionCloserThread.run(HRegionServer.java:1654)
  21.  
  22. "ResponseProcessor for block blk_7830202764694154483_625018" daemon prio=10 tid=0x00002aaeb2bb4800 nid=0x6902 runnable [0x00000000431a7000]
  23. java.lang.Thread.State: RUNNABLE
  24. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  25. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
  26. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  27. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  28. - locked <0x00002aaac8ddb6e8> (a sun.nio.ch.Util$1)
  29. - locked <0x00002aaac8ddb700> (a java.util.Collections$UnmodifiableSet)
  30. - locked <0x00002aaac090dd58> (a sun.nio.ch.EPollSelectorImpl)
  31. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  32. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:332)
  33. at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
  34. at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
  35. at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
  36. at java.io.DataInputStream.readFully(DataInputStream.java:178)
  37. at java.io.DataInputStream.readLong(DataInputStream.java:399)
  38. at org.apache.hadoop.hdfs.protocol.DataTransferProtocol$PipelineAck.readFields(DataTransferProtocol.java:119)
  39. at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$ResponseProcessor.run(DFSClient.java:2424)
  40.  
  41. "pool-1-thread-1" prio=10 tid=0x00002aaeb4448000 nid=0x6532 waiting on condition [0x0000000046fe5000]
  42. java.lang.Thread.State: TIMED_WAITING (parking)
  43. at sun.misc.Unsafe.park(Native Method)
  44. - parking to wait for <0x00002aaac05502f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  45. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
  46. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
  47. at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
  48. at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583)
  49. at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576)
  50. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
  51. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
  52. at java.lang.Thread.run(Thread.java:619)
  53.  
  54. "LruBlockCache.EvictionThread" daemon prio=10 tid=0x00002aaeb4444000 nid=0x6531 in Object.wait() [0x0000000046ee4000]
  55. java.lang.Thread.State: WAITING (on object monitor)
  56. at java.lang.Object.wait(Native Method)
  57. - waiting on <0x00002aaac0503f20> (a org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread)
  58. at java.lang.Object.wait(Object.java:485)
  59. at org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread.run(LruBlockCache.java:512)
  60. - locked <0x00002aaac0503f20> (a org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread)
  61.  
  62. "IPC Server handler 49 on 60020" daemon prio=10 tid=0x00002aaeb443f800 nid=0x6530 waiting on condition [0x0000000046de3000]
  63. java.lang.Thread.State: WAITING (parking)
  64. at sun.misc.Unsafe.park(Native Method)
  65. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  66. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  67. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  68. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  69. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  70.  
  71. "IPC Server handler 48 on 60020" daemon prio=10 tid=0x00002aaeb443d800 nid=0x652f runnable [0x0000000046ce2000]
  72. java.lang.Thread.State: RUNNABLE
  73. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  74. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  75. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  76. - locked <0x00002aaaaeb278a0> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  77. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  78. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  79. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  80. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  81. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  82. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  83. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  84. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  85. at java.lang.reflect.Method.invoke(Method.java:597)
  86. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  87. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  88.  
  89. "IPC Server handler 47 on 60020" daemon prio=10 tid=0x00002aaeb443c000 nid=0x652e waiting on condition [0x0000000046be1000]
  90. java.lang.Thread.State: WAITING (parking)
  91. at sun.misc.Unsafe.park(Native Method)
  92. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  93. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  94. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  95. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  96. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  97.  
  98. "IPC Server handler 46 on 60020" daemon prio=10 tid=0x00002aaeb443a000 nid=0x652d runnable [0x0000000046ae0000]
  99. java.lang.Thread.State: RUNNABLE
  100. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  101. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  102. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  103. - locked <0x00002aaaaead96d8> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  104. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  105. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  106. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  107. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  108. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  109. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  110. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  111. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  112. at java.lang.reflect.Method.invoke(Method.java:597)
  113. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  114. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  115.  
  116. "IPC Server handler 45 on 60020" daemon prio=10 tid=0x00002aaeb4363000 nid=0x652c waiting on condition [0x00000000469df000]
  117. java.lang.Thread.State: WAITING (parking)
  118. at sun.misc.Unsafe.park(Native Method)
  119. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  120. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  121. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  122. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  123. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  124.  
  125. "IPC Server handler 44 on 60020" daemon prio=10 tid=0x00002aaeb4361000 nid=0x652b runnable [0x00000000468de000]
  126. java.lang.Thread.State: RUNNABLE
  127. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  128. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  129. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  130. - locked <0x00002aaaae509c78> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  131. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  132. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  133. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  134. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  135. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  136. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  137. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  138. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  139. at java.lang.reflect.Method.invoke(Method.java:597)
  140. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  141. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  142.  
  143. "IPC Server handler 43 on 60020" daemon prio=10 tid=0x00002aaeb435f000 nid=0x652a waiting on condition [0x00000000467dd000]
  144. java.lang.Thread.State: WAITING (parking)
  145. at sun.misc.Unsafe.park(Native Method)
  146. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  147. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  148. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  149. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  150. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  151.  
  152. "IPC Server handler 42 on 60020" daemon prio=10 tid=0x00002aaeb435d000 nid=0x6529 waiting on condition [0x00000000466dc000]
  153. java.lang.Thread.State: WAITING (parking)
  154. at sun.misc.Unsafe.park(Native Method)
  155. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  156. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  157. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  158. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  159. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  160.  
  161. "IPC Server handler 41 on 60020" daemon prio=10 tid=0x00002aaeb435b000 nid=0x6528 waiting on condition [0x00000000465db000]
  162. java.lang.Thread.State: WAITING (parking)
  163. at sun.misc.Unsafe.park(Native Method)
  164. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  165. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  166. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  167. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  168. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  169.  
  170. "IPC Server handler 40 on 60020" daemon prio=10 tid=0x00002aaeb4359800 nid=0x6527 waiting on condition [0x00000000464da000]
  171. java.lang.Thread.State: WAITING (parking)
  172. at sun.misc.Unsafe.park(Native Method)
  173. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  174. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  175. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  176. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  177. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  178.  
  179. "IPC Server handler 39 on 60020" daemon prio=10 tid=0x00002aaeb4357800 nid=0x6526 runnable [0x00000000463d9000]
  180. java.lang.Thread.State: RUNNABLE
  181. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  182. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  183. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  184. - locked <0x00002aaad345bbc8> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  185. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  186. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  187. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  188. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  189. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  190. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  191. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  192. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  193. at java.lang.reflect.Method.invoke(Method.java:597)
  194. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  195. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  196.  
  197. "IPC Server handler 38 on 60020" daemon prio=10 tid=0x00002aaeb4355800 nid=0x6525 waiting on condition [0x00000000462d8000]
  198. java.lang.Thread.State: WAITING (parking)
  199. at sun.misc.Unsafe.park(Native Method)
  200. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  201. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  202. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  203. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  204. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  205.  
  206. "IPC Server handler 37 on 60020" daemon prio=10 tid=0x00002aaeb4353800 nid=0x6524 runnable [0x00000000461d7000]
  207. java.lang.Thread.State: RUNNABLE
  208. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  209. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  210. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  211. - locked <0x00002aaac6a5b5e0> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  212. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  213. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  214. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  215. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  216. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  217. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  218. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  219. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  220. at java.lang.reflect.Method.invoke(Method.java:597)
  221. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  222. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  223.  
  224. "IPC Server handler 36 on 60020" daemon prio=10 tid=0x00002aaeb4351800 nid=0x6523 waiting on condition [0x00000000460d6000]
  225. java.lang.Thread.State: WAITING (parking)
  226. at sun.misc.Unsafe.park(Native Method)
  227. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  228. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  229. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  230. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  231. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  232.  
  233. "IPC Server handler 35 on 60020" daemon prio=10 tid=0x00002aaeb434f800 nid=0x6522 runnable [0x0000000045fd5000]
  234. java.lang.Thread.State: RUNNABLE
  235. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  236. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  237. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  238. - locked <0x00002aaaaea40fa0> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  239. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  240. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  241. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  242. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  243. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  244. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  245. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  246. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  247. at java.lang.reflect.Method.invoke(Method.java:597)
  248. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  249. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  250.  
  251. "IPC Server handler 34 on 60020" daemon prio=10 tid=0x00002aaeb434d800 nid=0x6521 runnable [0x0000000045ed4000]
  252. java.lang.Thread.State: RUNNABLE
  253. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  254. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  255. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  256. - locked <0x00002aaac201ee00> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  257. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  258. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  259. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  260. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  261. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  262. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  263. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  264. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  265. at java.lang.reflect.Method.invoke(Method.java:597)
  266. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  267. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  268.  
  269. "IPC Server handler 33 on 60020" daemon prio=10 tid=0x00002aaeb434b800 nid=0x6520 waiting on condition [0x0000000045dd3000]
  270. java.lang.Thread.State: WAITING (parking)
  271. at sun.misc.Unsafe.park(Native Method)
  272. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  273. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  274. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  275. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  276. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  277.  
  278. "IPC Server handler 32 on 60020" daemon prio=10 tid=0x00002aaeb4349800 nid=0x651f runnable [0x0000000045cd1000]
  279. java.lang.Thread.State: RUNNABLE
  280. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  281. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  282. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  283. - locked <0x00002aaac08ac840> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  284. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  285. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  286. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  287. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  288. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  289. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  290. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  291. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  292. at java.lang.reflect.Method.invoke(Method.java:597)
  293. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  294. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  295.  
  296. "IPC Server handler 31 on 60020" daemon prio=10 tid=0x00002aaeb4347800 nid=0x651e runnable [0x0000000045bd1000]
  297. java.lang.Thread.State: RUNNABLE
  298. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  299. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  300. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  301. - locked <0x00002aaad0c481b8> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  302. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  303. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  304. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  305. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  306. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  307. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  308. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  309. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  310. at java.lang.reflect.Method.invoke(Method.java:597)
  311. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  312. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  313.  
  314. "IPC Server handler 30 on 60020" daemon prio=10 tid=0x00002aaeb4346000 nid=0x651d runnable [0x0000000045ad0000]
  315. java.lang.Thread.State: RUNNABLE
  316. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  317. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  318. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  319. - locked <0x00002aaaae4a45e8> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  320. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  321. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  322. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  323. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  324. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  325. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  326. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  327. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  328. at java.lang.reflect.Method.invoke(Method.java:597)
  329. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  330. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  331.  
  332. "IPC Server handler 29 on 60020" daemon prio=10 tid=0x00002aaeb433e800 nid=0x651c waiting on condition [0x00000000459cf000]
  333. java.lang.Thread.State: WAITING (parking)
  334. at sun.misc.Unsafe.park(Native Method)
  335. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  336. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  337. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  338. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  339. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  340.  
  341. "IPC Server handler 28 on 60020" daemon prio=10 tid=0x00002aaeb433c800 nid=0x651b waiting on condition [0x00000000458ce000]
  342. java.lang.Thread.State: WAITING (parking)
  343. at sun.misc.Unsafe.park(Native Method)
  344. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  345. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  346. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  347. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  348. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  349.  
  350. "IPC Server handler 27 on 60020" daemon prio=10 tid=0x00002aaeb433a800 nid=0x651a waiting on condition [0x00000000457cd000]
  351. java.lang.Thread.State: WAITING (parking)
  352. at sun.misc.Unsafe.park(Native Method)
  353. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  354. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  355. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  356. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  357. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  358.  
  359. "IPC Server handler 26 on 60020" daemon prio=10 tid=0x00002aaeb4338800 nid=0x6519 waiting on condition [0x00000000456cc000]
  360. java.lang.Thread.State: WAITING (parking)
  361. at sun.misc.Unsafe.park(Native Method)
  362. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  363. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  364. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  365. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  366. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  367.  
  368. "IPC Server handler 25 on 60020" daemon prio=10 tid=0x00002aaeb4336800 nid=0x6518 runnable [0x00000000455cb000]
  369. java.lang.Thread.State: RUNNABLE
  370. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  371. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  372. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  373. - locked <0x00002aaad0c45848> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  374. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  375. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  376. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  377. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  378. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  379. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  380. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  381. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  382. at java.lang.reflect.Method.invoke(Method.java:597)
  383. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  384. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  385.  
  386. "IPC Server handler 24 on 60020" daemon prio=10 tid=0x00002aaeb4334800 nid=0x6517 runnable [0x00000000454c9000]
  387. java.lang.Thread.State: RUNNABLE
  388. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  389. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  390. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  391. - locked <0x00002aaac83e2480> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  392. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  393. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  394. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  395. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  396. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  397. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  398. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  399. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  400. at java.lang.reflect.Method.invoke(Method.java:597)
  401. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  402. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  403.  
  404. "IPC Server handler 23 on 60020" daemon prio=10 tid=0x00002aaeb4332800 nid=0x6516 waiting on condition [0x00000000453c9000]
  405. java.lang.Thread.State: WAITING (parking)
  406. at sun.misc.Unsafe.park(Native Method)
  407. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  408. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  409. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  410. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  411. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  412.  
  413. "IPC Server handler 22 on 60020" daemon prio=10 tid=0x00002aaeb4331000 nid=0x6515 waiting on condition [0x00000000452c8000]
  414. java.lang.Thread.State: WAITING (parking)
  415. at sun.misc.Unsafe.park(Native Method)
  416. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  417. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  418. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  419. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  420. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  421.  
  422. "IPC Server handler 21 on 60020" daemon prio=10 tid=0x00002aaeb432f000 nid=0x6514 waiting on condition [0x00000000451c7000]
  423. java.lang.Thread.State: WAITING (parking)
  424. at sun.misc.Unsafe.park(Native Method)
  425. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  426. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  427. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  428. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  429. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  430.  
  431. "IPC Server handler 20 on 60020" daemon prio=10 tid=0x00002aaeb432d800 nid=0x6513 runnable [0x00000000450c6000]
  432. java.lang.Thread.State: RUNNABLE
  433. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  434. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  435. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  436. - locked <0x00002aaad0b60e18> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  437. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  438. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  439. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  440. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  441. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  442. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  443. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  444. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  445. at java.lang.reflect.Method.invoke(Method.java:597)
  446. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  447. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  448.  
  449. "IPC Server handler 19 on 60020" daemon prio=10 tid=0x00002aaeb430b000 nid=0x6512 waiting on condition [0x0000000044fc5000]
  450. java.lang.Thread.State: WAITING (parking)
  451. at sun.misc.Unsafe.park(Native Method)
  452. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  453. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  454. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  455. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  456. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  457.  
  458. "IPC Server handler 18 on 60020" daemon prio=10 tid=0x00002aaeb4309000 nid=0x6511 runnable [0x0000000044ec4000]
  459. java.lang.Thread.State: RUNNABLE
  460. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  461. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  462. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  463. - locked <0x00002aaad345bca8> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  464. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  465. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  466. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  467. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  468. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  469. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  470. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  471. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  472. at java.lang.reflect.Method.invoke(Method.java:597)
  473. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  474. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  475.  
  476. "IPC Server handler 17 on 60020" daemon prio=10 tid=0x00002aaeb4307000 nid=0x6510 waiting on condition [0x0000000044dc3000]
  477. java.lang.Thread.State: WAITING (parking)
  478. at sun.misc.Unsafe.park(Native Method)
  479. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  480. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  481. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  482. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  483. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  484.  
  485. "IPC Server handler 16 on 60020" daemon prio=10 tid=0x00002aaeb4305000 nid=0x650f waiting on condition [0x0000000044cc2000]
  486. java.lang.Thread.State: WAITING (parking)
  487. at sun.misc.Unsafe.park(Native Method)
  488. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  489. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  490. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  491. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  492. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  493.  
  494. "IPC Server handler 15 on 60020" daemon prio=10 tid=0x00002aaeb4303000 nid=0x650e waiting on condition [0x0000000044bc1000]
  495. java.lang.Thread.State: WAITING (parking)
  496. at sun.misc.Unsafe.park(Native Method)
  497. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  498. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  499. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  500. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  501. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  502.  
  503. "IPC Server handler 14 on 60020" daemon prio=10 tid=0x00002aaeb4301000 nid=0x650d waiting on condition [0x0000000044ac0000]
  504. java.lang.Thread.State: WAITING (parking)
  505. at sun.misc.Unsafe.park(Native Method)
  506. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  507. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  508. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  509. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  510. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  511.  
  512. "IPC Server handler 13 on 60020" daemon prio=10 tid=0x00002aaeb42ff000 nid=0x650c waiting on condition [0x00000000449bf000]
  513. java.lang.Thread.State: WAITING (parking)
  514. at sun.misc.Unsafe.park(Native Method)
  515. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  516. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  517. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  518. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  519. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  520.  
  521. "IPC Server handler 12 on 60020" daemon prio=10 tid=0x00002aaeb42fd000 nid=0x650b waiting on condition [0x00000000448be000]
  522. java.lang.Thread.State: WAITING (parking)
  523. at sun.misc.Unsafe.park(Native Method)
  524. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  525. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  526. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  527. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  528. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  529.  
  530. "IPC Server handler 11 on 60020" daemon prio=10 tid=0x00002aaeb42fb800 nid=0x650a runnable [0x00000000447bd000]
  531. java.lang.Thread.State: RUNNABLE
  532. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  533. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  534. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  535. - locked <0x00002aaad7d0f7b0> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  536. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  537. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  538. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  539. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  540. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  541. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  542. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  543. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  544. at java.lang.reflect.Method.invoke(Method.java:597)
  545. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  546. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  547.  
  548. "IPC Server handler 10 on 60020" daemon prio=10 tid=0x00002aaeb42f9800 nid=0x6509 waiting on condition [0x00000000446bc000]
  549. java.lang.Thread.State: WAITING (parking)
  550. at sun.misc.Unsafe.park(Native Method)
  551. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  552. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  553. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  554. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  555. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  556.  
  557. "IPC Server handler 9 on 60020" daemon prio=10 tid=0x00002aaeb42f7800 nid=0x6508 runnable [0x00000000445bb000]
  558. java.lang.Thread.State: RUNNABLE
  559. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  560. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  561. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  562. - locked <0x00002aaac855b5e8> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  563. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  564. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  565. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  566. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  567. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  568. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  569. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  570. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  571. at java.lang.reflect.Method.invoke(Method.java:597)
  572. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  573. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  574.  
  575. "IPC Server handler 8 on 60020" daemon prio=10 tid=0x00002aaeb4160000 nid=0x6507 runnable [0x00000000444ba000]
  576. java.lang.Thread.State: RUNNABLE
  577. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  578. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  579. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  580. - locked <0x00002aaad0c47b98> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  581. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  582. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  583. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  584. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  585. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  586. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  587. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  588. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  589. at java.lang.reflect.Method.invoke(Method.java:597)
  590. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  591. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  592.  
  593. "IPC Server handler 7 on 60020" daemon prio=10 tid=0x00002aaeb415e000 nid=0x6506 waiting on condition [0x00000000443b9000]
  594. java.lang.Thread.State: WAITING (parking)
  595. at sun.misc.Unsafe.park(Native Method)
  596. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  597. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  598. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  599. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  600. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  601.  
  602. "IPC Server handler 6 on 60020" daemon prio=10 tid=0x00002aaeb415c000 nid=0x6505 waiting on condition [0x00000000442b8000]
  603. java.lang.Thread.State: WAITING (parking)
  604. at sun.misc.Unsafe.park(Native Method)
  605. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  606. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  607. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  608. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  609. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  610.  
  611. "IPC Server handler 5 on 60020" daemon prio=10 tid=0x00002aaeb415a000 nid=0x6504 waiting on condition [0x00000000441b7000]
  612. java.lang.Thread.State: WAITING (parking)
  613. at sun.misc.Unsafe.park(Native Method)
  614. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  615. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  616. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  617. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  618. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  619.  
  620. "IPC Server handler 4 on 60020" daemon prio=10 tid=0x00002aaeb4158800 nid=0x6503 waiting on condition [0x00000000440b6000]
  621. java.lang.Thread.State: WAITING (parking)
  622. at sun.misc.Unsafe.park(Native Method)
  623. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  624. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  625. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  626. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  627. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  628.  
  629. "IPC Server handler 3 on 60020" daemon prio=10 tid=0x00002aaeb4156800 nid=0x6502 waiting on condition [0x0000000043fb5000]
  630. java.lang.Thread.State: WAITING (parking)
  631. at sun.misc.Unsafe.park(Native Method)
  632. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  633. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  634. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  635. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  636. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  637.  
  638. "IPC Server handler 2 on 60020" daemon prio=10 tid=0x00002aaeb4154800 nid=0x6501 waiting on condition [0x0000000043eb4000]
  639. java.lang.Thread.State: WAITING (parking)
  640. at sun.misc.Unsafe.park(Native Method)
  641. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  642. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  643. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  644. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  645. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  646.  
  647. "IPC Server handler 1 on 60020" daemon prio=10 tid=0x00002aaeb4153000 nid=0x6500 runnable [0x0000000043db3000]
  648. java.lang.Thread.State: RUNNABLE
  649. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  650. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  651. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  652. - locked <0x00002aaaaed7be80> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  653. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  654. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  655. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  656. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  657. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  658. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  659. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  660. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  661. at java.lang.reflect.Method.invoke(Method.java:597)
  662. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  663. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  664.  
  665. "IPC Server handler 0 on 60020" daemon prio=10 tid=0x00002aaeb40ef000 nid=0x64ff waiting on condition [0x0000000043cb2000]
  666. java.lang.Thread.State: WAITING (parking)
  667. at sun.misc.Unsafe.park(Native Method)
  668. - parking to wait for <0x00002aaac0282118> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  669. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  670. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  671. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  672. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:901)
  673.  
  674. "IPC Server listener on 60020" daemon prio=10 tid=0x00002aaeb40f1800 nid=0x64fe runnable [0x0000000043bb1000]
  675. java.lang.Thread.State: RUNNABLE
  676. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  677. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
  678. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  679. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  680. - locked <0x00002aaac02b1ee0> (a sun.nio.ch.Util$1)
  681. - locked <0x00002aaac02b1ef8> (a java.util.Collections$UnmodifiableSet)
  682. - locked <0x00002aaac02b0a90> (a sun.nio.ch.EPollSelectorImpl)
  683. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  684. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
  685. at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.run(HBaseServer.java:308)
  686.  
  687. "IPC Server Responder" daemon prio=10 tid=0x00002aaeb40ec000 nid=0x64fd runnable [0x0000000043ab0000]
  688. java.lang.Thread.State: RUNNABLE
  689. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  690. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
  691. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  692. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  693. - locked <0x00002aaac0282160> (a sun.nio.ch.Util$1)
  694. - locked <0x00002aaac0282178> (a java.util.Collections$UnmodifiableSet)
  695. - locked <0x00002aaac029f9f0> (a sun.nio.ch.EPollSelectorImpl)
  696. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  697. at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.run(HBaseServer.java:477)
  698.  
  699. "Timer-0" daemon prio=10 tid=0x00002aaeb439b800 nid=0x64fa in Object.wait() [0x00000000439af000]
  700. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  701. at java.lang.Object.wait(Native Method)
  702. - waiting on <0x00002aaac04ee038> (a java.util.TaskQueue)
  703. at java.util.TimerThread.mainLoop(Timer.java:509)
  704. - locked <0x00002aaac04ee038> (a java.util.TaskQueue)
  705. at java.util.TimerThread.run(Timer.java:462)
  706.  
  707. "1512109123@qtp0-1 - Acceptor0 [email protected]:60030" prio=10 tid=0x00002aaeb243b800 nid=0x64ad runnable [0x00000000438ae000]
  708. java.lang.Thread.State: RUNNABLE
  709. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  710. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
  711. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  712. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  713. - locked <0x00002aaac049cd68> (a sun.nio.ch.Util$1)
  714. - locked <0x00002aaac049cd80> (a java.util.Collections$UnmodifiableSet)
  715. - locked <0x00002aaac02d72c8> (a sun.nio.ch.EPollSelectorImpl)
  716. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  717. at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:429)
  718. at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:185)
  719. at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
  720. at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:707)
  721. at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
  722.  
  723. "2018147830@qtp0-0" prio=10 tid=0x00002aaeb22a0000 nid=0x64ac in Object.wait() [0x00000000437ad000]
  724. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  725. at java.lang.Object.wait(Native Method)
  726. - waiting on <0x00002aaac047ced8> (a org.mortbay.thread.QueuedThreadPool$PoolThread)
  727. at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:565)
  728. - locked <0x00002aaac047ced8> (a org.mortbay.thread.QueuedThreadPool$PoolThread)
  729.  
  730. "regionserver/193.0.23.14:60020.leaseChecker" prio=10 tid=0x00002aaeb2222800 nid=0x64ab waiting on condition [0x00000000436ac000]
  731. java.lang.Thread.State: TIMED_WAITING (parking)
  732. at sun.misc.Unsafe.park(Native Method)
  733. - parking to wait for <0x00002aaac029a7f0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  734. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
  735. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
  736. at java.util.concurrent.DelayQueue.poll(DelayQueue.java:201)
  737. at org.apache.hadoop.hbase.Leases.run(Leases.java:81)
  738.  
  739. "regionserver/193.0.23.14:60020.majorCompactionChecker" daemon prio=10 tid=0x00002aaeb260f000 nid=0x64aa in Object.wait() [0x00000000435ab000]
  740. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  741. at java.lang.Object.wait(Native Method)
  742. - waiting on <0x00002aaac028d5d0> (a java.lang.Object)
  743. at org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:89)
  744. - locked <0x00002aaac028d5d0> (a java.lang.Object)
  745. at org.apache.hadoop.hbase.Chore.run(Chore.java:76)
  746.  
  747. "regionserver/193.0.23.14:60020.worker" daemon prio=10 tid=0x00002aaeb260d000 nid=0x64a9 in Object.wait() [0x00000000434aa000]
  748. java.lang.Thread.State: WAITING (on object monitor)
  749. at java.lang.Object.wait(Native Method)
  750. - waiting on <0x00002aaab089d758> (a org.apache.hadoop.hbase.regionserver.HRegionServer$RegionCloserThread)
  751. at java.lang.Thread.join(Thread.java:1143)
  752. - locked <0x00002aaab089d758> (a org.apache.hadoop.hbase.regionserver.HRegionServer$RegionCloserThread)
  753. at java.lang.Thread.join(Thread.java:1196)
  754. at org.apache.hadoop.hbase.regionserver.HRegionServer.closeUserRegions(HRegionServer.java:1693)
  755. at org.apache.hadoop.hbase.regionserver.HRegionServer$Worker.run(HRegionServer.java:1446)
  756. at java.lang.Thread.run(Thread.java:619)
  757.  
  758. "regionserver/193.0.23.14:60020.compactor" daemon prio=10 tid=0x00002aaeb2580800 nid=0x64a8 waiting on condition [0x00000000433a9000]
  759. java.lang.Thread.State: TIMED_WAITING (parking)
  760. at sun.misc.Unsafe.park(Native Method)
  761. - parking to wait for <0x00002aaac03c0110> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  762. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
  763. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
  764. at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:395)
  765. at org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:85)
  766.  
  767. "regionserver/193.0.23.14:60020.cacheFlusher" daemon prio=10 tid=0x00002aaeb2173000 nid=0x64a7 waiting on condition [0x0000000040f88000]
  768. java.lang.Thread.State: TIMED_WAITING (parking)
  769. at sun.misc.Unsafe.park(Native Method)
  770. - parking to wait for <0x00002aaac03e8790> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  771. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
  772. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
  773. at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:395)
  774. at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:145)
  775.  
  776. "regionserver/193.0.23.14:60020.logFlusher" daemon prio=10 tid=0x00002aaeb2c49000 nid=0x64a6 in Object.wait() [0x0000000040e87000]
  777. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  778. at java.lang.Object.wait(Native Method)
  779. - waiting on <0x00002aaac028d5e8> (a java.lang.Object)
  780. at org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:89)
  781. - locked <0x00002aaac028d5e8> (a java.lang.Object)
  782. at org.apache.hadoop.hbase.Chore.run(Chore.java:76)
  783.  
  784. "regionserver/193.0.23.14:60020.logRoller" daemon prio=10 tid=0x00002aaeb2ae7800 nid=0x64a5 in Object.wait() [0x0000000040d86000]
  785. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  786. at java.lang.Object.wait(Native Method)
  787. - waiting on <0x00002aaac02a53c0> (a java.util.concurrent.atomic.AtomicBoolean)
  788. at org.apache.hadoop.hbase.regionserver.LogRoller.run(LogRoller.java:65)
  789. - locked <0x00002aaac02a53c0> (a java.util.concurrent.atomic.AtomicBoolean)
  790.  
  791. "LeaseChecker" daemon prio=10 tid=0x00002aaeb21e9800 nid=0x64a4 waiting on condition [0x00000000432a8000]
  792. java.lang.Thread.State: TIMED_WAITING (sleeping)
  793. at java.lang.Thread.sleep(Native Method)
  794. at org.apache.hadoop.hdfs.DFSClient$LeaseChecker.run(DFSClient.java:1077)
  795. at java.lang.Thread.run(Thread.java:619)
  796.  
  797. "DataStreamer for file /hbase/.logs/inrdb-worker2.ripe.net,60020,1273835052627/hlog.dat.1273835052947 block blk_7830202764694154483_625018" daemon prio=10 tid=0x00002aaeb21cb800 nid=0x64a3 in Object.wait() [0x000000004142c000]
  798. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  799. at java.lang.Object.wait(Native Method)
  800. - waiting on <0x00002aaac050a748> (a java.util.LinkedList)
  801. at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2271)
  802. - locked <0x00002aaac050a748> (a java.util.LinkedList)
  803.  
  804. "IPC Client (47) connection to /193.0.1.49:60000 from an unknown user" daemon prio=10 tid=0x00002aaeb4242000 nid=0x6495 in Object.wait() [0x0000000040c85000]
  805. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  806. at java.lang.Object.wait(Native Method)
  807. - waiting on <0x00002aaac031c410> (a org.apache.hadoop.hbase.ipc.HBaseClient$Connection)
  808. at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.waitForWork(HBaseClient.java:406)
  809. - locked <0x00002aaac031c410> (a org.apache.hadoop.hbase.ipc.HBaseClient$Connection)
  810. at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:449)
  811.  
  812. "DestroyJavaVM" prio=10 tid=0x00002aaeb41e3800 nid=0x6476 waiting on condition [0x0000000000000000]
  813. java.lang.Thread.State: RUNNABLE
  814.  
  815. "regionserver/193.0.23.14:60020" prio=10 tid=0x00002aaeb42bc800 nid=0x6494 waiting on condition [0x00000000430a6000]
  816. java.lang.Thread.State: TIMED_WAITING (parking)
  817. at sun.misc.Unsafe.park(Native Method)
  818. - parking to wait for <0x00002aaac028d600> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  819. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
  820. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
  821. at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:395)
  822. at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:596)
  823. at java.lang.Thread.run(Thread.java:619)
  824.  
  825. "main-EventThread" daemon prio=10 tid=0x00002aaeb42bb000 nid=0x6493 waiting on condition [0x0000000042fa5000]
  826. java.lang.Thread.State: WAITING (parking)
  827. at sun.misc.Unsafe.park(Native Method)
  828. - parking to wait for <0x00002aaac02a5870> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  829. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  830. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  831. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  832. at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:414)
  833.  
  834. "main-SendThread" daemon prio=10 tid=0x00002aaeb4286000 nid=0x6492 runnable [0x0000000042ea4000]
  835. java.lang.Thread.State: RUNNABLE
  836. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  837. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
  838. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  839. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  840. - locked <0x00002aaac02dba10> (a sun.nio.ch.Util$1)
  841. - locked <0x00002aaac02dba28> (a java.util.Collections$UnmodifiableSet)
  842. - locked <0x00002aaac02afc80> (a sun.nio.ch.EPollSelectorImpl)
  843. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  844. at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:921)
  845.  
  846. "Low Memory Detector" daemon prio=10 tid=0x00002aaeb4007000 nid=0x6490 runnable [0x0000000000000000]
  847. java.lang.Thread.State: RUNNABLE
  848.  
  849. "CompilerThread1" daemon prio=10 tid=0x00002aaeb4004800 nid=0x648f waiting on condition [0x0000000000000000]
  850. java.lang.Thread.State: RUNNABLE
  851.  
  852. "CompilerThread0" daemon prio=10 tid=0x00002aaeb4002800 nid=0x648e waiting on condition [0x0000000000000000]
  853. java.lang.Thread.State: RUNNABLE
  854.  
  855. "Signal Dispatcher" daemon prio=10 tid=0x00002aaeb1cb8000 nid=0x648d runnable [0x0000000000000000]
  856. java.lang.Thread.State: RUNNABLE
  857.  
  858. "Surrogate Locker Thread (CMS)" daemon prio=10 tid=0x00002aaeb1cb6000 nid=0x648c waiting on condition [0x0000000000000000]
  859. java.lang.Thread.State: RUNNABLE
  860.  
  861. "Finalizer" daemon prio=10 tid=0x00002aaeb1c93800 nid=0x648b in Object.wait() [0x000000004289e000]
  862. java.lang.Thread.State: WAITING (on object monitor)
  863. at java.lang.Object.wait(Native Method)
  864. - waiting on <0x00002aaac029a838> (a java.lang.ref.ReferenceQueue$Lock)
  865. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
  866. - locked <0x00002aaac029a838> (a java.lang.ref.ReferenceQueue$Lock)
  867. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
  868. at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
  869.  
  870. "Reference Handler" daemon prio=10 tid=0x00002aaeb1c91800 nid=0x648a in Object.wait() [0x000000004279d000]
  871. java.lang.Thread.State: WAITING (on object monitor)
  872. at java.lang.Object.wait(Native Method)
  873. - waiting on <0x00002aaac0284c08> (a java.lang.ref.Reference$Lock)
  874. at java.lang.Object.wait(Object.java:485)
  875. at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
  876. - locked <0x00002aaac0284c08> (a java.lang.ref.Reference$Lock)
  877.  
  878. "VM Thread" prio=10 tid=0x00002aaeb1c8a800 nid=0x6489 runnable
  879.  
  880. "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x000000004233c800 nid=0x6477 runnable
  881.  
  882. "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x000000004233e800 nid=0x6478 runnable
  883.  
  884. "Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x0000000042340800 nid=0x6479 runnable
  885.  
  886. "Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x0000000042342000 nid=0x647a runnable
  887.  
  888. "Gang worker#4 (Parallel GC Threads)" prio=10 tid=0x0000000042344000 nid=0x647b runnable
  889.  
  890. "Gang worker#5 (Parallel GC Threads)" prio=10 tid=0x0000000042346000 nid=0x647c runnable
  891.  
  892. "Gang worker#6 (Parallel GC Threads)" prio=10 tid=0x0000000042347800 nid=0x647d runnable
  893.  
  894. "Gang worker#7 (Parallel GC Threads)" prio=10 tid=0x0000000042349800 nid=0x647e runnable
  895.  
  896. "Gang worker#8 (Parallel GC Threads)" prio=10 tid=0x000000004234b800 nid=0x647f runnable
  897.  
  898. "Gang worker#9 (Parallel GC Threads)" prio=10 tid=0x000000004234d000 nid=0x6480 runnable
  899.  
  900. "Gang worker#10 (Parallel GC Threads)" prio=10 tid=0x000000004234f000 nid=0x6481 runnable
  901.  
  902. "Gang worker#11 (Parallel GC Threads)" prio=10 tid=0x0000000042351000 nid=0x6482 runnable
  903.  
  904. "Gang worker#12 (Parallel GC Threads)" prio=10 tid=0x0000000042352800 nid=0x6483 runnable
  905.  
  906. "Concurrent Mark-Sweep GC Thread" prio=10 tid=0x0000000042498800 nid=0x6488 runnable
  907. "Gang worker#0 (Parallel CMS Threads)" prio=10 tid=0x0000000042491000 nid=0x6484 runnable
  908.  
  909. "Gang worker#1 (Parallel CMS Threads)" prio=10 tid=0x0000000042492800 nid=0x6485 runnable
  910.  
  911. "Gang worker#2 (Parallel CMS Threads)" prio=10 tid=0x0000000042494800 nid=0x6486 runnable
  912.  
  913. "Gang worker#3 (Parallel CMS Threads)" prio=10 tid=0x0000000042496800 nid=0x6487 runnable
  914.  
  915. "VM Periodic Task Thread" prio=10 tid=0x00002aaeb4009800 nid=0x6491 waiting on condition
  916.  
  917. JNI global references: 692
  918.  
  919.  
  920.  
  921.  
  922.  
  923.  
  924.  
  925.  
  926.  
  927.  
  928.  
  929.  
  930.  
  931.  
  932.  
  933.  
  934.  
  935.  
  936.  
  937.  
  938.  
  939.  
  940.  
  941.  
  942.  
  943.  
  944.  
  945.  
  946. After shutdown command:
  947. 2010-05-14 13:53:07
  948. Full thread dump Java HotSpot(TM) 64-Bit Server VM (14.3-b01 mixed mode):
  949.  
  950. "Thread-17" prio=10 tid=0x00002aaeb22a0000 nid=0x7729 in Object.wait() [0x00000000436ac000]
  951. java.lang.Thread.State: WAITING (on object monitor)
  952. at java.lang.Object.wait(Native Method)
  953. - waiting on <0x00002aaac044fbe0> (a java.lang.Thread)
  954. at java.lang.Thread.join(Thread.java:1143)
  955. - locked <0x00002aaac044fbe0> (a java.lang.Thread)
  956. at org.apache.hadoop.hbase.util.Threads.shutdown(Threads.java:77)
  957. at org.apache.hadoop.hbase.util.Threads.shutdown(Threads.java:66)
  958. at org.apache.hadoop.hbase.regionserver.HRegionServer$ShutdownThread.run(HRegionServer.java:955)
  959.  
  960. "SIGTERM handler" daemon prio=10 tid=0x00002aaeb2173000 nid=0x7727 in Object.wait() [0x0000000040f88000]
  961. java.lang.Thread.State: WAITING (on object monitor)
  962. at java.lang.Object.wait(Native Method)
  963. - waiting on <0x00002aaac044ae48> (a org.apache.hadoop.hbase.regionserver.HRegionServer$ShutdownThread)
  964. at java.lang.Thread.join(Thread.java:1143)
  965. - locked <0x00002aaac044ae48> (a org.apache.hadoop.hbase.regionserver.HRegionServer$ShutdownThread)
  966. at java.lang.Thread.join(Thread.java:1196)
  967. at java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:79)
  968. at java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:24)
  969. at java.lang.Shutdown.runHooks(Shutdown.java:79)
  970. at java.lang.Shutdown.sequence(Shutdown.java:123)
  971. at java.lang.Shutdown.exit(Shutdown.java:168)
  972. - locked <0x00002aae963ddec8> (a java.lang.Class for java.lang.Shutdown)
  973. at java.lang.Terminator$1.handle(Terminator.java:35)
  974. at sun.misc.Signal$1.run(Signal.java:195)
  975. at java.lang.Thread.run(Thread.java:619)
  976.  
  977. "Attach Listener" daemon prio=10 tid=0x00002aaeb2ac8000 nid=0x766d waiting on condition [0x0000000000000000]
  978. java.lang.Thread.State: RUNNABLE
  979.  
  980. "regionserver/193.0.23.14:60020.worker.regionCloser.inrdb_reg_line,,1273476414312" daemon prio=10 tid=0x00002aaeb43f1800 nid=0x758a waiting for monitor entry [0x00000000487fd000]
  981. java.lang.Thread.State: BLOCKED (on object monitor)
  982. at org.apache.hadoop.hbase.regionserver.StoreScanner.updateReaders(StoreScanner.java:251)
  983. - waiting to lock <0x00002aaac855b5e8> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  984. at org.apache.hadoop.hbase.regionserver.Store.notifyChangedReadersObservers(Store.java:621)
  985. at org.apache.hadoop.hbase.regionserver.Store.updateStorefiles(Store.java:607)
  986. at org.apache.hadoop.hbase.regionserver.Store.access$200(Store.java:88)
  987. at org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.commit(Store.java:1609)
  988. at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1028)
  989. at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:521)
  990. - locked <0x00002aaac6b288f8> (a java.lang.Object)
  991. at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:456)
  992. at org.apache.hadoop.hbase.regionserver.HRegionServer$RegionCloserThread.run(HRegionServer.java:1654)
  993.  
  994. "ResponseProcessor for block blk_7830202764694154483_625018" daemon prio=10 tid=0x00002aaeb2bb4800 nid=0x6902 runnable [0x00000000431a7000]
  995. java.lang.Thread.State: RUNNABLE
  996. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  997. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
  998. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  999. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  1000. - locked <0x00002aaac8ddb6e8> (a sun.nio.ch.Util$1)
  1001. - locked <0x00002aaac8ddb700> (a java.util.Collections$UnmodifiableSet)
  1002. - locked <0x00002aaac090dd58> (a sun.nio.ch.EPollSelectorImpl)
  1003. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  1004. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:332)
  1005. at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
  1006. at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
  1007. at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
  1008. at java.io.DataInputStream.readFully(DataInputStream.java:178)
  1009. at java.io.DataInputStream.readLong(DataInputStream.java:399)
  1010. at org.apache.hadoop.hdfs.protocol.DataTransferProtocol$PipelineAck.readFields(DataTransferProtocol.java:119)
  1011. at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$ResponseProcessor.run(DFSClient.java:2424)
  1012.  
  1013. "LruBlockCache.EvictionThread" daemon prio=10 tid=0x00002aaeb4444000 nid=0x6531 in Object.wait() [0x0000000046ee4000]
  1014. java.lang.Thread.State: WAITING (on object monitor)
  1015. at java.lang.Object.wait(Native Method)
  1016. - waiting on <0x00002aaac0503f20> (a org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread)
  1017. at java.lang.Object.wait(Object.java:485)
  1018. at org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread.run(LruBlockCache.java:512)
  1019. - locked <0x00002aaac0503f20> (a org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread)
  1020.  
  1021. "IPC Server handler 48 on 60020" daemon prio=10 tid=0x00002aaeb443d800 nid=0x652f runnable [0x0000000046ce2000]
  1022. java.lang.Thread.State: RUNNABLE
  1023. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1024. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1025. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1026. - locked <0x00002aaaaeb278a0> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1027. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1028. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1029. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1030. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1031. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1032. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1033. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1034. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1035. at java.lang.reflect.Method.invoke(Method.java:597)
  1036. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1037. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1038.  
  1039. "IPC Server handler 46 on 60020" daemon prio=10 tid=0x00002aaeb443a000 nid=0x652d runnable [0x0000000046ae0000]
  1040. java.lang.Thread.State: RUNNABLE
  1041. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1042. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1043. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1044. - locked <0x00002aaaaead96d8> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1045. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1046. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1047. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1048. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1049. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1050. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1051. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1052. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1053. at java.lang.reflect.Method.invoke(Method.java:597)
  1054. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1055. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1056.  
  1057. "IPC Server handler 44 on 60020" daemon prio=10 tid=0x00002aaeb4361000 nid=0x652b runnable [0x00000000468de000]
  1058. java.lang.Thread.State: RUNNABLE
  1059. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1060. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1061. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1062. - locked <0x00002aaaae509c78> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1063. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1064. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1065. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1066. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1067. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1068. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1069. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1070. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1071. at java.lang.reflect.Method.invoke(Method.java:597)
  1072. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1073. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1074.  
  1075. "IPC Server handler 39 on 60020" daemon prio=10 tid=0x00002aaeb4357800 nid=0x6526 runnable [0x00000000463d9000]
  1076. java.lang.Thread.State: RUNNABLE
  1077. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1078. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1079. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1080. - locked <0x00002aaad345bbc8> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1081. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1082. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1083. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1084. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1085. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1086. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1087. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1088. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1089. at java.lang.reflect.Method.invoke(Method.java:597)
  1090. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1091. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1092.  
  1093. "IPC Server handler 37 on 60020" daemon prio=10 tid=0x00002aaeb4353800 nid=0x6524 runnable [0x00000000461d7000]
  1094. java.lang.Thread.State: RUNNABLE
  1095. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1096. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1097. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1098. - locked <0x00002aaac6a5b5e0> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1099. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1100. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1101. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1102. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1103. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1104. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1105. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1106. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1107. at java.lang.reflect.Method.invoke(Method.java:597)
  1108. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1109. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1110.  
  1111. "IPC Server handler 35 on 60020" daemon prio=10 tid=0x00002aaeb434f800 nid=0x6522 runnable [0x0000000045fd5000]
  1112. java.lang.Thread.State: RUNNABLE
  1113. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1114. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1115. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1116. - locked <0x00002aaaaea40fa0> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1117. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1118. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1119. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1120. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1121. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1122. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1123. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1124. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1125. at java.lang.reflect.Method.invoke(Method.java:597)
  1126. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1127. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1128.  
  1129. "IPC Server handler 34 on 60020" daemon prio=10 tid=0x00002aaeb434d800 nid=0x6521 runnable [0x0000000045ed4000]
  1130. java.lang.Thread.State: RUNNABLE
  1131. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1132. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1133. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1134. - locked <0x00002aaac201ee00> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1135. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1136. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1137. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1138. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1139. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1140. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1141. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1142. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1143. at java.lang.reflect.Method.invoke(Method.java:597)
  1144. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1145. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1146.  
  1147. "IPC Server handler 32 on 60020" daemon prio=10 tid=0x00002aaeb4349800 nid=0x651f runnable [0x0000000045cd1000]
  1148. java.lang.Thread.State: RUNNABLE
  1149. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1150. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1151. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1152. - locked <0x00002aaac08ac840> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1153. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1154. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1155. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1156. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1157. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1158. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1159. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1160. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1161. at java.lang.reflect.Method.invoke(Method.java:597)
  1162. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1163. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1164.  
  1165. "IPC Server handler 31 on 60020" daemon prio=10 tid=0x00002aaeb4347800 nid=0x651e runnable [0x0000000045bd1000]
  1166. java.lang.Thread.State: RUNNABLE
  1167. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1168. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1169. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1170. - locked <0x00002aaad0c481b8> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1171. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1172. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1173. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1174. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1175. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1176. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1177. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1178. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1179. at java.lang.reflect.Method.invoke(Method.java:597)
  1180. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1181. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1182.  
  1183. "IPC Server handler 30 on 60020" daemon prio=10 tid=0x00002aaeb4346000 nid=0x651d runnable [0x0000000045ad0000]
  1184. java.lang.Thread.State: RUNNABLE
  1185. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1186. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1187. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1188. - locked <0x00002aaaae4a45e8> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1189. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1190. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1191. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1192. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1193. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1194. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1195. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1196. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1197. at java.lang.reflect.Method.invoke(Method.java:597)
  1198. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1199. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1200.  
  1201. "IPC Server handler 25 on 60020" daemon prio=10 tid=0x00002aaeb4336800 nid=0x6518 runnable [0x00000000455cb000]
  1202. java.lang.Thread.State: RUNNABLE
  1203. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1204. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1205. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1206. - locked <0x00002aaad0c45848> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1207. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1208. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1209. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1210. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1211. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1212. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1213. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1214. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1215. at java.lang.reflect.Method.invoke(Method.java:597)
  1216. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1217. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1218.  
  1219. "IPC Server handler 24 on 60020" daemon prio=10 tid=0x00002aaeb4334800 nid=0x6517 runnable [0x00000000454c9000]
  1220. java.lang.Thread.State: RUNNABLE
  1221. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1222. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1223. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1224. - locked <0x00002aaac83e2480> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1225. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1226. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1227. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1228. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1229. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1230. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1231. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1232. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1233. at java.lang.reflect.Method.invoke(Method.java:597)
  1234. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1235. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1236.  
  1237. "IPC Server handler 20 on 60020" daemon prio=10 tid=0x00002aaeb432d800 nid=0x6513 runnable [0x00000000450c6000]
  1238. java.lang.Thread.State: RUNNABLE
  1239. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1240. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1241. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1242. - locked <0x00002aaad0b60e18> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1243. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1244. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1245. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1246. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1247. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1248. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1249. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1250. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1251. at java.lang.reflect.Method.invoke(Method.java:597)
  1252. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1253. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1254.  
  1255. "IPC Server handler 18 on 60020" daemon prio=10 tid=0x00002aaeb4309000 nid=0x6511 runnable [0x0000000044ec4000]
  1256. java.lang.Thread.State: RUNNABLE
  1257. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1258. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1259. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1260. - locked <0x00002aaad345bca8> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1261. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1262. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1263. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1264. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1265. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1266. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1267. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1268. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1269. at java.lang.reflect.Method.invoke(Method.java:597)
  1270. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1271. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1272.  
  1273. "IPC Server handler 11 on 60020" daemon prio=10 tid=0x00002aaeb42fb800 nid=0x650a runnable [0x00000000447bd000]
  1274. java.lang.Thread.State: RUNNABLE
  1275. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1276. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1277. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1278. - locked <0x00002aaad7d0f7b0> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1279. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1280. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1281. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1282. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1283. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1284. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1285. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1286. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1287. at java.lang.reflect.Method.invoke(Method.java:597)
  1288. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1289. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1290.  
  1291. "IPC Server handler 9 on 60020" daemon prio=10 tid=0x00002aaeb42f7800 nid=0x6508 runnable [0x00000000445bb000]
  1292. java.lang.Thread.State: RUNNABLE
  1293. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1294. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1295. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1296. - locked <0x00002aaac855b5e8> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1297. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1298. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1299. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1300. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1301. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1302. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1303. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1304. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1305. at java.lang.reflect.Method.invoke(Method.java:597)
  1306. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1307. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1308.  
  1309. "IPC Server handler 8 on 60020" daemon prio=10 tid=0x00002aaeb4160000 nid=0x6507 runnable [0x00000000444ba000]
  1310. java.lang.Thread.State: RUNNABLE
  1311. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1312. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1313. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1314. - locked <0x00002aaad0c47b98> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1315. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1316. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1317. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1318. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1319. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1320. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1321. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1322. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1323. at java.lang.reflect.Method.invoke(Method.java:597)
  1324. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1325. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1326.  
  1327. "IPC Server handler 1 on 60020" daemon prio=10 tid=0x00002aaeb4153000 nid=0x6500 runnable [0x0000000043db3000]
  1328. java.lang.Thread.State: RUNNABLE
  1329. at org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.checkColumn(ExplicitColumnTracker.java:128)
  1330. at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:165)
  1331. at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:176)
  1332. - locked <0x00002aaaaed7be80> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
  1333. at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
  1334. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1923)
  1335. at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1887)
  1336. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2507)
  1337. at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2493)
  1338. at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1742)
  1339. at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
  1340. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  1341. at java.lang.reflect.Method.invoke(Method.java:597)
  1342. at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
  1343. at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
  1344.  
  1345. "regionserver/193.0.23.14:60020.worker" daemon prio=10 tid=0x00002aaeb260d000 nid=0x64a9 in Object.wait() [0x00000000434aa000]
  1346. java.lang.Thread.State: WAITING (on object monitor)
  1347. at java.lang.Object.wait(Native Method)
  1348. - waiting on <0x00002aaab089d758> (a org.apache.hadoop.hbase.regionserver.HRegionServer$RegionCloserThread)
  1349. at java.lang.Thread.join(Thread.java:1143)
  1350. - locked <0x00002aaab089d758> (a org.apache.hadoop.hbase.regionserver.HRegionServer$RegionCloserThread)
  1351. at java.lang.Thread.join(Thread.java:1196)
  1352. at org.apache.hadoop.hbase.regionserver.HRegionServer.closeUserRegions(HRegionServer.java:1693)
  1353. at org.apache.hadoop.hbase.regionserver.HRegionServer$Worker.run(HRegionServer.java:1446)
  1354. at java.lang.Thread.run(Thread.java:619)
  1355.  
  1356. "LeaseChecker" daemon prio=10 tid=0x00002aaeb21e9800 nid=0x64a4 waiting on condition [0x00000000432a8000]
  1357. java.lang.Thread.State: TIMED_WAITING (sleeping)
  1358. at java.lang.Thread.sleep(Native Method)
  1359. at org.apache.hadoop.hdfs.DFSClient$LeaseChecker.run(DFSClient.java:1077)
  1360. at java.lang.Thread.run(Thread.java:619)
  1361.  
  1362. "DataStreamer for file /hbase/.logs/inrdb-worker2.ripe.net,60020,1273835052627/hlog.dat.1273835052947 block blk_7830202764694154483_625018" daemon prio=10 tid=0x00002aaeb21cb800 nid=0x64a3 in Object.wait() [0x000000004142c000]
  1363. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  1364. at java.lang.Object.wait(Native Method)
  1365. - waiting on <0x00002aaac050a748> (a java.util.LinkedList)
  1366. at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2271)
  1367. - locked <0x00002aaac050a748> (a java.util.LinkedList)
  1368.  
  1369. "DestroyJavaVM" prio=10 tid=0x00002aaeb41e3800 nid=0x6476 waiting on condition [0x0000000000000000]
  1370. java.lang.Thread.State: RUNNABLE
  1371.  
  1372. "regionserver/193.0.23.14:60020" prio=10 tid=0x00002aaeb42bc800 nid=0x6494 waiting on condition [0x00000000430a6000]
  1373. java.lang.Thread.State: WAITING (parking)
  1374. at sun.misc.Unsafe.park(Native Method)
  1375. - parking to wait for <0x00002aaac075ba20> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
  1376. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  1377. at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747)
  1378. at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:778)
  1379. at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1114)
  1380. at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
  1381. at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262)
  1382. at org.apache.hadoop.hbase.regionserver.HLog.close(HLog.java:542)
  1383. at org.apache.hadoop.hbase.regionserver.HLog.closeAndDelete(HLog.java:532)
  1384. at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:652)
  1385. at java.lang.Thread.run(Thread.java:619)
  1386.  
  1387. "main-EventThread" daemon prio=10 tid=0x00002aaeb42bb000 nid=0x6493 waiting on condition [0x0000000042fa5000]
  1388. java.lang.Thread.State: WAITING (parking)
  1389. at sun.misc.Unsafe.park(Native Method)
  1390. - parking to wait for <0x00002aaac02a5870> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1391. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  1392. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  1393. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  1394. at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:414)
  1395.  
  1396. "main-SendThread" daemon prio=10 tid=0x00002aaeb4286000 nid=0x6492 runnable [0x0000000042ea4000]
  1397. java.lang.Thread.State: RUNNABLE
  1398. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1399. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
  1400. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  1401. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  1402. - locked <0x00002aaac02dba10> (a sun.nio.ch.Util$1)
  1403. - locked <0x00002aaac02dba28> (a java.util.Collections$UnmodifiableSet)
  1404. - locked <0x00002aaac02afc80> (a sun.nio.ch.EPollSelectorImpl)
  1405. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  1406. at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:921)
  1407.  
  1408. "Low Memory Detector" daemon prio=10 tid=0x00002aaeb4007000 nid=0x6490 runnable [0x0000000000000000]
  1409. java.lang.Thread.State: RUNNABLE
  1410.  
  1411. "CompilerThread1" daemon prio=10 tid=0x00002aaeb4004800 nid=0x648f waiting on condition [0x0000000000000000]
  1412. java.lang.Thread.State: RUNNABLE
  1413.  
  1414. "CompilerThread0" daemon prio=10 tid=0x00002aaeb4002800 nid=0x648e waiting on condition [0x0000000000000000]
  1415. java.lang.Thread.State: RUNNABLE
  1416.  
  1417. "Signal Dispatcher" daemon prio=10 tid=0x00002aaeb1cb8000 nid=0x648d runnable [0x0000000000000000]
  1418. java.lang.Thread.State: RUNNABLE
  1419.  
  1420. "Surrogate Locker Thread (CMS)" daemon prio=10 tid=0x00002aaeb1cb6000 nid=0x648c waiting on condition [0x0000000000000000]
  1421. java.lang.Thread.State: RUNNABLE
  1422.  
  1423. "Finalizer" daemon prio=10 tid=0x00002aaeb1c93800 nid=0x648b in Object.wait() [0x000000004289e000]
  1424. java.lang.Thread.State: WAITING (on object monitor)
  1425. at java.lang.Object.wait(Native Method)
  1426. - waiting on <0x00002aaac029a838> (a java.lang.ref.ReferenceQueue$Lock)
  1427. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
  1428. - locked <0x00002aaac029a838> (a java.lang.ref.ReferenceQueue$Lock)
  1429. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
  1430. at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
  1431.  
  1432. "Reference Handler" daemon prio=10 tid=0x00002aaeb1c91800 nid=0x648a in Object.wait() [0x000000004279d000]
  1433. java.lang.Thread.State: WAITING (on object monitor)
  1434. at java.lang.Object.wait(Native Method)
  1435. - waiting on <0x00002aaac0284c08> (a java.lang.ref.Reference$Lock)
  1436. at java.lang.Object.wait(Object.java:485)
  1437. at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
  1438. - locked <0x00002aaac0284c08> (a java.lang.ref.Reference$Lock)
  1439.  
  1440. "VM Thread" prio=10 tid=0x00002aaeb1c8a800 nid=0x6489 runnable
  1441.  
  1442. "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x000000004233c800 nid=0x6477 runnable
  1443.  
  1444. "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x000000004233e800 nid=0x6478 runnable
  1445.  
  1446. "Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x0000000042340800 nid=0x6479 runnable
  1447.  
  1448. "Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x0000000042342000 nid=0x647a runnable
  1449.  
  1450. "Gang worker#4 (Parallel GC Threads)" prio=10 tid=0x0000000042344000 nid=0x647b runnable
  1451.  
  1452. "Gang worker#5 (Parallel GC Threads)" prio=10 tid=0x0000000042346000 nid=0x647c runnable
  1453.  
  1454. "Gang worker#6 (Parallel GC Threads)" prio=10 tid=0x0000000042347800 nid=0x647d runnable
  1455.  
  1456. "Gang worker#7 (Parallel GC Threads)" prio=10 tid=0x0000000042349800 nid=0x647e runnable
  1457.  
  1458. "Gang worker#8 (Parallel GC Threads)" prio=10 tid=0x000000004234b800 nid=0x647f runnable
  1459.  
  1460. "Gang worker#9 (Parallel GC Threads)" prio=10 tid=0x000000004234d000 nid=0x6480 runnable
  1461.  
  1462. "Gang worker#10 (Parallel GC Threads)" prio=10 tid=0x000000004234f000 nid=0x6481 runnable
  1463.  
  1464. "Gang worker#11 (Parallel GC Threads)" prio=10 tid=0x0000000042351000 nid=0x6482 runnable
  1465.  
  1466. "Gang worker#12 (Parallel GC Threads)" prio=10 tid=0x0000000042352800 nid=0x6483 runnable
  1467.  
  1468. "Concurrent Mark-Sweep GC Thread" prio=10 tid=0x0000000042498800 nid=0x6488 runnable
  1469. "Gang worker#0 (Parallel CMS Threads)" prio=10 tid=0x0000000042491000 nid=0x6484 runnable
  1470.  
  1471. "Gang worker#1 (Parallel CMS Threads)" prio=10 tid=0x0000000042492800 nid=0x6485 runnable
  1472.  
  1473. "Gang worker#2 (Parallel CMS Threads)" prio=10 tid=0x0000000042494800 nid=0x6486 runnable
  1474.  
  1475. "Gang worker#3 (Parallel CMS Threads)" prio=10 tid=0x0000000042496800 nid=0x6487 runnable
  1476.  
  1477. "VM Periodic Task Thread" prio=10 tid=0x00002aaeb4009800 nid=0x6491 waiting on condition
  1478.  
  1479. JNI global references: 693
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!