API tools faq
paste
Advertisement
Guest User

hbase hanging after cdh 5.3 upgrade

a guest
Jan 6th, 2015
263
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Java 26.27 KB | None | 0 0
raw download clone embed print report
  1. 2015-01-06 19:07:15
  2. Full thread dump Java HotSpot(TM) 64-Bit Server VM (20.45-b01 mixed mode):
  3.  
  4. "Attach Listener" daemon prio=10 tid=0x00007fbbfc06d000 nid=0x862f waiting on condition [0x0000000000000000]
  5.    java.lang.Thread.State: RUNNABLE
  6.  
  7. "IPC Client (1940813045) connection to node03.lab/10.7.202.83:60020 from antony.mayi" daemon prio=10 tid=0x00007fbc04acc000 nid=0x85ed runnable [0x00007fbbeb529000]
  8.    java.lang.Thread.State: RUNNABLE
  9.     at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  10.     at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  11.     at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  12.     at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  13.     - locked <0x00000000bc6fa7c8> (a sun.nio.ch.Util$2)
  14.     - locked <0x00000000bc6fa7b8> (a java.util.Collections$UnmodifiableSet)
  15.     - locked <0x00000000bc6fa5b0> (a sun.nio.ch.EPollSelectorImpl)
  16.     at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  17.     at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)
  18.     at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
  19.     at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
  20.     at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
  21.     at java.io.FilterInputStream.read(FilterInputStream.java:116)
  22.     at java.io.FilterInputStream.read(FilterInputStream.java:116)
  23.     at org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
  24.     at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  25.     at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
  26.     - locked <0x00000000bc7097b8> (a java.io.BufferedInputStream)
  27.     at java.io.DataInputStream.readInt(DataInputStream.java:370)
  28.     at org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
  29.     at org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)
  30.  
  31. "Thread-3" daemon prio=10 tid=0x00007fbc04ace800 nid=0x85e9 runnable [0x00007fbbeb62a000]
  32.    java.lang.Thread.State: RUNNABLE
  33.     at org.apache.hadoop.net.unix.DomainSocketWatcher.doPoll0(Native Method)
  34.     at org.apache.hadoop.net.unix.DomainSocketWatcher.access$900(DomainSocketWatcher.java:52)
  35.     at org.apache.hadoop.net.unix.DomainSocketWatcher$1.run(DomainSocketWatcher.java:474)
  36.     at java.lang.Thread.run(Thread.java:662)
  37.  
  38. "main-EventThread" daemon prio=10 tid=0x00007fbc05cbd000 nid=0x85e8 waiting on condition [0x00007fbbec089000]
  39.    java.lang.Thread.State: WAITING (parking)
  40.     at sun.misc.Unsafe.park(Native Method)
  41.     - parking to wait for  <0x00000000bd8bb2a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  42.     at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  43.     at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  44.     at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  45.     at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
  46.  
  47. "main-SendThread(node01.lab:2181)" daemon prio=10 tid=0x00007fbc0648a800 nid=0x85e7 runnable [0x00007fbbebc57000]
  48.    java.lang.Thread.State: RUNNABLE
  49.     at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  50.     at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  51.     at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  52.     at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  53.     - locked <0x00000000bd8c6550> (a sun.nio.ch.Util$2)
  54.     - locked <0x00000000bd8c6560> (a java.util.Collections$UnmodifiableSet)
  55.     - locked <0x00000000bd8c6508> (a sun.nio.ch.EPollSelectorImpl)
  56.     at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  57.     at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
  58.     at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
  59.  
  60. "Low Memory Detector" daemon prio=10 tid=0x00007fbc043b0800 nid=0x85c0 runnable [0x0000000000000000]
  61.    java.lang.Thread.State: RUNNABLE
  62.  
  63. "C2 CompilerThread1" daemon prio=10 tid=0x00007fbc043ab800 nid=0x85bf waiting on condition [0x0000000000000000]
  64.    java.lang.Thread.State: RUNNABLE
  65.  
  66. "C2 CompilerThread0" daemon prio=10 tid=0x00007fbc043a9800 nid=0x85be waiting on condition [0x0000000000000000]
  67.    java.lang.Thread.State: RUNNABLE
  68.  
  69. "Signal Dispatcher" daemon prio=10 tid=0x00007fbc043a7000 nid=0x85bd runnable [0x0000000000000000]
  70.    java.lang.Thread.State: RUNNABLE
  71.  
  72. "Surrogate Locker Thread (Concurrent GC)" daemon prio=10 tid=0x00007fbc043a5800 nid=0x85bc waiting on condition [0x0000000000000000]
  73.    java.lang.Thread.State: RUNNABLE
  74.  
  75. "Finalizer" daemon prio=10 tid=0x00007fbc04389000 nid=0x85bb in Object.wait() [0x00007fbbf365d000]
  76.    java.lang.Thread.State: WAITING (on object monitor)
  77.     at java.lang.Object.wait(Native Method)
  78.     - waiting on <0x00000000c4320848> (a java.lang.ref.ReferenceQueue$Lock)
  79.     at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
  80.     - locked <0x00000000c4320848> (a java.lang.ref.ReferenceQueue$Lock)
  81.     at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
  82.     at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:171)
  83.  
  84. "Reference Handler" daemon prio=10 tid=0x00007fbc04382000 nid=0x85ba in Object.wait() [0x00007fbbf375e000]
  85.    java.lang.Thread.State: WAITING (on object monitor)
  86.     at java.lang.Object.wait(Native Method)
  87.     - waiting on <0x00000000c4323d90> (a java.lang.ref.Reference$Lock)
  88.     at java.lang.Object.wait(Object.java:485)
  89.     at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
  90.     - locked <0x00000000c4323d90> (a java.lang.ref.Reference$Lock)
  91.  
  92. "main""VM Thread" prio=10 tid=0x00007fbc0437b000 nid=0x85b9 runnable
  93.  
  94. "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x00007fbc04022000 nid=0x85a1 runnable
  95.  
  96. "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x00007fbc04023800 nid=0x85a2 runnable
  97.  
  98. "Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x00007fbc04025800 nid=0x85a3 runnable
  99.  
  100. "Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x00007fbc04027800 nid=0x85a4 runnable
  101.  
  102. "Gang worker#4 (Parallel GC Threads)" prio=10 tid=0x00007fbc04029000 nid=0x85a5 runnable
  103.  
  104. "Gang worker#5 (Parallel GC Threads)" prio=10 tid=0x00007fbc0402b000 nid=0x85a6 runnable
  105.  
  106. "Gang worker#6 (Parallel GC Threads)" prio=10 tid=0x00007fbc0402d000 nid=0x85a7 runnable
  107.  
  108. "Gang worker#7 (Parallel GC Threads)" prio=10 tid=0x00007fbc0402e800 nid=0x85a8 runnable
  109.  
  110. "Gang worker#8 (Parallel GC Threads)" prio=10 tid=0x00007fbc04030800 nid=0x85a9 runnable
  111.  
  112. "Gang worker#9 (Parallel GC Threads)" prio=10 tid=0x00007fbc04032000 nid=0x85aa runnable
  113.  
  114. "Gang worker#10 (Parallel GC Threads)" prio=10 tid=0x00007fbc04034000 nid=0x85ab runnable
  115.  
  116. "Gang worker#11 (Parallel GC Threads)" prio=10 tid=0x00007fbc04036000 nid=0x85ac runnable
  117.  
  118. "Gang worker#12 (Parallel GC Threads)" prio=10 tid=0x00007fbc04037800 nid=0x85ad runnable
  119.  
  120. "Gang worker#13 (Parallel GC Threads)" prio=10 tid=0x00007fbc04039800 nid=0x85ae runnable
  121.  
  122. "Gang worker#14 (Parallel GC Threads)" prio=10 tid=0x00007fbc0403b800 nid=0x85af runnable
  123.  
  124. "Gang worker#15 (Parallel GC Threads)" prio=10 tid=0x00007fbc0403d000 nid=0x85b0 runnable
  125.  
  126. "Gang worker#16 (Parallel GC Threads)" prio=10 tid=0x00007fbc0403f000 nid=0x85b1 runnable
  127.  
  128. "Gang worker#17 (Parallel GC Threads)" prio=10 tid=0x00007fbc04041000 nid=0x85b2 runnable
  129.  
  130. "Concurrent Mark-Sweep GC Thread" prio=10 tid=0x00007fbc0422c800 nid=0x85b8 runnable
  131. "Gang worker#0 (Parallel CMS Threads)" prio=10 tid=0x00007fbc04222800 nid=0x85b3 runnable
  132.  
  133. "Gang worker#1 (Parallel CMS Threads)" prio=10 tid=0x00007fbc04224800 nid=0x85b4 runnable
  134.  
  135. "Gang worker#2 (Parallel CMS Threads)" prio=10 tid=0x00007fbc04226800 nid=0x85b5 runnable
  136.  
  137. "Gang worker#3 (Parallel CMS Threads)" prio=10 tid=0x00007fbc04228000 nid=0x85b6 runnable
  138.  
  139. "Gang worker#4 (Parallel CMS Threads)" prio=10 tid=0x00007fbc0422a000 nid=0x85b7 runnable
  140.  
  141. "VM Periodic Task Thread" prio=10 tid=0x00007fbc043e5800 nid=0x85c1 waiting on condition
  142.  
  143. JNI global references: 1385
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!