API tools faq
paste
ronbuckley

OCLC CDH4.1.1 Standby Namenode Stacktrace

ronbuckley
Nov 8th, 2012
182
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 27.30 KB | None | 0 0
raw download clone embed print report
  1.  
  2. finddev07.dev.oclc.org:/home/buckleyr/saved.cdh4NamenodeLogs>cat hadoop-findsup-namenode-find64-02.dev.oclc.org.out
  3. 2012-11-08 07:28:50
  4. Full thread dump Java HotSpot(TM) 64-Bit Server VM (10.0-b22 mixed mode):
  5.  
  6. "IPC Client (1551158018) connection to find64-01.dev.oclc.org/192.168.64.1:9008 from findsup" daemon prio=10 tid=0x000000004d8b2800 nid=0x15bf runnable [0x000000004128e000..0x000000004128fa90]
  7. java.lang.Thread.State: RUNNABLE
  8. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  9. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
  10. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  11. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  12. - locked <0x00002aab091bacd8> (a sun.nio.ch.Util$1)
  13. - locked <0x00002aab091bacc0> (a java.util.Collections$UnmodifiableSet)
  14. - locked <0x00002aab091ba960> (a sun.nio.ch.EPollSelectorImpl)
  15. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  16. at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:336)
  17. at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:158)
  18. at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:154)
  19. at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:127)
  20. at java.io.FilterInputStream.read(FilterInputStream.java:116)
  21. at java.io.FilterInputStream.read(FilterInputStream.java:116)
  22. at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:386)
  23. at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  24. at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
  25. - locked <0x00002aab091923e0> (a java.io.BufferedInputStream)
  26. at java.io.FilterInputStream.read(FilterInputStream.java:66)
  27. at com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:276)
  28. at com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:760)
  29. at com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:288)
  30. at com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:752)
  31. at org.apache.hadoop.ipc.protobuf.RpcPayloadHeaderProtos$RpcResponseHeaderProto.parseDelimitedFrom(RpcPayloadHeaderProtos.java:985)
  32. at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:882)
  33. at org.apache.hadoop.ipc.Client$Connection.run(Client.java:813)
  34.  
  35. "84277798@qtp-2063387354-2" prio=10 tid=0x00002aab3c43ac00 nid=0x10f4 in Object.wait() [0x00000000424a4000..0x00000000424a4d10]
  36. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  37. at java.lang.Object.wait(Native Method)
  38. - waiting on <0x00002aab0a630e40> (a org.mortbay.thread.QueuedThreadPool$PoolThread)
  39. at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
  40. - locked <0x00002aab0a630e40> (a org.mortbay.thread.QueuedThreadPool$PoolThread)
  41.  
  42. "Standby State Checkpointer" prio=10 tid=0x00002aab34634400 nid=0x1064 waiting on condition [0x00000000438b8000..0x00000000438b8b10]
  43. java.lang.Thread.State: TIMED_WAITING (sleeping)
  44. at java.lang.Thread.sleep(Native Method)
  45. at org.apache.hadoop.hdfs.server.namenode.ha.StandbyCheckpointer$CheckpointerThread.doWork(StandbyCheckpointer.java:256)
  46. at org.apache.hadoop.hdfs.server.namenode.ha.StandbyCheckpointer$CheckpointerThread.access$300(StandbyCheckpointer.java:210)
  47. at org.apache.hadoop.hdfs.server.namenode.ha.StandbyCheckpointer$CheckpointerThread$1.run(StandbyCheckpointer.java:230)
  48. at org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:452)
  49. at org.apache.hadoop.hdfs.server.namenode.ha.StandbyCheckpointer$CheckpointerThread.run(StandbyCheckpointer.java:226)
  50.  
  51. "Edit log tailer" prio=10 tid=0x00002aab347fd400 nid=0x1063 in Object.wait() [0x00000000437b7000..0x00000000437b7c90]
  52. java.lang.Thread.State: WAITING (on object monitor)
  53. at java.lang.Object.wait(Native Method)
  54. - waiting on <0x00002aab0918e890> (a org.apache.hadoop.ipc.Client$Call)
  55. at java.lang.Object.wait(Object.java:485)
  56. at org.apache.hadoop.ipc.Client.call(Client.java:1146)
  57. - locked <0x00002aab0918e890> (a org.apache.hadoop.ipc.Client$Call)
  58. at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
  59. at $Proxy12.rollEditLog(Unknown Source)
  60. at org.apache.hadoop.hdfs.protocolPB.NamenodeProtocolTranslatorPB.rollEditLog(NamenodeProtocolTranslatorPB.java:137)
  61. at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.triggerActiveLogRoll(EditLogTailer.java:268)
  62. at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.access$600(EditLogTailer.java:61)
  63. at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:310)
  64. at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$200(EditLogTailer.java:279)
  65. at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:296)
  66. at org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:452)
  67. at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:292)
  68.  
  69. "IPC Server handler 9 on 9008" daemon prio=10 tid=0x00002aab340a6400 nid=0x1062 waiting on condition [0x00000000436b6000..0x00000000436b6c10]
  70. java.lang.Thread.State: WAITING (parking)
  71. at sun.misc.Unsafe.park(Native Method)
  72. - parking to wait for <0x00002aaab596ffd8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  73. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  74. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  75. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  76. at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1670)
  77.  
  78. "IPC Server handler 8 on 9008" daemon prio=10 tid=0x00002aab3464d400 nid=0x1061 waiting on condition [0x00000000435b5000..0x00000000435b5d90]
  79. java.lang.Thread.State: WAITING (parking)
  80. at sun.misc.Unsafe.park(Native Method)
  81. - parking to wait for <0x00002aaab596ffd8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  82. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  83. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  84. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  85. at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1670)
  86.  
  87. "IPC Server handler 7 on 9008" daemon prio=10 tid=0x00002aab3464c000 nid=0x1060 waiting on condition [0x00000000434b4000..0x00000000434b4d10]
  88. java.lang.Thread.State: WAITING (parking)
  89. at sun.misc.Unsafe.park(Native Method)
  90. - parking to wait for <0x00002aaab596ffd8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  91. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  92. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  93. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  94. at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1670)
  95.  
  96. "IPC Server handler 6 on 9008" daemon prio=10 tid=0x00002aab345fac00 nid=0x105f waiting on condition [0x00000000433b3000..0x00000000433b3a90]
  97. java.lang.Thread.State: WAITING (parking)
  98. at sun.misc.Unsafe.park(Native Method)
  99. - parking to wait for <0x00002aaab596ffd8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  100. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  101. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  102. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  103. at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1670)
  104.  
  105. "IPC Server handler 5 on 9008" daemon prio=10 tid=0x00002aab340bd400 nid=0x105e waiting on condition [0x00000000432b2000..0x00000000432b2a10]
  106. java.lang.Thread.State: WAITING (parking)
  107. at sun.misc.Unsafe.park(Native Method)
  108. - parking to wait for <0x00002aaab596ffd8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  109. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  110. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  111. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  112. at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1670)
  113.  
  114. "IPC Server handler 4 on 9008" daemon prio=10 tid=0x00002aab340bc400 nid=0x105d waiting on condition [0x00000000431b1000..0x00000000431b1b90]
  115. java.lang.Thread.State: WAITING (parking)
  116. at sun.misc.Unsafe.park(Native Method)
  117. - parking to wait for <0x00002aaab596ffd8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  118. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  119. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  120. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  121. at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1670)
  122.  
  123. "IPC Server handler 3 on 9008" daemon prio=10 tid=0x00002aab344fb800 nid=0x105c waiting on condition [0x00000000430b0000..0x00000000430b0b10]
  124. java.lang.Thread.State: WAITING (parking)
  125. at sun.misc.Unsafe.park(Native Method)
  126. - parking to wait for <0x00002aaab596ffd8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  127. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  128. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  129. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  130. at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1670)
  131.  
  132. "IPC Server handler 2 on 9008" daemon prio=10 tid=0x00002aab345ae400 nid=0x105b waiting on condition [0x0000000042faf000..0x0000000042fafc90]
  133. java.lang.Thread.State: WAITING (parking)
  134. at sun.misc.Unsafe.park(Native Method)
  135. - parking to wait for <0x00002aaab596ffd8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  136. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  137. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  138. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  139. at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1670)
  140.  
  141. "IPC Server handler 1 on 9008" daemon prio=10 tid=0x00002aab34786c00 nid=0x105a waiting on condition [0x0000000042eae000..0x0000000042eaec10]
  142. java.lang.Thread.State: WAITING (parking)
  143. at sun.misc.Unsafe.park(Native Method)
  144. - parking to wait for <0x00002aaab596ffd8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  145. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  146. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  147. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  148. at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1670)
  149.  
  150. "IPC Server handler 0 on 9008" daemon prio=10 tid=0x00002aab340c4800 nid=0x1059 waiting on condition [0x0000000042dad000..0x0000000042dadd90]
  151. java.lang.Thread.State: WAITING (parking)
  152. at sun.misc.Unsafe.park(Native Method)
  153. - parking to wait for <0x00002aaab596ffd8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  154. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  155. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  156. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  157. at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1670)
  158.  
  159. "IPC Server listener on 9008" daemon prio=10 tid=0x00002aab34725c00 nid=0x1058 runnable [0x0000000042cac000..0x0000000042cacd10]
  160. java.lang.Thread.State: RUNNABLE
  161. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  162. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
  163. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  164. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  165. - locked <0x00002aaab599fc88> (a sun.nio.ch.Util$1)
  166. - locked <0x00002aaab599fc70> (a java.util.Collections$UnmodifiableSet)
  167. - locked <0x00002aaab599f548> (a sun.nio.ch.EPollSelectorImpl)
  168. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  169. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
  170. at org.apache.hadoop.ipc.Server$Listener.run(Server.java:612)
  171.  
  172. "IPC Server Responder" daemon prio=10 tid=0x00002aab34090000 nid=0x1057 runnable [0x0000000042bab000..0x0000000042baba90]
  173. java.lang.Thread.State: RUNNABLE
  174. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  175. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
  176. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  177. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  178. - locked <0x00002aaab59a06f0> (a sun.nio.ch.Util$1)
  179. - locked <0x00002aaab59a06d8> (a java.util.Collections$UnmodifiableSet)
  180. - locked <0x00002aaab59a0360> (a sun.nio.ch.EPollSelectorImpl)
  181. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  182. at org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:796)
  183. at org.apache.hadoop.ipc.Server$Responder.run(Server.java:779)
  184.  
  185. "Timer-0" daemon prio=10 tid=0x00002aab340e5800 nid=0x1056 in Object.wait() [0x0000000042aaa000..0x0000000042aaaa10]
  186. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  187. at java.lang.Object.wait(Native Method)
  188. - waiting on <0x00002aaab59c6af8> (a java.util.TaskQueue)
  189. at java.util.TimerThread.mainLoop(Timer.java:509)
  190. - locked <0x00002aaab59c6af8> (a java.util.TaskQueue)
  191. at java.util.TimerThread.run(Timer.java:462)
  192.  
  193. "1741604980@qtp-2063387354-1 - Acceptor0 [email protected]:9000" prio=10 tid=0x00002aab340e4c00 nid=0x1055 runnable [0x00000000429a9000..0x00000000429a9b90]
  194. java.lang.Thread.State: RUNNABLE
  195. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  196. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
  197. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  198. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  199. - locked <0x00002aaab594f038> (a sun.nio.ch.Util$1)
  200. - locked <0x00002aaab594f020> (a java.util.Collections$UnmodifiableSet)
  201. - locked <0x00002aaab594e910> (a sun.nio.ch.EPollSelectorImpl)
  202. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  203. at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:511)
  204. at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:193)
  205. at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
  206. at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
  207. at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  208.  
  209. "org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@6da5db4b" daemon prio=10 tid=0x00002aab34049400 nid=0x1053 waiting on condition [0x0000000040eca000..0x0000000040ecac90]
  210. java.lang.Thread.State: TIMED_WAITING (sleeping)
  211. at java.lang.Thread.sleep(Native Method)
  212. at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor.run(BlockManager.java:3025)
  213. at java.lang.Thread.run(Thread.java:619)
  214.  
  215. "org.apache.hadoop.hdfs.server.blockmanagement.HeartbeatManager$Monitor@51e67ac" daemon prio=10 tid=0x00002aab34048c00 nid=0x1052 waiting on condition [0x0000000040522000..0x0000000040522c10]
  216. java.lang.Thread.State: TIMED_WAITING (sleeping)
  217. at java.lang.Thread.sleep(Native Method)
  218. at org.apache.hadoop.hdfs.server.blockmanagement.HeartbeatManager$Monitor.run(HeartbeatManager.java:273)
  219. at java.lang.Thread.run(Thread.java:619)
  220.  
  221. "org.apache.hadoop.hdfs.server.blockmanagement.DecommissionManager$Monitor@52d306dd" daemon prio=10 tid=0x00002aab3475c000 nid=0x1051 waiting on condition [0x00000000428a8000..0x00000000428a8d90]
  222. java.lang.Thread.State: TIMED_WAITING (sleeping)
  223. at java.lang.Thread.sleep(Native Method)
  224. at org.apache.hadoop.hdfs.server.blockmanagement.DecommissionManager$Monitor.run(DecommissionManager.java:76)
  225. at java.lang.Thread.run(Thread.java:619)
  226.  
  227. "org.apache.hadoop.hdfs.server.blockmanagement.PendingReplicationBlocks$PendingReplicationMonitor@c191e4c" daemon prio=10 tid=0x00002aab3475cc00 nid=0x1050 waiting on condition [0x00000000427a7000..0x00000000427a7d10]
  228. java.lang.Thread.State: TIMED_WAITING (sleeping)
  229. at java.lang.Thread.sleep(Native Method)
  230. at org.apache.hadoop.hdfs.server.blockmanagement.PendingReplicationBlocks$PendingReplicationMonitor.run(PendingReplicationBlocks.java:201)
  231. at java.lang.Thread.run(Thread.java:619)
  232.  
  233. "Socket Reader #1 for port 9008" prio=10 tid=0x00002aab3404a800 nid=0x1045 runnable [0x00000000426a6000..0x00000000426a6b90]
  234. java.lang.Thread.State: RUNNABLE
  235. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  236. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
  237. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  238. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  239. - locked <0x00002aaab59a1e20> (a sun.nio.ch.Util$1)
  240. - locked <0x00002aaab59a1e08> (a java.util.Collections$UnmodifiableSet)
  241. - locked <0x00002aaab595f0a0> (a sun.nio.ch.EPollSelectorImpl)
  242. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  243. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
  244. at org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:499)
  245. - locked <0x00002aaab595eb48> (a org.apache.hadoop.ipc.Server$Listener$Reader)
  246. at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:485)
  247.  
  248. "Logger channel to find64-03.dev.oclc.org/192.168.64.3:8485" daemon prio=10 tid=0x00002aab34529400 nid=0xfe4 waiting on condition [0x00000000423a3000..0x00000000423a3d10]
  249. java.lang.Thread.State: WAITING (parking)
  250. at sun.misc.Unsafe.park(Native Method)
  251. - parking to wait for <0x00002aaab596ba20> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  252. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  253. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  254. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  255. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:946)
  256. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:906)
  257. at java.lang.Thread.run(Thread.java:619)
  258.  
  259. "Logger channel to find64-02.dev.oclc.org/192.168.64.2:8485" daemon prio=10 tid=0x00002aab34528800 nid=0xfe3 waiting on condition [0x00000000422a2000..0x00000000422a2a90]
  260. java.lang.Thread.State: WAITING (parking)
  261. at sun.misc.Unsafe.park(Native Method)
  262. - parking to wait for <0x00002aaab5955e28> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  263. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  264. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  265. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  266. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:946)
  267. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:906)
  268. at java.lang.Thread.run(Thread.java:619)
  269.  
  270. "Logger channel to find64-01.dev.oclc.org/192.168.64.1:8485" daemon prio=10 tid=0x00002aab345c8000 nid=0xfe2 waiting on condition [0x00000000421a1000..0x00000000421a1a10]
  271. java.lang.Thread.State: WAITING (parking)
  272. at sun.misc.Unsafe.park(Native Method)
  273. - parking to wait for <0x00002aaab5976da0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  274. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
  275. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
  276. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
  277. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:946)
  278. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:906)
  279. at java.lang.Thread.run(Thread.java:619)
  280.  
  281. "Timer for 'NameNode' metrics system" daemon prio=10 tid=0x000000004dc82800 nid=0xfe0 in Object.wait() [0x0000000040801000..0x0000000040801b90]
  282. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  283. at java.lang.Object.wait(Native Method)
  284. - waiting on <0x00002aaab5696050> (a java.util.TaskQueue)
  285. at java.util.TimerThread.mainLoop(Timer.java:509)
  286. - locked <0x00002aaab5696050> (a java.util.TaskQueue)
  287. at java.util.TimerThread.run(Timer.java:462)
  288.  
  289. "IncidentDistributionPoint-0/ProcessSyncQueue" daemon prio=10 tid=0x00002aab344c1400 nid=0xfdb in Object.wait() [0x000000004114e000..0x000000004114ec10]
  290. java.lang.Thread.State: WAITING (on object monitor)
  291. at java.lang.Object.wait(Native Method)
  292. - waiting on <0x00002aaab3559200> (a ORG.oclc.infra.util.SyncQueue)
  293. at java.lang.Object.wait(Object.java:485)
  294. at ORG.oclc.infra.util.SyncQueue.waitForEntries(SyncQueue.java:276)
  295. - locked <0x00002aaab3559200> (a ORG.oclc.infra.util.SyncQueue)
  296. at ORG.oclc.infra.drax.DestinationList.execute(DestinationList.java:115)
  297. at ORG.oclc.infra.drax.DistributionPoint.run(DistributionPoint.java:134)
  298. at java.lang.Thread.run(Thread.java:619)
  299.  
  300. "RMI TCP Accept-0" daemon prio=10 tid=0x00002aab34454000 nid=0xfc5 runnable [0x0000000041ca6000..0x0000000041ca6a90]
  301. java.lang.Thread.State: RUNNABLE
  302. at java.net.PlainSocketImpl.socketAccept(Native Method)
  303. at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
  304. - locked <0x00002aaab5676f78> (a java.net.SocksSocketImpl)
  305. at java.net.ServerSocket.implAccept(ServerSocket.java:453)
  306. at java.net.ServerSocket.accept(ServerSocket.java:421)
  307. at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
  308. at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
  309. at java.lang.Thread.run(Thread.java:619)
  310.  
  311. "Low Memory Detector" daemon prio=10 tid=0x00002aab34085400 nid=0xfc2 runnable [0x0000000000000000..0x0000000000000000]
  312. java.lang.Thread.State: RUNNABLE
  313.  
  314. "CompilerThread1" daemon prio=10 tid=0x00002aab34082c00 nid=0xfc1 waiting on condition [0x0000000000000000..0x0000000041a98430]
  315. java.lang.Thread.State: RUNNABLE
  316.  
  317. "CompilerThread0" daemon prio=10 tid=0x00002aab34081000 nid=0xfc0 waiting on condition [0x0000000000000000..0x00000000419973b0]
  318. java.lang.Thread.State: RUNNABLE
  319.  
  320. "Signal Dispatcher" daemon prio=10 tid=0x00002aab3407fc00 nid=0xfbf runnable [0x0000000000000000..0x0000000000000000]
  321. java.lang.Thread.State: RUNNABLE
  322.  
  323. "Finalizer" daemon prio=10 tid=0x00002aab3405a000 nid=0xfbe in Object.wait() [0x0000000041796000..0x0000000041796c10]
  324. java.lang.Thread.State: WAITING (on object monitor)
  325. at java.lang.Object.wait(Native Method)
  326. - waiting on <0x00002aaab566c558> (a java.lang.ref.ReferenceQueue$Lock)
  327. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
  328. - locked <0x00002aaab566c558> (a java.lang.ref.ReferenceQueue$Lock)
  329. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
  330. at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
  331.  
  332. "Reference Handler" daemon prio=10 tid=0x00002aab34058800 nid=0xfbd in Object.wait() [0x0000000041695000..0x0000000041695d90]
  333. java.lang.Thread.State: WAITING (on object monitor)
  334. at java.lang.Object.wait(Native Method)
  335. - waiting on <0x00002aaab560ffa8> (a java.lang.ref.Reference$Lock)
  336. at java.lang.Object.wait(Object.java:485)
  337. at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
  338. - locked <0x00002aaab560ffa8> (a java.lang.ref.Reference$Lock)
  339.  
  340. "main" prio=10 tid=0x000000004d50ac00 nid=0xfb7 in Object.wait() [0x0000000040209000..0x0000000040209ed0]
  341. java.lang.Thread.State: WAITING (on object monitor)
  342. at java.lang.Object.wait(Native Method)
  343. - waiting on <0x00002aaab59602e8> (a org.apache.hadoop.ipc.ProtobufRpcEngine$Server)
  344. at java.lang.Object.wait(Object.java:485)
  345. at org.apache.hadoop.ipc.Server.join(Server.java:2019)
  346. - locked <0x00002aaab59602e8> (a org.apache.hadoop.ipc.ProtobufRpcEngine$Server)
  347. at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.join(NameNodeRpcServer.java:289)
  348. at org.apache.hadoop.hdfs.server.namenode.NameNode.join(NameNode.java:630)
  349. at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1194)
  350.  
  351. "VM Thread" prio=10 tid=0x00002aab34053800 nid=0xfbc runnable
  352.  
  353. "GC task thread#0 (ParallelGC)" prio=10 tid=0x000000004d515000 nid=0xfb8 runnable
  354.  
  355. "GC task thread#1 (ParallelGC)" prio=10 tid=0x000000004d516400 nid=0xfb9 runnable
  356.  
  357. "GC task thread#2 (ParallelGC)" prio=10 tid=0x000000004d517800 nid=0xfba runnable
  358.  
  359. "GC task thread#3 (ParallelGC)" prio=10 tid=0x000000004d518c00 nid=0xfbb runnable
  360.  
  361. "VM Periodic Task Thread" prio=10 tid=0x00002aab34464c00 nid=0xfc6 waiting on condition
  362.  
  363. JNI global references: 844
  364.  
  365. Heap
  366. PSYoungGen total 32000K, used 27239K [0x00002aab08980000, 0x00002aab0af30000, 0x00002aab33420000)
  367. eden space 29376K, 90% used [0x00002aab08980000,0x00002aab0a391e98,0x00002aab0a630000)
  368. from space 2624K, 20% used [0x00002aab0a630000,0x00002aab0a6b8000,0x00002aab0a8c0000)
  369. to space 2624K, 0% used [0x00002aab0aca0000,0x00002aab0aca0000,0x00002aab0af30000)
  370. PSOldGen total 56256K, used 39789K [0x00002aaab3420000, 0x00002aaab6b10000, 0x00002aab08980000)
  371. object space 56256K, 70% used [0x00002aaab3420000,0x00002aaab5afb448,0x00002aaab6b10000)
  372. PSPermGen total 47616K, used 32712K [0x00002aaaae020000, 0x00002aaab0ea0000, 0x00002aaab3420000)
  373. object space 47616K, 68% used [0x00002aaaae020000,0x00002aaab0012198,0x00002aaab0ea0000)
  374.  
  375. finddev07.dev.oclc.org:/home/buckleyr/saved.cdh4NamenodeLogs>
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!