API tools faq
paste
Advertisement
eremaijala

Solr 5.1.0 highlighter stack

eremaijala
May 5th, 2015
73
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 139.76 KB | None | 0 0
raw download clone embed print report
  1. [solr@finna-dev-fe ~]$ jstack 28259
  2. 2015-05-05 10:54:31
  3. Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.75-b04 mixed mode):
  4.  
  5. "Attach Listener" daemon prio=10 tid=0x00007f8238002800 nid=0x200a waiting on condition [0x0000000000000000]
  6. java.lang.Thread.State: RUNNABLE
  7.  
  8. "commitScheduler-13-thread-1" prio=10 tid=0x00007f820c014800 nid=0x4973 waiting on condition [0x00007f8268175000]
  9. java.lang.Thread.State: WAITING (parking)
  10. at sun.misc.Unsafe.park(Native Method)
  11. - parking to wait for <0x00007f8636e55110> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  12. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  13. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  14. at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
  15. at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
  16. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  17. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  18. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  19. at java.lang.Thread.run(Thread.java:745)
  20.  
  21. "searcherExecutor-11-thread-1" prio=10 tid=0x00007f8200129000 nid=0x48b0 waiting on condition [0x00007f823c397000]
  22. java.lang.Thread.State: WAITING (parking)
  23. at sun.misc.Unsafe.park(Native Method)
  24. - parking to wait for <0x00007f8635fd1208> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  25. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  26. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  27. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  28. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  29. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  30. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  31. at java.lang.Thread.run(Thread.java:745)
  32.  
  33. "DestroyJavaVM" prio=10 tid=0x00007f8aa4009800 nid=0x6e6a waiting on condition [0x0000000000000000]
  34. java.lang.Thread.State: RUNNABLE
  35.  
  36. "coreZkRegister-1-thread-1" prio=10 tid=0x00007f8aa52c0800 nid=0x6ee7 waiting on condition [0x00007f8268071000]
  37. java.lang.Thread.State: WAITING (parking)
  38. at sun.misc.Unsafe.park(Native Method)
  39. - parking to wait for <0x00007f848ee2da50> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  40. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  41. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  42. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  43. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  44. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  45. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  46. at java.lang.Thread.run(Thread.java:745)
  47.  
  48. "Thread-14" prio=10 tid=0x00007f8aa52bf000 nid=0x6ee6 in Object.wait() [0x00007f82680b2000]
  49. java.lang.Thread.State: WAITING (on object monitor)
  50. at java.lang.Object.wait(Native Method)
  51. - waiting on <0x00007f848ee26aa8> (a java.lang.Object)
  52. at java.lang.Object.wait(Object.java:503)
  53. at org.apache.solr.core.CloserThread.run(CoreContainer.java:907)
  54. - locked <0x00007f848ee26aa8> (a java.lang.Object)
  55.  
  56. "main-EventThread" daemon prio=10 tid=0x00007f8aa523a800 nid=0x6ec4 waiting on condition [0x00007f82681b6000]
  57. java.lang.Thread.State: WAITING (parking)
  58. at sun.misc.Unsafe.park(Native Method)
  59. - parking to wait for <0x00007f848ee2d678> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  60. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  61. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  62. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  63. at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
  64.  
  65. "main-SendThread(finna-dev-fe.csc.fi:2181)" daemon prio=10 tid=0x00007f8aa5260800 nid=0x6ec3 runnable [0x00007f82681f7000]
  66. java.lang.Thread.State: RUNNABLE
  67. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  68. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
  69. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
  70. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
  71. - locked <0x00007f848ee304b0> (a sun.nio.ch.Util$2)
  72. - locked <0x00007f848ee304c8> (a java.util.Collections$UnmodifiableSet)
  73. - locked <0x00007f848ee37250> (a sun.nio.ch.EPollSelectorImpl)
  74. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
  75. at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
  76. at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
  77.  
  78. "HashSessionScavenger-0" daemon prio=10 tid=0x00007f8aa5048000 nid=0x6ebb in Object.wait() [0x00007f826889c000]
  79. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  80. at java.lang.Object.wait(Native Method)
  81. at java.util.TimerThread.mainLoop(Timer.java:552)
  82. - locked <0x00007f848ee37c50> (a java.util.TaskQueue)
  83. at java.util.TimerThread.run(Timer.java:505)
  84.  
  85. "qtp125683540-20" prio=10 tid=0x00007f8aa4e8e800 nid=0x6e93 waiting on condition [0x00007f8268b76000]
  86. java.lang.Thread.State: TIMED_WAITING (parking)
  87. at sun.misc.Unsafe.park(Native Method)
  88. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  89. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  90. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  91. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  92. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  93. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  94. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  95. at java.lang.Thread.run(Thread.java:745)
  96.  
  97. "qtp125683540-19" prio=10 tid=0x00007f8aa4e8c800 nid=0x6e92 waiting on condition [0x00007f8268bb7000]
  98. java.lang.Thread.State: TIMED_WAITING (parking)
  99. at sun.misc.Unsafe.park(Native Method)
  100. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  101. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  102. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  103. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  104. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  105. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  106. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  107. at java.lang.Thread.run(Thread.java:745)
  108.  
  109. "qtp125683540-18" prio=10 tid=0x00007f8aa4e8a800 nid=0x6e91 waiting on condition [0x00007f8268bf8000]
  110. java.lang.Thread.State: TIMED_WAITING (parking)
  111. at sun.misc.Unsafe.park(Native Method)
  112. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  113. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  114. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  115. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  116. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  117. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  118. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  119. at java.lang.Thread.run(Thread.java:745)
  120.  
  121. "qtp125683540-17" prio=10 tid=0x00007f8aa4e88800 nid=0x6e90 waiting on condition [0x00007f8268c39000]
  122. java.lang.Thread.State: TIMED_WAITING (parking)
  123. at sun.misc.Unsafe.park(Native Method)
  124. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  125. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  126. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  127. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  128. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  129. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  130. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  131. at java.lang.Thread.run(Thread.java:745)
  132.  
  133. "qtp125683540-16" prio=10 tid=0x00007f8aa4e86800 nid=0x6e8f waiting on condition [0x00007f8268c7a000]
  134. java.lang.Thread.State: TIMED_WAITING (parking)
  135. at sun.misc.Unsafe.park(Native Method)
  136. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  137. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  138. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  139. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  140. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  141. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  142. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  143. at java.lang.Thread.run(Thread.java:745)
  144.  
  145. "qtp125683540-15" prio=10 tid=0x00007f8aa4e84000 nid=0x6e8e waiting on condition [0x00007f8268cbb000]
  146. java.lang.Thread.State: TIMED_WAITING (parking)
  147. at sun.misc.Unsafe.park(Native Method)
  148. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  149. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  150. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  151. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  152. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  153. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  154. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  155. at java.lang.Thread.run(Thread.java:745)
  156.  
  157. "qtp125683540-14" prio=10 tid=0x00007f8aa4e83000 nid=0x6e8d waiting on condition [0x00007f8268cfc000]
  158. java.lang.Thread.State: TIMED_WAITING (parking)
  159. at sun.misc.Unsafe.park(Native Method)
  160. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  161. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  162. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  163. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  164. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  165. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  166. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  167. at java.lang.Thread.run(Thread.java:745)
  168.  
  169. "qtp125683540-13" prio=10 tid=0x00007f8aa4e81000 nid=0x6e8c runnable [0x00007f8268d3c000]
  170. java.lang.Thread.State: RUNNABLE
  171. at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1067)
  172. at org.apache.lucene.index.MultiFields.getMergedFieldInfos(MultiFields.java:244)
  173. at org.apache.lucene.index.SlowCompositeReaderWrapper.getFieldInfos(SlowCompositeReaderWrapper.java:239)
  174. at org.apache.solr.highlight.DefaultSolrHighlighter.getSpanQueryScorer(DefaultSolrHighlighter.java:214)
  175. at org.apache.solr.highlight.DefaultSolrHighlighter.getPhraseHighlighter(DefaultSolrHighlighter.java:176)
  176. at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:504)
  177. at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:397)
  178. at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:143)
  179. at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:230)
  180. at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
  181. at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)
  182. at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:829)
  183. at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:446)
  184. at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
  185. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
  186. at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
  187. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
  188. at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
  189. at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
  190. at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
  191. at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
  192. at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
  193. at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
  194. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
  195. at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
  196. at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
  197. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
  198. at org.eclipse.jetty.server.Server.handle(Server.java:368)
  199. at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
  200. at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
  201. at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
  202. at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
  203. at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
  204. at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
  205. at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
  206. at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
  207. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
  208. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
  209. at java.lang.Thread.run(Thread.java:745)
  210.  
  211. "qtp125683540-12" prio=10 tid=0x00007f8aa4e7f800 nid=0x6e8b waiting on condition [0x00007f8268d7e000]
  212. java.lang.Thread.State: TIMED_WAITING (parking)
  213. at sun.misc.Unsafe.park(Native Method)
  214. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  215. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  216. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  217. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  218. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  219. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  220. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  221. at java.lang.Thread.run(Thread.java:745)
  222.  
  223. "qtp125683540-11 Acceptor0 [email protected]:8984" prio=10 tid=0x00007f8aa4e7a000 nid=0x6e8a runnable [0x00007f8268dbf000]
  224. java.lang.Thread.State: RUNNABLE
  225. at java.net.PlainSocketImpl.socketAccept(Native Method)
  226. at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
  227. at java.net.ServerSocket.implAccept(ServerSocket.java:530)
  228. at java.net.ServerSocket.accept(ServerSocket.java:498)
  229. at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
  230. at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
  231. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
  232. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
  233. at java.lang.Thread.run(Thread.java:745)
  234.  
  235. "ShutdownMonitor" daemon prio=10 tid=0x00007f8aa4e6f800 nid=0x6e89 runnable [0x00007f8268e00000]
  236. java.lang.Thread.State: RUNNABLE
  237. at java.net.PlainSocketImpl.socketAccept(Native Method)
  238. at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
  239. at java.net.ServerSocket.implAccept(ServerSocket.java:530)
  240. at java.net.ServerSocket.accept(ServerSocket.java:498)
  241. at org.eclipse.jetty.server.ShutdownMonitor.run(ShutdownMonitor.java:165)
  242.  
  243. "Service Thread" daemon prio=10 tid=0x00007f8aa4143800 nid=0x6e7d runnable [0x0000000000000000]
  244. java.lang.Thread.State: RUNNABLE
  245.  
  246. "C2 CompilerThread1" daemon prio=10 tid=0x00007f8aa4141000 nid=0x6e7c waiting on condition [0x0000000000000000]
  247. java.lang.Thread.State: RUNNABLE
  248.  
  249. "C2 CompilerThread0" daemon prio=10 tid=0x00007f8aa413e800 nid=0x6e7b waiting on condition [0x0000000000000000]
  250. java.lang.Thread.State: RUNNABLE
  251.  
  252. "Signal Dispatcher" daemon prio=10 tid=0x00007f8aa413c800 nid=0x6e7a runnable [0x0000000000000000]
  253. java.lang.Thread.State: RUNNABLE
  254.  
  255. "Surrogate Locker Thread (Concurrent GC)" daemon prio=10 tid=0x00007f8aa413a800 nid=0x6e79 waiting on condition [0x0000000000000000]
  256. java.lang.Thread.State: RUNNABLE
  257.  
  258. "Finalizer" daemon prio=10 tid=0x00007f8aa4119800 nid=0x6e78 in Object.wait() [0x00007f828808e000]
  259. java.lang.Thread.State: WAITING (on object monitor)
  260. at java.lang.Object.wait(Native Method)
  261. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
  262. - locked <0x00007f848f39db00> (a java.lang.ref.ReferenceQueue$Lock)
  263. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
  264. at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
  265.  
  266. "Reference Handler" daemon prio=10 tid=0x00007f8aa4117800 nid=0x6e77 in Object.wait() [0x00007f82880cf000]
  267. java.lang.Thread.State: WAITING (on object monitor)
  268. at java.lang.Object.wait(Native Method)
  269. at java.lang.Object.wait(Object.java:503)
  270. at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
  271. - locked <0x00007f848f39db48> (a java.lang.ref.Reference$Lock)
  272.  
  273. "VM Thread" prio=10 tid=0x00007f8aa4113000 nid=0x6e76 runnable
  274.  
  275. "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401b000 nid=0x6e6b runnable
  276.  
  277. "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401d000 nid=0x6e6c runnable
  278.  
  279. "Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401e800 nid=0x6e6d runnable
  280.  
  281. "Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x00007f8aa4020800 nid=0x6e6e runnable
  282.  
  283. "Concurrent Mark-Sweep GC Thread" prio=10 tid=0x00007f8aa40a8800 nid=0x6e75 runnable
  284. "Gang worker#0 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a1000 nid=0x6e71 runnable
  285.  
  286. "Gang worker#1 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a2800 nid=0x6e72 runnable
  287.  
  288. "Gang worker#2 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a4800 nid=0x6e73 runnable
  289.  
  290. "Gang worker#3 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a6800 nid=0x6e74 runnable
  291.  
  292. "VM Periodic Task Thread" prio=10 tid=0x00007f8aa414e800 nid=0x6e7e waiting on condition
  293.  
  294. JNI global references: 344
  295.  
  296. [solr@finna-dev-fe ~]$ jstack 28259
  297. 2015-05-05 10:54:32
  298. Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.75-b04 mixed mode):
  299.  
  300. "Attach Listener" daemon prio=10 tid=0x00007f8238002800 nid=0x200a waiting on condition [0x0000000000000000]
  301. java.lang.Thread.State: RUNNABLE
  302.  
  303. "commitScheduler-13-thread-1" prio=10 tid=0x00007f820c014800 nid=0x4973 waiting on condition [0x00007f8268175000]
  304. java.lang.Thread.State: WAITING (parking)
  305. at sun.misc.Unsafe.park(Native Method)
  306. - parking to wait for <0x00007f8636e55110> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  307. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  308. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  309. at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
  310. at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
  311. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  312. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  313. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  314. at java.lang.Thread.run(Thread.java:745)
  315.  
  316. "searcherExecutor-11-thread-1" prio=10 tid=0x00007f8200129000 nid=0x48b0 waiting on condition [0x00007f823c397000]
  317. java.lang.Thread.State: WAITING (parking)
  318. at sun.misc.Unsafe.park(Native Method)
  319. - parking to wait for <0x00007f8635fd1208> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  320. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  321. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  322. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  323. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  324. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  325. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  326. at java.lang.Thread.run(Thread.java:745)
  327.  
  328. "DestroyJavaVM" prio=10 tid=0x00007f8aa4009800 nid=0x6e6a waiting on condition [0x0000000000000000]
  329. java.lang.Thread.State: RUNNABLE
  330.  
  331. "coreZkRegister-1-thread-1" prio=10 tid=0x00007f8aa52c0800 nid=0x6ee7 waiting on condition [0x00007f8268071000]
  332. java.lang.Thread.State: WAITING (parking)
  333. at sun.misc.Unsafe.park(Native Method)
  334. - parking to wait for <0x00007f848ee2da50> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  335. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  336. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  337. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  338. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  339. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  340. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  341. at java.lang.Thread.run(Thread.java:745)
  342.  
  343. "Thread-14" prio=10 tid=0x00007f8aa52bf000 nid=0x6ee6 in Object.wait() [0x00007f82680b2000]
  344. java.lang.Thread.State: WAITING (on object monitor)
  345. at java.lang.Object.wait(Native Method)
  346. - waiting on <0x00007f848ee26aa8> (a java.lang.Object)
  347. at java.lang.Object.wait(Object.java:503)
  348. at org.apache.solr.core.CloserThread.run(CoreContainer.java:907)
  349. - locked <0x00007f848ee26aa8> (a java.lang.Object)
  350.  
  351. "main-EventThread" daemon prio=10 tid=0x00007f8aa523a800 nid=0x6ec4 waiting on condition [0x00007f82681b6000]
  352. java.lang.Thread.State: WAITING (parking)
  353. at sun.misc.Unsafe.park(Native Method)
  354. - parking to wait for <0x00007f848ee2d678> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  355. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  356. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  357. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  358. at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
  359.  
  360. "main-SendThread(finna-dev-fe.csc.fi:2181)" daemon prio=10 tid=0x00007f8aa5260800 nid=0x6ec3 runnable [0x00007f82681f7000]
  361. java.lang.Thread.State: RUNNABLE
  362. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  363. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
  364. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
  365. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
  366. - locked <0x00007f848ee304b0> (a sun.nio.ch.Util$2)
  367. - locked <0x00007f848ee304c8> (a java.util.Collections$UnmodifiableSet)
  368. - locked <0x00007f848ee37250> (a sun.nio.ch.EPollSelectorImpl)
  369. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
  370. at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
  371. at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
  372.  
  373. "HashSessionScavenger-0" daemon prio=10 tid=0x00007f8aa5048000 nid=0x6ebb in Object.wait() [0x00007f826889c000]
  374. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  375. at java.lang.Object.wait(Native Method)
  376. at java.util.TimerThread.mainLoop(Timer.java:552)
  377. - locked <0x00007f848ee37c50> (a java.util.TaskQueue)
  378. at java.util.TimerThread.run(Timer.java:505)
  379.  
  380. "qtp125683540-20" prio=10 tid=0x00007f8aa4e8e800 nid=0x6e93 waiting on condition [0x00007f8268b76000]
  381. java.lang.Thread.State: TIMED_WAITING (parking)
  382. at sun.misc.Unsafe.park(Native Method)
  383. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  384. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  385. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  386. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  387. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  388. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  389. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  390. at java.lang.Thread.run(Thread.java:745)
  391.  
  392. "qtp125683540-19" prio=10 tid=0x00007f8aa4e8c800 nid=0x6e92 waiting on condition [0x00007f8268bb7000]
  393. java.lang.Thread.State: TIMED_WAITING (parking)
  394. at sun.misc.Unsafe.park(Native Method)
  395. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  396. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  397. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  398. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  399. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  400. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  401. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  402. at java.lang.Thread.run(Thread.java:745)
  403.  
  404. "qtp125683540-18" prio=10 tid=0x00007f8aa4e8a800 nid=0x6e91 waiting on condition [0x00007f8268bf8000]
  405. java.lang.Thread.State: TIMED_WAITING (parking)
  406. at sun.misc.Unsafe.park(Native Method)
  407. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  408. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  409. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  410. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  411. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  412. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  413. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  414. at java.lang.Thread.run(Thread.java:745)
  415.  
  416. "qtp125683540-17" prio=10 tid=0x00007f8aa4e88800 nid=0x6e90 waiting on condition [0x00007f8268c39000]
  417. java.lang.Thread.State: TIMED_WAITING (parking)
  418. at sun.misc.Unsafe.park(Native Method)
  419. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  420. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  421. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  422. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  423. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  424. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  425. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  426. at java.lang.Thread.run(Thread.java:745)
  427.  
  428. "qtp125683540-16" prio=10 tid=0x00007f8aa4e86800 nid=0x6e8f waiting on condition [0x00007f8268c7a000]
  429. java.lang.Thread.State: TIMED_WAITING (parking)
  430. at sun.misc.Unsafe.park(Native Method)
  431. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  432. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  433. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  434. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  435. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  436. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  437. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  438. at java.lang.Thread.run(Thread.java:745)
  439.  
  440. "qtp125683540-15" prio=10 tid=0x00007f8aa4e84000 nid=0x6e8e waiting on condition [0x00007f8268cbb000]
  441. java.lang.Thread.State: TIMED_WAITING (parking)
  442. at sun.misc.Unsafe.park(Native Method)
  443. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  444. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  445. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  446. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  447. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  448. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  449. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  450. at java.lang.Thread.run(Thread.java:745)
  451.  
  452. "qtp125683540-14" prio=10 tid=0x00007f8aa4e83000 nid=0x6e8d waiting on condition [0x00007f8268cfc000]
  453. java.lang.Thread.State: TIMED_WAITING (parking)
  454. at sun.misc.Unsafe.park(Native Method)
  455. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  456. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  457. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  458. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  459. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  460. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  461. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  462. at java.lang.Thread.run(Thread.java:745)
  463.  
  464. "qtp125683540-13" prio=10 tid=0x00007f8aa4e81000 nid=0x6e8c runnable [0x00007f8268d3c000]
  465. java.lang.Thread.State: RUNNABLE
  466. at org.apache.lucene.index.FieldInfos$Builder.addOrUpdateInternal(FieldInfos.java:305)
  467. at org.apache.lucene.index.FieldInfos$Builder.add(FieldInfos.java:338)
  468. at org.apache.lucene.index.FieldInfos$Builder.add(FieldInfos.java:276)
  469. at org.apache.lucene.index.MultiFields.getMergedFieldInfos(MultiFields.java:245)
  470. at org.apache.lucene.index.SlowCompositeReaderWrapper.getFieldInfos(SlowCompositeReaderWrapper.java:239)
  471. at org.apache.solr.highlight.DefaultSolrHighlighter.getSpanQueryScorer(DefaultSolrHighlighter.java:214)
  472. at org.apache.solr.highlight.DefaultSolrHighlighter.getPhraseHighlighter(DefaultSolrHighlighter.java:176)
  473. at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:504)
  474. at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:397)
  475. at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:143)
  476. at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:230)
  477. at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
  478. at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)
  479. at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:829)
  480. at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:446)
  481. at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
  482. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
  483. at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
  484. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
  485. at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
  486. at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
  487. at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
  488. at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
  489. at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
  490. at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
  491. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
  492. at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
  493. at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
  494. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
  495. at org.eclipse.jetty.server.Server.handle(Server.java:368)
  496. at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
  497. at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
  498. at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
  499. at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
  500. at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
  501. at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
  502. at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
  503. at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
  504. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
  505. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
  506. at java.lang.Thread.run(Thread.java:745)
  507.  
  508. "qtp125683540-12" prio=10 tid=0x00007f8aa4e7f800 nid=0x6e8b waiting on condition [0x00007f8268d7e000]
  509. java.lang.Thread.State: TIMED_WAITING (parking)
  510. at sun.misc.Unsafe.park(Native Method)
  511. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  512. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  513. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  514. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  515. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  516. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  517. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  518. at java.lang.Thread.run(Thread.java:745)
  519.  
  520. "qtp125683540-11 Acceptor0 [email protected]:8984" prio=10 tid=0x00007f8aa4e7a000 nid=0x6e8a runnable [0x00007f8268dbf000]
  521. java.lang.Thread.State: RUNNABLE
  522. at java.net.PlainSocketImpl.socketAccept(Native Method)
  523. at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
  524. at java.net.ServerSocket.implAccept(ServerSocket.java:530)
  525. at java.net.ServerSocket.accept(ServerSocket.java:498)
  526. at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
  527. at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
  528. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
  529. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
  530. at java.lang.Thread.run(Thread.java:745)
  531.  
  532. "ShutdownMonitor" daemon prio=10 tid=0x00007f8aa4e6f800 nid=0x6e89 runnable [0x00007f8268e00000]
  533. java.lang.Thread.State: RUNNABLE
  534. at java.net.PlainSocketImpl.socketAccept(Native Method)
  535. at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
  536. at java.net.ServerSocket.implAccept(ServerSocket.java:530)
  537. at java.net.ServerSocket.accept(ServerSocket.java:498)
  538. at org.eclipse.jetty.server.ShutdownMonitor.run(ShutdownMonitor.java:165)
  539.  
  540. "Service Thread" daemon prio=10 tid=0x00007f8aa4143800 nid=0x6e7d runnable [0x0000000000000000]
  541. java.lang.Thread.State: RUNNABLE
  542.  
  543. "C2 CompilerThread1" daemon prio=10 tid=0x00007f8aa4141000 nid=0x6e7c waiting on condition [0x0000000000000000]
  544. java.lang.Thread.State: RUNNABLE
  545.  
  546. "C2 CompilerThread0" daemon prio=10 tid=0x00007f8aa413e800 nid=0x6e7b waiting on condition [0x0000000000000000]
  547. java.lang.Thread.State: RUNNABLE
  548.  
  549. "Signal Dispatcher" daemon prio=10 tid=0x00007f8aa413c800 nid=0x6e7a runnable [0x0000000000000000]
  550. java.lang.Thread.State: RUNNABLE
  551.  
  552. "Surrogate Locker Thread (Concurrent GC)" daemon prio=10 tid=0x00007f8aa413a800 nid=0x6e79 waiting on condition [0x0000000000000000]
  553. java.lang.Thread.State: RUNNABLE
  554.  
  555. "Finalizer" daemon prio=10 tid=0x00007f8aa4119800 nid=0x6e78 in Object.wait() [0x00007f828808e000]
  556. java.lang.Thread.State: WAITING (on object monitor)
  557. at java.lang.Object.wait(Native Method)
  558. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
  559. - locked <0x00007f848f39db00> (a java.lang.ref.ReferenceQueue$Lock)
  560. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
  561. at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
  562.  
  563. "Reference Handler" daemon prio=10 tid=0x00007f8aa4117800 nid=0x6e77 in Object.wait() [0x00007f82880cf000]
  564. java.lang.Thread.State: WAITING (on object monitor)
  565. at java.lang.Object.wait(Native Method)
  566. at java.lang.Object.wait(Object.java:503)
  567. at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
  568. - locked <0x00007f848f39db48> (a java.lang.ref.Reference$Lock)
  569.  
  570. "VM Thread" prio=10 tid=0x00007f8aa4113000 nid=0x6e76 runnable
  571.  
  572. "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401b000 nid=0x6e6b runnable
  573.  
  574. "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401d000 nid=0x6e6c runnable
  575.  
  576. "Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401e800 nid=0x6e6d runnable
  577.  
  578. "Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x00007f8aa4020800 nid=0x6e6e runnable
  579.  
  580. "Concurrent Mark-Sweep GC Thread" prio=10 tid=0x00007f8aa40a8800 nid=0x6e75 runnable
  581. "Gang worker#0 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a1000 nid=0x6e71 runnable
  582.  
  583. "Gang worker#1 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a2800 nid=0x6e72 runnable
  584.  
  585. "Gang worker#2 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a4800 nid=0x6e73 runnable
  586.  
  587. "Gang worker#3 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a6800 nid=0x6e74 runnable
  588.  
  589. "VM Periodic Task Thread" prio=10 tid=0x00007f8aa414e800 nid=0x6e7e waiting on condition
  590.  
  591. JNI global references: 344
  592.  
  593. [solr@finna-dev-fe ~]$ jstack 28259
  594. 2015-05-05 10:54:33
  595. Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.75-b04 mixed mode):
  596.  
  597. "Attach Listener" daemon prio=10 tid=0x00007f8238002800 nid=0x200a waiting on condition [0x0000000000000000]
  598. java.lang.Thread.State: RUNNABLE
  599.  
  600. "commitScheduler-13-thread-1" prio=10 tid=0x00007f820c014800 nid=0x4973 waiting on condition [0x00007f8268175000]
  601. java.lang.Thread.State: WAITING (parking)
  602. at sun.misc.Unsafe.park(Native Method)
  603. - parking to wait for <0x00007f8636e55110> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  604. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  605. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  606. at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
  607. at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
  608. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  609. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  610. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  611. at java.lang.Thread.run(Thread.java:745)
  612.  
  613. "searcherExecutor-11-thread-1" prio=10 tid=0x00007f8200129000 nid=0x48b0 waiting on condition [0x00007f823c397000]
  614. java.lang.Thread.State: WAITING (parking)
  615. at sun.misc.Unsafe.park(Native Method)
  616. - parking to wait for <0x00007f8635fd1208> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  617. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  618. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  619. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  620. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  621. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  622. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  623. at java.lang.Thread.run(Thread.java:745)
  624.  
  625. "DestroyJavaVM" prio=10 tid=0x00007f8aa4009800 nid=0x6e6a waiting on condition [0x0000000000000000]
  626. java.lang.Thread.State: RUNNABLE
  627.  
  628. "coreZkRegister-1-thread-1" prio=10 tid=0x00007f8aa52c0800 nid=0x6ee7 waiting on condition [0x00007f8268071000]
  629. java.lang.Thread.State: WAITING (parking)
  630. at sun.misc.Unsafe.park(Native Method)
  631. - parking to wait for <0x00007f848ee2da50> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  632. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  633. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  634. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  635. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  636. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  637. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  638. at java.lang.Thread.run(Thread.java:745)
  639.  
  640. "Thread-14" prio=10 tid=0x00007f8aa52bf000 nid=0x6ee6 in Object.wait() [0x00007f82680b2000]
  641. java.lang.Thread.State: WAITING (on object monitor)
  642. at java.lang.Object.wait(Native Method)
  643. - waiting on <0x00007f848ee26aa8> (a java.lang.Object)
  644. at java.lang.Object.wait(Object.java:503)
  645. at org.apache.solr.core.CloserThread.run(CoreContainer.java:907)
  646. - locked <0x00007f848ee26aa8> (a java.lang.Object)
  647.  
  648. "main-EventThread" daemon prio=10 tid=0x00007f8aa523a800 nid=0x6ec4 waiting on condition [0x00007f82681b6000]
  649. java.lang.Thread.State: WAITING (parking)
  650. at sun.misc.Unsafe.park(Native Method)
  651. - parking to wait for <0x00007f848ee2d678> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  652. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  653. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  654. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  655. at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
  656.  
  657. "main-SendThread(finna-dev-fe.csc.fi:2181)" daemon prio=10 tid=0x00007f8aa5260800 nid=0x6ec3 runnable [0x00007f82681f7000]
  658. java.lang.Thread.State: RUNNABLE
  659. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  660. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
  661. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
  662. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
  663. - locked <0x00007f848ee304b0> (a sun.nio.ch.Util$2)
  664. - locked <0x00007f848ee304c8> (a java.util.Collections$UnmodifiableSet)
  665. - locked <0x00007f848ee37250> (a sun.nio.ch.EPollSelectorImpl)
  666. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
  667. at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
  668. at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
  669.  
  670. "HashSessionScavenger-0" daemon prio=10 tid=0x00007f8aa5048000 nid=0x6ebb in Object.wait() [0x00007f826889c000]
  671. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  672. at java.lang.Object.wait(Native Method)
  673. at java.util.TimerThread.mainLoop(Timer.java:552)
  674. - locked <0x00007f848ee37c50> (a java.util.TaskQueue)
  675. at java.util.TimerThread.run(Timer.java:505)
  676.  
  677. "qtp125683540-20" prio=10 tid=0x00007f8aa4e8e800 nid=0x6e93 waiting on condition [0x00007f8268b76000]
  678. java.lang.Thread.State: TIMED_WAITING (parking)
  679. at sun.misc.Unsafe.park(Native Method)
  680. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  681. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  682. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  683. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  684. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  685. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  686. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  687. at java.lang.Thread.run(Thread.java:745)
  688.  
  689. "qtp125683540-19" prio=10 tid=0x00007f8aa4e8c800 nid=0x6e92 waiting on condition [0x00007f8268bb7000]
  690. java.lang.Thread.State: TIMED_WAITING (parking)
  691. at sun.misc.Unsafe.park(Native Method)
  692. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  693. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  694. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  695. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  696. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  697. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  698. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  699. at java.lang.Thread.run(Thread.java:745)
  700.  
  701. "qtp125683540-18" prio=10 tid=0x00007f8aa4e8a800 nid=0x6e91 waiting on condition [0x00007f8268bf8000]
  702. java.lang.Thread.State: TIMED_WAITING (parking)
  703. at sun.misc.Unsafe.park(Native Method)
  704. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  705. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  706. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  707. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  708. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  709. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  710. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  711. at java.lang.Thread.run(Thread.java:745)
  712.  
  713. "qtp125683540-17" prio=10 tid=0x00007f8aa4e88800 nid=0x6e90 waiting on condition [0x00007f8268c39000]
  714. java.lang.Thread.State: TIMED_WAITING (parking)
  715. at sun.misc.Unsafe.park(Native Method)
  716. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  717. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  718. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  719. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  720. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  721. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  722. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  723. at java.lang.Thread.run(Thread.java:745)
  724.  
  725. "qtp125683540-16" prio=10 tid=0x00007f8aa4e86800 nid=0x6e8f waiting on condition [0x00007f8268c7a000]
  726. java.lang.Thread.State: TIMED_WAITING (parking)
  727. at sun.misc.Unsafe.park(Native Method)
  728. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  729. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  730. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  731. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  732. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  733. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  734. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  735. at java.lang.Thread.run(Thread.java:745)
  736.  
  737. "qtp125683540-15" prio=10 tid=0x00007f8aa4e84000 nid=0x6e8e waiting on condition [0x00007f8268cbb000]
  738. java.lang.Thread.State: TIMED_WAITING (parking)
  739. at sun.misc.Unsafe.park(Native Method)
  740. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  741. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  742. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  743. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  744. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  745. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  746. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  747. at java.lang.Thread.run(Thread.java:745)
  748.  
  749. "qtp125683540-14" prio=10 tid=0x00007f8aa4e83000 nid=0x6e8d waiting on condition [0x00007f8268cfc000]
  750. java.lang.Thread.State: TIMED_WAITING (parking)
  751. at sun.misc.Unsafe.park(Native Method)
  752. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  753. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  754. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  755. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  756. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  757. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  758. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  759. at java.lang.Thread.run(Thread.java:745)
  760.  
  761. "qtp125683540-13" prio=10 tid=0x00007f8aa4e81000 nid=0x6e8c runnable [0x00007f8268d3c000]
  762. java.lang.Thread.State: RUNNABLE
  763. at org.apache.lucene.index.FieldInfos$Builder.addOrUpdateInternal(FieldInfos.java:305)
  764. at org.apache.lucene.index.FieldInfos$Builder.add(FieldInfos.java:338)
  765. at org.apache.lucene.index.FieldInfos$Builder.add(FieldInfos.java:276)
  766. at org.apache.lucene.index.MultiFields.getMergedFieldInfos(MultiFields.java:245)
  767. at org.apache.lucene.index.SlowCompositeReaderWrapper.getFieldInfos(SlowCompositeReaderWrapper.java:239)
  768. at org.apache.solr.highlight.DefaultSolrHighlighter.getSpanQueryScorer(DefaultSolrHighlighter.java:214)
  769. at org.apache.solr.highlight.DefaultSolrHighlighter.getPhraseHighlighter(DefaultSolrHighlighter.java:176)
  770. at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:504)
  771. at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:397)
  772. at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:143)
  773. at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:230)
  774. at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
  775. at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)
  776. at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:829)
  777. at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:446)
  778. at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
  779. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
  780. at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
  781. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
  782. at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
  783. at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
  784. at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
  785. at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
  786. at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
  787. at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
  788. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
  789. at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
  790. at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
  791. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
  792. at org.eclipse.jetty.server.Server.handle(Server.java:368)
  793. at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
  794. at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
  795. at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
  796. at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
  797. at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
  798. at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
  799. at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
  800. at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
  801. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
  802. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
  803. at java.lang.Thread.run(Thread.java:745)
  804.  
  805. "qtp125683540-12" prio=10 tid=0x00007f8aa4e7f800 nid=0x6e8b waiting on condition [0x00007f8268d7e000]
  806. java.lang.Thread.State: TIMED_WAITING (parking)
  807. at sun.misc.Unsafe.park(Native Method)
  808. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  809. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  810. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  811. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  812. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  813. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  814. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  815. at java.lang.Thread.run(Thread.java:745)
  816.  
  817. "qtp125683540-11 Acceptor0 [email protected]:8984" prio=10 tid=0x00007f8aa4e7a000 nid=0x6e8a runnable [0x00007f8268dbf000]
  818. java.lang.Thread.State: RUNNABLE
  819. at java.net.PlainSocketImpl.socketAccept(Native Method)
  820. at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
  821. at java.net.ServerSocket.implAccept(ServerSocket.java:530)
  822. at java.net.ServerSocket.accept(ServerSocket.java:498)
  823. at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
  824. at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
  825. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
  826. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
  827. at java.lang.Thread.run(Thread.java:745)
  828.  
  829. "ShutdownMonitor" daemon prio=10 tid=0x00007f8aa4e6f800 nid=0x6e89 runnable [0x00007f8268e00000]
  830. java.lang.Thread.State: RUNNABLE
  831. at java.net.PlainSocketImpl.socketAccept(Native Method)
  832. at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
  833. at java.net.ServerSocket.implAccept(ServerSocket.java:530)
  834. at java.net.ServerSocket.accept(ServerSocket.java:498)
  835. at org.eclipse.jetty.server.ShutdownMonitor.run(ShutdownMonitor.java:165)
  836.  
  837. "Service Thread" daemon prio=10 tid=0x00007f8aa4143800 nid=0x6e7d runnable [0x0000000000000000]
  838. java.lang.Thread.State: RUNNABLE
  839.  
  840. "C2 CompilerThread1" daemon prio=10 tid=0x00007f8aa4141000 nid=0x6e7c waiting on condition [0x0000000000000000]
  841. java.lang.Thread.State: RUNNABLE
  842.  
  843. "C2 CompilerThread0" daemon prio=10 tid=0x00007f8aa413e800 nid=0x6e7b waiting on condition [0x0000000000000000]
  844. java.lang.Thread.State: RUNNABLE
  845.  
  846. "Signal Dispatcher" daemon prio=10 tid=0x00007f8aa413c800 nid=0x6e7a runnable [0x0000000000000000]
  847. java.lang.Thread.State: RUNNABLE
  848.  
  849. "Surrogate Locker Thread (Concurrent GC)" daemon prio=10 tid=0x00007f8aa413a800 nid=0x6e79 waiting on condition [0x0000000000000000]
  850. java.lang.Thread.State: RUNNABLE
  851.  
  852. "Finalizer" daemon prio=10 tid=0x00007f8aa4119800 nid=0x6e78 in Object.wait() [0x00007f828808e000]
  853. java.lang.Thread.State: WAITING (on object monitor)
  854. at java.lang.Object.wait(Native Method)
  855. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
  856. - locked <0x00007f848f39db00> (a java.lang.ref.ReferenceQueue$Lock)
  857. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
  858. at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
  859.  
  860. "Reference Handler" daemon prio=10 tid=0x00007f8aa4117800 nid=0x6e77 in Object.wait() [0x00007f82880cf000]
  861. java.lang.Thread.State: WAITING (on object monitor)
  862. at java.lang.Object.wait(Native Method)
  863. at java.lang.Object.wait(Object.java:503)
  864. at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
  865. - locked <0x00007f848f39db48> (a java.lang.ref.Reference$Lock)
  866.  
  867. "VM Thread" prio=10 tid=0x00007f8aa4113000 nid=0x6e76 runnable
  868.  
  869. "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401b000 nid=0x6e6b runnable
  870.  
  871. "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401d000 nid=0x6e6c runnable
  872.  
  873. "Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401e800 nid=0x6e6d runnable
  874.  
  875. "Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x00007f8aa4020800 nid=0x6e6e runnable
  876.  
  877. "Concurrent Mark-Sweep GC Thread" prio=10 tid=0x00007f8aa40a8800 nid=0x6e75 runnable
  878. "Gang worker#0 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a1000 nid=0x6e71 runnable
  879.  
  880. "Gang worker#1 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a2800 nid=0x6e72 runnable
  881.  
  882. "Gang worker#2 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a4800 nid=0x6e73 runnable
  883.  
  884. "Gang worker#3 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a6800 nid=0x6e74 runnable
  885.  
  886. "VM Periodic Task Thread" prio=10 tid=0x00007f8aa414e800 nid=0x6e7e waiting on condition
  887.  
  888. JNI global references: 344
  889.  
  890. [solr@finna-dev-fe ~]$ jstack 28259
  891. 2015-05-05 10:54:34
  892. Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.75-b04 mixed mode):
  893.  
  894. "Attach Listener" daemon prio=10 tid=0x00007f8238002800 nid=0x200a waiting on condition [0x0000000000000000]
  895. java.lang.Thread.State: RUNNABLE
  896.  
  897. "commitScheduler-13-thread-1" prio=10 tid=0x00007f820c014800 nid=0x4973 waiting on condition [0x00007f8268175000]
  898. java.lang.Thread.State: WAITING (parking)
  899. at sun.misc.Unsafe.park(Native Method)
  900. - parking to wait for <0x00007f8636e55110> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  901. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  902. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  903. at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
  904. at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
  905. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  906. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  907. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  908. at java.lang.Thread.run(Thread.java:745)
  909.  
  910. "searcherExecutor-11-thread-1" prio=10 tid=0x00007f8200129000 nid=0x48b0 waiting on condition [0x00007f823c397000]
  911. java.lang.Thread.State: WAITING (parking)
  912. at sun.misc.Unsafe.park(Native Method)
  913. - parking to wait for <0x00007f8635fd1208> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  914. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  915. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  916. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  917. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  918. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  919. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  920. at java.lang.Thread.run(Thread.java:745)
  921.  
  922. "DestroyJavaVM" prio=10 tid=0x00007f8aa4009800 nid=0x6e6a waiting on condition [0x0000000000000000]
  923. java.lang.Thread.State: RUNNABLE
  924.  
  925. "coreZkRegister-1-thread-1" prio=10 tid=0x00007f8aa52c0800 nid=0x6ee7 waiting on condition [0x00007f8268071000]
  926. java.lang.Thread.State: WAITING (parking)
  927. at sun.misc.Unsafe.park(Native Method)
  928. - parking to wait for <0x00007f848ee2da50> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  929. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  930. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  931. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  932. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  933. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  934. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  935. at java.lang.Thread.run(Thread.java:745)
  936.  
  937. "Thread-14" prio=10 tid=0x00007f8aa52bf000 nid=0x6ee6 in Object.wait() [0x00007f82680b2000]
  938. java.lang.Thread.State: WAITING (on object monitor)
  939. at java.lang.Object.wait(Native Method)
  940. - waiting on <0x00007f848ee26aa8> (a java.lang.Object)
  941. at java.lang.Object.wait(Object.java:503)
  942. at org.apache.solr.core.CloserThread.run(CoreContainer.java:907)
  943. - locked <0x00007f848ee26aa8> (a java.lang.Object)
  944.  
  945. "main-EventThread" daemon prio=10 tid=0x00007f8aa523a800 nid=0x6ec4 waiting on condition [0x00007f82681b6000]
  946. java.lang.Thread.State: WAITING (parking)
  947. at sun.misc.Unsafe.park(Native Method)
  948. - parking to wait for <0x00007f848ee2d678> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  949. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  950. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  951. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  952. at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
  953.  
  954. "main-SendThread(finna-dev-fe.csc.fi:2181)" daemon prio=10 tid=0x00007f8aa5260800 nid=0x6ec3 runnable [0x00007f82681f7000]
  955. java.lang.Thread.State: RUNNABLE
  956. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  957. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
  958. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
  959. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
  960. - locked <0x00007f848ee304b0> (a sun.nio.ch.Util$2)
  961. - locked <0x00007f848ee304c8> (a java.util.Collections$UnmodifiableSet)
  962. - locked <0x00007f848ee37250> (a sun.nio.ch.EPollSelectorImpl)
  963. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
  964. at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
  965. at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
  966.  
  967. "HashSessionScavenger-0" daemon prio=10 tid=0x00007f8aa5048000 nid=0x6ebb in Object.wait() [0x00007f826889c000]
  968. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  969. at java.lang.Object.wait(Native Method)
  970. at java.util.TimerThread.mainLoop(Timer.java:552)
  971. - locked <0x00007f848ee37c50> (a java.util.TaskQueue)
  972. at java.util.TimerThread.run(Timer.java:505)
  973.  
  974. "qtp125683540-20" prio=10 tid=0x00007f8aa4e8e800 nid=0x6e93 waiting on condition [0x00007f8268b76000]
  975. java.lang.Thread.State: TIMED_WAITING (parking)
  976. at sun.misc.Unsafe.park(Native Method)
  977. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  978. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  979. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  980. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  981. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  982. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  983. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  984. at java.lang.Thread.run(Thread.java:745)
  985.  
  986. "qtp125683540-19" prio=10 tid=0x00007f8aa4e8c800 nid=0x6e92 waiting on condition [0x00007f8268bb7000]
  987. java.lang.Thread.State: TIMED_WAITING (parking)
  988. at sun.misc.Unsafe.park(Native Method)
  989. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  990. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  991. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  992. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  993. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  994. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  995. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  996. at java.lang.Thread.run(Thread.java:745)
  997.  
  998. "qtp125683540-18" prio=10 tid=0x00007f8aa4e8a800 nid=0x6e91 waiting on condition [0x00007f8268bf8000]
  999. java.lang.Thread.State: TIMED_WAITING (parking)
  1000. at sun.misc.Unsafe.park(Native Method)
  1001. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1002. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1003. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1004. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1005. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1006. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1007. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1008. at java.lang.Thread.run(Thread.java:745)
  1009.  
  1010. "qtp125683540-17" prio=10 tid=0x00007f8aa4e88800 nid=0x6e90 waiting on condition [0x00007f8268c39000]
  1011. java.lang.Thread.State: TIMED_WAITING (parking)
  1012. at sun.misc.Unsafe.park(Native Method)
  1013. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1014. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1015. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1016. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1017. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1018. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1019. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1020. at java.lang.Thread.run(Thread.java:745)
  1021.  
  1022. "qtp125683540-16" prio=10 tid=0x00007f8aa4e86800 nid=0x6e8f waiting on condition [0x00007f8268c7a000]
  1023. java.lang.Thread.State: TIMED_WAITING (parking)
  1024. at sun.misc.Unsafe.park(Native Method)
  1025. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1026. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1027. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1028. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1029. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1030. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1031. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1032. at java.lang.Thread.run(Thread.java:745)
  1033.  
  1034. "qtp125683540-15" prio=10 tid=0x00007f8aa4e84000 nid=0x6e8e waiting on condition [0x00007f8268cbb000]
  1035. java.lang.Thread.State: TIMED_WAITING (parking)
  1036. at sun.misc.Unsafe.park(Native Method)
  1037. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1038. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1039. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1040. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1041. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1042. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1043. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1044. at java.lang.Thread.run(Thread.java:745)
  1045.  
  1046. "qtp125683540-14" prio=10 tid=0x00007f8aa4e83000 nid=0x6e8d waiting on condition [0x00007f8268cfc000]
  1047. java.lang.Thread.State: TIMED_WAITING (parking)
  1048. at sun.misc.Unsafe.park(Native Method)
  1049. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1050. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1051. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1052. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1053. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1054. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1055. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1056. at java.lang.Thread.run(Thread.java:745)
  1057.  
  1058. "qtp125683540-13" prio=10 tid=0x00007f8aa4e81000 nid=0x6e8c runnable [0x00007f8268d3c000]
  1059. java.lang.Thread.State: RUNNABLE
  1060. at org.apache.lucene.index.FieldInfos$Builder.addOrUpdateInternal(FieldInfos.java:305)
  1061. at org.apache.lucene.index.FieldInfos$Builder.add(FieldInfos.java:338)
  1062. at org.apache.lucene.index.FieldInfos$Builder.add(FieldInfos.java:276)
  1063. at org.apache.lucene.index.MultiFields.getMergedFieldInfos(MultiFields.java:245)
  1064. at org.apache.lucene.index.SlowCompositeReaderWrapper.getFieldInfos(SlowCompositeReaderWrapper.java:239)
  1065. at org.apache.solr.highlight.DefaultSolrHighlighter.getSpanQueryScorer(DefaultSolrHighlighter.java:214)
  1066. at org.apache.solr.highlight.DefaultSolrHighlighter.getPhraseHighlighter(DefaultSolrHighlighter.java:176)
  1067. at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:504)
  1068. at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:397)
  1069. at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:143)
  1070. at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:230)
  1071. at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
  1072. at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)
  1073. at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:829)
  1074. at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:446)
  1075. at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
  1076. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
  1077. at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
  1078. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
  1079. at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
  1080. at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
  1081. at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
  1082. at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
  1083. at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
  1084. at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
  1085. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
  1086. at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
  1087. at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
  1088. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
  1089. at org.eclipse.jetty.server.Server.handle(Server.java:368)
  1090. at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
  1091. at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
  1092. at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
  1093. at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
  1094. at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
  1095. at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
  1096. at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
  1097. at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
  1098. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
  1099. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
  1100. at java.lang.Thread.run(Thread.java:745)
  1101.  
  1102. "qtp125683540-12" prio=10 tid=0x00007f8aa4e7f800 nid=0x6e8b waiting on condition [0x00007f8268d7e000]
  1103. java.lang.Thread.State: TIMED_WAITING (parking)
  1104. at sun.misc.Unsafe.park(Native Method)
  1105. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1106. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1107. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1108. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1109. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1110. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1111. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1112. at java.lang.Thread.run(Thread.java:745)
  1113.  
  1114. "qtp125683540-11 Acceptor0 [email protected]:8984" prio=10 tid=0x00007f8aa4e7a000 nid=0x6e8a runnable [0x00007f8268dbf000]
  1115. java.lang.Thread.State: RUNNABLE
  1116. at java.net.PlainSocketImpl.socketAccept(Native Method)
  1117. at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
  1118. at java.net.ServerSocket.implAccept(ServerSocket.java:530)
  1119. at java.net.ServerSocket.accept(ServerSocket.java:498)
  1120. at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
  1121. at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
  1122. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
  1123. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
  1124. at java.lang.Thread.run(Thread.java:745)
  1125.  
  1126. "ShutdownMonitor" daemon prio=10 tid=0x00007f8aa4e6f800 nid=0x6e89 runnable [0x00007f8268e00000]
  1127. java.lang.Thread.State: RUNNABLE
  1128. at java.net.PlainSocketImpl.socketAccept(Native Method)
  1129. at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
  1130. at java.net.ServerSocket.implAccept(ServerSocket.java:530)
  1131. at java.net.ServerSocket.accept(ServerSocket.java:498)
  1132. at org.eclipse.jetty.server.ShutdownMonitor.run(ShutdownMonitor.java:165)
  1133.  
  1134. "Service Thread" daemon prio=10 tid=0x00007f8aa4143800 nid=0x6e7d runnable [0x0000000000000000]
  1135. java.lang.Thread.State: RUNNABLE
  1136.  
  1137. "C2 CompilerThread1" daemon prio=10 tid=0x00007f8aa4141000 nid=0x6e7c waiting on condition [0x0000000000000000]
  1138. java.lang.Thread.State: RUNNABLE
  1139.  
  1140. "C2 CompilerThread0" daemon prio=10 tid=0x00007f8aa413e800 nid=0x6e7b waiting on condition [0x0000000000000000]
  1141. java.lang.Thread.State: RUNNABLE
  1142.  
  1143. "Signal Dispatcher" daemon prio=10 tid=0x00007f8aa413c800 nid=0x6e7a runnable [0x0000000000000000]
  1144. java.lang.Thread.State: RUNNABLE
  1145.  
  1146. "Surrogate Locker Thread (Concurrent GC)" daemon prio=10 tid=0x00007f8aa413a800 nid=0x6e79 waiting on condition [0x0000000000000000]
  1147. java.lang.Thread.State: RUNNABLE
  1148.  
  1149. "Finalizer" daemon prio=10 tid=0x00007f8aa4119800 nid=0x6e78 in Object.wait() [0x00007f828808e000]
  1150. java.lang.Thread.State: WAITING (on object monitor)
  1151. at java.lang.Object.wait(Native Method)
  1152. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
  1153. - locked <0x00007f848f39db00> (a java.lang.ref.ReferenceQueue$Lock)
  1154. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
  1155. at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
  1156.  
  1157. "Reference Handler" daemon prio=10 tid=0x00007f8aa4117800 nid=0x6e77 in Object.wait() [0x00007f82880cf000]
  1158. java.lang.Thread.State: WAITING (on object monitor)
  1159. at java.lang.Object.wait(Native Method)
  1160. at java.lang.Object.wait(Object.java:503)
  1161. at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
  1162. - locked <0x00007f848f39db48> (a java.lang.ref.Reference$Lock)
  1163.  
  1164. "VM Thread" prio=10 tid=0x00007f8aa4113000 nid=0x6e76 runnable
  1165.  
  1166. "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401b000 nid=0x6e6b runnable
  1167.  
  1168. "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401d000 nid=0x6e6c runnable
  1169.  
  1170. "Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401e800 nid=0x6e6d runnable
  1171.  
  1172. "Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x00007f8aa4020800 nid=0x6e6e runnable
  1173.  
  1174. "Concurrent Mark-Sweep GC Thread" prio=10 tid=0x00007f8aa40a8800 nid=0x6e75 runnable
  1175. "Gang worker#0 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a1000 nid=0x6e71 runnable
  1176.  
  1177. "Gang worker#1 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a2800 nid=0x6e72 runnable
  1178.  
  1179. "Gang worker#2 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a4800 nid=0x6e73 runnable
  1180.  
  1181. "Gang worker#3 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a6800 nid=0x6e74 runnable
  1182.  
  1183. "VM Periodic Task Thread" prio=10 tid=0x00007f8aa414e800 nid=0x6e7e waiting on condition
  1184.  
  1185. JNI global references: 344
  1186.  
  1187. [solr@finna-dev-fe ~]$ jstack 28259
  1188. 2015-05-05 10:54:35
  1189. Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.75-b04 mixed mode):
  1190.  
  1191. "Attach Listener" daemon prio=10 tid=0x00007f8238002800 nid=0x200a waiting on condition [0x0000000000000000]
  1192. java.lang.Thread.State: RUNNABLE
  1193.  
  1194. "commitScheduler-13-thread-1" prio=10 tid=0x00007f820c014800 nid=0x4973 waiting on condition [0x00007f8268175000]
  1195. java.lang.Thread.State: WAITING (parking)
  1196. at sun.misc.Unsafe.park(Native Method)
  1197. - parking to wait for <0x00007f8636e55110> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1198. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  1199. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  1200. at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
  1201. at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
  1202. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  1203. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  1204. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  1205. at java.lang.Thread.run(Thread.java:745)
  1206.  
  1207. "searcherExecutor-11-thread-1" prio=10 tid=0x00007f8200129000 nid=0x48b0 waiting on condition [0x00007f823c397000]
  1208. java.lang.Thread.State: WAITING (parking)
  1209. at sun.misc.Unsafe.park(Native Method)
  1210. - parking to wait for <0x00007f8635fd1208> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1211. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  1212. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  1213. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  1214. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  1215. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  1216. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  1217. at java.lang.Thread.run(Thread.java:745)
  1218.  
  1219. "DestroyJavaVM" prio=10 tid=0x00007f8aa4009800 nid=0x6e6a waiting on condition [0x0000000000000000]
  1220. java.lang.Thread.State: RUNNABLE
  1221.  
  1222. "coreZkRegister-1-thread-1" prio=10 tid=0x00007f8aa52c0800 nid=0x6ee7 waiting on condition [0x00007f8268071000]
  1223. java.lang.Thread.State: WAITING (parking)
  1224. at sun.misc.Unsafe.park(Native Method)
  1225. - parking to wait for <0x00007f848ee2da50> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1226. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  1227. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  1228. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  1229. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  1230. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  1231. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  1232. at java.lang.Thread.run(Thread.java:745)
  1233.  
  1234. "Thread-14" prio=10 tid=0x00007f8aa52bf000 nid=0x6ee6 in Object.wait() [0x00007f82680b2000]
  1235. java.lang.Thread.State: WAITING (on object monitor)
  1236. at java.lang.Object.wait(Native Method)
  1237. - waiting on <0x00007f848ee26aa8> (a java.lang.Object)
  1238. at java.lang.Object.wait(Object.java:503)
  1239. at org.apache.solr.core.CloserThread.run(CoreContainer.java:907)
  1240. - locked <0x00007f848ee26aa8> (a java.lang.Object)
  1241.  
  1242. "main-EventThread" daemon prio=10 tid=0x00007f8aa523a800 nid=0x6ec4 waiting on condition [0x00007f82681b6000]
  1243. java.lang.Thread.State: WAITING (parking)
  1244. at sun.misc.Unsafe.park(Native Method)
  1245. - parking to wait for <0x00007f848ee2d678> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1246. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  1247. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  1248. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  1249. at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
  1250.  
  1251. "main-SendThread(finna-dev-fe.csc.fi:2181)" daemon prio=10 tid=0x00007f8aa5260800 nid=0x6ec3 runnable [0x00007f82681f7000]
  1252. java.lang.Thread.State: RUNNABLE
  1253. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1254. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
  1255. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
  1256. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
  1257. - locked <0x00007f848ee304b0> (a sun.nio.ch.Util$2)
  1258. - locked <0x00007f848ee304c8> (a java.util.Collections$UnmodifiableSet)
  1259. - locked <0x00007f848ee37250> (a sun.nio.ch.EPollSelectorImpl)
  1260. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
  1261. at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
  1262. at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
  1263.  
  1264. "HashSessionScavenger-0" daemon prio=10 tid=0x00007f8aa5048000 nid=0x6ebb in Object.wait() [0x00007f826889c000]
  1265. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  1266. at java.lang.Object.wait(Native Method)
  1267. at java.util.TimerThread.mainLoop(Timer.java:552)
  1268. - locked <0x00007f848ee37c50> (a java.util.TaskQueue)
  1269. at java.util.TimerThread.run(Timer.java:505)
  1270.  
  1271. "qtp125683540-20" prio=10 tid=0x00007f8aa4e8e800 nid=0x6e93 waiting on condition [0x00007f8268b76000]
  1272. java.lang.Thread.State: TIMED_WAITING (parking)
  1273. at sun.misc.Unsafe.park(Native Method)
  1274. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1275. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1276. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1277. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1278. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1279. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1280. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1281. at java.lang.Thread.run(Thread.java:745)
  1282.  
  1283. "qtp125683540-19" prio=10 tid=0x00007f8aa4e8c800 nid=0x6e92 waiting on condition [0x00007f8268bb7000]
  1284. java.lang.Thread.State: TIMED_WAITING (parking)
  1285. at sun.misc.Unsafe.park(Native Method)
  1286. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1287. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1288. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1289. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1290. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1291. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1292. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1293. at java.lang.Thread.run(Thread.java:745)
  1294.  
  1295. "qtp125683540-18" prio=10 tid=0x00007f8aa4e8a800 nid=0x6e91 waiting on condition [0x00007f8268bf8000]
  1296. java.lang.Thread.State: TIMED_WAITING (parking)
  1297. at sun.misc.Unsafe.park(Native Method)
  1298. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1299. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1300. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1301. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1302. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1303. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1304. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1305. at java.lang.Thread.run(Thread.java:745)
  1306.  
  1307. "qtp125683540-17" prio=10 tid=0x00007f8aa4e88800 nid=0x6e90 waiting on condition [0x00007f8268c39000]
  1308. java.lang.Thread.State: TIMED_WAITING (parking)
  1309. at sun.misc.Unsafe.park(Native Method)
  1310. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1311. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1312. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1313. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1314. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1315. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1316. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1317. at java.lang.Thread.run(Thread.java:745)
  1318.  
  1319. "qtp125683540-16" prio=10 tid=0x00007f8aa4e86800 nid=0x6e8f waiting on condition [0x00007f8268c7a000]
  1320. java.lang.Thread.State: TIMED_WAITING (parking)
  1321. at sun.misc.Unsafe.park(Native Method)
  1322. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1323. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1324. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1325. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1326. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1327. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1328. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1329. at java.lang.Thread.run(Thread.java:745)
  1330.  
  1331. "qtp125683540-15" prio=10 tid=0x00007f8aa4e84000 nid=0x6e8e waiting on condition [0x00007f8268cbb000]
  1332. java.lang.Thread.State: TIMED_WAITING (parking)
  1333. at sun.misc.Unsafe.park(Native Method)
  1334. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1335. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1336. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1337. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1338. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1339. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1340. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1341. at java.lang.Thread.run(Thread.java:745)
  1342.  
  1343. "qtp125683540-14" prio=10 tid=0x00007f8aa4e83000 nid=0x6e8d waiting on condition [0x00007f8268cfc000]
  1344. java.lang.Thread.State: TIMED_WAITING (parking)
  1345. at sun.misc.Unsafe.park(Native Method)
  1346. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1347. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1348. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1349. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1350. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1351. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1352. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1353. at java.lang.Thread.run(Thread.java:745)
  1354.  
  1355. "qtp125683540-13" prio=10 tid=0x00007f8aa4e81000 nid=0x6e8c runnable [0x00007f8268d3c000]
  1356. java.lang.Thread.State: RUNNABLE
  1357. at org.apache.lucene.index.FieldInfos$Builder.addOrUpdateInternal(FieldInfos.java:305)
  1358. at org.apache.lucene.index.FieldInfos$Builder.add(FieldInfos.java:338)
  1359. at org.apache.lucene.index.FieldInfos$Builder.add(FieldInfos.java:276)
  1360. at org.apache.lucene.index.MultiFields.getMergedFieldInfos(MultiFields.java:245)
  1361. at org.apache.lucene.index.SlowCompositeReaderWrapper.getFieldInfos(SlowCompositeReaderWrapper.java:239)
  1362. at org.apache.solr.highlight.DefaultSolrHighlighter.getSpanQueryScorer(DefaultSolrHighlighter.java:214)
  1363. at org.apache.solr.highlight.DefaultSolrHighlighter.getPhraseHighlighter(DefaultSolrHighlighter.java:176)
  1364. at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:504)
  1365. at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:397)
  1366. at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:143)
  1367. at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:230)
  1368. at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
  1369. at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)
  1370. at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:829)
  1371. at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:446)
  1372. at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
  1373. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
  1374. at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
  1375. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
  1376. at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
  1377. at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
  1378. at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
  1379. at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
  1380. at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
  1381. at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
  1382. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
  1383. at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
  1384. at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
  1385. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
  1386. at org.eclipse.jetty.server.Server.handle(Server.java:368)
  1387. at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
  1388. at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
  1389. at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
  1390. at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
  1391. at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
  1392. at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
  1393. at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
  1394. at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
  1395. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
  1396. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
  1397. at java.lang.Thread.run(Thread.java:745)
  1398.  
  1399. "qtp125683540-12" prio=10 tid=0x00007f8aa4e7f800 nid=0x6e8b waiting on condition [0x00007f8268d7e000]
  1400. java.lang.Thread.State: TIMED_WAITING (parking)
  1401. at sun.misc.Unsafe.park(Native Method)
  1402. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1403. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1404. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1405. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1406. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1407. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1408. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1409. at java.lang.Thread.run(Thread.java:745)
  1410.  
  1411. "qtp125683540-11 Acceptor0 [email protected]:8984" prio=10 tid=0x00007f8aa4e7a000 nid=0x6e8a runnable [0x00007f8268dbf000]
  1412. java.lang.Thread.State: RUNNABLE
  1413. at java.net.PlainSocketImpl.socketAccept(Native Method)
  1414. at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
  1415. at java.net.ServerSocket.implAccept(ServerSocket.java:530)
  1416. at java.net.ServerSocket.accept(ServerSocket.java:498)
  1417. at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
  1418. at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
  1419. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
  1420. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
  1421. at java.lang.Thread.run(Thread.java:745)
  1422.  
  1423. "ShutdownMonitor" daemon prio=10 tid=0x00007f8aa4e6f800 nid=0x6e89 runnable [0x00007f8268e00000]
  1424. java.lang.Thread.State: RUNNABLE
  1425. at java.net.PlainSocketImpl.socketAccept(Native Method)
  1426. at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
  1427. at java.net.ServerSocket.implAccept(ServerSocket.java:530)
  1428. at java.net.ServerSocket.accept(ServerSocket.java:498)
  1429. at org.eclipse.jetty.server.ShutdownMonitor.run(ShutdownMonitor.java:165)
  1430.  
  1431. "Service Thread" daemon prio=10 tid=0x00007f8aa4143800 nid=0x6e7d runnable [0x0000000000000000]
  1432. java.lang.Thread.State: RUNNABLE
  1433.  
  1434. "C2 CompilerThread1" daemon prio=10 tid=0x00007f8aa4141000 nid=0x6e7c waiting on condition [0x0000000000000000]
  1435. java.lang.Thread.State: RUNNABLE
  1436.  
  1437. "C2 CompilerThread0" daemon prio=10 tid=0x00007f8aa413e800 nid=0x6e7b waiting on condition [0x0000000000000000]
  1438. java.lang.Thread.State: RUNNABLE
  1439.  
  1440. "Signal Dispatcher" daemon prio=10 tid=0x00007f8aa413c800 nid=0x6e7a runnable [0x0000000000000000]
  1441. java.lang.Thread.State: RUNNABLE
  1442.  
  1443. "Surrogate Locker Thread (Concurrent GC)" daemon prio=10 tid=0x00007f8aa413a800 nid=0x6e79 waiting on condition [0x0000000000000000]
  1444. java.lang.Thread.State: RUNNABLE
  1445.  
  1446. "Finalizer" daemon prio=10 tid=0x00007f8aa4119800 nid=0x6e78 in Object.wait() [0x00007f828808e000]
  1447. java.lang.Thread.State: WAITING (on object monitor)
  1448. at java.lang.Object.wait(Native Method)
  1449. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
  1450. - locked <0x00007f848f39db00> (a java.lang.ref.ReferenceQueue$Lock)
  1451. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
  1452. at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
  1453.  
  1454. "Reference Handler" daemon prio=10 tid=0x00007f8aa4117800 nid=0x6e77 in Object.wait() [0x00007f82880cf000]
  1455. java.lang.Thread.State: WAITING (on object monitor)
  1456. at java.lang.Object.wait(Native Method)
  1457. at java.lang.Object.wait(Object.java:503)
  1458. at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
  1459. - locked <0x00007f848f39db48> (a java.lang.ref.Reference$Lock)
  1460.  
  1461. "VM Thread" prio=10 tid=0x00007f8aa4113000 nid=0x6e76 runnable
  1462.  
  1463. "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401b000 nid=0x6e6b runnable
  1464.  
  1465. "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401d000 nid=0x6e6c runnable
  1466.  
  1467. "Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401e800 nid=0x6e6d runnable
  1468.  
  1469. "Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x00007f8aa4020800 nid=0x6e6e runnable
  1470.  
  1471. "Concurrent Mark-Sweep GC Thread" prio=10 tid=0x00007f8aa40a8800 nid=0x6e75 runnable
  1472. "Gang worker#0 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a1000 nid=0x6e71 runnable
  1473.  
  1474. "Gang worker#1 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a2800 nid=0x6e72 runnable
  1475.  
  1476. "Gang worker#2 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a4800 nid=0x6e73 runnable
  1477.  
  1478. "Gang worker#3 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a6800 nid=0x6e74 runnable
  1479.  
  1480. "VM Periodic Task Thread" prio=10 tid=0x00007f8aa414e800 nid=0x6e7e waiting on condition
  1481.  
  1482. JNI global references: 344
  1483.  
  1484. [solr@finna-dev-fe ~]$ jstack 28259
  1485. 2015-05-05 10:54:36
  1486. Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.75-b04 mixed mode):
  1487.  
  1488. "Attach Listener" daemon prio=10 tid=0x00007f8238002800 nid=0x200a waiting on condition [0x0000000000000000]
  1489. java.lang.Thread.State: RUNNABLE
  1490.  
  1491. "commitScheduler-13-thread-1" prio=10 tid=0x00007f820c014800 nid=0x4973 waiting on condition [0x00007f8268175000]
  1492. java.lang.Thread.State: WAITING (parking)
  1493. at sun.misc.Unsafe.park(Native Method)
  1494. - parking to wait for <0x00007f8636e55110> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1495. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  1496. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  1497. at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
  1498. at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
  1499. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  1500. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  1501. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  1502. at java.lang.Thread.run(Thread.java:745)
  1503.  
  1504. "searcherExecutor-11-thread-1" prio=10 tid=0x00007f8200129000 nid=0x48b0 waiting on condition [0x00007f823c397000]
  1505. java.lang.Thread.State: WAITING (parking)
  1506. at sun.misc.Unsafe.park(Native Method)
  1507. - parking to wait for <0x00007f8635fd1208> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1508. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  1509. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  1510. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  1511. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  1512. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  1513. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  1514. at java.lang.Thread.run(Thread.java:745)
  1515.  
  1516. "DestroyJavaVM" prio=10 tid=0x00007f8aa4009800 nid=0x6e6a waiting on condition [0x0000000000000000]
  1517. java.lang.Thread.State: RUNNABLE
  1518.  
  1519. "coreZkRegister-1-thread-1" prio=10 tid=0x00007f8aa52c0800 nid=0x6ee7 waiting on condition [0x00007f8268071000]
  1520. java.lang.Thread.State: WAITING (parking)
  1521. at sun.misc.Unsafe.park(Native Method)
  1522. - parking to wait for <0x00007f848ee2da50> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1523. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  1524. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  1525. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  1526. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  1527. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  1528. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  1529. at java.lang.Thread.run(Thread.java:745)
  1530.  
  1531. "Thread-14" prio=10 tid=0x00007f8aa52bf000 nid=0x6ee6 in Object.wait() [0x00007f82680b2000]
  1532. java.lang.Thread.State: WAITING (on object monitor)
  1533. at java.lang.Object.wait(Native Method)
  1534. - waiting on <0x00007f848ee26aa8> (a java.lang.Object)
  1535. at java.lang.Object.wait(Object.java:503)
  1536. at org.apache.solr.core.CloserThread.run(CoreContainer.java:907)
  1537. - locked <0x00007f848ee26aa8> (a java.lang.Object)
  1538.  
  1539. "main-EventThread" daemon prio=10 tid=0x00007f8aa523a800 nid=0x6ec4 waiting on condition [0x00007f82681b6000]
  1540. java.lang.Thread.State: WAITING (parking)
  1541. at sun.misc.Unsafe.park(Native Method)
  1542. - parking to wait for <0x00007f848ee2d678> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1543. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  1544. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  1545. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  1546. at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
  1547.  
  1548. "main-SendThread(finna-dev-fe.csc.fi:2181)" daemon prio=10 tid=0x00007f8aa5260800 nid=0x6ec3 runnable [0x00007f82681f7000]
  1549. java.lang.Thread.State: RUNNABLE
  1550. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1551. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
  1552. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
  1553. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
  1554. - locked <0x00007f848ee304b0> (a sun.nio.ch.Util$2)
  1555. - locked <0x00007f848ee304c8> (a java.util.Collections$UnmodifiableSet)
  1556. - locked <0x00007f848ee37250> (a sun.nio.ch.EPollSelectorImpl)
  1557. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
  1558. at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
  1559. at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
  1560.  
  1561. "HashSessionScavenger-0" daemon prio=10 tid=0x00007f8aa5048000 nid=0x6ebb in Object.wait() [0x00007f826889c000]
  1562. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  1563. at java.lang.Object.wait(Native Method)
  1564. at java.util.TimerThread.mainLoop(Timer.java:552)
  1565. - locked <0x00007f848ee37c50> (a java.util.TaskQueue)
  1566. at java.util.TimerThread.run(Timer.java:505)
  1567.  
  1568. "qtp125683540-20" prio=10 tid=0x00007f8aa4e8e800 nid=0x6e93 waiting on condition [0x00007f8268b76000]
  1569. java.lang.Thread.State: TIMED_WAITING (parking)
  1570. at sun.misc.Unsafe.park(Native Method)
  1571. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1572. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1573. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1574. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1575. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1576. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1577. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1578. at java.lang.Thread.run(Thread.java:745)
  1579.  
  1580. "qtp125683540-19" prio=10 tid=0x00007f8aa4e8c800 nid=0x6e92 waiting on condition [0x00007f8268bb7000]
  1581. java.lang.Thread.State: TIMED_WAITING (parking)
  1582. at sun.misc.Unsafe.park(Native Method)
  1583. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1584. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1585. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1586. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1587. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1588. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1589. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1590. at java.lang.Thread.run(Thread.java:745)
  1591.  
  1592. "qtp125683540-18" prio=10 tid=0x00007f8aa4e8a800 nid=0x6e91 waiting on condition [0x00007f8268bf8000]
  1593. java.lang.Thread.State: TIMED_WAITING (parking)
  1594. at sun.misc.Unsafe.park(Native Method)
  1595. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1596. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1597. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1598. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1599. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1600. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1601. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1602. at java.lang.Thread.run(Thread.java:745)
  1603.  
  1604. "qtp125683540-17" prio=10 tid=0x00007f8aa4e88800 nid=0x6e90 waiting on condition [0x00007f8268c39000]
  1605. java.lang.Thread.State: TIMED_WAITING (parking)
  1606. at sun.misc.Unsafe.park(Native Method)
  1607. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1608. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1609. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1610. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1611. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1612. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1613. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1614. at java.lang.Thread.run(Thread.java:745)
  1615.  
  1616. "qtp125683540-16" prio=10 tid=0x00007f8aa4e86800 nid=0x6e8f waiting on condition [0x00007f8268c7a000]
  1617. java.lang.Thread.State: TIMED_WAITING (parking)
  1618. at sun.misc.Unsafe.park(Native Method)
  1619. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1620. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1621. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1622. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1623. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1624. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1625. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1626. at java.lang.Thread.run(Thread.java:745)
  1627.  
  1628. "qtp125683540-15" prio=10 tid=0x00007f8aa4e84000 nid=0x6e8e waiting on condition [0x00007f8268cbb000]
  1629. java.lang.Thread.State: TIMED_WAITING (parking)
  1630. at sun.misc.Unsafe.park(Native Method)
  1631. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1632. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1633. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1634. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1635. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1636. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1637. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1638. at java.lang.Thread.run(Thread.java:745)
  1639.  
  1640. "qtp125683540-14" prio=10 tid=0x00007f8aa4e83000 nid=0x6e8d waiting on condition [0x00007f8268cfc000]
  1641. java.lang.Thread.State: TIMED_WAITING (parking)
  1642. at sun.misc.Unsafe.park(Native Method)
  1643. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1644. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1645. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1646. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1647. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1648. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1649. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1650. at java.lang.Thread.run(Thread.java:745)
  1651.  
  1652. "qtp125683540-13" prio=10 tid=0x00007f8aa4e81000 nid=0x6e8c runnable [0x00007f8268d3c000]
  1653. java.lang.Thread.State: RUNNABLE
  1654. at org.apache.lucene.index.FieldInfos$Builder.addOrUpdateInternal(FieldInfos.java:305)
  1655. at org.apache.lucene.index.FieldInfos$Builder.add(FieldInfos.java:338)
  1656. at org.apache.lucene.index.FieldInfos$Builder.add(FieldInfos.java:276)
  1657. at org.apache.lucene.index.MultiFields.getMergedFieldInfos(MultiFields.java:245)
  1658. at org.apache.lucene.index.SlowCompositeReaderWrapper.getFieldInfos(SlowCompositeReaderWrapper.java:239)
  1659. at org.apache.solr.highlight.DefaultSolrHighlighter.getSpanQueryScorer(DefaultSolrHighlighter.java:214)
  1660. at org.apache.solr.highlight.DefaultSolrHighlighter.getPhraseHighlighter(DefaultSolrHighlighter.java:176)
  1661. at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:504)
  1662. at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:397)
  1663. at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:143)
  1664. at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:230)
  1665. at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
  1666. at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)
  1667. at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:829)
  1668. at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:446)
  1669. at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
  1670. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
  1671. at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
  1672. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
  1673. at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
  1674. at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
  1675. at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
  1676. at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
  1677. at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
  1678. at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
  1679. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
  1680. at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
  1681. at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
  1682. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
  1683. at org.eclipse.jetty.server.Server.handle(Server.java:368)
  1684. at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
  1685. at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
  1686. at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
  1687. at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
  1688. at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
  1689. at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
  1690. at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
  1691. at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
  1692. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
  1693. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
  1694. at java.lang.Thread.run(Thread.java:745)
  1695.  
  1696. "qtp125683540-12" prio=10 tid=0x00007f8aa4e7f800 nid=0x6e8b waiting on condition [0x00007f8268d7e000]
  1697. java.lang.Thread.State: TIMED_WAITING (parking)
  1698. at sun.misc.Unsafe.park(Native Method)
  1699. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1700. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1701. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1702. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1703. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1704. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1705. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1706. at java.lang.Thread.run(Thread.java:745)
  1707.  
  1708. "qtp125683540-11 Acceptor0 [email protected]:8984" prio=10 tid=0x00007f8aa4e7a000 nid=0x6e8a runnable [0x00007f8268dbf000]
  1709. java.lang.Thread.State: RUNNABLE
  1710. at java.net.PlainSocketImpl.socketAccept(Native Method)
  1711. at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
  1712. at java.net.ServerSocket.implAccept(ServerSocket.java:530)
  1713. at java.net.ServerSocket.accept(ServerSocket.java:498)
  1714. at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
  1715. at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
  1716. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
  1717. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
  1718. at java.lang.Thread.run(Thread.java:745)
  1719.  
  1720. "ShutdownMonitor" daemon prio=10 tid=0x00007f8aa4e6f800 nid=0x6e89 runnable [0x00007f8268e00000]
  1721. java.lang.Thread.State: RUNNABLE
  1722. at java.net.PlainSocketImpl.socketAccept(Native Method)
  1723. at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
  1724. at java.net.ServerSocket.implAccept(ServerSocket.java:530)
  1725. at java.net.ServerSocket.accept(ServerSocket.java:498)
  1726. at org.eclipse.jetty.server.ShutdownMonitor.run(ShutdownMonitor.java:165)
  1727.  
  1728. "Service Thread" daemon prio=10 tid=0x00007f8aa4143800 nid=0x6e7d runnable [0x0000000000000000]
  1729. java.lang.Thread.State: RUNNABLE
  1730.  
  1731. "C2 CompilerThread1" daemon prio=10 tid=0x00007f8aa4141000 nid=0x6e7c waiting on condition [0x0000000000000000]
  1732. java.lang.Thread.State: RUNNABLE
  1733.  
  1734. "C2 CompilerThread0" daemon prio=10 tid=0x00007f8aa413e800 nid=0x6e7b waiting on condition [0x0000000000000000]
  1735. java.lang.Thread.State: RUNNABLE
  1736.  
  1737. "Signal Dispatcher" daemon prio=10 tid=0x00007f8aa413c800 nid=0x6e7a runnable [0x0000000000000000]
  1738. java.lang.Thread.State: RUNNABLE
  1739.  
  1740. "Surrogate Locker Thread (Concurrent GC)" daemon prio=10 tid=0x00007f8aa413a800 nid=0x6e79 waiting on condition [0x0000000000000000]
  1741. java.lang.Thread.State: RUNNABLE
  1742.  
  1743. "Finalizer" daemon prio=10 tid=0x00007f8aa4119800 nid=0x6e78 in Object.wait() [0x00007f828808e000]
  1744. java.lang.Thread.State: WAITING (on object monitor)
  1745. at java.lang.Object.wait(Native Method)
  1746. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
  1747. - locked <0x00007f848f39db00> (a java.lang.ref.ReferenceQueue$Lock)
  1748. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
  1749. at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
  1750.  
  1751. "Reference Handler" daemon prio=10 tid=0x00007f8aa4117800 nid=0x6e77 in Object.wait() [0x00007f82880cf000]
  1752. java.lang.Thread.State: WAITING (on object monitor)
  1753. at java.lang.Object.wait(Native Method)
  1754. at java.lang.Object.wait(Object.java:503)
  1755. at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
  1756. - locked <0x00007f848f39db48> (a java.lang.ref.Reference$Lock)
  1757.  
  1758. "VM Thread" prio=10 tid=0x00007f8aa4113000 nid=0x6e76 runnable
  1759.  
  1760. "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401b000 nid=0x6e6b runnable
  1761.  
  1762. "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401d000 nid=0x6e6c runnable
  1763.  
  1764. "Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401e800 nid=0x6e6d runnable
  1765.  
  1766. "Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x00007f8aa4020800 nid=0x6e6e runnable
  1767.  
  1768. "Concurrent Mark-Sweep GC Thread" prio=10 tid=0x00007f8aa40a8800 nid=0x6e75 runnable
  1769. "Gang worker#0 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a1000 nid=0x6e71 runnable
  1770.  
  1771. "Gang worker#1 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a2800 nid=0x6e72 runnable
  1772.  
  1773. "Gang worker#2 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a4800 nid=0x6e73 runnable
  1774.  
  1775. "Gang worker#3 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a6800 nid=0x6e74 runnable
  1776.  
  1777. "VM Periodic Task Thread" prio=10 tid=0x00007f8aa414e800 nid=0x6e7e waiting on condition
  1778.  
  1779. JNI global references: 344
  1780.  
  1781. [solr@finna-dev-fe ~]$ jstack 28259
  1782. 2015-05-05 10:54:37
  1783. Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.75-b04 mixed mode):
  1784.  
  1785. "Attach Listener" daemon prio=10 tid=0x00007f8238002800 nid=0x200a waiting on condition [0x0000000000000000]
  1786. java.lang.Thread.State: RUNNABLE
  1787.  
  1788. "commitScheduler-13-thread-1" prio=10 tid=0x00007f820c014800 nid=0x4973 waiting on condition [0x00007f8268175000]
  1789. java.lang.Thread.State: WAITING (parking)
  1790. at sun.misc.Unsafe.park(Native Method)
  1791. - parking to wait for <0x00007f8636e55110> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1792. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  1793. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  1794. at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
  1795. at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
  1796. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  1797. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  1798. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  1799. at java.lang.Thread.run(Thread.java:745)
  1800.  
  1801. "searcherExecutor-11-thread-1" prio=10 tid=0x00007f8200129000 nid=0x48b0 waiting on condition [0x00007f823c397000]
  1802. java.lang.Thread.State: WAITING (parking)
  1803. at sun.misc.Unsafe.park(Native Method)
  1804. - parking to wait for <0x00007f8635fd1208> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1805. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  1806. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  1807. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  1808. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  1809. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  1810. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  1811. at java.lang.Thread.run(Thread.java:745)
  1812.  
  1813. "DestroyJavaVM" prio=10 tid=0x00007f8aa4009800 nid=0x6e6a waiting on condition [0x0000000000000000]
  1814. java.lang.Thread.State: RUNNABLE
  1815.  
  1816. "coreZkRegister-1-thread-1" prio=10 tid=0x00007f8aa52c0800 nid=0x6ee7 waiting on condition [0x00007f8268071000]
  1817. java.lang.Thread.State: WAITING (parking)
  1818. at sun.misc.Unsafe.park(Native Method)
  1819. - parking to wait for <0x00007f848ee2da50> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1820. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  1821. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  1822. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  1823. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  1824. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  1825. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  1826. at java.lang.Thread.run(Thread.java:745)
  1827.  
  1828. "Thread-14" prio=10 tid=0x00007f8aa52bf000 nid=0x6ee6 in Object.wait() [0x00007f82680b2000]
  1829. java.lang.Thread.State: WAITING (on object monitor)
  1830. at java.lang.Object.wait(Native Method)
  1831. - waiting on <0x00007f848ee26aa8> (a java.lang.Object)
  1832. at java.lang.Object.wait(Object.java:503)
  1833. at org.apache.solr.core.CloserThread.run(CoreContainer.java:907)
  1834. - locked <0x00007f848ee26aa8> (a java.lang.Object)
  1835.  
  1836. "main-EventThread" daemon prio=10 tid=0x00007f8aa523a800 nid=0x6ec4 waiting on condition [0x00007f82681b6000]
  1837. java.lang.Thread.State: WAITING (parking)
  1838. at sun.misc.Unsafe.park(Native Method)
  1839. - parking to wait for <0x00007f848ee2d678> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1840. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  1841. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  1842. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  1843. at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
  1844.  
  1845. "main-SendThread(finna-dev-fe.csc.fi:2181)" daemon prio=10 tid=0x00007f8aa5260800 nid=0x6ec3 runnable [0x00007f82681f7000]
  1846. java.lang.Thread.State: RUNNABLE
  1847. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1848. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
  1849. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
  1850. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
  1851. - locked <0x00007f848ee304b0> (a sun.nio.ch.Util$2)
  1852. - locked <0x00007f848ee304c8> (a java.util.Collections$UnmodifiableSet)
  1853. - locked <0x00007f848ee37250> (a sun.nio.ch.EPollSelectorImpl)
  1854. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
  1855. at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
  1856. at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
  1857.  
  1858. "HashSessionScavenger-0" daemon prio=10 tid=0x00007f8aa5048000 nid=0x6ebb in Object.wait() [0x00007f826889c000]
  1859. java.lang.Thread.State: TIMED_WAITING (on object monitor)
  1860. at java.lang.Object.wait(Native Method)
  1861. at java.util.TimerThread.mainLoop(Timer.java:552)
  1862. - locked <0x00007f848ee37c50> (a java.util.TaskQueue)
  1863. at java.util.TimerThread.run(Timer.java:505)
  1864.  
  1865. "qtp125683540-20" prio=10 tid=0x00007f8aa4e8e800 nid=0x6e93 waiting on condition [0x00007f8268b76000]
  1866. java.lang.Thread.State: TIMED_WAITING (parking)
  1867. at sun.misc.Unsafe.park(Native Method)
  1868. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1869. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1870. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1871. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1872. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1873. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1874. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1875. at java.lang.Thread.run(Thread.java:745)
  1876.  
  1877. "qtp125683540-19" prio=10 tid=0x00007f8aa4e8c800 nid=0x6e92 waiting on condition [0x00007f8268bb7000]
  1878. java.lang.Thread.State: TIMED_WAITING (parking)
  1879. at sun.misc.Unsafe.park(Native Method)
  1880. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1881. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1882. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1883. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1884. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1885. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1886. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1887. at java.lang.Thread.run(Thread.java:745)
  1888.  
  1889. "qtp125683540-18" prio=10 tid=0x00007f8aa4e8a800 nid=0x6e91 waiting on condition [0x00007f8268bf8000]
  1890. java.lang.Thread.State: TIMED_WAITING (parking)
  1891. at sun.misc.Unsafe.park(Native Method)
  1892. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1893. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1894. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1895. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1896. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1897. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1898. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1899. at java.lang.Thread.run(Thread.java:745)
  1900.  
  1901. "qtp125683540-17" prio=10 tid=0x00007f8aa4e88800 nid=0x6e90 waiting on condition [0x00007f8268c39000]
  1902. java.lang.Thread.State: TIMED_WAITING (parking)
  1903. at sun.misc.Unsafe.park(Native Method)
  1904. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1905. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1906. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1907. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1908. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1909. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1910. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1911. at java.lang.Thread.run(Thread.java:745)
  1912.  
  1913. "qtp125683540-16" prio=10 tid=0x00007f8aa4e86800 nid=0x6e8f waiting on condition [0x00007f8268c7a000]
  1914. java.lang.Thread.State: TIMED_WAITING (parking)
  1915. at sun.misc.Unsafe.park(Native Method)
  1916. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1917. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1918. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1919. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1920. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1921. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1922. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1923. at java.lang.Thread.run(Thread.java:745)
  1924.  
  1925. "qtp125683540-15" prio=10 tid=0x00007f8aa4e84000 nid=0x6e8e waiting on condition [0x00007f8268cbb000]
  1926. java.lang.Thread.State: TIMED_WAITING (parking)
  1927. at sun.misc.Unsafe.park(Native Method)
  1928. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1929. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1930. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1931. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1932. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1933. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1934. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1935. at java.lang.Thread.run(Thread.java:745)
  1936.  
  1937. "qtp125683540-14" prio=10 tid=0x00007f8aa4e83000 nid=0x6e8d waiting on condition [0x00007f8268cfc000]
  1938. java.lang.Thread.State: TIMED_WAITING (parking)
  1939. at sun.misc.Unsafe.park(Native Method)
  1940. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  1941. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  1942. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  1943. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  1944. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  1945. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  1946. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1947. at java.lang.Thread.run(Thread.java:745)
  1948.  
  1949. "qtp125683540-13" prio=10 tid=0x00007f8aa4e81000 nid=0x6e8c runnable [0x00007f8268d3c000]
  1950. java.lang.Thread.State: RUNNABLE
  1951. at java.util.HashMap.put(HashMap.java:494)
  1952. at org.apache.lucene.index.FieldInfos$FieldNumbers.addOrGet(FieldInfos.java:213)
  1953. - locked <0x00007f82db5e33e0> (a org.apache.lucene.index.FieldInfos$FieldNumbers)
  1954. at org.apache.lucene.index.FieldInfos$Builder.addOrUpdateInternal(FieldInfos.java:312)
  1955. at org.apache.lucene.index.FieldInfos$Builder.add(FieldInfos.java:338)
  1956. at org.apache.lucene.index.FieldInfos$Builder.add(FieldInfos.java:276)
  1957. at org.apache.lucene.index.MultiFields.getMergedFieldInfos(MultiFields.java:245)
  1958. at org.apache.lucene.index.SlowCompositeReaderWrapper.getFieldInfos(SlowCompositeReaderWrapper.java:239)
  1959. at org.apache.solr.highlight.DefaultSolrHighlighter.getSpanQueryScorer(DefaultSolrHighlighter.java:214)
  1960. at org.apache.solr.highlight.DefaultSolrHighlighter.getPhraseHighlighter(DefaultSolrHighlighter.java:176)
  1961. at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:504)
  1962. at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:397)
  1963. at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:143)
  1964. at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:230)
  1965. at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
  1966. at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)
  1967. at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:829)
  1968. at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:446)
  1969. at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
  1970. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
  1971. at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
  1972. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
  1973. at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
  1974. at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
  1975. at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
  1976. at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
  1977. at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
  1978. at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
  1979. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
  1980. at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
  1981. at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
  1982. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
  1983. at org.eclipse.jetty.server.Server.handle(Server.java:368)
  1984. at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
  1985. at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
  1986. at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
  1987. at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
  1988. at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
  1989. at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
  1990. at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
  1991. at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
  1992. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
  1993. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
  1994. at java.lang.Thread.run(Thread.java:745)
  1995.  
  1996. "qtp125683540-12" prio=10 tid=0x00007f8aa4e7f800 nid=0x6e8b waiting on condition [0x00007f8268d7e000]
  1997. java.lang.Thread.State: TIMED_WAITING (parking)
  1998. at sun.misc.Unsafe.park(Native Method)
  1999. - parking to wait for <0x00007f848ee2d6a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  2000. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  2001. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  2002. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  2003. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  2004. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  2005. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  2006. at java.lang.Thread.run(Thread.java:745)
  2007.  
  2008. "qtp125683540-11 Acceptor0 [email protected]:8984" prio=10 tid=0x00007f8aa4e7a000 nid=0x6e8a runnable [0x00007f8268dbf000]
  2009. java.lang.Thread.State: RUNNABLE
  2010. at java.net.PlainSocketImpl.socketAccept(Native Method)
  2011. at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
  2012. at java.net.ServerSocket.implAccept(ServerSocket.java:530)
  2013. at java.net.ServerSocket.accept(ServerSocket.java:498)
  2014. at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
  2015. at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
  2016. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
  2017. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
  2018. at java.lang.Thread.run(Thread.java:745)
  2019.  
  2020. "ShutdownMonitor" daemon prio=10 tid=0x00007f8aa4e6f800 nid=0x6e89 runnable [0x00007f8268e00000]
  2021. java.lang.Thread.State: RUNNABLE
  2022. at java.net.PlainSocketImpl.socketAccept(Native Method)
  2023. at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
  2024. at java.net.ServerSocket.implAccept(ServerSocket.java:530)
  2025. at java.net.ServerSocket.accept(ServerSocket.java:498)
  2026. at org.eclipse.jetty.server.ShutdownMonitor.run(ShutdownMonitor.java:165)
  2027.  
  2028. "Service Thread" daemon prio=10 tid=0x00007f8aa4143800 nid=0x6e7d runnable [0x0000000000000000]
  2029. java.lang.Thread.State: RUNNABLE
  2030.  
  2031. "C2 CompilerThread1" daemon prio=10 tid=0x00007f8aa4141000 nid=0x6e7c waiting on condition [0x0000000000000000]
  2032. java.lang.Thread.State: RUNNABLE
  2033.  
  2034. "C2 CompilerThread0" daemon prio=10 tid=0x00007f8aa413e800 nid=0x6e7b waiting on condition [0x0000000000000000]
  2035. java.lang.Thread.State: RUNNABLE
  2036.  
  2037. "Signal Dispatcher" daemon prio=10 tid=0x00007f8aa413c800 nid=0x6e7a runnable [0x0000000000000000]
  2038. java.lang.Thread.State: RUNNABLE
  2039.  
  2040. "Surrogate Locker Thread (Concurrent GC)" daemon prio=10 tid=0x00007f8aa413a800 nid=0x6e79 waiting on condition [0x0000000000000000]
  2041. java.lang.Thread.State: RUNNABLE
  2042.  
  2043. "Finalizer" daemon prio=10 tid=0x00007f8aa4119800 nid=0x6e78 in Object.wait() [0x00007f828808e000]
  2044. java.lang.Thread.State: WAITING (on object monitor)
  2045. at java.lang.Object.wait(Native Method)
  2046. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
  2047. - locked <0x00007f848f39db00> (a java.lang.ref.ReferenceQueue$Lock)
  2048. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
  2049. at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
  2050.  
  2051. "Reference Handler" daemon prio=10 tid=0x00007f8aa4117800 nid=0x6e77 in Object.wait() [0x00007f82880cf000]
  2052. java.lang.Thread.State: WAITING (on object monitor)
  2053. at java.lang.Object.wait(Native Method)
  2054. at java.lang.Object.wait(Object.java:503)
  2055. at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
  2056. - locked <0x00007f848f39db48> (a java.lang.ref.Reference$Lock)
  2057.  
  2058. "VM Thread" prio=10 tid=0x00007f8aa4113000 nid=0x6e76 runnable
  2059.  
  2060. "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401b000 nid=0x6e6b runnable
  2061.  
  2062. "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401d000 nid=0x6e6c runnable
  2063.  
  2064. "Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x00007f8aa401e800 nid=0x6e6d runnable
  2065.  
  2066. "Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x00007f8aa4020800 nid=0x6e6e runnable
  2067.  
  2068. "Concurrent Mark-Sweep GC Thread" prio=10 tid=0x00007f8aa40a8800 nid=0x6e75 runnable
  2069. "Gang worker#0 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a1000 nid=0x6e71 runnable
  2070.  
  2071. "Gang worker#1 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a2800 nid=0x6e72 runnable
  2072.  
  2073. "Gang worker#2 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a4800 nid=0x6e73 runnable
  2074.  
  2075. "Gang worker#3 (Parallel CMS Threads)" prio=10 tid=0x00007f8aa40a6800 nid=0x6e74 runnable
  2076.  
  2077. "VM Periodic Task Thread" prio=10 tid=0x00007f8aa414e800 nid=0x6e7e waiting on condition
  2078.  
  2079. JNI global references: 344
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!