API tools faq
paste
Guest User

Spark thread dump

a guest
Mar 19th, 2014
263
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 81.05 KB | None | 0 0
raw download clone embed print report
  1. 2014-03-19 08:41:53
  2. Full thread dump OpenJDK 64-Bit Server VM (23.7-b01 mixed mode):
  3.  
  4. "spark-akka.actor.default-dispatcher-1588" daemon prio=10 tid=0x00000000019c2800 nid=0x6aec waiting on condition [0x00007f0c337e8000]
  5. java.lang.Thread.State: TIMED_WAITING (parking)
  6. at sun.misc.Unsafe.park(Native Method)
  7. - parking to wait for <0x00007f0cfbd6f8e8> (a akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinPool)
  8. at akka.jsr166y.ForkJoinPool.idleAwaitWork(ForkJoinPool.java:1626)
  9. at akka.jsr166y.ForkJoinPool.scan(ForkJoinPool.java:1579)
  10. at akka.jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1479)
  11. at akka.jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
  12.  
  13. "spark-2870" daemon prio=10 tid=0x00007f0c5c8be800 nid=0x6aeb waiting on condition [0x00007f0c3368b000]
  14. java.lang.Thread.State: TIMED_WAITING (parking)
  15. at sun.misc.Unsafe.park(Native Method)
  16. - parking to wait for <0x00007f0cfbed7478> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  17. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  18. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  19. at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
  20. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  21. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  22. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  23. at java.lang.Thread.run(Thread.java:724)
  24.  
  25. "spark-2869" daemon prio=10 tid=0x00007f0c5c8f1800 nid=0x6aea waiting on condition [0x00007f0c33548000]
  26. java.lang.Thread.State: TIMED_WAITING (parking)
  27. at sun.misc.Unsafe.park(Native Method)
  28. - parking to wait for <0x00007f0cfbed7478> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  29. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  30. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  31. at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
  32. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  33. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  34. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  35. at java.lang.Thread.run(Thread.java:724)
  36.  
  37. "sparkExecutor-1522" daemon prio=10 tid=0x00007f0c54218000 nid=0x6ad7 waiting on condition [0x00007f0c6195c000]
  38. java.lang.Thread.State: TIMED_WAITING (parking)
  39. at sun.misc.Unsafe.park(Native Method)
  40. - parking to wait for <0x00007f0cfbb31788> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  41. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  42. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  43. at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
  44. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  45. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  46. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  47. at java.lang.Thread.run(Thread.java:724)
  48.  
  49. "sparkExecutor-1521" daemon prio=10 tid=0x00007f0ce0ce5800 nid=0x6ad6 waiting on condition [0x00007f0c5b5f4000]
  50. java.lang.Thread.State: TIMED_WAITING (parking)
  51. at sun.misc.Unsafe.park(Native Method)
  52. - parking to wait for <0x00007f0cfbb31788> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  53. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  54. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  55. at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
  56. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  57. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  58. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  59. at java.lang.Thread.run(Thread.java:724)
  60.  
  61. "spark-2868" daemon prio=10 tid=0x00007f0c5c54d800 nid=0x6ad2 waiting on condition [0x00007f0c5b8f7000]
  62. java.lang.Thread.State: TIMED_WAITING (parking)
  63. at sun.misc.Unsafe.park(Native Method)
  64. - parking to wait for <0x00007f0cfbed7478> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  65. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  66. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  67. at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
  68. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  69. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  70. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  71. at java.lang.Thread.run(Thread.java:724)
  72.  
  73. "spark-2867" daemon prio=10 tid=0x00007f0c5c675000 nid=0x6ad1 waiting on condition [0x00007f0c3184d000]
  74. java.lang.Thread.State: TIMED_WAITING (parking)
  75. at sun.misc.Unsafe.park(Native Method)
  76. - parking to wait for <0x00007f0cfbed7478> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  77. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  78. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  79. at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
  80. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  81. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  82. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  83. at java.lang.Thread.run(Thread.java:724)
  84.  
  85. "spark-2866" daemon prio=10 tid=0x00007f0c5c8f1000 nid=0x6ac6 waiting on condition [0x00007f0c5b9f8000]
  86. java.lang.Thread.State: TIMED_WAITING (parking)
  87. at sun.misc.Unsafe.park(Native Method)
  88. - parking to wait for <0x00007f0cfbed7478> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  89. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  90. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  91. at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
  92. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  93. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  94. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  95. at java.lang.Thread.run(Thread.java:724)
  96.  
  97. "spark-2865" daemon prio=10 tid=0x00007f0c5c676000 nid=0x6ac5 waiting on condition [0x00007f0c5a5dd000]
  98. java.lang.Thread.State: TIMED_WAITING (parking)
  99. at sun.misc.Unsafe.park(Native Method)
  100. - parking to wait for <0x00007f0cfbed7478> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  101. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  102. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  103. at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
  104. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  105. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  106. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  107. at java.lang.Thread.run(Thread.java:724)
  108.  
  109. "sparkExecutor-1520" daemon prio=10 tid=0x00007f0c5406d000 nid=0x6ac4 waiting on condition [0x00007f0c331a1000]
  110. java.lang.Thread.State: TIMED_WAITING (parking)
  111. at sun.misc.Unsafe.park(Native Method)
  112. - parking to wait for <0x00007f0cfbb31788> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  113. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  114. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  115. at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
  116. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  117. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  118. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  119. at java.lang.Thread.run(Thread.java:724)
  120.  
  121. "sparkExecutor-1519" daemon prio=10 tid=0x00007f0ce0aab800 nid=0x6ac3 waiting on condition [0x00007f0c33409000]
  122. java.lang.Thread.State: TIMED_WAITING (parking)
  123. at sun.misc.Unsafe.park(Native Method)
  124. - parking to wait for <0x00007f0cfbb31788> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  125. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  126. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  127. at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
  128. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  129. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  130. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  131. at java.lang.Thread.run(Thread.java:724)
  132.  
  133. "spark-2864" daemon prio=10 tid=0x00007f0c5c811800 nid=0x6abf waiting on condition [0x00007f0c44001000]
  134. java.lang.Thread.State: TIMED_WAITING (parking)
  135. at sun.misc.Unsafe.park(Native Method)
  136. - parking to wait for <0x00007f0cfbed7478> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  137. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  138. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  139. at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
  140. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  141. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  142. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  143. at java.lang.Thread.run(Thread.java:724)
  144.  
  145. "spark-2863" daemon prio=10 tid=0x00007f0c5c5d4000 nid=0x6abe waiting on condition [0x00007f0c4e7be000]
  146. java.lang.Thread.State: TIMED_WAITING (parking)
  147. at sun.misc.Unsafe.park(Native Method)
  148. - parking to wait for <0x00007f0cfbed7478> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  149. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  150. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  151. at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
  152. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  153. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  154. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  155. at java.lang.Thread.run(Thread.java:724)
  156.  
  157. "Attach Listener" daemon prio=10 tid=0x0000000001674800 nid=0x6ab2 waiting on condition [0x0000000000000000]
  158. java.lang.Thread.State: RUNNABLE
  159.  
  160. "spark-akka.actor.default-dispatcher-1479" daemon prio=10 tid=0x0000000001daf000 nid=0x6892 waiting on condition [0x00007f0c5b4f3000]
  161. java.lang.Thread.State: WAITING (parking)
  162. at sun.misc.Unsafe.park(Native Method)
  163. - parking to wait for <0x00007f0cfbd6f8e8> (a akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinPool)
  164. at akka.jsr166y.ForkJoinPool.scan(ForkJoinPool.java:1594)
  165. at akka.jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1479)
  166. at akka.jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
  167.  
  168. "pool-1-thread-20" prio=10 tid=0x00007f0c5425c000 nid=0x4c78 waiting on condition [0x00007f0c4ac98000]
  169. java.lang.Thread.State: WAITING (parking)
  170. at sun.misc.Unsafe.park(Native Method)
  171. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  172. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  173. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  174. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  175. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  176. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  177. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  178. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  179. at java.lang.Thread.run(Thread.java:724)
  180.  
  181. "pool-1-thread-19" prio=10 tid=0x00007f0c54087000 nid=0x4c77 waiting on condition [0x00007f0c4ad99000]
  182. java.lang.Thread.State: WAITING (parking)
  183. at sun.misc.Unsafe.park(Native Method)
  184. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  185. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  186. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  187. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  188. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  189. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  190. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  191. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  192. at java.lang.Thread.run(Thread.java:724)
  193.  
  194. "pool-1-thread-18" prio=10 tid=0x00007f0c54086000 nid=0x4c76 waiting on condition [0x00007f0c4c12a000]
  195. java.lang.Thread.State: WAITING (parking)
  196. at sun.misc.Unsafe.park(Native Method)
  197. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  198. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  199. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  200. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  201. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  202. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  203. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  204. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  205. at java.lang.Thread.run(Thread.java:724)
  206.  
  207. "pool-1-thread-17" prio=10 tid=0x00007f0c542ee000 nid=0x4c75 waiting on condition [0x00007f0c4c22b000]
  208. java.lang.Thread.State: WAITING (parking)
  209. at sun.misc.Unsafe.park(Native Method)
  210. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  211. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  212. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  213. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  214. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  215. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  216. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  217. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  218. at java.lang.Thread.run(Thread.java:724)
  219.  
  220. "pool-1-thread-16" prio=10 tid=0x00007f0c54580800 nid=0x4c74 waiting on condition [0x00007f0c4c32c000]
  221. java.lang.Thread.State: WAITING (parking)
  222. at sun.misc.Unsafe.park(Native Method)
  223. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  224. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  225. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  226. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  227. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  228. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  229. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  230. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  231. at java.lang.Thread.run(Thread.java:724)
  232.  
  233. "pool-1-thread-15" prio=10 tid=0x00007f0c5c616800 nid=0x4c73 waiting on condition [0x00007f0c4a15e000]
  234. java.lang.Thread.State: WAITING (parking)
  235. at sun.misc.Unsafe.park(Native Method)
  236. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  237. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  238. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  239. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  240. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  241. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  242. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  243. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  244. at java.lang.Thread.run(Thread.java:724)
  245.  
  246. "pool-1-thread-14" prio=10 tid=0x00007f0c5cad6000 nid=0x4c72 waiting on condition [0x00007f0c4a293000]
  247. java.lang.Thread.State: WAITING (parking)
  248. at sun.misc.Unsafe.park(Native Method)
  249. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  250. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  251. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  252. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  253. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  254. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  255. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  256. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  257. at java.lang.Thread.run(Thread.java:724)
  258.  
  259. "pool-1-thread-13" prio=10 tid=0x00007f0c5caea000 nid=0x4c71 waiting on condition [0x00007f0c4a3b8000]
  260. java.lang.Thread.State: WAITING (parking)
  261. at sun.misc.Unsafe.park(Native Method)
  262. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  263. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  264. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  265. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  266. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  267. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  268. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  269. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  270. at java.lang.Thread.run(Thread.java:724)
  271.  
  272. "pool-1-thread-12" prio=10 tid=0x0000000002009000 nid=0x4c70 waiting on condition [0x00007f0c4a4d0000]
  273. java.lang.Thread.State: WAITING (parking)
  274. at sun.misc.Unsafe.park(Native Method)
  275. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  276. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  277. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  278. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  279. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  280. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  281. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  282. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  283. at java.lang.Thread.run(Thread.java:724)
  284.  
  285. "pool-1-thread-11" prio=10 tid=0x0000000001761800 nid=0x4c6f waiting on condition [0x00007f0c4a5e3000]
  286. java.lang.Thread.State: WAITING (parking)
  287. at sun.misc.Unsafe.park(Native Method)
  288. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  289. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  290. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  291. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  292. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  293. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  294. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  295. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  296. at java.lang.Thread.run(Thread.java:724)
  297.  
  298. "pool-1-thread-10" prio=10 tid=0x0000000001fc2000 nid=0x4c6e waiting on condition [0x00007f0c58997000]
  299. java.lang.Thread.State: WAITING (parking)
  300. at sun.misc.Unsafe.park(Native Method)
  301. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  302. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  303. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  304. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  305. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  306. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  307. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  308. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  309. at java.lang.Thread.run(Thread.java:724)
  310.  
  311. "pool-3-thread-1" prio=10 tid=0x00007f0c5c8bb800 nid=0x4c6d waiting on condition [0x00007f0c5bbfa000]
  312. java.lang.Thread.State: WAITING (parking)
  313. at sun.misc.Unsafe.park(Native Method)
  314. - parking to wait for <0x00007f0cfbf12590> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  315. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  316. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  317. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  318. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  319. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  320. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  321. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  322. at java.lang.Thread.run(Thread.java:724)
  323.  
  324. "pool-1-thread-9" prio=10 tid=0x00000000020bf800 nid=0x4c68 waiting on condition [0x00007f0c53cfb000]
  325. java.lang.Thread.State: WAITING (parking)
  326. at sun.misc.Unsafe.park(Native Method)
  327. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  328. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  329. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  330. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  331. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  332. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  333. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  334. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  335. at java.lang.Thread.run(Thread.java:724)
  336.  
  337. "pool-1-thread-8" prio=10 tid=0x00007f0c5cd06000 nid=0x4c67 waiting on condition [0x00007f0c53dfc000]
  338. java.lang.Thread.State: WAITING (parking)
  339. at sun.misc.Unsafe.park(Native Method)
  340. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  341. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  342. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  343. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  344. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  345. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  346. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  347. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  348. at java.lang.Thread.run(Thread.java:724)
  349.  
  350. "pool-1-thread-7" prio=10 tid=0x00007f0c5cd04000 nid=0x4c66 waiting on condition [0x00007f0c53efd000]
  351. java.lang.Thread.State: WAITING (parking)
  352. at sun.misc.Unsafe.park(Native Method)
  353. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  354. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  355. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  356. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  357. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  358. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  359. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  360. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  361. at java.lang.Thread.run(Thread.java:724)
  362.  
  363. "pool-1-thread-6" prio=10 tid=0x0000000001fec800 nid=0x4c65 waiting on condition [0x00007f0c53ffe000]
  364. java.lang.Thread.State: WAITING (parking)
  365. at sun.misc.Unsafe.park(Native Method)
  366. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  367. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  368. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  369. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  370. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  371. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  372. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  373. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  374. at java.lang.Thread.run(Thread.java:724)
  375.  
  376. "pool-1-thread-5" prio=10 tid=0x0000000001feb800 nid=0x4c64 waiting on condition [0x00007f0c5818f000]
  377. java.lang.Thread.State: WAITING (parking)
  378. at sun.misc.Unsafe.park(Native Method)
  379. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  380. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  381. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  382. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  383. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  384. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  385. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  386. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  387. at java.lang.Thread.run(Thread.java:724)
  388.  
  389. "pool-1-thread-4" prio=10 tid=0x00007f0c5c3b9000 nid=0x4c63 waiting on condition [0x00007f0c58290000]
  390. java.lang.Thread.State: WAITING (parking)
  391. at sun.misc.Unsafe.park(Native Method)
  392. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  393. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  394. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  395. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  396. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  397. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  398. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  399. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  400. at java.lang.Thread.run(Thread.java:724)
  401.  
  402. "pool-1-thread-3" prio=10 tid=0x00007f0c5c3b8000 nid=0x4c62 waiting on condition [0x00007f0c58391000]
  403. java.lang.Thread.State: WAITING (parking)
  404. at sun.misc.Unsafe.park(Native Method)
  405. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  406. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  407. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  408. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  409. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  410. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  411. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  412. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  413. at java.lang.Thread.run(Thread.java:724)
  414.  
  415. "pool-1-thread-2" prio=10 tid=0x00007f0c5c3b7800 nid=0x4c61 waiting on condition [0x00007f0c58492000]
  416. java.lang.Thread.State: WAITING (parking)
  417. at sun.misc.Unsafe.park(Native Method)
  418. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  419. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  420. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  421. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  422. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  423. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  424. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  425. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  426. at java.lang.Thread.run(Thread.java:724)
  427.  
  428. "pool-1-thread-1" prio=10 tid=0x0000000001d11800 nid=0x4c60 waiting on condition [0x00007f0c60b4a000]
  429. java.lang.Thread.State: WAITING (parking)
  430. at sun.misc.Unsafe.park(Native Method)
  431. - parking to wait for <0x00007f0cfbf11ee0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  432. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  433. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  434. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  435. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  436. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  437. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  438. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  439. at java.lang.Thread.run(Thread.java:724)
  440.  
  441. "pool-2-thread-4" prio=10 tid=0x0000000002189000 nid=0x4c5f waiting on condition [0x00007f0c5baf9000]
  442. java.lang.Thread.State: WAITING (parking)
  443. at sun.misc.Unsafe.park(Native Method)
  444. - parking to wait for <0x00007f0cfbf12238> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  445. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  446. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  447. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  448. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  449. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  450. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  451. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  452. at java.lang.Thread.run(Thread.java:724)
  453.  
  454. "pool-2-thread-3" prio=10 tid=0x00000000018d6800 nid=0x4c5e waiting on condition [0x00007f0c58593000]
  455. java.lang.Thread.State: WAITING (parking)
  456. at sun.misc.Unsafe.park(Native Method)
  457. - parking to wait for <0x00007f0cfbf12238> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  458. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  459. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  460. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  461. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  462. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  463. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  464. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  465. at java.lang.Thread.run(Thread.java:724)
  466.  
  467. "pool-2-thread-2" prio=10 tid=0x0000000001f0c000 nid=0x4c5d waiting on condition [0x00007f0c6185b000]
  468. java.lang.Thread.State: WAITING (parking)
  469. at sun.misc.Unsafe.park(Native Method)
  470. - parking to wait for <0x00007f0cfbf12238> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  471. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  472. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  473. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  474. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  475. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  476. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  477. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  478. at java.lang.Thread.run(Thread.java:724)
  479.  
  480. "pool-2-thread-1" prio=10 tid=0x00000000019d4800 nid=0x4c5a waiting on condition [0x00007f0c58896000]
  481. java.lang.Thread.State: WAITING (parking)
  482. at sun.misc.Unsafe.park(Native Method)
  483. - parking to wait for <0x00007f0cfbf12238> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  484. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  485. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  486. at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  487. at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
  488. at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
  489. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  490. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  491. at java.lang.Thread.run(Thread.java:724)
  492.  
  493. "ResultTask cleanup timer" daemon prio=10 tid=0x00007f0c5c7e8000 nid=0x4c51 in Object.wait() [0x00007f0c58694000]
  494. java.lang.Thread.State: WAITING (on object monitor)
  495. at java.lang.Object.wait(Native Method)
  496. - waiting on <0x00007f0d184f7120> (a java.util.TaskQueue)
  497. at java.lang.Object.wait(Object.java:503)
  498. at java.util.TimerThread.mainLoop(Timer.java:526)
  499. - locked <0x00007f0d184f7120> (a java.util.TaskQueue)
  500. at java.util.TimerThread.run(Timer.java:505)
  501.  
  502. "MultiThreadedHttpConnectionManager cleanup" daemon prio=10 tid=0x00007f0ce0900000 nid=0x4aaa in Object.wait() [0x00007f0c60a49000]
  503. java.lang.Thread.State: WAITING (on object monitor)
  504. at java.lang.Object.wait(Native Method)
  505. - waiting on <0x00007f0cfc1ee3f0> (a java.lang.ref.ReferenceQueue$Lock)
  506. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
  507. - locked <0x00007f0cfc1ee3f0> (a java.lang.ref.ReferenceQueue$Lock)
  508. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
  509. at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ReferenceQueueThread.run(MultiThreadedHttpConnectionManager.java:1122)
  510.  
  511. "ShuffleMapTask cleanup timer" daemon prio=10 tid=0x00007f0c5c37f000 nid=0x4aa9 in Object.wait() [0x00007f0c5a6de000]
  512. java.lang.Thread.State: WAITING (on object monitor)
  513. at java.lang.Object.wait(Native Method)
  514. - waiting on <0x00007f0cfc39df68> (a java.util.TaskQueue)
  515. at java.lang.Object.wait(Object.java:503)
  516. at java.util.TimerThread.mainLoop(Timer.java:526)
  517. - locked <0x00007f0cfc39df68> (a java.util.TaskQueue)
  518. at java.util.TimerThread.run(Timer.java:505)
  519.  
  520. "Executor task launch worker-7" daemon prio=10 tid=0x00000000015a2800 nid=0x4a9b waiting on condition [0x00007f0c5aae8000]
  521. java.lang.Thread.State: WAITING (parking)
  522. at sun.misc.Unsafe.park(Native Method)
  523. - parking to wait for <0x00007f0fb0ff45f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  524. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  525. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  526. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  527. at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator.next(BlockFetcherIterator.scala:247)
  528. at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator.next(BlockFetcherIterator.scala:71)
  529. at scala.collection.Iterator$$anon$21.hasNext(Iterator.scala:440)
  530. at org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:26)
  531. at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:27)
  532. at org.apache.spark.Aggregator.combineValuesByKey(Aggregator.scala:40)
  533. at org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:103)
  534. at org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:102)
  535. at org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:465)
  536. at org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:465)
  537. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:34)
  538. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:237)
  539. at org.apache.spark.rdd.RDD.iterator(RDD.scala:226)
  540. at org.apache.spark.rdd.FlatMappedRDD.compute(FlatMappedRDD.scala:32)
  541. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:237)
  542. at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:71)
  543. at org.apache.spark.rdd.RDD.iterator(RDD.scala:224)
  544. at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:107)
  545. at org.apache.spark.scheduler.Task.run(Task.scala:53)
  546. at org.apache.spark.executor.Executor$TaskRunner$$anonfun$run$1.apply$mcV$sp(Executor.scala:215)
  547. at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:47)
  548. at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:46)
  549. at java.security.AccessController.doPrivileged(Native Method)
  550. at javax.security.auth.Subject.doAs(Subject.java:415)
  551. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
  552. at org.apache.spark.deploy.SparkHadoopUtil.runAsUser(SparkHadoopUtil.scala:46)
  553. at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:182)
  554. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  555. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  556. at java.lang.Thread.run(Thread.java:724)
  557.  
  558. "Executor task launch worker-6" daemon prio=10 tid=0x00000000015a0000 nid=0x4a9a waiting on condition [0x00007f0c5abe9000]
  559. java.lang.Thread.State: WAITING (parking)
  560. at sun.misc.Unsafe.park(Native Method)
  561. - parking to wait for <0x00007f0faa72f510> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  562. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  563. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  564. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  565. at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator.next(BlockFetcherIterator.scala:247)
  566. at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator.next(BlockFetcherIterator.scala:71)
  567. at scala.collection.Iterator$$anon$21.hasNext(Iterator.scala:440)
  568. at org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:26)
  569. at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:27)
  570. at org.apache.spark.Aggregator.combineValuesByKey(Aggregator.scala:40)
  571. at org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:103)
  572. at org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:102)
  573. at org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:465)
  574. at org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:465)
  575. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:34)
  576. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:237)
  577. at org.apache.spark.rdd.RDD.iterator(RDD.scala:226)
  578. at org.apache.spark.rdd.FlatMappedRDD.compute(FlatMappedRDD.scala:32)
  579. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:237)
  580. at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:71)
  581. at org.apache.spark.rdd.RDD.iterator(RDD.scala:224)
  582. at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:107)
  583. at org.apache.spark.scheduler.Task.run(Task.scala:53)
  584. at org.apache.spark.executor.Executor$TaskRunner$$anonfun$run$1.apply$mcV$sp(Executor.scala:215)
  585. at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:47)
  586. at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:46)
  587. at java.security.AccessController.doPrivileged(Native Method)
  588. at javax.security.auth.Subject.doAs(Subject.java:415)
  589. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
  590. at org.apache.spark.deploy.SparkHadoopUtil.runAsUser(SparkHadoopUtil.scala:46)
  591. at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:182)
  592. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  593. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  594. at java.lang.Thread.run(Thread.java:724)
  595.  
  596. "Executor task launch worker-5" daemon prio=10 tid=0x00000000016f2800 nid=0x4a99 waiting on condition [0x00007f0c60745000]
  597. java.lang.Thread.State: WAITING (parking)
  598. at sun.misc.Unsafe.park(Native Method)
  599. - parking to wait for <0x00007f0fad74f0f0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  600. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  601. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  602. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  603. at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator.next(BlockFetcherIterator.scala:247)
  604. at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator.next(BlockFetcherIterator.scala:71)
  605. at scala.collection.Iterator$$anon$21.hasNext(Iterator.scala:440)
  606. at org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:26)
  607. at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:27)
  608. at org.apache.spark.Aggregator.combineValuesByKey(Aggregator.scala:40)
  609. at org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:103)
  610. at org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:102)
  611. at org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:465)
  612. at org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:465)
  613. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:34)
  614. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:237)
  615. at org.apache.spark.rdd.RDD.iterator(RDD.scala:226)
  616. at org.apache.spark.rdd.FlatMappedRDD.compute(FlatMappedRDD.scala:32)
  617. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:237)
  618. at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:71)
  619. at org.apache.spark.rdd.RDD.iterator(RDD.scala:224)
  620. at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:107)
  621. at org.apache.spark.scheduler.Task.run(Task.scala:53)
  622. at org.apache.spark.executor.Executor$TaskRunner$$anonfun$run$1.apply$mcV$sp(Executor.scala:215)
  623. at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:47)
  624. at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:46)
  625. at java.security.AccessController.doPrivileged(Native Method)
  626. at javax.security.auth.Subject.doAs(Subject.java:415)
  627. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
  628. at org.apache.spark.deploy.SparkHadoopUtil.runAsUser(SparkHadoopUtil.scala:46)
  629. at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:182)
  630. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  631. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  632. at java.lang.Thread.run(Thread.java:724)
  633.  
  634. "Executor task launch worker-4" daemon prio=10 tid=0x00000000016f1000 nid=0x4a98 waiting on condition [0x00007f0c6114f000]
  635. java.lang.Thread.State: WAITING (parking)
  636. at sun.misc.Unsafe.park(Native Method)
  637. - parking to wait for <0x00007f0fad1ebc28> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  638. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  639. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  640. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  641. at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator.next(BlockFetcherIterator.scala:247)
  642. at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator.next(BlockFetcherIterator.scala:71)
  643. at scala.collection.Iterator$$anon$21.hasNext(Iterator.scala:440)
  644. at org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:26)
  645. at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:27)
  646. at org.apache.spark.Aggregator.combineValuesByKey(Aggregator.scala:40)
  647. at org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:103)
  648. at org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:102)
  649. at org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:465)
  650. at org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:465)
  651. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:34)
  652. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:237)
  653. at org.apache.spark.rdd.RDD.iterator(RDD.scala:226)
  654. at org.apache.spark.rdd.FlatMappedRDD.compute(FlatMappedRDD.scala:32)
  655. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:237)
  656. at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:71)
  657. at org.apache.spark.rdd.RDD.iterator(RDD.scala:224)
  658. at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:107)
  659. at org.apache.spark.scheduler.Task.run(Task.scala:53)
  660. at org.apache.spark.executor.Executor$TaskRunner$$anonfun$run$1.apply$mcV$sp(Executor.scala:215)
  661. at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:47)
  662. at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:46)
  663. at java.security.AccessController.doPrivileged(Native Method)
  664. at javax.security.auth.Subject.doAs(Subject.java:415)
  665. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
  666. at org.apache.spark.deploy.SparkHadoopUtil.runAsUser(SparkHadoopUtil.scala:46)
  667. at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:182)
  668. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  669. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  670. at java.lang.Thread.run(Thread.java:724)
  671.  
  672. "Executor task launch worker-3" daemon prio=10 tid=0x00000000016ef800 nid=0x4a97 waiting on condition [0x00007f0c60d4b000]
  673. java.lang.Thread.State: WAITING (parking)
  674. at sun.misc.Unsafe.park(Native Method)
  675. - parking to wait for <0x00007f0fac577d80> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  676. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  677. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  678. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  679. at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator.next(BlockFetcherIterator.scala:247)
  680. at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator.next(BlockFetcherIterator.scala:71)
  681. at scala.collection.Iterator$$anon$21.hasNext(Iterator.scala:440)
  682. at org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:26)
  683. at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:27)
  684. at org.apache.spark.Aggregator.combineValuesByKey(Aggregator.scala:40)
  685. at org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:103)
  686. at org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:102)
  687. at org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:465)
  688. at org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:465)
  689. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:34)
  690. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:237)
  691. at org.apache.spark.rdd.RDD.iterator(RDD.scala:226)
  692. at org.apache.spark.rdd.FlatMappedRDD.compute(FlatMappedRDD.scala:32)
  693. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:237)
  694. at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:71)
  695. at org.apache.spark.rdd.RDD.iterator(RDD.scala:224)
  696. at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:107)
  697. at org.apache.spark.scheduler.Task.run(Task.scala:53)
  698. at org.apache.spark.executor.Executor$TaskRunner$$anonfun$run$1.apply$mcV$sp(Executor.scala:215)
  699. at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:47)
  700. at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:46)
  701. at java.security.AccessController.doPrivileged(Native Method)
  702. at javax.security.auth.Subject.doAs(Subject.java:415)
  703. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
  704. at org.apache.spark.deploy.SparkHadoopUtil.runAsUser(SparkHadoopUtil.scala:46)
  705. at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:182)
  706. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  707. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  708. at java.lang.Thread.run(Thread.java:724)
  709.  
  710. "Executor task launch worker-2" daemon prio=10 tid=0x00000000016ee800 nid=0x4a96 waiting on condition [0x00007f0c60e4c000]
  711. java.lang.Thread.State: WAITING (parking)
  712. at sun.misc.Unsafe.park(Native Method)
  713. - parking to wait for <0x00007f0fae130758> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  714. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  715. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  716. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  717. at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator.next(BlockFetcherIterator.scala:247)
  718. at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator.next(BlockFetcherIterator.scala:71)
  719. at scala.collection.Iterator$$anon$21.hasNext(Iterator.scala:440)
  720. at org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:26)
  721. at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:27)
  722. at org.apache.spark.Aggregator.combineValuesByKey(Aggregator.scala:40)
  723. at org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:103)
  724. at org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:102)
  725. at org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:465)
  726. at org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:465)
  727. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:34)
  728. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:237)
  729. at org.apache.spark.rdd.RDD.iterator(RDD.scala:226)
  730. at org.apache.spark.rdd.FlatMappedRDD.compute(FlatMappedRDD.scala:32)
  731. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:237)
  732. at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:71)
  733. at org.apache.spark.rdd.RDD.iterator(RDD.scala:224)
  734. at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:107)
  735. at org.apache.spark.scheduler.Task.run(Task.scala:53)
  736. at org.apache.spark.executor.Executor$TaskRunner$$anonfun$run$1.apply$mcV$sp(Executor.scala:215)
  737. at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:47)
  738. at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:46)
  739. at java.security.AccessController.doPrivileged(Native Method)
  740. at javax.security.auth.Subject.doAs(Subject.java:415)
  741. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
  742. at org.apache.spark.deploy.SparkHadoopUtil.runAsUser(SparkHadoopUtil.scala:46)
  743. at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:182)
  744. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  745. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  746. at java.lang.Thread.run(Thread.java:724)
  747.  
  748. "Executor task launch worker-1" daemon prio=10 tid=0x00000000016ed800 nid=0x4a95 waiting on condition [0x00007f0c6104e000]
  749. java.lang.Thread.State: WAITING (parking)
  750. at sun.misc.Unsafe.park(Native Method)
  751. - parking to wait for <0x00007f0fb327ace8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  752. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  753. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  754. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  755. at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator.next(BlockFetcherIterator.scala:247)
  756. at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator.next(BlockFetcherIterator.scala:71)
  757. at scala.collection.Iterator$$anon$21.hasNext(Iterator.scala:440)
  758. at org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:26)
  759. at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:27)
  760. at org.apache.spark.Aggregator.combineValuesByKey(Aggregator.scala:40)
  761. at org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:103)
  762. at org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:102)
  763. at org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:465)
  764. at org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:465)
  765. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:34)
  766. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:237)
  767. at org.apache.spark.rdd.RDD.iterator(RDD.scala:226)
  768. at org.apache.spark.rdd.FlatMappedRDD.compute(FlatMappedRDD.scala:32)
  769. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:237)
  770. at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:71)
  771. at org.apache.spark.rdd.RDD.iterator(RDD.scala:224)
  772. at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:107)
  773. at org.apache.spark.scheduler.Task.run(Task.scala:53)
  774. at org.apache.spark.executor.Executor$TaskRunner$$anonfun$run$1.apply$mcV$sp(Executor.scala:215)
  775. at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:47)
  776. at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:46)
  777. at java.security.AccessController.doPrivileged(Native Method)
  778. at javax.security.auth.Subject.doAs(Subject.java:415)
  779. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
  780. at org.apache.spark.deploy.SparkHadoopUtil.runAsUser(SparkHadoopUtil.scala:46)
  781. at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:182)
  782. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  783. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  784. at java.lang.Thread.run(Thread.java:724)
  785.  
  786. "Executor task launch worker-0" daemon prio=10 tid=0x0000000001951800 nid=0x4a94 waiting on condition [0x00007f0c60c4a000]
  787. java.lang.Thread.State: WAITING (parking)
  788. at sun.misc.Unsafe.park(Native Method)
  789. - parking to wait for <0x00007f0faead0b90> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  790. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  791. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  792. at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  793. at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator.next(BlockFetcherIterator.scala:247)
  794. at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator.next(BlockFetcherIterator.scala:71)
  795. at scala.collection.Iterator$$anon$21.hasNext(Iterator.scala:440)
  796. at org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:26)
  797. at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:27)
  798. at org.apache.spark.Aggregator.combineValuesByKey(Aggregator.scala:40)
  799. at org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:103)
  800. at org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:102)
  801. at org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:465)
  802. at org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:465)
  803. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:34)
  804. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:237)
  805. at org.apache.spark.rdd.RDD.iterator(RDD.scala:226)
  806. at org.apache.spark.rdd.FlatMappedRDD.compute(FlatMappedRDD.scala:32)
  807. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:237)
  808. at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:71)
  809. at org.apache.spark.rdd.RDD.iterator(RDD.scala:224)
  810. at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:107)
  811. at org.apache.spark.scheduler.Task.run(Task.scala:53)
  812. at org.apache.spark.executor.Executor$TaskRunner$$anonfun$run$1.apply$mcV$sp(Executor.scala:215)
  813. at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:47)
  814. at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:46)
  815. at java.security.AccessController.doPrivileged(Native Method)
  816. at javax.security.auth.Subject.doAs(Subject.java:415)
  817. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
  818. at org.apache.spark.deploy.SparkHadoopUtil.runAsUser(SparkHadoopUtil.scala:46)
  819. at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:182)
  820. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  821. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  822. at java.lang.Thread.run(Thread.java:724)
  823.  
  824. "qtp758550709-53" daemon prio=10 tid=0x00007f0ce012e800 nid=0x4a79 waiting on condition [0x00007f0c5aceb000]
  825. java.lang.Thread.State: TIMED_WAITING (parking)
  826. at sun.misc.Unsafe.park(Native Method)
  827. - parking to wait for <0x00007f0cfbf13f20> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  828. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  829. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  830. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  831. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  832. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  833. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  834. at java.lang.Thread.run(Thread.java:724)
  835.  
  836. "qtp758550709-52" daemon prio=10 tid=0x00007f0ce012c800 nid=0x4a78 waiting on condition [0x00007f0c5adec000]
  837. java.lang.Thread.State: TIMED_WAITING (parking)
  838. at sun.misc.Unsafe.park(Native Method)
  839. - parking to wait for <0x00007f0cfbf13f20> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  840. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  841. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  842. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  843. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  844. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  845. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  846. at java.lang.Thread.run(Thread.java:724)
  847.  
  848. "qtp758550709-51" daemon prio=10 tid=0x00007f0ce0473000 nid=0x4a77 waiting on condition [0x00007f0c5aeed000]
  849. java.lang.Thread.State: TIMED_WAITING (parking)
  850. at sun.misc.Unsafe.park(Native Method)
  851. - parking to wait for <0x00007f0cfbf13f20> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  852. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  853. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  854. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  855. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  856. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  857. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  858. at java.lang.Thread.run(Thread.java:724)
  859.  
  860. "qtp758550709-50" daemon prio=10 tid=0x00007f0ce0471000 nid=0x4a76 waiting on condition [0x00007f0c5afee000]
  861. java.lang.Thread.State: TIMED_WAITING (parking)
  862. at sun.misc.Unsafe.park(Native Method)
  863. - parking to wait for <0x00007f0cfbf13f20> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  864. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  865. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  866. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  867. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  868. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  869. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  870. at java.lang.Thread.run(Thread.java:724)
  871.  
  872. "qtp758550709-49" daemon prio=10 tid=0x00007f0ce046e000 nid=0x4a75 waiting on condition [0x00007f0c5b0ef000]
  873. java.lang.Thread.State: TIMED_WAITING (parking)
  874. at sun.misc.Unsafe.park(Native Method)
  875. - parking to wait for <0x00007f0cfbf13f20> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  876. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  877. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  878. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  879. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  880. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  881. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  882. at java.lang.Thread.run(Thread.java:724)
  883.  
  884. "qtp758550709-48" daemon prio=10 tid=0x00007f0ce046c800 nid=0x4a74 waiting on condition [0x00007f0c5b1f0000]
  885. java.lang.Thread.State: TIMED_WAITING (parking)
  886. at sun.misc.Unsafe.park(Native Method)
  887. - parking to wait for <0x00007f0cfbf13f20> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  888. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  889. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  890. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  891. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  892. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  893. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  894. at java.lang.Thread.run(Thread.java:724)
  895.  
  896. "qtp758550709-47" daemon prio=10 tid=0x00007f0ce0065000 nid=0x4a73 waiting on condition [0x00007f0c5b2f1000]
  897. java.lang.Thread.State: TIMED_WAITING (parking)
  898. at sun.misc.Unsafe.park(Native Method)
  899. - parking to wait for <0x00007f0cfbf13f20> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  900. at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
  901. at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
  902. at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
  903. at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
  904. at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
  905. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  906. at java.lang.Thread.run(Thread.java:724)
  907.  
  908. "qtp758550709-46 Acceptor0 [email protected]:56470" daemon prio=10 tid=0x00007f0ce0476000 nid=0x4a72 runnable [0x00007f0c5b3f2000]
  909. java.lang.Thread.State: RUNNABLE
  910. at java.net.PlainSocketImpl.socketAccept(Native Method)
  911. at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
  912. at java.net.ServerSocket.implAccept(ServerSocket.java:530)
  913. at java.net.ServerSocket.accept(ServerSocket.java:498)
  914. at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
  915. at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
  916. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
  917. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
  918. at java.lang.Thread.run(Thread.java:724)
  919.  
  920. "MapOutputTracker cleanup timer" daemon prio=10 tid=0x00007f0ce06f7000 nid=0x4a6f in Object.wait() [0x00007f0c5b6f5000]
  921. java.lang.Thread.State: WAITING (on object monitor)
  922. at java.lang.Object.wait(Native Method)
  923. - waiting on <0x00007f0cfbf0c910> (a java.util.TaskQueue)
  924. at java.lang.Object.wait(Object.java:503)
  925. at java.util.TimerThread.mainLoop(Timer.java:526)
  926. - locked <0x00007f0cfbf0c910> (a java.util.TaskQueue)
  927. at java.util.TimerThread.run(Timer.java:505)
  928.  
  929. "HttpBroadcast cleanup timer" daemon prio=10 tid=0x00007f0ce071c800 nid=0x4a6e in Object.wait() [0x00007f0c5b7f6000]
  930. java.lang.Thread.State: WAITING (on object monitor)
  931. at java.lang.Object.wait(Native Method)
  932. - waiting on <0x00007f0cfc39e1f8> (a java.util.TaskQueue)
  933. at java.lang.Object.wait(Object.java:503)
  934. at java.util.TimerThread.mainLoop(Timer.java:526)
  935. - locked <0x00007f0cfc39e1f8> (a java.util.TaskQueue)
  936. at java.util.TimerThread.run(Timer.java:505)
  937.  
  938. "New I/O worker #33" daemon prio=10 tid=0x000000000188d000 nid=0x4a69 runnable [0x00007f0c5bcfb000]
  939. java.lang.Thread.State: RUNNABLE
  940. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  941. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228)
  942. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:81)
  943. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
  944. - locked <0x00007f0cfbecd7c8> (a sun.nio.ch.Util$2)
  945. - locked <0x00007f0cfbecd7b0> (a java.util.Collections$UnmodifiableSet)
  946. - locked <0x00007f0cfbecd110> (a sun.nio.ch.EPollSelectorImpl)
  947. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
  948. at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:52)
  949. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:223)
  950. at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
  951. at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)
  952. at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
  953. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  954. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  955. at java.lang.Thread.run(Thread.java:724)
  956.  
  957. "Hashed wheel timer #2" daemon prio=10 tid=0x00007f0ce0712000 nid=0x4a66 waiting on condition [0x00007f0c5bffe000]
  958. java.lang.Thread.State: TIMED_WAITING (sleeping)
  959. at java.lang.Thread.sleep(Native Method)
  960. at org.jboss.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:464)
  961. at org.jboss.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:373)
  962. at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)
  963. at java.lang.Thread.run(Thread.java:724)
  964.  
  965. "BroadcastVars cleanup timer" daemon prio=10 tid=0x00007f0ce0711000 nid=0x4a65 in Object.wait() [0x00007f0c60140000]
  966. java.lang.Thread.State: WAITING (on object monitor)
  967. at java.lang.Object.wait(Native Method)
  968. - waiting on <0x00007f0cfc39e8a0> (a java.util.TaskQueue)
  969. at java.lang.Object.wait(Object.java:503)
  970. at java.util.TimerThread.mainLoop(Timer.java:526)
  971. - locked <0x00007f0cfc39e8a0> (a java.util.TaskQueue)
  972. at java.util.TimerThread.run(Timer.java:505)
  973.  
  974. "BlockManager cleanup timer" daemon prio=10 tid=0x00007f0ce0704000 nid=0x4a64 in Object.wait() [0x00007f0c60241000]
  975. java.lang.Thread.State: WAITING (on object monitor)
  976. at java.lang.Object.wait(Native Method)
  977. - waiting on <0x00007f0cfc39ef48> (a java.util.TaskQueue)
  978. at java.lang.Object.wait(Object.java:503)
  979. at java.util.TimerThread.mainLoop(Timer.java:526)
  980. - locked <0x00007f0cfc39ef48> (a java.util.TaskQueue)
  981. at java.util.TimerThread.run(Timer.java:505)
  982.  
  983. "connection-manager-thread" daemon prio=10 tid=0x00007f0ce0723000 nid=0x4a63 runnable [0x00007f0c60342000]
  984. java.lang.Thread.State: RUNNABLE
  985. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  986. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228)
  987. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:81)
  988. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
  989. - locked <0x00007f0cfbf11b10> (a sun.nio.ch.Util$2)
  990. - locked <0x00007f0cfbf11af8> (a java.util.Collections$UnmodifiableSet)
  991. - locked <0x00007f0cfbf0d1f0> (a sun.nio.ch.EPollSelectorImpl)
  992. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
  993. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
  994. at org.apache.spark.network.ConnectionManager.run(ConnectionManager.scala:283)
  995. at org.apache.spark.network.ConnectionManager$$anon$3.run(ConnectionManager.scala:98)
  996.  
  997. "DiskBlockManager cleanup timer" daemon prio=10 tid=0x00007f0ce0709000 nid=0x4a61 in Object.wait() [0x00007f0c60443000]
  998. java.lang.Thread.State: WAITING (on object monitor)
  999. at java.lang.Object.wait(Native Method)
  1000. - waiting on <0x00007f0cfc39f5f8> (a java.util.TaskQueue)
  1001. at java.lang.Object.wait(Object.java:503)
  1002. at java.util.TimerThread.mainLoop(Timer.java:526)
  1003. - locked <0x00007f0cfc39f5f8> (a java.util.TaskQueue)
  1004. at java.util.TimerThread.run(Timer.java:505)
  1005.  
  1006. "New I/O server boss #2 ([id: 0x319117be, /10.45.168.151:36351])" daemon prio=10 tid=0x00007f0ce04a8000 nid=0x4a60 runnable [0x00007f0c60544000]
  1007. java.lang.Thread.State: RUNNABLE
  1008. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1009. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228)
  1010. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:81)
  1011. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
  1012. - locked <0x00007f0cfbefa460> (a sun.nio.ch.Util$2)
  1013. - locked <0x00007f0cfbefa448> (a java.util.Collections$UnmodifiableSet)
  1014. - locked <0x00007f0cfbef9d70> (a sun.nio.ch.EPollSelectorImpl)
  1015. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
  1016. at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink$Boss.run(NioServerSocketPipelineSink.java:229)
  1017. at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)
  1018. at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
  1019. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  1020. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  1021. at java.lang.Thread.run(Thread.java:724)
  1022.  
  1023. "spark-scheduler-5" daemon prio=10 tid=0x00007f0ce04a5800 nid=0x4a5f waiting on condition [0x00007f0c60645000]
  1024. java.lang.Thread.State: TIMED_WAITING (sleeping)
  1025. at java.lang.Thread.sleep(Native Method)
  1026. at org.jboss.netty.akka.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:418)
  1027. at org.jboss.netty.akka.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:318)
  1028. at java.lang.Thread.run(Thread.java:724)
  1029.  
  1030. "New I/O worker #1" daemon prio=10 tid=0x000000000193b000 nid=0x4a56 runnable [0x00007f0c60f4e000]
  1031. java.lang.Thread.State: RUNNABLE
  1032. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1033. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228)
  1034. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:81)
  1035. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
  1036. - locked <0x00007f0cfbc00d68> (a sun.nio.ch.Util$2)
  1037. - locked <0x00007f0cfbc00d50> (a java.util.Collections$UnmodifiableSet)
  1038. - locked <0x00007f0cfbc005f0> (a sun.nio.ch.EPollSelectorImpl)
  1039. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
  1040. at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:52)
  1041. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:223)
  1042. at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
  1043. at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)
  1044. at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
  1045. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  1046. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  1047. at java.lang.Thread.run(Thread.java:724)
  1048.  
  1049. "Hashed wheel timer #1" daemon prio=10 tid=0x00007f0c5c2b9800 nid=0x4a53 waiting on condition [0x00007f0c61251000]
  1050. java.lang.Thread.State: TIMED_WAITING (sleeping)
  1051. at java.lang.Thread.sleep(Native Method)
  1052. at org.jboss.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:464)
  1053. at org.jboss.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:373)
  1054. at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)
  1055. at java.lang.Thread.run(Thread.java:724)
  1056.  
  1057. "New I/O server boss #1 ([id: 0x0afdd280, /10.45.168.151:40445])" daemon prio=10 tid=0x00007f0c5c290800 nid=0x4a52 runnable [0x00007f0c61352000]
  1058. java.lang.Thread.State: RUNNABLE
  1059. at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1060. at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228)
  1061. at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:81)
  1062. at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
  1063. - locked <0x00007f0cfbc9a3a8> (a sun.nio.ch.Util$2)
  1064. - locked <0x00007f0cfbc9a390> (a java.util.Collections$UnmodifiableSet)
  1065. - locked <0x00007f0cfbc99cb8> (a sun.nio.ch.EPollSelectorImpl)
  1066. at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
  1067. at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink$Boss.run(NioServerSocketPipelineSink.java:229)
  1068. at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)
  1069. at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
  1070. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  1071. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  1072. at java.lang.Thread.run(Thread.java:724)
  1073.  
  1074. "sparkExecutor-scheduler-2" daemon prio=10 tid=0x00007f0c5c004000 nid=0x4a51 waiting on condition [0x00007f0c61659000]
  1075. java.lang.Thread.State: TIMED_WAITING (sleeping)
  1076. at java.lang.Thread.sleep(Native Method)
  1077. at org.jboss.netty.akka.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:418)
  1078. at org.jboss.netty.akka.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:318)
  1079. at java.lang.Thread.run(Thread.java:724)
  1080.  
  1081. "sparkExecutor-akka.actor.default-dispatcher-5" daemon prio=10 tid=0x00007f0ce03e4800 nid=0x4a50 waiting on condition [0x00007f0c6175a000]
  1082. java.lang.Thread.State: WAITING (parking)
  1083. at sun.misc.Unsafe.park(Native Method)
  1084. - parking to wait for <0x00007f0cfc282dd0> (a akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinPool)
  1085. at akka.jsr166y.ForkJoinPool.scan(ForkJoinPool.java:1594)
  1086. at akka.jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1479)
  1087. at akka.jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
  1088.  
  1089. "Service Thread" daemon prio=10 tid=0x00007f0ce0060800 nid=0x4a4c runnable [0x0000000000000000]
  1090. java.lang.Thread.State: RUNNABLE
  1091.  
  1092. "C2 CompilerThread1" daemon prio=10 tid=0x00007f0ce005e000 nid=0x4a4b waiting on condition [0x0000000000000000]
  1093. java.lang.Thread.State: RUNNABLE
  1094.  
  1095. "C2 CompilerThread0" daemon prio=10 tid=0x00007f0ce005b000 nid=0x4a4a waiting on condition [0x0000000000000000]
  1096. java.lang.Thread.State: RUNNABLE
  1097.  
  1098. "Signal Dispatcher" daemon prio=10 tid=0x00007f0ce0058800 nid=0x4a49 runnable [0x0000000000000000]
  1099. java.lang.Thread.State: RUNNABLE
  1100.  
  1101. "Finalizer" daemon prio=10 tid=0x00007f0ce0001000 nid=0x4a48 in Object.wait() [0x00007f0ce459a000]
  1102. java.lang.Thread.State: WAITING (on object monitor)
  1103. at java.lang.Object.wait(Native Method)
  1104. - waiting on <0x00007f0cfb886868> (a java.lang.ref.ReferenceQueue$Lock)
  1105. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
  1106. - locked <0x00007f0cfb886868> (a java.lang.ref.ReferenceQueue$Lock)
  1107. at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
  1108. at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
  1109.  
  1110. "Reference Handler" daemon prio=10 tid=0x0000000001596000 nid=0x4a47 in Object.wait() [0x00007f0ce469b000]
  1111. java.lang.Thread.State: WAITING (on object monitor)
  1112. at java.lang.Object.wait(Native Method)
  1113. - waiting on <0x00007f0cfd06b2b0> (a java.lang.ref.Reference$Lock)
  1114. at java.lang.Object.wait(Object.java:503)
  1115. at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
  1116. - locked <0x00007f0cfd06b2b0> (a java.lang.ref.Reference$Lock)
  1117.  
  1118. "main" prio=10 tid=0x00000000014e6800 nid=0x4a3d waiting on condition [0x00007f1941ac3000]
  1119. java.lang.Thread.State: WAITING (parking)
  1120. at sun.misc.Unsafe.park(Native Method)
  1121. - parking to wait for <0x00007f0cfc286080> (a java.util.concurrent.CountDownLatch$Sync)
  1122. at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  1123. at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
  1124. at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
  1125. at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
  1126. at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
  1127. at akka.actor.ActorSystemImpl$TerminationCallbacks.ready(ActorSystem.scala:736)
  1128. at akka.actor.ActorSystemImpl$TerminationCallbacks.ready(ActorSystem.scala:705)
  1129. at akka.dispatch.Await$.ready(Future.scala:64)
  1130. at akka.actor.ActorSystemImpl.awaitTermination(ActorSystem.scala:604)
  1131. at akka.actor.ActorSystemImpl.awaitTermination(ActorSystem.scala:605)
  1132. at org.apache.spark.executor.CoarseGrainedExecutorBackend$.run(CoarseGrainedExecutorBackend.scala:109)
  1133. at org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:120)
  1134. at org.apache.spark.executor.CoarseGrainedExecutorBackend.main(CoarseGrainedExecutorBackend.scala)
  1135.  
  1136. "VM Thread" prio=10 tid=0x000000000158d800 nid=0x4a46 runnable
  1137.  
  1138. "GC task thread#0 (ParallelGC)" prio=10 tid=0x00000000014f4800 nid=0x4a3e runnable
  1139.  
  1140. "GC task thread#1 (ParallelGC)" prio=10 tid=0x00000000014f6000 nid=0x4a3f runnable
  1141.  
  1142. "GC task thread#2 (ParallelGC)" prio=10 tid=0x00000000014f8000 nid=0x4a40 runnable
  1143.  
  1144. "GC task thread#3 (ParallelGC)" prio=10 tid=0x00000000014fa000 nid=0x4a41 runnable
  1145.  
  1146. "GC task thread#4 (ParallelGC)" prio=10 tid=0x00000000014fb800 nid=0x4a42 runnable
  1147.  
  1148. "GC task thread#5 (ParallelGC)" prio=10 tid=0x00000000014fd800 nid=0x4a43 runnable
  1149.  
  1150. "GC task thread#6 (ParallelGC)" prio=10 tid=0x00000000014ff800 nid=0x4a44 runnable
  1151.  
  1152. "GC task thread#7 (ParallelGC)" prio=10 tid=0x0000000001501800 nid=0x4a45 runnable
  1153.  
  1154. "VM Periodic Task Thread" prio=10 tid=0x00007f0ce006b000 nid=0x4a4d waiting on condition
  1155.  
  1156. JNI global references: 279
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!