API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Jan 25th, 2019
188
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 26.66 KB | None | 0 0
raw download clone embed print report
  1. [2019-01-25T10:43:14,316][INFO ][o.e.n.Node ] [] initializing ...
  2. [2019-01-25T10:43:14,424][INFO ][o.e.e.NodeEnvironment ] [lJgJwYh] using [1] data paths, mounts [[/ (/dev/sda1)]], net usable_space [1.5tb], net total_space [1.7tb], spins? [possibly], types [ext4]
  3. [2019-01-25T10:43:14,425][INFO ][o.e.e.NodeEnvironment ] [lJgJwYh] heap size [1.9gb], compressed ordinary object pointers [true]
  4. [2019-01-25T10:43:14,442][INFO ][o.e.n.Node ] node name [lJgJwYh] derived from node ID [lJgJwYhmQA-fucaOBHD5Vg]; set [node.name] to override
  5. [2019-01-25T10:43:14,442][INFO ][o.e.n.Node ] version[5.6.9], pid[8804], build[877a590/2018-04-12T16:25:14.838Z], OS[Linux/4.9.0-8-amd64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_131/25.131-b11]
  6. [2019-01-25T10:43:14,442][INFO ][o.e.n.Node ] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/home/christian/elasticsearch-5.6.9-2]
  7. [2019-01-25T10:43:14,912][INFO ][o.e.p.PluginsService ] [lJgJwYh] loaded module [aggs-matrix-stats]
  8. [2019-01-25T10:43:14,913][INFO ][o.e.p.PluginsService ] [lJgJwYh] loaded module [ingest-common]
  9. [2019-01-25T10:43:14,913][INFO ][o.e.p.PluginsService ] [lJgJwYh] loaded module [lang-expression]
  10. [2019-01-25T10:43:14,913][INFO ][o.e.p.PluginsService ] [lJgJwYh] loaded module [lang-groovy]
  11. [2019-01-25T10:43:14,913][INFO ][o.e.p.PluginsService ] [lJgJwYh] loaded module [lang-mustache]
  12. [2019-01-25T10:43:14,913][INFO ][o.e.p.PluginsService ] [lJgJwYh] loaded module [lang-painless]
  13. [2019-01-25T10:43:14,913][INFO ][o.e.p.PluginsService ] [lJgJwYh] loaded module [parent-join]
  14. [2019-01-25T10:43:14,913][INFO ][o.e.p.PluginsService ] [lJgJwYh] loaded module [percolator]
  15. [2019-01-25T10:43:14,913][INFO ][o.e.p.PluginsService ] [lJgJwYh] loaded module [reindex]
  16. [2019-01-25T10:43:14,913][INFO ][o.e.p.PluginsService ] [lJgJwYh] loaded module [transport-netty3]
  17. [2019-01-25T10:43:14,913][INFO ][o.e.p.PluginsService ] [lJgJwYh] loaded module [transport-netty4]
  18. [2019-01-25T10:43:14,913][INFO ][o.e.p.PluginsService ] [lJgJwYh] no plugins loaded
  19. [2019-01-25T10:43:15,793][INFO ][o.e.d.DiscoveryModule ] [lJgJwYh] using discovery type [zen]
  20. [2019-01-25T10:43:16,265][INFO ][o.e.n.Node ] initialized
  21. [2019-01-25T10:43:16,266][INFO ][o.e.n.Node ] [lJgJwYh] starting ...
  22. [2019-01-25T10:43:16,363][INFO ][o.e.t.TransportService ] [lJgJwYh] publish_address {10.231.83.146:9301}, bound_addresses {10.231.83.146:9301}
  23. [2019-01-25T10:43:16,372][INFO ][o.e.b.BootstrapChecks ] [lJgJwYh] bound or publishing to a non-loopback address, enforcing bootstrap checks
  24. [2019-01-25T10:43:19,403][WARN ][o.e.d.z.ZenDiscovery ] [lJgJwYh] failed to connect to master [{wAVZzky}{wAVZzkyrSfyBmuGRw0Bltw}{r1L7UXpESgiM_Kxq_jFUjw}{10.231.83.146}{10.231.83.146:9302}], retrying...
  25. org.elasticsearch.transport.ConnectTransportException: [wAVZzky][10.231.83.146:9302] connect_timeout[30s]
  26. at org.elasticsearch.transport.netty4.Netty4Transport.connectToChannels(Netty4Transport.java:363) ~[?:?]
  27. at org.elasticsearch.transport.TcpTransport.openConnection(TcpTransport.java:570) ~[elasticsearch-5.6.9.jar:5.6.9]
  28. at org.elasticsearch.transport.TcpTransport.connectToNode(TcpTransport.java:473) ~[elasticsearch-5.6.9.jar:5.6.9]
  29. at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:342) ~[elasticsearch-5.6.9.jar:5.6.9]
  30. at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:329) ~[elasticsearch-5.6.9.jar:5.6.9]
  31. at org.elasticsearch.discovery.zen.ZenDiscovery.joinElectedMaster(ZenDiscovery.java:458) [elasticsearch-5.6.9.jar:5.6.9]
  32. at org.elasticsearch.discovery.zen.ZenDiscovery.innerJoinCluster(ZenDiscovery.java:410) [elasticsearch-5.6.9.jar:5.6.9]
  33. at org.elasticsearch.discovery.zen.ZenDiscovery.access$4100(ZenDiscovery.java:82) [elasticsearch-5.6.9.jar:5.6.9]
  34. at org.elasticsearch.discovery.zen.ZenDiscovery$JoinThreadControl$1.run(ZenDiscovery.java:1188) [elasticsearch-5.6.9.jar:5.6.9]
  35. at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:575) [elasticsearch-5.6.9.jar:5.6.9]
  36. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
  37. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
  38. at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
  39. Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: 10.231.83.146/10.231.83.146:9302
  40. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
  41. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:?]
  42. at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:352) ~[?:?]
  43. at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340) ~[?:?]
  44. at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:632) ~[?:?]
  45. at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) ~[?:?]
  46. at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) ~[?:?]
  47. at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) ~[?:?]
  48. at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) ~[?:?]
  49. ... 1 more
  50. Caused by: java.net.ConnectException: Connection refused
  51. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
  52. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:?]
  53. at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:352) ~[?:?]
  54. at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340) ~[?:?]
  55. at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:632) ~[?:?]
  56. at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) ~[?:?]
  57. at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) ~[?:?]
  58. at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) ~[?:?]
  59. at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) ~[?:?]
  60. ... 1 more
  61. [2019-01-25T10:43:22,583][INFO ][o.e.c.s.ClusterService ] [lJgJwYh] detected_master {yRj9xlW}{yRj9xlW8Q-WgQ0j6H4DbOQ}{SAn1BH7ES42OhsAknxVe0Q}{10.231.83.238}{10.231.83.238:9300}, added {{yRj9xlW}{yRj9xlW8Q-WgQ0j6H4DbOQ}{SAn1BH7ES42OhsAknxVe0Q}{10.231.83.238}{10.231.83.238:9300},{wAVZzky}{wAVZzkyrSfyBmuGRw0Bltw}{r1L7UXpESgiM_Kxq_jFUjw}{10.231.83.146}{10.231.83.146:9302},}, reason: zen-disco-receive(from master [master {yRj9xlW}{yRj9xlW8Q-WgQ0j6H4DbOQ}{SAn1BH7ES42OhsAknxVe0Q}{10.231.83.238}{10.231.83.238:9300} committed version [183]])
  62. [2019-01-25T10:43:22,604][WARN ][o.e.c.NodeConnectionsService] [lJgJwYh] failed to connect to node {wAVZzky}{wAVZzkyrSfyBmuGRw0Bltw}{r1L7UXpESgiM_Kxq_jFUjw}{10.231.83.146}{10.231.83.146:9302} (tried [1] times)
  63. org.elasticsearch.transport.ConnectTransportException: [wAVZzky][10.231.83.146:9302] handshake failed. unexpected remote node {wAVZzky}{wAVZzkyrSfyBmuGRw0Bltw}{Qc8kODytQreZFj_24t5O7g}{10.231.83.146}{10.231.83.146:9302}
  64. at org.elasticsearch.transport.TransportService.lambda$connectToNode$3(TransportService.java:346) ~[elasticsearch-5.6.9.jar:5.6.9]
  65. at org.elasticsearch.transport.TcpTransport.connectToNode(TcpTransport.java:474) ~[elasticsearch-5.6.9.jar:5.6.9]
  66. at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:342) ~[elasticsearch-5.6.9.jar:5.6.9]
  67. at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:329) ~[elasticsearch-5.6.9.jar:5.6.9]
  68. at org.elasticsearch.cluster.NodeConnectionsService.validateAndConnectIfNeeded(NodeConnectionsService.java:154) [elasticsearch-5.6.9.jar:5.6.9]
  69. at org.elasticsearch.cluster.NodeConnectionsService$1.doRun(NodeConnectionsService.java:107) [elasticsearch-5.6.9.jar:5.6.9]
  70. at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:674) [elasticsearch-5.6.9.jar:5.6.9]
  71. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.6.9.jar:5.6.9]
  72. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
  73. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
  74. at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
  75. [2019-01-25T10:43:23,023][INFO ][o.e.h.n.Netty4HttpServerTransport] [lJgJwYh] publish_address {10.231.83.146:9201}, bound_addresses {10.231.83.146:9201}
  76. [2019-01-25T10:43:23,023][INFO ][o.e.n.Node ] [lJgJwYh] started
  77. [2019-01-25T10:43:23,057][INFO ][o.e.c.s.ClusterService ] [lJgJwYh] removed {{wAVZzky}{wAVZzkyrSfyBmuGRw0Bltw}{r1L7UXpESgiM_Kxq_jFUjw}{10.231.83.146}{10.231.83.146:9302},}, reason: zen-disco-receive(from master [master {yRj9xlW}{yRj9xlW8Q-WgQ0j6H4DbOQ}{SAn1BH7ES42OhsAknxVe0Q}{10.231.83.238}{10.231.83.238:9300} committed version [184]])
  78. [2019-01-25T10:43:24,494][INFO ][o.e.c.s.ClusterService ] [lJgJwYh] added {{wAVZzky}{wAVZzkyrSfyBmuGRw0Bltw}{Qc8kODytQreZFj_24t5O7g}{10.231.83.146}{10.231.83.146:9302},}, reason: zen-disco-receive(from master [master {yRj9xlW}{yRj9xlW8Q-WgQ0j6H4DbOQ}{SAn1BH7ES42OhsAknxVe0Q}{10.231.83.238}{10.231.83.238:9300} committed version [188]])
  79. [2019-01-25T10:44:30,153][INFO ][o.e.d.z.ZenDiscovery ] [lJgJwYh] master_left [{yRj9xlW}{yRj9xlW8Q-WgQ0j6H4DbOQ}{SAn1BH7ES42OhsAknxVe0Q}{10.231.83.238}{10.231.83.238:9300}], reason [failed to ping, tried [3] times, each with maximum [4s] timeout]
  80. [2019-01-25T10:44:30,156][WARN ][o.e.d.z.ZenDiscovery ] [lJgJwYh] master left (reason = failed to ping, tried [3] times, each with maximum [4s] timeout), current nodes: nodes:
  81. {yRj9xlW}{yRj9xlW8Q-WgQ0j6H4DbOQ}{SAn1BH7ES42OhsAknxVe0Q}{10.231.83.238}{10.231.83.238:9300}, master
  82. {lJgJwYh}{lJgJwYhmQA-fucaOBHD5Vg}{lbN7QcyiSWCOBYNL4jNkdg}{10.231.83.146}{10.231.83.146:9301}, local
  83. {wAVZzky}{wAVZzkyrSfyBmuGRw0Bltw}{Qc8kODytQreZFj_24t5O7g}{10.231.83.146}{10.231.83.146:9302}
  84.  
  85. [2019-01-25T10:44:33,319][INFO ][o.e.c.s.ClusterService ] [lJgJwYh] new_master {lJgJwYh}{lJgJwYhmQA-fucaOBHD5Vg}{lbN7QcyiSWCOBYNL4jNkdg}{10.231.83.146}{10.231.83.146:9301}, reason: zen-disco-elected-as-master ([1] nodes joined)[, ]
  86. [2019-01-25T10:44:33,327][DEBUG][o.e.a.a.c.h.TransportClusterHealthAction] [lJgJwYh] no known master node, scheduling a retry
  87. [2019-01-25T10:44:33,327][DEBUG][o.e.a.a.c.h.TransportClusterHealthAction] [lJgJwYh] no known master node, scheduling a retry
  88. [2019-01-25T10:44:33,910][WARN ][o.e.d.z.UnicastZenPing ] [lJgJwYh] failed to send ping to [{yRj9xlW}{yRj9xlW8Q-WgQ0j6H4DbOQ}{SAn1BH7ES42OhsAknxVe0Q}{10.231.83.238}{10.231.83.238:9300}]
  89. org.elasticsearch.transport.ReceiveTimeoutTransportException: [yRj9xlW][10.231.83.238:9300][internal:discovery/zen/unicast] request_id [111] timed out after [3750ms]
  90. at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:961) [elasticsearch-5.6.9.jar:5.6.9]
  91. at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:575) [elasticsearch-5.6.9.jar:5.6.9]
  92. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
  93. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
  94. at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
  95. [2019-01-25T10:44:34,717][DEBUG][o.e.a.a.c.h.TransportClusterHealthAction] [lJgJwYh] no known master node, scheduling a retry
  96. [2019-01-25T10:44:34,910][WARN ][o.e.d.z.UnicastZenPing ] [lJgJwYh] failed to send ping to [{yRj9xlW}{yRj9xlW8Q-WgQ0j6H4DbOQ}{SAn1BH7ES42OhsAknxVe0Q}{10.231.83.238}{10.231.83.238:9300}]
  97. org.elasticsearch.transport.ReceiveTimeoutTransportException: [yRj9xlW][10.231.83.238:9300][internal:discovery/zen/unicast] request_id [114] timed out after [3751ms]
  98. at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:961) [elasticsearch-5.6.9.jar:5.6.9]
  99. at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:575) [elasticsearch-5.6.9.jar:5.6.9]
  100. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
  101. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
  102. at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
  103. [2019-01-25T10:44:35,910][WARN ][o.e.d.z.UnicastZenPing ] [lJgJwYh] failed to send ping to [{yRj9xlW}{yRj9xlW8Q-WgQ0j6H4DbOQ}{SAn1BH7ES42OhsAknxVe0Q}{10.231.83.238}{10.231.83.238:9300}]
  104. org.elasticsearch.transport.ReceiveTimeoutTransportException: [yRj9xlW][10.231.83.238:9300][internal:discovery/zen/unicast] request_id [118] timed out after [3750ms]
  105. at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:961) [elasticsearch-5.6.9.jar:5.6.9]
  106. at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:575) [elasticsearch-5.6.9.jar:5.6.9]
  107. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
  108. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
  109. at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
  110. [2019-01-25T10:44:36,727][DEBUG][o.e.a.a.c.h.TransportClusterHealthAction] [lJgJwYh] no known master node, scheduling a retry
  111. [2019-01-25T10:44:38,738][DEBUG][o.e.a.a.c.h.TransportClusterHealthAction] [lJgJwYh] no known master node, scheduling a retry
  112. [2019-01-25T10:44:40,748][DEBUG][o.e.a.a.c.h.TransportClusterHealthAction] [lJgJwYh] no known master node, scheduling a retry
  113. [2019-01-25T10:44:42,758][DEBUG][o.e.a.a.c.h.TransportClusterHealthAction] [lJgJwYh] no known master node, scheduling a retry
  114. [2019-01-25T10:44:44,767][DEBUG][o.e.a.a.c.h.TransportClusterHealthAction] [lJgJwYh] no known master node, scheduling a retry
  115. [2019-01-25T10:45:03,322][WARN ][o.e.d.z.PublishClusterStateAction] [lJgJwYh] timed out waiting for all nodes to process published state [200] (timeout [30s], pending nodes: [{yRj9xlW}{yRj9xlW8Q-WgQ0j6H4DbOQ}{SAn1BH7ES42OhsAknxVe0Q}{10.231.83.238}{10.231.83.238:9300}])
  116. [2019-01-25T10:45:03,327][WARN ][o.e.c.s.ClusterService ] [lJgJwYh] cluster state update task [zen-disco-elected-as-master ([1] nodes joined)[, ]] took [30s] above the warn threshold of 30s
  117. ^C[2019-01-25T10:45:14,479][INFO ][o.e.n.Node ] [lJgJwYh] stopping ...
  118. [2019-01-25T10:45:16,277][WARN ][o.e.c.NodeConnectionsService] [lJgJwYh] failed to connect to node {wAVZzky}{wAVZzkyrSfyBmuGRw0Bltw}{Qc8kODytQreZFj_24t5O7g}{10.231.83.146}{10.231.83.146:9302} (tried [1] times)
  119. org.elasticsearch.transport.ConnectTransportException: [wAVZzky][10.231.83.146:9302] connect_timeout[30s]
  120. at org.elasticsearch.transport.netty4.Netty4Transport.connectToChannels(Netty4Transport.java:363) ~[?:?]
  121. at org.elasticsearch.transport.TcpTransport.openConnection(TcpTransport.java:570) ~[elasticsearch-5.6.9.jar:5.6.9]
  122. at org.elasticsearch.transport.TcpTransport.connectToNode(TcpTransport.java:473) ~[elasticsearch-5.6.9.jar:5.6.9]
  123. at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:342) ~[elasticsearch-5.6.9.jar:5.6.9]
  124. at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:329) ~[elasticsearch-5.6.9.jar:5.6.9]
  125. at org.elasticsearch.cluster.NodeConnectionsService.validateAndConnectIfNeeded(NodeConnectionsService.java:154) [elasticsearch-5.6.9.jar:5.6.9]
  126. at org.elasticsearch.cluster.NodeConnectionsService$ConnectionChecker.doRun(NodeConnectionsService.java:183) [elasticsearch-5.6.9.jar:5.6.9]
  127. at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:674) [elasticsearch-5.6.9.jar:5.6.9]
  128. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.6.9.jar:5.6.9]
  129. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
  130. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
  131. at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
  132. Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: 10.231.83.146/10.231.83.146:9302
  133. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
  134. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:?]
  135. at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:352) ~[?:?]
  136. at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340) ~[?:?]
  137. at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:632) ~[?:?]
  138. at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) ~[?:?]
  139. at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) ~[?:?]
  140. at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) ~[?:?]
  141. at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) ~[?:?]
  142. ... 1 more
  143. Caused by: java.net.ConnectException: Connection refused
  144. at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
  145. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:?]
  146. at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:352) ~[?:?]
  147. at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340) ~[?:?]
  148. at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:632) ~[?:?]
  149. at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) ~[?:?]
  150. at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) ~[?:?]
  151. at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) ~[?:?]
  152. at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) ~[?:?]
  153. ... 1 more
  154. [2019-01-25T10:45:18,333][DEBUG][o.e.a.a.c.n.s.TransportNodesStatsAction] [lJgJwYh] failed to execute on node [yRj9xlW8Q-WgQ0j6H4DbOQ]
  155. org.elasticsearch.transport.ReceiveTimeoutTransportException: [yRj9xlW][10.231.83.238:9300][cluster:monitor/nodes/stats[n]] request_id [128] timed out after [15000ms]
  156. at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:961) [elasticsearch-5.6.9.jar:5.6.9]
  157. at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:575) [elasticsearch-5.6.9.jar:5.6.9]
  158. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
  159. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
  160. at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
  161. [2019-01-25T10:45:24,502][WARN ][o.e.e.NodeEnvironment ] [lJgJwYh] lock assertion failed
  162. java.nio.channels.ClosedByInterruptException: null
  163. at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) ~[?:1.8.0_131]
  164. at sun.nio.ch.FileChannelImpl.size(FileChannelImpl.java:315) ~[?:?]
  165. at org.apache.lucene.store.NativeFSLockFactory$NativeFSLock.ensureValid(NativeFSLockFactory.java:170) ~[lucene-core-6.6.1.jar:6.6.1 9aa465a89b64ff2dabe7b4d50c472de32c298683 - varunthacker - 2017-08-29 21:54:39]
  166. at org.elasticsearch.env.NodeEnvironment.assertEnvIsLocked(NodeEnvironment.java:941) ~[elasticsearch-5.6.9.jar:5.6.9]
  167. at org.elasticsearch.env.NodeEnvironment.availableIndexFolders(NodeEnvironment.java:820) ~[elasticsearch-5.6.9.jar:5.6.9]
  168. at org.elasticsearch.gateway.MetaStateService.loadIndicesStates(MetaStateService.java:89) ~[elasticsearch-5.6.9.jar:5.6.9]
  169. at org.elasticsearch.gateway.DanglingIndicesState.findNewDanglingIndices(DanglingIndicesState.java:131) ~[elasticsearch-5.6.9.jar:5.6.9]
  170. at org.elasticsearch.gateway.DanglingIndicesState.findNewAndAddDanglingIndices(DanglingIndicesState.java:116) ~[elasticsearch-5.6.9.jar:5.6.9]
  171. at org.elasticsearch.gateway.DanglingIndicesState.processDanglingIndices(DanglingIndicesState.java:81) ~[elasticsearch-5.6.9.jar:5.6.9]
  172. at org.elasticsearch.gateway.DanglingIndicesState.clusterChanged(DanglingIndicesState.java:185) ~[elasticsearch-5.6.9.jar:5.6.9]
  173. at org.elasticsearch.cluster.service.ClusterService.lambda$publishAndApplyChanges$7(ClusterService.java:777) ~[elasticsearch-5.6.9.jar:5.6.9]
  174. at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) [?:1.8.0_131]
  175. at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:742) [?:1.8.0_131]
  176. at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) [?:1.8.0_131]
  177. at org.elasticsearch.cluster.service.ClusterService.publishAndApplyChanges(ClusterService.java:774) [elasticsearch-5.6.9.jar:5.6.9]
  178. at org.elasticsearch.cluster.service.ClusterService.runTasks(ClusterService.java:587) [elasticsearch-5.6.9.jar:5.6.9]
  179. at org.elasticsearch.cluster.service.ClusterService$ClusterServiceTaskBatcher.run(ClusterService.java:263) [elasticsearch-5.6.9.jar:5.6.9]
  180. at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-5.6.9.jar:5.6.9]
  181. at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-5.6.9.jar:5.6.9]
  182. at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:575) [elasticsearch-5.6.9.jar:5.6.9]
  183. at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:247) [elasticsearch-5.6.9.jar:5.6.9]
  184. at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:210) [elasticsearch-5.6.9.jar:5.6.9]
  185. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
  186. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
  187. at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
  188. [2019-01-25T10:45:24,505][WARN ][o.e.c.s.ClusterService ] [lJgJwYh] failed to notify ClusterStateListener
  189. java.lang.IllegalStateException: environment is not locked
  190. at org.elasticsearch.env.NodeEnvironment.assertEnvIsLocked(NodeEnvironment.java:944) ~[elasticsearch-5.6.9.jar:5.6.9]
  191. at org.elasticsearch.env.NodeEnvironment.availableIndexFolders(NodeEnvironment.java:820) ~[elasticsearch-5.6.9.jar:5.6.9]
  192. at org.elasticsearch.gateway.MetaStateService.loadIndicesStates(MetaStateService.java:89) ~[elasticsearch-5.6.9.jar:5.6.9]
  193. at org.elasticsearch.gateway.DanglingIndicesState.findNewDanglingIndices(DanglingIndicesState.java:131) ~[elasticsearch-5.6.9.jar:5.6.9]
  194. at org.elasticsearch.gateway.DanglingIndicesState.findNewAndAddDanglingIndices(DanglingIndicesState.java:116) ~[elasticsearch-5.6.9.jar:5.6.9]
  195. at org.elasticsearch.gateway.DanglingIndicesState.processDanglingIndices(DanglingIndicesState.java:81) ~[elasticsearch-5.6.9.jar:5.6.9]
  196. at org.elasticsearch.gateway.DanglingIndicesState.clusterChanged(DanglingIndicesState.java:185) ~[elasticsearch-5.6.9.jar:5.6.9]
  197. at org.elasticsearch.cluster.service.ClusterService.lambda$publishAndApplyChanges$7(ClusterService.java:777) ~[elasticsearch-5.6.9.jar:5.6.9]
  198. at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) [?:1.8.0_131]
  199. at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:742) [?:1.8.0_131]
  200. at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) [?:1.8.0_131]
  201. at org.elasticsearch.cluster.service.ClusterService.publishAndApplyChanges(ClusterService.java:774) [elasticsearch-5.6.9.jar:5.6.9]
  202. at org.elasticsearch.cluster.service.ClusterService.runTasks(ClusterService.java:587) [elasticsearch-5.6.9.jar:5.6.9]
  203. at org.elasticsearch.cluster.service.ClusterService$ClusterServiceTaskBatcher.run(ClusterService.java:263) [elasticsearch-5.6.9.jar:5.6.9]
  204. at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-5.6.9.jar:5.6.9]
  205. at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-5.6.9.jar:5.6.9]
  206. at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:575) [elasticsearch-5.6.9.jar:5.6.9]
  207. at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:247) [elasticsearch-5.6.9.jar:5.6.9]
  208. at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:210) [elasticsearch-5.6.9.jar:5.6.9]
  209. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
  210. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
  211. at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
  212. Caused by: java.nio.channels.ClosedByInterruptException
  213. at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) ~[?:1.8.0_131]
  214. at sun.nio.ch.FileChannelImpl.size(FileChannelImpl.java:315) ~[?:?]
  215. at org.apache.lucene.store.NativeFSLockFactory$NativeFSLock.ensureValid(NativeFSLockFactory.java:170) ~[lucene-core-6.6.1.jar:6.6.1 9aa465a89b64ff2dabe7b4d50c472de32c298683 - varunthacker - 2017-08-29 21:54:39]
  216. at org.elasticsearch.env.NodeEnvironment.assertEnvIsLocked(NodeEnvironment.java:941) ~[elasticsearch-5.6.9.jar:5.6.9]
  217. ... 21 more
  218. [2019-01-25T10:45:24,509][DEBUG][o.e.a.a.i.s.TransportIndicesStatsAction] [lJgJwYh] failed to execute [indices:monitor/stats] on node [yRj9xlW8Q-WgQ0j6H4DbOQ]
  219. org.elasticsearch.transport.NodeDisconnectedException: [yRj9xlW][10.231.83.238:9300][indices:monitor/stats[n]] disconnected
  220. [2019-01-25T10:45:24,518][WARN ][o.e.t.n.Netty4Transport ] [lJgJwYh] write and flush on the network layer failed (channel: [id: 0xa0aaf523, L:0.0.0.0/0.0.0.0:9301 ! R:/10.231.83.146:44348])
  221. java.nio.channels.ClosedChannelException: null
  222. at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source) ~[?:?]
  223. [2019-01-25T10:45:24,528][INFO ][o.e.n.Node ] [lJgJwYh] stopped
  224. [2019-01-25T10:45:24,528][INFO ][o.e.n.Node ] [lJgJwYh] closing ...
  225. [2019-01-25T10:45:24,534][INFO ][o.e.n.Node ] [lJgJwYh] closed
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!