API tools faq
paste
Advertisement
tatdat171

Elastic error upgrade 5.6.2 to 6.0.0

tatdat171
Nov 24th, 2017
235
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 40.21 KB | None | 0 0
raw download clone embed print report
  1. [2017-11-25T00:19:33,552][INFO ][o.e.n.Node ] [Elastic-03] initializing ...
  2. [2017-11-25T00:19:33,697][INFO ][o.e.e.NodeEnvironment ] [Elastic-03] using [1] data paths, mounts [[/data (/dev/sdb1)]], net usable_space [536gb], net total_space [1.8tb], spins? [possibly], types [ext4]
  3. [2017-11-25T00:19:33,698][INFO ][o.e.e.NodeEnvironment ] [Elastic-03] heap size [19.8gb], compressed ordinary object pointers [true]
  4. [2017-11-25T00:19:34,064][INFO ][o.e.n.Node ] [Elastic-03] node name [Elastic-03], node ID [DTbL2JaQQTGTcRPDC5c-iA]
  5. [2017-11-25T00:19:34,065][INFO ][o.e.n.Node ] [Elastic-03] version[5.6.2], pid[12063], build[57e20f3/2017-09-23T13:16:45.703Z], OS[Linux/3.10.0-229.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_121/25.121-b13]
  6. [2017-11-25T00:19:34,065][INFO ][o.e.n.Node ] [Elastic-03] JVM arguments [-Xms20g, -Xmx20g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+DisableExplicitGC, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/usr/share/elasticsearch]
  7. [2017-11-25T00:19:35,329][INFO ][o.e.p.PluginsService ] [Elastic-03] loaded module [aggs-matrix-stats]
  8. [2017-11-25T00:19:35,330][INFO ][o.e.p.PluginsService ] [Elastic-03] loaded module [ingest-common]
  9. [2017-11-25T00:19:35,330][INFO ][o.e.p.PluginsService ] [Elastic-03] loaded module [lang-expression]
  10. [2017-11-25T00:19:35,330][INFO ][o.e.p.PluginsService ] [Elastic-03] loaded module [lang-groovy]
  11. [2017-11-25T00:19:35,330][INFO ][o.e.p.PluginsService ] [Elastic-03] loaded module [lang-mustache]
  12. [2017-11-25T00:19:35,330][INFO ][o.e.p.PluginsService ] [Elastic-03] loaded module [lang-painless]
  13. [2017-11-25T00:19:35,330][INFO ][o.e.p.PluginsService ] [Elastic-03] loaded module [parent-join]
  14. [2017-11-25T00:19:35,330][INFO ][o.e.p.PluginsService ] [Elastic-03] loaded module [percolator]
  15. [2017-11-25T00:19:35,330][INFO ][o.e.p.PluginsService ] [Elastic-03] loaded module [reindex]
  16. [2017-11-25T00:19:35,330][INFO ][o.e.p.PluginsService ] [Elastic-03] loaded module [transport-netty3]
  17. [2017-11-25T00:19:35,330][INFO ][o.e.p.PluginsService ] [Elastic-03] loaded module [transport-netty4]
  18. [2017-11-25T00:19:35,331][INFO ][o.e.p.PluginsService ] [Elastic-03] loaded plugin [x-pack]
  19. [2017-11-25T00:19:37,723][INFO ][o.e.d.DiscoveryModule ] [Elastic-03] using discovery type [zen]
  20. [2017-11-25T00:19:37,725][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/12154] [Main.cc@128] controller (64 bit): Version 5.6.2 (Build 228329870d1c63) Copyright (c) 2017 Elasticsearch BV
  21. [2017-11-25T00:19:39,854][INFO ][o.e.n.Node ] [Elastic-03] initialized
  22. [2017-11-25T00:19:39,855][INFO ][o.e.n.Node ] [Elastic-03] starting ...
  23. [2017-11-25T00:19:40,067][INFO ][o.e.t.TransportService ] [Elastic-03] publish_address {10.1.6.197:9300}, bound_addresses {10.1.6.197:9300}
  24. [2017-11-25T00:19:40,082][INFO ][o.e.b.BootstrapChecks ] [Elastic-03] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
  25. [2017-11-25T00:19:40,279][WARN ][o.e.d.z.UnicastZenPing ] [Elastic-03] [1] failed send ping to {#zen_unicast_10.1.6.196_0#}{DwGkoSRlSY2Fy5QBLZncpQ}{10.1.6.196}{10.1.6.196:9300}
  26. java.lang.IllegalStateException: handshake failed with {#zen_unicast_10.1.6.196_0#}{DwGkoSRlSY2Fy5QBLZncpQ}{10.1.6.196}{10.1.6.196:9300}
  27. at org.elasticsearch.transport.TransportService.handshake(TransportService.java:403) ~[elasticsearch-5.6.2.jar:5.6.2]
  28. at org.elasticsearch.transport.TransportService.handshake(TransportService.java:370) ~[elasticsearch-5.6.2.jar:5.6.2]
  29. at org.elasticsearch.discovery.zen.UnicastZenPing$PingingRound.getOrConnect(UnicastZenPing.java:400) ~[elasticsearch-5.6.2.jar:5.6.2]
  30. at org.elasticsearch.discovery.zen.UnicastZenPing$3.doRun(UnicastZenPing.java:507) [elasticsearch-5.6.2.jar:5.6.2]
  31. at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) [elasticsearch-5.6.2.jar:5.6.2]
  32. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.6.2.jar:5.6.2]
  33. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
  34. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
  35. at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
  36. Caused by: org.elasticsearch.transport.RemoteTransportException: [Elastic-02][10.1.6.196:9300][internal:transport/handshake]
  37. Caused by: org.elasticsearch.ElasticsearchSecurityException: missing authentication token for action [internal:transport/handshake]
  38. at org.elasticsearch.xpack.security.support.Exceptions.authenticationError(Exceptions.java:39) ~[?:?]
  39. at org.elasticsearch.xpack.security.authc.DefaultAuthenticationFailureHandler.missingToken(DefaultAuthenticationFailureHandler.java:74) ~[?:?]
  40. at org.elasticsearch.xpack.security.authc.AuthenticationService$AuditableTransportRequest.anonymousAccessDenied(AuthenticationService.java:553) ~[?:?]
  41. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$handleNullToken$16(AuthenticationService.java:363) ~[?:?]
  42. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.handleNullToken(AuthenticationService.java:368) ~[?:?]
  43. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.consumeToken(AuthenticationService.java:295) ~[?:?]
  44. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$extractToken$7(AuthenticationService.java:267) ~[?:?]
  45. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.extractToken(AuthenticationService.java:284) ~[?:?]
  46. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$null$0(AuthenticationService.java:218) ~[?:?]
  47. at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:59) ~[elasticsearch-5.6.2.jar:5.6.2]
  48. at org.elasticsearch.xpack.security.authc.TokenService.getAndValidateToken(TokenService.java:218) ~[?:?]
  49. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$2(AuthenticationService.java:214) ~[?:?]
  50. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$4(AuthenticationService.java:246) ~[?:?]
  51. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:257) ~[?:?]
  52. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:210) ~[?:?]
  53. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:159) ~[?:?]
  54. at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:122) ~[?:?]
  55. at org.elasticsearch.xpack.security.transport.ServerTransportFilter$NodeProfile.inbound(ServerTransportFilter.java:146) ~[?:?]
  56. at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:314) ~[?:?]
  57. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.6.2.jar:5.6.2]
  58. at org.elasticsearch.transport.TcpTransport$RequestHandler.doRun(TcpTransport.java:1533) ~[elasticsearch-5.6.2.jar:5.6.2]
  59. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.6.2.jar:5.6.2]
  60. at org.elasticsearch.common.util.concurrent.EsExecutors$1.execute(EsExecutors.java:110) ~[elasticsearch-5.6.2.jar:5.6.2]
  61. at org.elasticsearch.transport.TcpTransport.handleRequest(TcpTransport.java:1490) ~[elasticsearch-5.6.2.jar:5.6.2]
  62. at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1374) ~[elasticsearch-5.6.2.jar:5.6.2]
  63. at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:74) ~[?:?]
  64. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
  65. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
  66. at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?]
  67. at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) ~[?:?]
  68. at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:297) ~[?:?]
  69. at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:413) ~[?:?]
  70. at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[?:?]
  71. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
  72. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
  73. at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?]
  74. at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) ~[?:?]
  75. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
  76. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
  77. at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?]
  78. at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) ~[?:?]
  79. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
  80. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
  81. at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) ~[?:?]
  82. at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) ~[?:?]
  83. at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) ~[?:?]
  84. at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) ~[?:?]
  85. at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) ~[?:?]
  86. at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) ~[?:?]
  87. at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) ~[?:?]
  88. ... 1 more
  89. [2017-11-25T00:19:41,155][WARN ][o.e.d.z.UnicastZenPing ] [Elastic-03] [1] failed send ping to {#zen_unicast_10.1.6.196_0#}{DwGkoSRlSY2Fy5QBLZncpQ}{10.1.6.196}{10.1.6.196:9300}
  90. java.lang.IllegalStateException: handshake failed with {#zen_unicast_10.1.6.196_0#}{DwGkoSRlSY2Fy5QBLZncpQ}{10.1.6.196}{10.1.6.196:9300}
  91. at org.elasticsearch.transport.TransportService.handshake(TransportService.java:403) ~[elasticsearch-5.6.2.jar:5.6.2]
  92. at org.elasticsearch.transport.TransportService.handshake(TransportService.java:370) ~[elasticsearch-5.6.2.jar:5.6.2]
  93. at org.elasticsearch.discovery.zen.UnicastZenPing$PingingRound.getOrConnect(UnicastZenPing.java:400) ~[elasticsearch-5.6.2.jar:5.6.2]
  94. at org.elasticsearch.discovery.zen.UnicastZenPing$3.doRun(UnicastZenPing.java:507) [elasticsearch-5.6.2.jar:5.6.2]
  95. at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) [elasticsearch-5.6.2.jar:5.6.2]
  96. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.6.2.jar:5.6.2]
  97. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
  98. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
  99. at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
  100. Caused by: org.elasticsearch.transport.RemoteTransportException: [Elastic-02][10.1.6.196:9300][internal:transport/handshake]
  101. Caused by: org.elasticsearch.ElasticsearchSecurityException: missing authentication token for action [internal:transport/handshake]
  102. at org.elasticsearch.xpack.security.support.Exceptions.authenticationError(Exceptions.java:39) ~[?:?]
  103. at org.elasticsearch.xpack.security.authc.DefaultAuthenticationFailureHandler.missingToken(DefaultAuthenticationFailureHandler.java:74) ~[?:?]
  104. at org.elasticsearch.xpack.security.authc.AuthenticationService$AuditableTransportRequest.anonymousAccessDenied(AuthenticationService.java:553) ~[?:?]
  105. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$handleNullToken$16(AuthenticationService.java:363) ~[?:?]
  106. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.handleNullToken(AuthenticationService.java:368) ~[?:?]
  107. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.consumeToken(AuthenticationService.java:295) ~[?:?]
  108. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$extractToken$7(AuthenticationService.java:267) ~[?:?]
  109. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.extractToken(AuthenticationService.java:284) ~[?:?]
  110. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$null$0(AuthenticationService.java:218) ~[?:?]
  111. at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:59) ~[elasticsearch-5.6.2.jar:5.6.2]
  112. at org.elasticsearch.xpack.security.authc.TokenService.getAndValidateToken(TokenService.java:218) ~[?:?]
  113. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$2(AuthenticationService.java:214) ~[?:?]
  114. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$4(AuthenticationService.java:246) ~[?:?]
  115. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:257) ~[?:?]
  116. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:210) ~[?:?]
  117. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:159) ~[?:?]
  118. at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:122) ~[?:?]
  119. at org.elasticsearch.xpack.security.transport.ServerTransportFilter$NodeProfile.inbound(ServerTransportFilter.java:146) ~[?:?]
  120. at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:314) ~[?:?]
  121. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.6.2.jar:5.6.2]
  122. at org.elasticsearch.transport.TcpTransport$RequestHandler.doRun(TcpTransport.java:1533) ~[elasticsearch-5.6.2.jar:5.6.2]
  123. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.6.2.jar:5.6.2]
  124. at org.elasticsearch.common.util.concurrent.EsExecutors$1.execute(EsExecutors.java:110) ~[elasticsearch-5.6.2.jar:5.6.2]
  125. at org.elasticsearch.transport.TcpTransport.handleRequest(TcpTransport.java:1490) ~[elasticsearch-5.6.2.jar:5.6.2]
  126. at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1374) ~[elasticsearch-5.6.2.jar:5.6.2]
  127. at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:74) ~[?:?]
  128. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
  129. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
  130. at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?]
  131. at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) ~[?:?]
  132. at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:297) ~[?:?]
  133. at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:413) ~[?:?]
  134. at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[?:?]
  135. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
  136. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
  137. at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?]
  138. at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) ~[?:?]
  139. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
  140. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
  141. at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?]
  142. at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) ~[?:?]
  143. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
  144. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
  145. at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) ~[?:?]
  146. at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) ~[?:?]
  147. at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) ~[?:?]
  148. at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) ~[?:?]
  149. at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) ~[?:?]
  150. at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) ~[?:?]
  151. at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) ~[?:?]
  152. ... 1 more
  153. [2017-11-25T00:19:42,144][WARN ][o.e.d.z.UnicastZenPing ] [Elastic-03] [1] failed send ping to {#zen_unicast_10.1.6.196_0#}{DwGkoSRlSY2Fy5QBLZncpQ}{10.1.6.196}{10.1.6.196:9300}
  154. java.lang.IllegalStateException: handshake failed with {#zen_unicast_10.1.6.196_0#}{DwGkoSRlSY2Fy5QBLZncpQ}{10.1.6.196}{10.1.6.196:9300}
  155. at org.elasticsearch.transport.TransportService.handshake(TransportService.java:403) ~[elasticsearch-5.6.2.jar:5.6.2]
  156. at org.elasticsearch.transport.TransportService.handshake(TransportService.java:370) ~[elasticsearch-5.6.2.jar:5.6.2]
  157. at org.elasticsearch.discovery.zen.UnicastZenPing$PingingRound.getOrConnect(UnicastZenPing.java:400) ~[elasticsearch-5.6.2.jar:5.6.2]
  158. at org.elasticsearch.discovery.zen.UnicastZenPing$3.doRun(UnicastZenPing.java:507) [elasticsearch-5.6.2.jar:5.6.2]
  159. at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) [elasticsearch-5.6.2.jar:5.6.2]
  160. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.6.2.jar:5.6.2]
  161. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
  162. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
  163. at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
  164. Caused by: org.elasticsearch.transport.RemoteTransportException: [Elastic-02][10.1.6.196:9300][internal:transport/handshake]
  165. Caused by: org.elasticsearch.ElasticsearchSecurityException: missing authentication token for action [internal:transport/handshake]
  166. at org.elasticsearch.xpack.security.support.Exceptions.authenticationError(Exceptions.java:39) ~[?:?]
  167. at org.elasticsearch.xpack.security.authc.DefaultAuthenticationFailureHandler.missingToken(DefaultAuthenticationFailureHandler.java:74) ~[?:?]
  168. at org.elasticsearch.xpack.security.authc.AuthenticationService$AuditableTransportRequest.anonymousAccessDenied(AuthenticationService.java:553) ~[?:?]
  169. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$handleNullToken$16(AuthenticationService.java:363) ~[?:?]
  170. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.handleNullToken(AuthenticationService.java:368) ~[?:?]
  171. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.consumeToken(AuthenticationService.java:295) ~[?:?]
  172. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$extractToken$7(AuthenticationService.java:267) ~[?:?]
  173. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.extractToken(AuthenticationService.java:284) ~[?:?]
  174. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$null$0(AuthenticationService.java:218) ~[?:?]
  175. at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:59) ~[elasticsearch-5.6.2.jar:5.6.2]
  176. at org.elasticsearch.xpack.security.authc.TokenService.getAndValidateToken(TokenService.java:218) ~[?:?]
  177. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$2(AuthenticationService.java:214) ~[?:?]
  178. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$4(AuthenticationService.java:246) ~[?:?]
  179. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:257) ~[?:?]
  180. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:210) ~[?:?]
  181. at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:159) ~[?:?]
  182. at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:122) ~[?:?]
  183. at org.elasticsearch.xpack.security.transport.ServerTransportFilter$NodeProfile.inbound(ServerTransportFilter.java:146) ~[?:?]
  184. at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:314) ~[?:?]
  185. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.6.2.jar:5.6.2]
  186. at org.elasticsearch.transport.TcpTransport$RequestHandler.doRun(TcpTransport.java:1533) ~[elasticsearch-5.6.2.jar:5.6.2]
  187. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.6.2.jar:5.6.2]
  188. at org.elasticsearch.common.util.concurrent.EsExecutors$1.execute(EsExecutors.java:110) ~[elasticsearch-5.6.2.jar:5.6.2]
  189. at org.elasticsearch.transport.TcpTransport.handleRequest(TcpTransport.java:1490) ~[elasticsearch-5.6.2.jar:5.6.2]
  190. at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1374) ~[elasticsearch-5.6.2.jar:5.6.2]
  191. at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:74) ~[?:?]
  192. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
  193. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
  194. at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?]
  195. at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) ~[?:?]
  196. at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:297) ~[?:?]
  197. at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:413) ~[?:?]
  198. at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[?:?]
  199. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
  200. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
  201. at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?]
  202. at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) ~[?:?]
  203. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
  204. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
  205. at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?]
  206. at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) ~[?:?]
  207. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
  208. at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
  209. at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) ~[?:?]
  210. at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) ~[?:?]
  211. at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) ~[?:?]
  212. at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) ~[?:?]
  213. at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) ~[?:?]
  214. at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) ~[?:?]
  215. at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) ~[?:?]
  216. ... 1 more
  217. [2017-11-25T00:19:43,140][INFO ][o.e.c.s.ClusterService ] [Elastic-03] new_master {Elastic-03}{DTbL2JaQQTGTcRPDC5c-iA}{NXEKY5o0QqqdLfaJRRw3UA}{10.1.6.197}{10.1.6.197:9300}{ml.max_open_jobs=10, ml.enabled=true}, added {{Elastic-02}{WSqN5IOoQbq-MAbwEoZDBA}{05ohG5xiTQioVJvZb9tnmg}{10.1.6.196}{10.1.6.196:9300}{ml.max_open_jobs=10, ml.enabled=true},}, reason: zen-disco-elected-as-master ([1] nodes joined)[{Elastic-02}{WSqN5IOoQbq-MAbwEoZDBA}{05ohG5xiTQioVJvZb9tnmg}{10.1.6.196}{10.1.6.196:9300}{ml.max_open_jobs=10, ml.enabled=true}]
  218. [2017-11-25T00:19:43,184][WARN ][o.e.d.z.ElectMasterService] [Elastic-03] value for setting "discovery.zen.minimum_master_nodes" is too low. This can result in data loss! Please set it to at least a quorum of master-eligible nodes (current value: [1], total number of master-eligible nodes used for publishing in this round: [2])
  219. [2017-11-25T00:19:43,215][INFO ][o.e.c.s.ClusterService ] [Elastic-03] removed {{Elastic-02}{WSqN5IOoQbq-MAbwEoZDBA}{05ohG5xiTQioVJvZb9tnmg}{10.1.6.196}{10.1.6.196:9300}{ml.max_open_jobs=10, ml.enabled=true},}, reason: zen-disco-node-failed({Elastic-02}{WSqN5IOoQbq-MAbwEoZDBA}{05ohG5xiTQioVJvZb9tnmg}{10.1.6.196}{10.1.6.196:9300}{ml.max_open_jobs=10, ml.enabled=true}), reason(failed to ping, tried [3] times, each with maximum [30s] timeout)[{Elastic-02}{WSqN5IOoQbq-MAbwEoZDBA}{05ohG5xiTQioVJvZb9tnmg}{10.1.6.196}{10.1.6.196:9300}{ml.max_open_jobs=10, ml.enabled=true} failed to ping, tried [3] times, each with maximum [30s] timeout]
  220. [2017-11-25T00:19:43,232][DEBUG][o.e.a.a.c.n.s.TransportNodesStatsAction] [Elastic-03] failed to execute on node [WSqN5IOoQbq-MAbwEoZDBA]
  221. org.elasticsearch.transport.NodeDisconnectedException: [Elastic-02][10.1.6.196:9300][cluster:monitor/nodes/stats[n]] disconnected
  222. [2017-11-25T00:19:43,235][INFO ][o.e.h.n.Netty4HttpServerTransport] [Elastic-03] publish_address {10.1.6.197:9200}, bound_addresses {10.1.6.197:9200}
  223. [2017-11-25T00:19:43,235][INFO ][o.e.n.Node ] [Elastic-03] started
  224. [2017-11-25T00:19:43,433][WARN ][o.e.g.Gateway ] [Elastic-03] failed to fetch state from node
  225. org.elasticsearch.action.FailedNodeException: Failed node [WSqN5IOoQbq-MAbwEoZDBA]
  226. at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction.onFailure(TransportNodesAction.java:239) ~[elasticsearch-5.6.2.jar:5.6.2]
  227. at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction.access$200(TransportNodesAction.java:153) ~[elasticsearch-5.6.2.jar:5.6.2]
  228. at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction$1.handleException(TransportNodesAction.java:211) ~[elasticsearch-5.6.2.jar:5.6.2]
  229. at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1067) ~[elasticsearch-5.6.2.jar:5.6.2]
  230. at org.elasticsearch.transport.TransportService$Adapter.lambda$onConnectionClosed$6(TransportService.java:893) ~[elasticsearch-5.6.2.jar:5.6.2]
  231. at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569) ~[elasticsearch-5.6.2.jar:5.6.2]
  232. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
  233. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
  234. at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
  235. Caused by: org.elasticsearch.transport.NodeDisconnectedException: [Elastic-02][10.1.6.196:9300][internal:gateway/local/meta_state[n]] disconnected
  236. [2017-11-25T00:19:45,651][INFO ][o.e.c.s.ClusterSettings ] [Elastic-03] updating [indices.store.throttle.max_bytes_per_sec] from [0b] to [100mb]
  237. [2017-11-25T00:19:49,895][ERROR][o.e.x.m.c.i.IndexRecoveryCollector] [Elastic-03] collector [index-recovery] failed to collect data
  238. org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
  239. at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:165) ~[elasticsearch-5.6.2.jar:5.6.2]
  240. at org.elasticsearch.action.admin.indices.recovery.TransportRecoveryAction.checkGlobalBlock(TransportRecoveryAction.java:114) ~[elasticsearch-5.6.2.jar:5.6.2]
  241. at org.elasticsearch.action.admin.indices.recovery.TransportRecoveryAction.checkGlobalBlock(TransportRecoveryAction.java:52) ~[elasticsearch-5.6.2.jar:5.6.2]
  242. at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$AsyncAction.<init>(TransportBroadcastByNodeAction.java:256) ~[elasticsearch-5.6.2.jar:5.6.2]
  243. at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction.doExecute(TransportBroadcastByNodeAction.java:234) ~[elasticsearch-5.6.2.jar:5.6.2]
  244. at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction.doExecute(TransportBroadcastByNodeAction.java:79) ~[elasticsearch-5.6.2.jar:5.6.2]
  245. at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:170) ~[elasticsearch-5.6.2.jar:5.6.2]
  246. at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.6.2.jar:5.6.2]
  247. at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.6.2.jar:5.6.2]
  248. at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.6.2.jar:5.6.2]
  249. at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.6.2.jar:5.6.2]
  250. at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:408) ~[elasticsearch-5.6.2.jar:5.6.2]
  251. at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) ~[elasticsearch-5.6.2.jar:5.6.2]
  252. at org.elasticsearch.xpack.security.InternalClient.doExecute(InternalClient.java:91) ~[?:?]
  253. at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:408) ~[elasticsearch-5.6.2.jar:5.6.2]
  254. at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1256) ~[elasticsearch-5.6.2.jar:5.6.2]
  255. at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:80) ~[elasticsearch-5.6.2.jar:5.6.2]
  256. at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:54) ~[elasticsearch-5.6.2.jar:5.6.2]
  257. at org.elasticsearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:69) ~[elasticsearch-5.6.2.jar:5.6.2]
  258. at org.elasticsearch.xpack.monitoring.collector.indices.IndexRecoveryCollector.doCollect(IndexRecoveryCollector.java:65) ~[x-pack-5.6.2.jar:5.6.2]
  259. at org.elasticsearch.xpack.monitoring.collector.Collector.collect(Collector.java:83) [x-pack-5.6.2.jar:5.6.2]
  260. at org.elasticsearch.xpack.monitoring.MonitoringService$MonitoringExecution$1.doRun(MonitoringService.java:195) [x-pack-5.6.2.jar:5.6.2]
  261. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.6.2.jar:5.6.2]
  262. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_121]
  263. at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_121]
  264. at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569) [elasticsearch-5.6.2.jar:5.6.2]
  265. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
  266. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
  267. at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
  268. [2017-11-25T00:19:49,910][ERROR][o.e.x.m.c.i.IndexStatsCollector] [Elastic-03] collector [index-stats] failed to collect data
  269. org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
  270. at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:165) ~[elasticsearch-5.6.2.jar:5.6.2]
  271. at org.elasticsearch.action.admin.indices.stats.TransportIndicesStatsAction.checkGlobalBlock(TransportIndicesStatsAction.java:70) ~[elasticsearch-5.6.2.jar:5.6.2]
  272. at org.elasticsearch.action.admin.indices.stats.TransportIndicesStatsAction.checkGlobalBlock(TransportIndicesStatsAction.java:47) ~[elasticsearch-5.6.2.jar:5.6.2]
  273. at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$AsyncAction.<init>(TransportBroadcastByNodeAction.java:256) ~[elasticsearch-5.6.2.jar:5.6.2]
  274. at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction.doExecute(TransportBroadcastByNodeAction.java:234) ~[elasticsearch-5.6.2.jar:5.6.2]
  275. at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction.doExecute(TransportBroadcastByNodeAction.java:79) ~[elasticsearch-5.6.2.jar:5.6.2]
  276. at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:170) ~[elasticsearch-5.6.2.jar:5.6.2]
  277. at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.6.2.jar:5.6.2]
  278. at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.6.2.jar:5.6.2]
  279. at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.6.2.jar:5.6.2]
  280. at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.6.2.jar:5.6.2]
  281. at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:408) ~[elasticsearch-5.6.2.jar:5.6.2]
  282. at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:67) ~[elasticsearch-5.6.2.jar:5.6.2]
  283. at org.elasticsearch.xpack.security.InternalClient.doExecute(InternalClient.java:91) ~[?:?]
  284. at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:408) ~[elasticsearch-5.6.2.jar:5.6.2]
  285. at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1256) ~[elasticsearch-5.6.2.jar:5.6.2]
  286. at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:80) ~[elasticsearch-5.6.2.jar:5.6.2]
  287. at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:54) ~[elasticsearch-5.6.2.jar:5.6.2]
  288. at org.elasticsearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:69) ~[elasticsearch-5.6.2.jar:5.6.2]
  289. at org.elasticsearch.xpack.monitoring.collector.indices.IndexStatsCollector.doCollect(IndexStatsCollector.java:77) ~[x-pack-5.6.2.jar:5.6.2]
  290. at org.elasticsearch.xpack.monitoring.collector.Collector.collect(Collector.java:83) [x-pack-5.6.2.jar:5.6.2]
  291. at org.elasticsearch.xpack.monitoring.MonitoringService$MonitoringExecution$1.doRun(MonitoringService.java:195) [x-pack-5.6.2.jar:5.6.2]
  292. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.6.2.jar:5.6.2]
  293. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_121]
  294. at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_121]
  295. at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569) [elasticsearch-5.6.2.jar:5.6.2]
  296. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
  297. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
  298. at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
  299. [2017-11-25T00:19:54,082][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [Elastic-03] fatal error in thread [elasticsearch[Elastic-03][clusterService#updateTask][T#1]], exiting
  300. java.lang.ClassFormatError: Incompatible magic value 1347093252 in class file org/elasticsearch/license/LicenseVerifier
  301. at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_121]
  302. at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_121]
  303. at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_121]
  304. at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) ~[?:1.8.0_121]
  305. at java.net.URLClassLoader.access$100(URLClassLoader.java:73) ~[?:1.8.0_121]
  306. at java.net.URLClassLoader$1.run(URLClassLoader.java:368) ~[?:1.8.0_121]
  307. at java.net.URLClassLoader$1.run(URLClassLoader.java:362) ~[?:1.8.0_121]
  308. at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_121]
  309. at java.net.URLClassLoader.findClass(URLClassLoader.java:361) ~[?:1.8.0_121]
  310. at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_121]
  311. at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:814) ~[?:1.8.0_121]
  312. at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_121]
  313. at org.elasticsearch.license.LicenseService.getLicense(LicenseService.java:472) ~[?:?]
  314. at org.elasticsearch.license.LicenseService.onUpdate(LicenseService.java:420) ~[?:?]
  315. at org.elasticsearch.license.LicenseService.clusterChanged(LicenseService.java:372) ~[?:?]
  316. at org.elasticsearch.cluster.service.ClusterService.lambda$publishAndApplyChanges$7(ClusterService.java:777) ~[elasticsearch-5.6.2.jar:5.6.2]
  317. at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) ~[?:1.8.0_121]
  318. at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:742) ~[?:1.8.0_121]
  319. at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) ~[?:1.8.0_121]
  320. at org.elasticsearch.cluster.service.ClusterService.publishAndApplyChanges(ClusterService.java:774) ~[elasticsearch-5.6.2.jar:5.6.2]
  321. at org.elasticsearch.cluster.service.ClusterService.runTasks(ClusterService.java:587) ~[elasticsearch-5.6.2.jar:5.6.2]
  322. at org.elasticsearch.cluster.service.ClusterService$ClusterServiceTaskBatcher.run(ClusterService.java:263) ~[elasticsearch-5.6.2.jar:5.6.2]
  323. at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) ~[elasticsearch-5.6.2.jar:5.6.2]
  324. at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) ~[elasticsearch-5.6.2.jar:5.6.2]
  325. at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569) ~[elasticsearch-5.6.2.jar:5.6.2]
  326. at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:247) ~[elasticsearch-5.6.2.jar:5.6.2]
  327. at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:210) ~[elasticsearch-5.6.2.jar:5.6.2]
  328. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_121]
  329. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_121]
  330. at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!