API tools faq
paste
Guest User

es1-02

a guest
Nov 14th, 2011
115
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 26.71 KB | None | 0 0
raw download clone embed print report
  1. [2011-11-01 00:12:39,777][INFO ][discovery.zen ] [es1-02] master_left [[es1-01][PrGr5OlPR_y3TBdyRCy8KA][inet[/10.1.101.151:9300]]], reason [failed to ping, tried [3] times, each with maximum [30s] timeout]
  2. [2011-11-01 00:12:39,832][INFO ][cluster.service ] [es1-02] master {new [es1-02][UGX0Wq5mSv6g-GPVXHVMnA][inet[/10.1.101.152:9300]], previous [es1-01][PrGr5OlPR_y3TBdyRCy8KA][inet[/10.1.101.151:9300]]}, removed {[es1-01][PrGr5OlPR_y3TBdyRCy8KA][inet[/10.1.101.151:9300]],}, reason: zen-disco-master_failed ([es1-01][PrGr5OlPR_y3TBdyRCy8KA][inet[/10.1.101.151:9300]])
  3. [2011-11-01 00:12:48,244][DEBUG][action.admin.cluster.node.stats] [es1-02] failed to execute on node [PrGr5OlPR_y3TBdyRCy8KA]
  4. org.elasticsearch.transport.NodeDisconnectedException: [es1-01][inet[/10.1.101.151:9300]][/cluster/nodes/stats/node] disconnected
  5. [2011-11-01 00:12:48,280][DEBUG][action.admin.cluster.node.stats] [es1-02] failed to execute on node [PrGr5OlPR_y3TBdyRCy8KA]
  6. org.elasticsearch.transport.NodeDisconnectedException: [es1-01][inet[/10.1.101.151:9300]][/cluster/nodes/stats/node] disconnected
  7. [2011-11-01 00:12:56,392][WARN ][cluster.metadata ] [es1-02] [5930452_0] re-syncing mappings with cluster state for types [[^profile, profile]]
  8. [2011-11-01 00:13:04,048][WARN ][cluster.metadata ] [es1-02] [6090102_m] re-syncing mappings with cluster state for types [[profile]]
  9. [2011-11-01 00:13:11,686][WARN ][cluster.metadata ] [es1-02] [6090072_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  10. [2011-11-01 00:13:19,020][WARN ][cluster.metadata ] [es1-02] [6076412_xl] re-syncing mappings with cluster state for types [[^profile, profile]]
  11. [2011-11-01 00:13:26,927][WARN ][cluster.metadata ] [es1-02] [6090232_m] re-syncing mappings with cluster state for types [[profile]]
  12. [2011-11-01 00:13:33,918][WARN ][cluster.metadata ] [es1-02] [988542_xs] re-syncing mappings with cluster state for types [[test, index1, settings2, test888, date, test10, noId2, test12, test1, type1, my_identities, test777, test17, type2, test92, settings3, test19, ???, profile, sometype, test89, index, test88, dataType_1, test999, test8, test7, ^profile, test2, test3, test25, test24, ???????, index4, account, obj, myalbums, myphotos]]
  13. [2011-11-01 00:13:34,019][WARN ][cluster.metadata ] [es1-02] [1069621_xl] re-syncing mappings with cluster state for types [[dataType_1, date, ^profile, profile]]
  14. [2011-11-01 00:13:34,144][WARN ][cluster.metadata ] [es1-02] [6090172_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  15. [2011-11-01 00:13:34,249][WARN ][cluster.metadata ] [es1-02] [6090222_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  16. [2011-11-01 00:13:34,341][WARN ][cluster.metadata ] [es1-02] [6090152_m] re-syncing mappings with cluster state for types [[profile]]
  17. [2011-11-01 00:13:34,455][WARN ][cluster.metadata ] [es1-02] [5930442_l] re-syncing mappings with cluster state for types [[^profile, profile]]
  18. [2011-11-01 00:13:34,544][WARN ][cluster.metadata ] [es1-02] [6090182_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  19. [2011-11-01 00:13:34,640][WARN ][cluster.metadata ] [es1-02] [6090032_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  20. [2011-11-01 00:13:34,725][WARN ][cluster.metadata ] [es1-02] [5930442_0] re-syncing mappings with cluster state for types [[^profile, profile]]
  21. [2011-11-01 00:13:34,808][WARN ][cluster.metadata ] [es1-02] [5931372_xl] re-syncing mappings with cluster state for types [[^profile, profile]]
  22. [2011-11-01 00:13:35,178][WARN ][cluster.metadata ] [es1-02] [5930432_0] re-syncing mappings with cluster state for types [[^profile, profile]]
  23. [2011-11-01 00:13:35,258][WARN ][cluster.metadata ] [es1-02] [5925212_0] re-syncing mappings with cluster state for types [[^profile, profile]]
  24. [2011-11-01 00:13:35,336][WARN ][cluster.metadata ] [es1-02] [6090052_m] re-syncing mappings with cluster state for types [[profile]]
  25. [2011-11-01 00:13:35,413][WARN ][cluster.metadata ] [es1-02] [6087422_s] re-syncing mappings with cluster state for types [[^profile, profile]]
  26. [2011-11-01 00:13:35,487][WARN ][cluster.metadata ] [es1-02] [5795421_s] re-syncing mappings with cluster state for types [[^profile, profile]]
  27. [2011-11-01 00:13:35,563][WARN ][cluster.metadata ] [es1-02] [6064272_xs] re-syncing mappings with cluster state for types [[^profile, profile]]
  28. [2011-11-01 00:13:35,777][WARN ][cluster.metadata ] [es1-02] [load_test] re-syncing mappings with cluster state for types [[^progile, ^profile, profile]]
  29. [2011-11-01 00:13:35,869][WARN ][cluster.metadata ] [es1-02] [6055102_xl] re-syncing mappings with cluster state for types [[^profile, profile]]
  30. [2011-11-01 00:13:36,117][WARN ][cluster.metadata ] [es1-02] [104908_s] re-syncing mappings with cluster state for types [[^profile, profile]]
  31. [2011-11-01 00:13:36,219][WARN ][cluster.metadata ] [es1-02] [6090082_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  32. [2011-11-01 00:13:36,349][WARN ][cluster.metadata ] [es1-02] [6098262_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  33. [2011-11-01 00:13:36,432][WARN ][cluster.metadata ] [es1-02] [6090142_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  34. [2011-11-01 00:13:36,511][WARN ][cluster.metadata ] [es1-02] [6090212_m] re-syncing mappings with cluster state for types [[profile]]
  35. [2011-11-01 00:13:36,636][WARN ][cluster.metadata ] [es1-02] [6090132_m] re-syncing mappings with cluster state for types [[profile]]
  36. [2011-11-01 00:13:36,711][WARN ][cluster.metadata ] [es1-02] [raviv_s] re-syncing mappings with cluster state for types [[^profile, profile]]
  37. [2011-11-01 00:13:36,807][WARN ][cluster.metadata ] [es1-02] [6060152_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  38. [2011-11-01 00:13:36,887][WARN ][cluster.metadata ] [es1-02] [6060153_s] re-syncing mappings with cluster state for types [[^profile, profile]]
  39. [2011-11-01 00:13:36,955][WARN ][cluster.metadata ] [es1-02] [6088372_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  40. [2011-11-01 00:13:37,019][WARN ][cluster.metadata ] [es1-02] [6060152_s] re-syncing mappings with cluster state for types [[^profile, profile]]
  41. [2011-11-01 00:13:37,100][WARN ][cluster.metadata ] [es1-02] [6090162_m] re-syncing mappings with cluster state for types [[profile]]
  42. [2011-11-01 00:13:37,157][WARN ][cluster.metadata ] [es1-02] [1071234_s] re-syncing mappings with cluster state for types [[^profile, profile]]
  43. [2011-11-01 00:13:37,210][WARN ][cluster.metadata ] [es1-02] [6060152_l] re-syncing mappings with cluster state for types [[profile]]
  44. [2011-11-01 00:13:37,261][WARN ][cluster.metadata ] [es1-02] [6090092_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  45. [2011-11-01 00:13:37,323][WARN ][cluster.metadata ] [es1-02] [5315901_xs] re-syncing mappings with cluster state for types [[^profile, profile]]
  46. [2011-11-01 00:13:37,375][WARN ][cluster.metadata ] [es1-02] [6059962_xs] re-syncing mappings with cluster state for types [[^profile, profile]]
  47. [2011-11-01 00:13:37,414][WARN ][cluster.metadata ] [es1-02] [6060151_s] re-syncing mappings with cluster state for types [[^profile, profile]]
  48. [2011-11-01 00:13:37,451][WARN ][cluster.metadata ] [es1-02] [6090042_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  49. [2011-11-01 00:13:37,484][WARN ][cluster.metadata ] [es1-02] [6090122_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  50. [2011-11-01 00:13:37,550][WARN ][cluster.metadata ] [es1-02] [134492_xl] re-syncing mappings with cluster state for types [[^profile, profile]]
  51. [2011-11-01 00:13:37,575][WARN ][cluster.metadata ] [es1-02] [104885_xs] re-syncing mappings with cluster state for types [[^profile, profile]]
  52. [2011-11-01 00:13:37,595][WARN ][cluster.metadata ] [es1-02] [6090112_m] re-syncing mappings with cluster state for types [[profile]]
  53. [2011-11-01 00:13:37,613][WARN ][cluster.metadata ] [es1-02] [6088492_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  54. [2011-11-01 00:13:37,627][WARN ][cluster.metadata ] [es1-02] [locad_test_s] re-syncing mappings with cluster state for types [[^profile, profile]]
  55. [2011-11-01 00:13:37,639][WARN ][cluster.metadata ] [es1-02] [585391_xs] re-syncing mappings with cluster state for types [[^profile, profile]]
  56. [2011-11-01 00:13:37,646][WARN ][cluster.metadata ] [es1-02] [6090062_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  57. [2011-11-01 00:21:37,668][INFO ][cluster.service ] [es1-02] added {[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]],}, reason: zen-disco-receive(join from node[[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]])
  58. [2011-11-01 00:22:37,530][WARN ][transport ] [es1-02] Received response for a request that has timed out, sent [58518ms] ago, timed out [28518ms] ago, action [discovery/zen/fd/ping], node [[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]], id [95581008]
  59. [2011-11-01 00:22:37,628][WARN ][transport ] [es1-02] Received response for a request that has timed out, sent [57085ms] ago, timed out [27085ms] ago, action [/cluster/nodes/indices/shard/store/node], node [[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]], id [95581009]
  60. [2011-11-01 00:23:14,051][WARN ][transport ] [es1-02] Received response for a request that has timed out, sent [34520ms] ago, timed out [4520ms] ago, action [discovery/zen/fd/ping], node [[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]], id [95581020]
  61. [2011-11-01 00:34:00,677][INFO ][cluster.service ] [es1-02] removed {[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]],}, reason: zen-disco-node_failed([es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]), reason failed to ping, tried [3] times, each with maximum [30s] timeout
  62. [2011-11-01 00:34:20,682][INFO ][cluster.service ] [es1-02] added {[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]],}, reason: zen-disco-receive(join from node[[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]])
  63. [2011-11-01 00:44:38,867][WARN ][common.jna ] Unknown mlockall error 0
  64. [2011-11-01 00:44:38,925][INFO ][node ] [es1-02] {elasticsearch/0.17.6}[14991]: initializing ...
  65. [2011-11-01 00:44:38,951][INFO ][plugins ] [es1-02] loaded [], sites []
  66. [2011-11-01 00:44:50,694][INFO ][node ] [es1-02] {elasticsearch/0.17.6}[14991]: initialized
  67. [2011-11-01 00:44:50,695][INFO ][node ] [es1-02] {elasticsearch/0.17.6}[14991]: starting ...
  68. [2011-11-01 00:44:51,834][INFO ][transport ] [es1-02] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/10.1.101.152:9300]}
  69. [2011-11-01 00:44:55,197][INFO ][cluster.service ] [es1-02] detected_master [es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]], added {[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]],}, reason: zen-disco-receive(from master [[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]])
  70. [2011-11-01 00:44:55,599][INFO ][discovery ] [es1-02] gcs/zi6VNqehTCaZjNS0nbGUhg
  71. [2011-11-01 00:44:57,448][INFO ][http ] [es1-02] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/10.1.101.152:9200]}
  72. [2011-11-01 00:44:58,421][INFO ][jmx ] [es1-02] bound_address {service:jmx:rmi:///jndi/rmi://:9400/jmxrmi}, publish_address {service:jmx:rmi:///jndi/rmi://10.1.101.152:9400/jmxrmi}
  73. [2011-11-01 00:44:58,421][INFO ][node ] [es1-02] {elasticsearch/0.17.6}[14991]: started
  74. [2011-11-01 00:57:51,032][WARN ][http.netty ] [es1-02] Caught exception while handling client http traffic, closing connection
  75. java.lang.IllegalStateException: cannot send more responses than requests
  76. at org.elasticsearch.common.netty.handler.codec.http.HttpContentEncoder.writeRequested(HttpContentEncoder.java:102)
  77. at org.elasticsearch.common.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:266)
  78. at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
  79. at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582)
  80. at org.elasticsearch.common.netty.channel.Channels.write(Channels.java:611)
  81. at org.elasticsearch.common.netty.channel.Channels.write(Channels.java:578)
  82. at org.elasticsearch.common.netty.channel.AbstractChannel.write(AbstractChannel.java:251)
  83. at org.elasticsearch.http.netty.NettyHttpChannel.sendResponse(NettyHttpChannel.java:156)
  84. at org.elasticsearch.rest.action.index.RestIndexAction$1.onResponse(RestIndexAction.java:123)
  85. at org.elasticsearch.rest.action.index.RestIndexAction$1.onResponse(RestIndexAction.java:101)
  86. at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$4.finishIfPossible(TransportShardReplicationOperationAction.java:538)
  87. at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$4.handleResponse(TransportShardReplicationOperationAction.java:525)
  88. at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$4.handleResponse(TransportShardReplicationOperationAction.java:523)
  89. at org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:132)
  90. at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:103)
  91. at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
  92. at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
  93. at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:783)
  94. at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:302)
  95. at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317)
  96. at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299)
  97. at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
  98. at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
  99. at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
  100. at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
  101. at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:274)
  102. at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:261)
  103. at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:349)
  104. at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:280)
  105. at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:200)
  106. at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
  107. at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
  108. at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
  109. at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
  110. at java.lang.Thread.run(Unknown Source)
  111. [2011-11-01 00:57:51,035][WARN ][http.netty ] [es1-02] Caught exception while handling client http traffic, closing connection
  112. java.lang.IllegalStateException: cannot send more responses than requests
  113. at org.elasticsearch.common.netty.handler.codec.http.HttpContentEncoder.writeRequested(HttpContentEncoder.java:102)
  114. at org.elasticsearch.common.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:266)
  115. at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
  116. at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582)
  117. at org.elasticsearch.common.netty.channel.Channels.write(Channels.java:611)
  118. at org.elasticsearch.common.netty.channel.Channels.write(Channels.java:578)
  119. at org.elasticsearch.common.netty.channel.AbstractChannel.write(AbstractChannel.java:251)
  120. at org.elasticsearch.http.netty.NettyHttpChannel.sendResponse(NettyHttpChannel.java:156)
  121. at org.elasticsearch.rest.action.index.RestIndexAction$1.onResponse(RestIndexAction.java:123)
  122. at org.elasticsearch.rest.action.index.RestIndexAction$1.onResponse(RestIndexAction.java:101)
  123. at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$4.finishIfPossible(TransportShardReplicationOperationAction.java:538)
  124. at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$4.handleResponse(TransportShardReplicationOperationAction.java:525)
  125. at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$4.handleResponse(TransportShardReplicationOperationAction.java:523)
  126. at org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:132)
  127. at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:103)
  128. at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
  129. at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
  130. at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:783)
  131. at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:302)
  132. at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317)
  133. at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299)
  134. at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
  135. at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
  136. at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
  137. at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
  138. at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:274)
  139. at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:261)
  140. at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:349)
  141. at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:280)
  142. at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:200)
  143. at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
  144. at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
  145. at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
  146. at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
  147. at java.lang.Thread.run(Unknown Source)
  148.  
  149.  
  150.  
  151.  
  152. ................. some uninteresting events here .........................................
  153.  
  154.  
  155.  
  156. [2011-11-01 06:20:04,134][WARN ][transport ] [es1-02] Received response for a request that has timed out, sent [44780ms] ago, timed out [14779ms] ago, action [discovery/zen/fd/masterPing], node [[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]], id [148088]
  157. [2011-11-01 06:43:09,864][INFO ][discovery.zen ] [es1-02] master_left [[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]], reason [failed to ping, tried [3] times, each with maximum [30s] timeout]
  158. [2011-11-01 06:43:09,872][INFO ][cluster.service ] [es1-02] master {new [es1-02][zi6VNqehTCaZjNS0nbGUhg][inet[/10.1.101.152:9300]], previous [es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]}, removed {[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]],}, reason: zen-disco-master_failed ([es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]])
  159. [2011-11-01 06:43:15,303][DEBUG][action.admin.cluster.node.stats] [es1-02] failed to execute on node [j0VcAKNRSsKnbHyaeof6pQ]
  160. org.elasticsearch.transport.NodeDisconnectedException: [es1-01][inet[/10.1.101.151:9300]][/cluster/nodes/stats/node] disconnected
  161. [2011-11-01 06:43:15,304][DEBUG][action.admin.cluster.node.stats] [es1-02] failed to execute on node [j0VcAKNRSsKnbHyaeof6pQ]
  162. org.elasticsearch.transport.NodeDisconnectedException: [es1-01][inet[/10.1.101.151:9300]][/cluster/nodes/stats/node] disconnected
  163. [2011-11-01 06:43:20,137][WARN ][cluster.metadata ] [es1-02] [5930452_0] re-syncing mappings with cluster state for types [[^profile, profile]]
  164. [2011-11-01 06:43:24,722][WARN ][cluster.metadata ] [es1-02] [6090102_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  165. [2011-11-01 06:43:29,305][WARN ][cluster.metadata ] [es1-02] [6090072_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  166. [2011-11-01 06:43:33,993][WARN ][cluster.metadata ] [es1-02] [6076412_xl] re-syncing mappings with cluster state for types [[^profile, profile]]
  167. [2011-11-01 06:43:38,809][WARN ][cluster.metadata ] [es1-02] [6090232_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  168. [2011-11-01 06:43:43,376][WARN ][cluster.metadata ] [es1-02] [988542_xs] re-syncing mappings with cluster state for types [[test, index1, settings2, test888, date, test10, noId2, test12, test1, type1, test777, my_identities, test17, type2, settings3, test92, test19, ???, profile, sometype, test89, index, test88, dataType_1, test999, test8, test7, ^profile, test2, test3, test25, test24, ???????, index4, account, obj, myalbums, myphotos]]
  169. [2011-11-01 06:43:43,402][WARN ][cluster.metadata ] [es1-02] [1069621_xl] re-syncing mappings with cluster state for types [[dataType_1, date, ^profile, profile]]
  170. [2011-11-01 06:43:43,663][WARN ][cluster.metadata ] [es1-02] [6090172_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  171. [2011-11-01 06:43:43,680][WARN ][cluster.metadata ] [es1-02] [6090222_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  172. [2011-11-01 06:43:43,693][WARN ][cluster.metadata ] [es1-02] [6090152_m] re-syncing mappings with cluster state for types [[^profile, profile]]
  173. [2011-11-01 06:43:43,702][WARN ][cluster.metadata ] [es1-02] [5930442_l] re-syncing mappings with cluster state for types [[^profile, profile]]
  174.  
  175.  
  176.  
  177. ................. some uninteresting events here .........................................
  178.  
  179.  
  180.  
  181. [2011-11-01 12:00:02,108][WARN ][discovery.zen ] [es1-02] master should not receive new cluster state from [[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]]
  182. [2011-11-01 12:00:05,391][WARN ][discovery.zen ] [es1-02] master should not receive new cluster state from [[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]]
  183. [2011-11-01 18:00:01,733][WARN ][discovery.zen ] [es1-02] master should not receive new cluster state from [[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]]
  184. [2011-11-01 18:00:05,485][WARN ][discovery.zen ] [es1-02] master should not receive new cluster state from [[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]]
  185.  
  186.  
  187.  
  188. ............. no other events here, but a new log file was started ..................
  189.  
  190.  
  191. [2011-11-02 00:00:01,577][WARN ][discovery.zen ] [es1-02] master should not receive new cluster state from [[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]]
  192. [2011-11-02 00:00:05,022][WARN ][discovery.zen ] [es1-02] master should not receive new cluster state from [[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]]
  193. [2011-11-02 03:45:39,618][WARN ][discovery.zen ] [es1-02] master should not receive new cluster state from [[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]]
  194. [2011-11-02 03:46:23,692][WARN ][discovery.zen ] [es1-02] master should not receive new cluster state from [[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]]
  195. [2011-11-02 03:47:09,127][WARN ][discovery.zen ] [es1-02] master should not receive new cluster state from [[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]]
  196. [2011-11-02 03:47:42,686][WARN ][discovery.zen ] [es1-02] master should not receive new cluster state from [[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]]
  197. [2011-11-02 03:47:51,739][WARN ][discovery.zen ] [es1-02] master should not receive new cluster state from [[es1-01][j0VcAKNRSsKnbHyaeof6pQ][inet[/10.1.101.151:9300]]]
  198. [2011-11-02 04:24:36,739][INFO ][node ] [es1-02] {elasticsearch/0.17.6}[14991]: stopping ...
  199. [2011-11-02 04:24:37,236][INFO ][node ] [es1-02] {elasticsearch/0.17.6}[14991]: stopped
  200. [2011-11-02 04:24:37,236][INFO ][node ] [es1-02] {elasticsearch/0.17.6}[14991]: closing ...
  201. [2011-11-02 04:24:37,325][INFO ][node ] [es1-02] {elasticsearch/0.17.6}[14991]: closed
  202. [2011-11-02 04:24:41,194][WARN ][common.jna ] Unknown mlockall error 0
  203. [2011-11-02 04:24:41,236][INFO ][node ] [es1-02] {elasticsearch/0.17.6}[1321]: initializing ...
  204. [2011-11-02 04:24:41,243][INFO ][plugins ] [es1-02] loaded [], sites []
  205. [2011-11-02 04:24:43,750][INFO ][node ] [es1-02] {elasticsearch/0.17.6}[1321]: initialized
  206. [2011-11-02 04:24:43,751][INFO ][node ] [es1-02] {elasticsearch/0.17.6}[1321]: starting ...
  207.  
  208.  
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!