Advertisement
franek

es console

Jun 14th, 2016
375
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 47.81 KB | None | 0 0
  1. [2016-06-14 08:46:31,308][INFO ][node ] [Infant Terrible] version[2.3.3], pid[82244], build[218bdf1/2016-05-17T15:40:04Z]
  2. [2016-06-14 08:46:31,309][INFO ][node ] [Infant Terrible] initializing ...
  3. [2016-06-14 08:46:33,144][INFO ][plugins ] [Infant Terrible] modules [reindex, lang-expression, lang-groovy], plugins [whatson, kopf], sites [kopf, whatson]
  4. [2016-06-14 08:46:33,287][INFO ][env ] [Infant Terrible] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [66.6gb], net total_space [233.1gb], spins? [unknown], types [hfs]
  5. [2016-06-14 08:46:33,288][INFO ][env ] [Infant Terrible] heap size [1.9gb], compressed ordinary object pointers [true]
  6. [2016-06-14 08:46:33,289][WARN ][env ] [Infant Terrible] max file descriptors [10240] for elasticsearch process likely too low, consider increasing to at least [65536]
  7. [2016-06-14 08:46:38,748][INFO ][node ] [Infant Terrible] initialized
  8. [2016-06-14 08:46:38,749][INFO ][node ] [Infant Terrible] starting ...
  9. [2016-06-14 08:46:39,020][INFO ][transport ] [Infant Terrible] publish_address {127.0.0.1:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300}, {127.94.0.2:9300}, {127.94.0.1:9300}
  10. [2016-06-14 08:46:39,027][INFO ][discovery ] [Infant Terrible] address-reputation/cXEBEbAQQkK5qZXQRi2Gtg
  11. [2016-06-14 08:46:42,068][INFO ][cluster.service ] [Infant Terrible] new_master {Infant Terrible}{cXEBEbAQQkK5qZXQRi2Gtg}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
  12. [2016-06-14 08:46:42,085][INFO ][http ] [Infant Terrible] publish_address {127.0.0.1:9200}, bound_addresses {[fe80::1]:9200}, {[::1]:9200}, {127.0.0.1:9200}, {127.94.0.2:9200}, {127.94.0.1:9200}
  13. [2016-06-14 08:46:42,085][INFO ][node ] [Infant Terrible] started
  14. [2016-06-14 08:46:42,133][INFO ][gateway ] [Infant Terrible] recovered [1] indices into cluster_state
  15. [2016-06-14 08:46:43,524][INFO ][cluster.routing.allocation] [Infant Terrible] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[address-reputation-data-full][3]] ...]).
  16. [2016-06-14 08:50:40,554][INFO ][cluster.metadata ] [Infant Terrible] [address-reputation-data] creating index, cause [api], templates [], shards [5]/[0], mappings [address]
  17. [2016-06-14 08:50:40,660][INFO ][cluster.routing.allocation] [Infant Terrible] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[address-reputation-data][4]] ...]).
  18. java.lang.OutOfMemoryError: Java heap space
  19. Dumping heap to java_pid82244.hprof ...
  20. Heap dump file created [2129241713 bytes in 8.721 secs]
  21. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  22. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  23. [2016-06-14 09:22:44,100][ERROR][rest.action.support ] failed to send failure response
  24. java.lang.OutOfMemoryError: Java heap space
  25. [2016-06-14 09:22:50,281][WARN ][action.bulk ] [Infant Terrible] unexpected error during the primary phase for action [indices:data/write/bulk[s]], request [BulkShardRequest to [address-reputation-data] containing [230] requests]
  26. java.lang.OutOfMemoryError: Java heap space
  27. [2016-06-14 09:22:48,704][DEBUG][action.bulk ] [Infant Terrible] [address-reputation-data][2] failed to execute bulk item (index) index {[address-reputation-data][address][GB906087880], source[{"postcode":"EH4 1SB","line1":"3 East Werberside Place","id":"GB906087880","line2":"","town":"Edinburgh","line3":""}]}
  28. MapperParsingException[failed to parse]; nested: OutOfMemoryError[Java heap space];
  29. at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:154)
  30. at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:309)
  31. at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:580)
  32. at org.elasticsearch.index.shard.IndexShard.prepareIndexOnPrimary(IndexShard.java:559)
  33. at org.elasticsearch.action.index.TransportIndexAction.prepareIndexOperationOnPrimary(TransportIndexAction.java:212)
  34. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:224)
  35. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  36. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  37. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  38. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  39. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  40. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  41. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  42. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  43. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  44. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  45. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  46. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  47. at java.lang.Thread.run(Thread.java:745)
  48. Caused by: java.lang.OutOfMemoryError: Java heap space
  49. Exception in thread "RMI TCP Connection(idle)" [2016-06-14 09:22:49,799][WARN ][rest.suppressed ] /_cluster/health Params: {}
  50. java.lang.OutOfMemoryError: Java heap space
  51. at com.fasterxml.jackson.core.util.BufferRecycler.balloc(BufferRecycler.java:155)
  52. at com.fasterxml.jackson.core.util.BufferRecycler.allocByteBuffer(BufferRecycler.java:96)
  53. at com.fasterxml.jackson.core.util.BufferRecycler.allocByteBuffer(BufferRecycler.java:86)
  54. at com.fasterxml.jackson.core.io.IOContext.allocWriteEncodingBuffer(IOContext.java:160)
  55. at com.fasterxml.jackson.core.json.UTF8JsonGenerator.<init>(UTF8JsonGenerator.java:118)
  56. at com.fasterxml.jackson.core.JsonFactory._createUTF8Generator(JsonFactory.java:1333)
  57. at com.fasterxml.jackson.core.JsonFactory.createGenerator(JsonFactory.java:1065)
  58. at org.elasticsearch.common.xcontent.json.JsonXContent.createGenerator(JsonXContent.java:73)
  59. at org.elasticsearch.common.xcontent.XContentBuilder.<init>(XContentBuilder.java:113)
  60. at org.elasticsearch.rest.RestChannel.newBuilder(RestChannel.java:73)
  61. at org.elasticsearch.rest.RestChannel.newBuilder(RestChannel.java:51)
  62. at org.elasticsearch.rest.action.support.RestStatusToXContentListener.buildResponse(RestStatusToXContentListener.java:38)
  63. at org.elasticsearch.rest.action.support.RestStatusToXContentListener.buildResponse(RestStatusToXContentListener.java:30)
  64. at org.elasticsearch.rest.action.support.RestResponseListener.processResponse(RestResponseListener.java:43)
  65. at org.elasticsearch.rest.action.support.RestActionListener.onResponse(RestActionListener.java:49)
  66. at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:89)
  67. at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:85)
  68. at org.elasticsearch.action.support.ThreadedActionListener$1.doRun(ThreadedActionListener.java:89)
  69. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  70. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  71. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  72. at java.lang.Thread.run(Thread.java:745)
  73. java.lang.OutOfMemoryError: Java heap space
  74. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  75. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  76. [2016-06-14 09:22:49,799][ERROR][rest.action.support ] failed to send failure response
  77. java.lang.OutOfMemoryError: Java heap space
  78. [2016-06-14 09:22:49,799][DEBUG][action.bulk ] [Infant Terrible] [address-reputation-data][0] failed to execute bulk item (index) index {[address-reputation-data][address][GB100021443846], source[{"postcode":"HA6 2TB","line1":"18 Fringewood Close","id":"GB100021443846","line2":"","town":"Northwood","line3":""}]}
  79. MapperParsingException[failed to parse]; nested: OutOfMemoryError[Java heap space];
  80. at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:154)
  81. at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:309)
  82. at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:580)
  83. at org.elasticsearch.index.shard.IndexShard.prepareIndexOnPrimary(IndexShard.java:559)
  84. at org.elasticsearch.action.index.TransportIndexAction.prepareIndexOperationOnPrimary(TransportIndexAction.java:212)
  85. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:224)
  86. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  87. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  88. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  89. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  90. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  91. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  92. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  93. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  94. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  95. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  96. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  97. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  98. at java.lang.Thread.run(Thread.java:745)
  99. Caused by: java.lang.OutOfMemoryError: Java heap space
  100. at com.fasterxml.jackson.core.util.BufferRecycler.balloc(BufferRecycler.java:155)
  101. at com.fasterxml.jackson.core.util.BufferRecycler.allocByteBuffer(BufferRecycler.java:96)
  102. at com.fasterxml.jackson.core.util.BufferRecycler.allocByteBuffer(BufferRecycler.java:86)
  103. at com.fasterxml.jackson.core.io.IOContext.allocReadIOBuffer(IOContext.java:147)
  104. at com.fasterxml.jackson.core.json.ByteSourceJsonBootstrapper.<init>(ByteSourceJsonBootstrapper.java:87)
  105. at com.fasterxml.jackson.core.JsonFactory._createParser(JsonFactory.java:1240)
  106. at com.fasterxml.jackson.core.JsonFactory.createParser(JsonFactory.java:802)
  107. at org.elasticsearch.common.xcontent.json.JsonXContent.createParser(JsonXContent.java:83)
  108. at org.elasticsearch.common.xcontent.XContentHelper.createParser(XContentHelper.java:59)
  109. at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:90)
  110. ... 18 more
  111. [2016-06-14 09:22:49,100][ERROR][rest.action.support ] failed to send failure response
  112. java.lang.OutOfMemoryError: Java heap space
  113. [2016-06-14 09:22:48,704][WARN ][netty.channel.socket.nio.AbstractNioSelector] Unexpected exception in the selector loop.
  114. java.lang.OutOfMemoryError: Java heap space
  115. [2016-06-14 09:23:06,870][DEBUG][action.bulk ] [Infant Terrible] [address-reputation-data][0] failed to execute bulk item (index) index {[address-reputation-data][address][GB100070956282], source[{"postcode":"B37 6TP","line1":"59 Cornfield Croft","id":"GB100070956282","line2":"","town":"Birmingham","line3":""}]}
  116. [address-reputation-data][[address-reputation-data][0]] IndexFailedEngineException[Index failed for [address#GB100070956282]]; nested: AlreadyClosedException[this IndexWriter is closed]; nested: OutOfMemoryError[Java heap space];
  117. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:462)
  118. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:601)
  119. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  120. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  121. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  122. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  123. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  124. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  125. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  126. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  127. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  128. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  129. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  130. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  131. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  132. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  133. at java.lang.Thread.run(Thread.java:745)
  134. Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
  135. at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
  136. at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
  137. at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
  138. at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
  139. at org.elasticsearch.index.engine.InternalEngine.innerIndex(InternalEngine.java:530)
  140. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:457)
  141. ... 16 more
  142. Caused by: java.lang.OutOfMemoryError: Java heap space
  143. [2016-06-14 09:23:06,885][DEBUG][action.bulk ] [Infant Terrible] failed to execute [BulkShardRequest to [address-reputation-data] containing [206] requests] on [[address-reputation-data][0]]
  144. [address-reputation-data][[address-reputation-data][0]] EngineClosedException[CurrentState[CLOSED] Closed]
  145. at org.elasticsearch.index.engine.Engine.ensureOpen(Engine.java:329)
  146. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:451)
  147. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:601)
  148. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  149. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  150. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  151. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  152. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  153. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  154. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  155. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  156. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  157. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  158. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  159. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  160. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  161. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  162. at java.lang.Thread.run(Thread.java:745)
  163. [2016-06-14 09:23:06,852][WARN ][http.netty ] [Infant Terrible] Caught exception while handling client http traffic, closing connection [id: 0x29ef5f8f, /0:0:0:0:0:0:0:1:50443 => /0:0:0:0:0:0:0:1:9200]
  164. java.lang.OutOfMemoryError: Java heap space
  165. at java.util.Arrays.copyOfRange(Arrays.java:3664)
  166. at java.lang.String.<init>(String.java:207)
  167. at java.lang.StringBuilder.toString(StringBuilder.java:407)
  168. at org.jboss.netty.handler.codec.http.HttpMessageDecoder.readHeader(HttpMessageDecoder.java:633)
  169. at org.jboss.netty.handler.codec.http.HttpMessageDecoder.readHeaders(HttpMessageDecoder.java:531)
  170. at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:195)
  171. at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:102)
  172. at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500)
  173. at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
  174. at org.jboss.Jun 14, 2016 9:22:56 AM ServerCommunicatorAdmin reqIncoming
  175. WARNING: The server has decided to close this client connection.
  176. netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
  177. at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
  178. at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
  179. at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:75)
  180. at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
  181. at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
  182. at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
  183. at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
  184. at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
  185. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
  186. at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
  187. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
  188. at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
  189. at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
  190. at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
  191. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  192. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  193. at java.lang.Thread.run(Thread.java:745)
  194. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  195. [2016-06-14 09:23:06,893][DEBUG][action.bulk ] [Infant Terrible] failed to execute [BulkShardRequest to [address-reputation-data] containing [206] requests] on [[address-reputation-data][0]]
  196. [address-reputation-data][[address-reputation-data][0]] EngineClosedException[CurrentState[CLOSED] Closed]
  197. at org.elasticsearch.index.engine.Engine.ensureOpen(Engine.java:329)
  198. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:451)
  199. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:601)
  200. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  201. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  202. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  203. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  204. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  205. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  206. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  207. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  208. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  209. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  210. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  211. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  212. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  213. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  214. at java.lang.Thread.run(Thread.java:745)
  215. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  216. [2016-06-14 09:23:06,898][DEBUG][action.bulk ] [Infant Terrible] failed to execute [BulkShardRequest to [address-reputation-data] containing [230] requests] on [[address-reputation-data][0]]
  217. [address-reputation-data][[address-reputation-data][0]] EngineClosedException[CurrentState[CLOSED] Closed]
  218. at org.elasticsearch.index.engine.Engine.ensureOpen(Engine.java:329)
  219. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:451)
  220. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:601)
  221. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  222. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  223. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  224. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  225. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  226. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  227. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  228. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  229. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  230. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  231. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  232. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  233. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  234. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  235. at java.lang.Thread.run(Thread.java:745)
  236. [2016-06-14 09:23:06,892][WARN ][index.engine ] [Infant Terrible] [address-reputation-data][0] failed engine [already closed by tragic event on the index writer]
  237. java.lang.OutOfMemoryError: Java heap space
  238. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  239. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  240. [2016-06-14 09:23:06,917][WARN ][indices.cluster ] [Infant Terrible] [[address-reputation-data][0]] marking and sending shard failed due to [engine failure, reason [already closed by tragic event on the index writer]]
  241. java.lang.OutOfMemoryError: Java heap space
  242. [2016-06-14 09:23:06,918][WARN ][cluster.action.shard ] [Infant Terrible] [address-reputation-data][0] received shard failed for target shard [[address-reputation-data][0], node[cXEBEbAQQkK5qZXQRi2Gtg], [P], v[2], s[STARTED], a[id=CgBkNgr3RxCNjK6dYIZFVg]], indexUUID [RI06vpNWRh6pMzN3MZHuew], message [engine failure, reason [already closed by tragic event on the index writer]], failure [OutOfMemoryError[Java heap space]]
  243. java.lang.OutOfMemoryError: Java heap space
  244. [2016-06-14 09:23:06,886][DEBUG][action.bulk ] [Infant Terrible] failed to execute [BulkShardRequest to [address-reputation-data] containing [210] requests] on [[address-reputation-data][0]]
  245. [address-reputation-data][[address-reputation-data][0]] EngineClosedException[CurrentState[CLOSED] Closed]
  246. at org.elasticsearch.index.engine.Engine.ensureOpen(Engine.java:329)
  247. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:451)
  248. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:601)
  249. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  250. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  251. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  252. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  253. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  254. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  255. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  256. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  257. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  258. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  259. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  260. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  261. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  262. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  263. at java.lang.Thread.run(Thread.java:745)
  264. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  265. [2016-06-14 09:23:06,914][DEBUG][action.bulk ] [Infant Terrible] [address-reputation-data][0] failed to execute bulk item (index) index {[address-reputation-data][address][GB100091479761], source[{"postcode":"IG10 4QY","line1":"30 Grange Court Upper Park","id":"GB100091479761","line2":"","town":"Loughton","line3":""}]}
  266. [address-reputation-data][[address-reputation-data][0]] IndexFailedEngineException[Index failed for [address#GB100091479761]]; nested: AlreadyClosedException[this IndexWriter is closed]; nested: OutOfMemoryError[Java heap space];
  267. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:462)
  268. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:601)
  269. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  270. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  271. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  272. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  273. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  274. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  275. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  276. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  277. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  278. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  279. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  280. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  281. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  282. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  283. at java.lang.Thread.run(Thread.java:745)
  284. Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
  285. at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
  286. at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
  287. at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
  288. at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
  289. at org.elasticsearch.index.engine.InternalEngine.innerIndex(InternalEngine.java:530)
  290. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:457)
  291. ... 16 more
  292. Caused by: java.lang.OutOfMemoryError: Java heap space
  293. [2016-06-14 09:23:06,895][DEBUG][action.bulk ] [Infant Terrible] [address-reputation-data][0] failed to execute bulk item (index) index {[address-reputation-data][address][GB100080910698], source[{"postcode":"WD3 8BP","line1":"36 Maxwell Close","id":"GB100080910698","line2":"Mill End","town":"Rickmansworth","line3":""}]}
  294. [address-reputation-data][[address-reputation-data][0]] IndexFailedEngineException[Index failed for [address#GB100080910698]]; nested: OutOfMemoryError[Java heap space];
  295. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:462)
  296. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:601)
  297. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  298. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  299. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  300. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  301. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  302. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  303. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  304. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  305. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  306. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  307. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  308. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  309. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  310. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  311. at java.lang.Thread.run(Thread.java:745)
  312. Caused by: java.lang.OutOfMemoryError: Java heap space
  313. [2016-06-14 09:23:06,930][DEBUG][action.bulk ] [Infant Terrible] failed to execute [BulkShardRequest to [address-reputation-data] containing [212] requests] on [[address-reputation-data][0]]
  314. [address-reputation-data][[address-reputation-data][0]] IllegalIndexShardStateException[CurrentState[CLOSED] operation only allowed when started/recovering, origin [PRIMARY]]
  315. at org.elasticsearch.index.shard.IndexShard.ensureWriteAllowed(IndexShard.java:1062)
  316. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:593)
  317. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  318. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  319. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  320. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  321. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  322. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  323. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  324. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  325. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  326. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  327. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  328. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  329. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  330. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  331. at java.lang.Thread.run(Thread.java:745)
  332. [2016-06-14 09:23:06,926][DEBUG][action.bulk ] [Infant Terrible] failed to execute [BulkShardRequest to [address-reputation-data] containing [195] requests] on [[address-reputation-data][0]]
  333. [address-reputation-data][[address-reputation-data][0]] ShardNotFoundException[no such shard]
  334. at org.elasticsearch.index.IndexService.shardSafe(IndexService.java:197)
  335. at org.elasticsearch.action.support.replication.TransportReplicationAction.getIndexShardOperationsCounter(TransportReplicationAction.java:746)
  336. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:638)
  337. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  338. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  339. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  340. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  341. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  342. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  343. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  344. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  345. at java.lang.Thread.run(Thread.java:745)
  346. [2016-06-14 09:23:06,937][INFO ][cluster.routing.allocation] [Infant Terrible] Cluster health status changed from [YELLOW] to [RED] (reason: [shards failed [[address-reputation-data][0]] ...]).
  347. [2016-06-14 09:23:07,644][DEBUG][action.bulk ] [Infant Terrible] failed to execute [BulkShardRequest to [address-reputation-data] containing [235] requests] on [[address-reputation-data][0]]
  348. [address-reputation-data][[address-reputation-data][0]] ShardNotFoundException[no such shard]
  349. at org.elasticsearch.index.IndexService.shardSafe(IndexService.java:197)
  350. at org.elasticsearch.action.support.replication.TransportReplicationAction.getIndexShardOperationsCounter(TransportReplicationAction.java:746)
  351. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:638)
  352. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  353. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  354. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  355. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  356. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  357. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  358. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  359. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  360. at java.lang.Thread.run(Thread.java:745)
  361. [2016-06-14 09:23:06,932][DEBUG][action.bulk ] [Infant Terrible] failed to execute [BulkShardRequest to [address-reputation-data] containing [245] requests] on [[address-reputation-data][0]]
  362. [address-reputation-data][[address-reputation-data][0]] IllegalIndexShardStateException[CurrentState[CLOSED] operation only allowed when started/recovering, origin [PRIMARY]]
  363. at org.elasticsearch.index.shard.IndexShard.ensureWriteAllowed(IndexShard.java:1062)
  364. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:593)
  365. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  366. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  367. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  368. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  369. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  370. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  371. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  372. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  373. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  374. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  375. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  376. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  377. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  378. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  379. at java.lang.Thread.run(Thread.java:745)
  380. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  381. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  382. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  383. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  384. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  385. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  386. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  387. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  388. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  389. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  390. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  391. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  392. [2016-06-14 09:23:15,496][INFO ][node ] [Infant Terrible] stopping ...
  393. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  394. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  395. [2016-06-14 09:23:17,041][WARN ][transport.netty ] [Infant Terrible] exception caught on transport layer [[id: 0x4ad0c069, /127.0.0.1:50553 => /127.0.0.1:9300]], closing connection
  396. java.io.IOException: Protocol wrong type for socket
  397. at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
  398. at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
  399. at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
  400. at sun.nio.ch.IOUtil.write(IOUtil.java:65)
  401. at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
  402. at org.jboss.netty.channel.socket.nio.SocketSendBufferPool$UnpooledSendBuffer.transferTo(SocketSendBufferPool.java:203)
  403. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.write0(AbstractNioWorker.java:201)
  404. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.writeFromSelectorLoop(AbstractNioWorker.java:158)
  405. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:114)
  406. at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstException in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  407. ractNioSelector.java:337)
  408. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
  409. at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
  410. at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
  411. at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
  412. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  413. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  414. at java.lang.Thread.run(Thread.java:745)
  415. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  416. [2016-06-14 09:23:17,465][WARN ][transport.netty ] [Infant Terrible] exception caught on transport layer [[id: 0xd2df1f91, /127.0.0.1:50552 => /127.0.0.1:9300]], closing connection
  417. java.io.IOException: Protocol wrong type for socket
  418. at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
  419. at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
  420. at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
  421. at sun.nio.ch.IOUtil.write(IOUtil.java:65)
  422. at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
  423. at org.jboss.netty.channel.socket.nio.SocketSendBufferPool$UnpooledSendBuffer.transferTo(SocketSendBufferPool.java:203)
  424. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.write0(AbstractNioWorker.java:201)
  425. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.writeFromSelectorLoop(AbstractNioWorker.java:158)
  426. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:114)
  427. at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
  428. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
  429. at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
  430. at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
  431. at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
  432. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  433. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  434. at java.lang.Thread.run(Thread.java:745)
  435. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  436. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  437. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  438. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  439. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  440. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  441. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  442. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  443. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  444. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  445. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  446. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  447. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  448. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  449. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  450. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  451. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  452. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  453. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  454. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  455. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  456. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  457. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  458. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  459. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  460. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  461. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  462. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  463. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  464. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  465. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  466. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  467. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  468. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  469. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  470. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  471. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  472. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  473. [1] 82244 killed bin/elasticsearch:
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement