API tools faq
paste
Advertisement
franek

es console

franek
Jun 14th, 2016
400
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 47.81 KB | None | 0 0
raw download clone embed print report
  1. [2016-06-14 08:46:31,308][INFO ][node ] [Infant Terrible] version[2.3.3], pid[82244], build[218bdf1/2016-05-17T15:40:04Z]
  2. [2016-06-14 08:46:31,309][INFO ][node ] [Infant Terrible] initializing ...
  3. [2016-06-14 08:46:33,144][INFO ][plugins ] [Infant Terrible] modules [reindex, lang-expression, lang-groovy], plugins [whatson, kopf], sites [kopf, whatson]
  4. [2016-06-14 08:46:33,287][INFO ][env ] [Infant Terrible] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [66.6gb], net total_space [233.1gb], spins? [unknown], types [hfs]
  5. [2016-06-14 08:46:33,288][INFO ][env ] [Infant Terrible] heap size [1.9gb], compressed ordinary object pointers [true]
  6. [2016-06-14 08:46:33,289][WARN ][env ] [Infant Terrible] max file descriptors [10240] for elasticsearch process likely too low, consider increasing to at least [65536]
  7. [2016-06-14 08:46:38,748][INFO ][node ] [Infant Terrible] initialized
  8. [2016-06-14 08:46:38,749][INFO ][node ] [Infant Terrible] starting ...
  9. [2016-06-14 08:46:39,020][INFO ][transport ] [Infant Terrible] publish_address {127.0.0.1:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300}, {127.94.0.2:9300}, {127.94.0.1:9300}
  10. [2016-06-14 08:46:39,027][INFO ][discovery ] [Infant Terrible] address-reputation/cXEBEbAQQkK5qZXQRi2Gtg
  11. [2016-06-14 08:46:42,068][INFO ][cluster.service ] [Infant Terrible] new_master {Infant Terrible}{cXEBEbAQQkK5qZXQRi2Gtg}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
  12. [2016-06-14 08:46:42,085][INFO ][http ] [Infant Terrible] publish_address {127.0.0.1:9200}, bound_addresses {[fe80::1]:9200}, {[::1]:9200}, {127.0.0.1:9200}, {127.94.0.2:9200}, {127.94.0.1:9200}
  13. [2016-06-14 08:46:42,085][INFO ][node ] [Infant Terrible] started
  14. [2016-06-14 08:46:42,133][INFO ][gateway ] [Infant Terrible] recovered [1] indices into cluster_state
  15. [2016-06-14 08:46:43,524][INFO ][cluster.routing.allocation] [Infant Terrible] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[address-reputation-data-full][3]] ...]).
  16. [2016-06-14 08:50:40,554][INFO ][cluster.metadata ] [Infant Terrible] [address-reputation-data] creating index, cause [api], templates [], shards [5]/[0], mappings [address]
  17. [2016-06-14 08:50:40,660][INFO ][cluster.routing.allocation] [Infant Terrible] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[address-reputation-data][4]] ...]).
  18. java.lang.OutOfMemoryError: Java heap space
  19. Dumping heap to java_pid82244.hprof ...
  20. Heap dump file created [2129241713 bytes in 8.721 secs]
  21. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  22. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  23. [2016-06-14 09:22:44,100][ERROR][rest.action.support ] failed to send failure response
  24. java.lang.OutOfMemoryError: Java heap space
  25. [2016-06-14 09:22:50,281][WARN ][action.bulk ] [Infant Terrible] unexpected error during the primary phase for action [indices:data/write/bulk[s]], request [BulkShardRequest to [address-reputation-data] containing [230] requests]
  26. java.lang.OutOfMemoryError: Java heap space
  27. [2016-06-14 09:22:48,704][DEBUG][action.bulk ] [Infant Terrible] [address-reputation-data][2] failed to execute bulk item (index) index {[address-reputation-data][address][GB906087880], source[{"postcode":"EH4 1SB","line1":"3 East Werberside Place","id":"GB906087880","line2":"","town":"Edinburgh","line3":""}]}
  28. MapperParsingException[failed to parse]; nested: OutOfMemoryError[Java heap space];
  29. at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:154)
  30. at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:309)
  31. at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:580)
  32. at org.elasticsearch.index.shard.IndexShard.prepareIndexOnPrimary(IndexShard.java:559)
  33. at org.elasticsearch.action.index.TransportIndexAction.prepareIndexOperationOnPrimary(TransportIndexAction.java:212)
  34. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:224)
  35. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  36. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  37. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  38. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  39. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  40. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  41. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  42. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  43. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  44. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  45. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  46. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  47. at java.lang.Thread.run(Thread.java:745)
  48. Caused by: java.lang.OutOfMemoryError: Java heap space
  49. Exception in thread "RMI TCP Connection(idle)" [2016-06-14 09:22:49,799][WARN ][rest.suppressed ] /_cluster/health Params: {}
  50. java.lang.OutOfMemoryError: Java heap space
  51. at com.fasterxml.jackson.core.util.BufferRecycler.balloc(BufferRecycler.java:155)
  52. at com.fasterxml.jackson.core.util.BufferRecycler.allocByteBuffer(BufferRecycler.java:96)
  53. at com.fasterxml.jackson.core.util.BufferRecycler.allocByteBuffer(BufferRecycler.java:86)
  54. at com.fasterxml.jackson.core.io.IOContext.allocWriteEncodingBuffer(IOContext.java:160)
  55. at com.fasterxml.jackson.core.json.UTF8JsonGenerator.<init>(UTF8JsonGenerator.java:118)
  56. at com.fasterxml.jackson.core.JsonFactory._createUTF8Generator(JsonFactory.java:1333)
  57. at com.fasterxml.jackson.core.JsonFactory.createGenerator(JsonFactory.java:1065)
  58. at org.elasticsearch.common.xcontent.json.JsonXContent.createGenerator(JsonXContent.java:73)
  59. at org.elasticsearch.common.xcontent.XContentBuilder.<init>(XContentBuilder.java:113)
  60. at org.elasticsearch.rest.RestChannel.newBuilder(RestChannel.java:73)
  61. at org.elasticsearch.rest.RestChannel.newBuilder(RestChannel.java:51)
  62. at org.elasticsearch.rest.action.support.RestStatusToXContentListener.buildResponse(RestStatusToXContentListener.java:38)
  63. at org.elasticsearch.rest.action.support.RestStatusToXContentListener.buildResponse(RestStatusToXContentListener.java:30)
  64. at org.elasticsearch.rest.action.support.RestResponseListener.processResponse(RestResponseListener.java:43)
  65. at org.elasticsearch.rest.action.support.RestActionListener.onResponse(RestActionListener.java:49)
  66. at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:89)
  67. at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:85)
  68. at org.elasticsearch.action.support.ThreadedActionListener$1.doRun(ThreadedActionListener.java:89)
  69. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  70. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  71. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  72. at java.lang.Thread.run(Thread.java:745)
  73. java.lang.OutOfMemoryError: Java heap space
  74. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  75. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  76. [2016-06-14 09:22:49,799][ERROR][rest.action.support ] failed to send failure response
  77. java.lang.OutOfMemoryError: Java heap space
  78. [2016-06-14 09:22:49,799][DEBUG][action.bulk ] [Infant Terrible] [address-reputation-data][0] failed to execute bulk item (index) index {[address-reputation-data][address][GB100021443846], source[{"postcode":"HA6 2TB","line1":"18 Fringewood Close","id":"GB100021443846","line2":"","town":"Northwood","line3":""}]}
  79. MapperParsingException[failed to parse]; nested: OutOfMemoryError[Java heap space];
  80. at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:154)
  81. at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:309)
  82. at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:580)
  83. at org.elasticsearch.index.shard.IndexShard.prepareIndexOnPrimary(IndexShard.java:559)
  84. at org.elasticsearch.action.index.TransportIndexAction.prepareIndexOperationOnPrimary(TransportIndexAction.java:212)
  85. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:224)
  86. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  87. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  88. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  89. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  90. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  91. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  92. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  93. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  94. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  95. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  96. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  97. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  98. at java.lang.Thread.run(Thread.java:745)
  99. Caused by: java.lang.OutOfMemoryError: Java heap space
  100. at com.fasterxml.jackson.core.util.BufferRecycler.balloc(BufferRecycler.java:155)
  101. at com.fasterxml.jackson.core.util.BufferRecycler.allocByteBuffer(BufferRecycler.java:96)
  102. at com.fasterxml.jackson.core.util.BufferRecycler.allocByteBuffer(BufferRecycler.java:86)
  103. at com.fasterxml.jackson.core.io.IOContext.allocReadIOBuffer(IOContext.java:147)
  104. at com.fasterxml.jackson.core.json.ByteSourceJsonBootstrapper.<init>(ByteSourceJsonBootstrapper.java:87)
  105. at com.fasterxml.jackson.core.JsonFactory._createParser(JsonFactory.java:1240)
  106. at com.fasterxml.jackson.core.JsonFactory.createParser(JsonFactory.java:802)
  107. at org.elasticsearch.common.xcontent.json.JsonXContent.createParser(JsonXContent.java:83)
  108. at org.elasticsearch.common.xcontent.XContentHelper.createParser(XContentHelper.java:59)
  109. at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:90)
  110. ... 18 more
  111. [2016-06-14 09:22:49,100][ERROR][rest.action.support ] failed to send failure response
  112. java.lang.OutOfMemoryError: Java heap space
  113. [2016-06-14 09:22:48,704][WARN ][netty.channel.socket.nio.AbstractNioSelector] Unexpected exception in the selector loop.
  114. java.lang.OutOfMemoryError: Java heap space
  115. [2016-06-14 09:23:06,870][DEBUG][action.bulk ] [Infant Terrible] [address-reputation-data][0] failed to execute bulk item (index) index {[address-reputation-data][address][GB100070956282], source[{"postcode":"B37 6TP","line1":"59 Cornfield Croft","id":"GB100070956282","line2":"","town":"Birmingham","line3":""}]}
  116. [address-reputation-data][[address-reputation-data][0]] IndexFailedEngineException[Index failed for [address#GB100070956282]]; nested: AlreadyClosedException[this IndexWriter is closed]; nested: OutOfMemoryError[Java heap space];
  117. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:462)
  118. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:601)
  119. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  120. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  121. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  122. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  123. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  124. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  125. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  126. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  127. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  128. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  129. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  130. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  131. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  132. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  133. at java.lang.Thread.run(Thread.java:745)
  134. Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
  135. at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
  136. at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
  137. at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
  138. at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
  139. at org.elasticsearch.index.engine.InternalEngine.innerIndex(InternalEngine.java:530)
  140. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:457)
  141. ... 16 more
  142. Caused by: java.lang.OutOfMemoryError: Java heap space
  143. [2016-06-14 09:23:06,885][DEBUG][action.bulk ] [Infant Terrible] failed to execute [BulkShardRequest to [address-reputation-data] containing [206] requests] on [[address-reputation-data][0]]
  144. [address-reputation-data][[address-reputation-data][0]] EngineClosedException[CurrentState[CLOSED] Closed]
  145. at org.elasticsearch.index.engine.Engine.ensureOpen(Engine.java:329)
  146. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:451)
  147. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:601)
  148. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  149. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  150. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  151. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  152. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  153. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  154. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  155. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  156. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  157. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  158. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  159. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  160. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  161. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  162. at java.lang.Thread.run(Thread.java:745)
  163. [2016-06-14 09:23:06,852][WARN ][http.netty ] [Infant Terrible] Caught exception while handling client http traffic, closing connection [id: 0x29ef5f8f, /0:0:0:0:0:0:0:1:50443 => /0:0:0:0:0:0:0:1:9200]
  164. java.lang.OutOfMemoryError: Java heap space
  165. at java.util.Arrays.copyOfRange(Arrays.java:3664)
  166. at java.lang.String.<init>(String.java:207)
  167. at java.lang.StringBuilder.toString(StringBuilder.java:407)
  168. at org.jboss.netty.handler.codec.http.HttpMessageDecoder.readHeader(HttpMessageDecoder.java:633)
  169. at org.jboss.netty.handler.codec.http.HttpMessageDecoder.readHeaders(HttpMessageDecoder.java:531)
  170. at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:195)
  171. at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:102)
  172. at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500)
  173. at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
  174. at org.jboss.Jun 14, 2016 9:22:56 AM ServerCommunicatorAdmin reqIncoming
  175. WARNING: The server has decided to close this client connection.
  176. netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
  177. at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
  178. at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
  179. at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:75)
  180. at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
  181. at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
  182. at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
  183. at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
  184. at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
  185. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
  186. at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
  187. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
  188. at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
  189. at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
  190. at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
  191. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  192. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  193. at java.lang.Thread.run(Thread.java:745)
  194. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  195. [2016-06-14 09:23:06,893][DEBUG][action.bulk ] [Infant Terrible] failed to execute [BulkShardRequest to [address-reputation-data] containing [206] requests] on [[address-reputation-data][0]]
  196. [address-reputation-data][[address-reputation-data][0]] EngineClosedException[CurrentState[CLOSED] Closed]
  197. at org.elasticsearch.index.engine.Engine.ensureOpen(Engine.java:329)
  198. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:451)
  199. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:601)
  200. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  201. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  202. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  203. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  204. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  205. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  206. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  207. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  208. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  209. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  210. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  211. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  212. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  213. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  214. at java.lang.Thread.run(Thread.java:745)
  215. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  216. [2016-06-14 09:23:06,898][DEBUG][action.bulk ] [Infant Terrible] failed to execute [BulkShardRequest to [address-reputation-data] containing [230] requests] on [[address-reputation-data][0]]
  217. [address-reputation-data][[address-reputation-data][0]] EngineClosedException[CurrentState[CLOSED] Closed]
  218. at org.elasticsearch.index.engine.Engine.ensureOpen(Engine.java:329)
  219. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:451)
  220. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:601)
  221. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  222. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  223. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  224. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  225. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  226. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  227. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  228. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  229. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  230. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  231. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  232. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  233. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  234. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  235. at java.lang.Thread.run(Thread.java:745)
  236. [2016-06-14 09:23:06,892][WARN ][index.engine ] [Infant Terrible] [address-reputation-data][0] failed engine [already closed by tragic event on the index writer]
  237. java.lang.OutOfMemoryError: Java heap space
  238. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  239. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  240. [2016-06-14 09:23:06,917][WARN ][indices.cluster ] [Infant Terrible] [[address-reputation-data][0]] marking and sending shard failed due to [engine failure, reason [already closed by tragic event on the index writer]]
  241. java.lang.OutOfMemoryError: Java heap space
  242. [2016-06-14 09:23:06,918][WARN ][cluster.action.shard ] [Infant Terrible] [address-reputation-data][0] received shard failed for target shard [[address-reputation-data][0], node[cXEBEbAQQkK5qZXQRi2Gtg], [P], v[2], s[STARTED], a[id=CgBkNgr3RxCNjK6dYIZFVg]], indexUUID [RI06vpNWRh6pMzN3MZHuew], message [engine failure, reason [already closed by tragic event on the index writer]], failure [OutOfMemoryError[Java heap space]]
  243. java.lang.OutOfMemoryError: Java heap space
  244. [2016-06-14 09:23:06,886][DEBUG][action.bulk ] [Infant Terrible] failed to execute [BulkShardRequest to [address-reputation-data] containing [210] requests] on [[address-reputation-data][0]]
  245. [address-reputation-data][[address-reputation-data][0]] EngineClosedException[CurrentState[CLOSED] Closed]
  246. at org.elasticsearch.index.engine.Engine.ensureOpen(Engine.java:329)
  247. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:451)
  248. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:601)
  249. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  250. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  251. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  252. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  253. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  254. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  255. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  256. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  257. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  258. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  259. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  260. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  261. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  262. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  263. at java.lang.Thread.run(Thread.java:745)
  264. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  265. [2016-06-14 09:23:06,914][DEBUG][action.bulk ] [Infant Terrible] [address-reputation-data][0] failed to execute bulk item (index) index {[address-reputation-data][address][GB100091479761], source[{"postcode":"IG10 4QY","line1":"30 Grange Court Upper Park","id":"GB100091479761","line2":"","town":"Loughton","line3":""}]}
  266. [address-reputation-data][[address-reputation-data][0]] IndexFailedEngineException[Index failed for [address#GB100091479761]]; nested: AlreadyClosedException[this IndexWriter is closed]; nested: OutOfMemoryError[Java heap space];
  267. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:462)
  268. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:601)
  269. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  270. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  271. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  272. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  273. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  274. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  275. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  276. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  277. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  278. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  279. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  280. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  281. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  282. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  283. at java.lang.Thread.run(Thread.java:745)
  284. Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
  285. at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
  286. at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
  287. at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
  288. at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
  289. at org.elasticsearch.index.engine.InternalEngine.innerIndex(InternalEngine.java:530)
  290. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:457)
  291. ... 16 more
  292. Caused by: java.lang.OutOfMemoryError: Java heap space
  293. [2016-06-14 09:23:06,895][DEBUG][action.bulk ] [Infant Terrible] [address-reputation-data][0] failed to execute bulk item (index) index {[address-reputation-data][address][GB100080910698], source[{"postcode":"WD3 8BP","line1":"36 Maxwell Close","id":"GB100080910698","line2":"Mill End","town":"Rickmansworth","line3":""}]}
  294. [address-reputation-data][[address-reputation-data][0]] IndexFailedEngineException[Index failed for [address#GB100080910698]]; nested: OutOfMemoryError[Java heap space];
  295. at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:462)
  296. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:601)
  297. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  298. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  299. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  300. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  301. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  302. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  303. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  304. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  305. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  306. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  307. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  308. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  309. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  310. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  311. at java.lang.Thread.run(Thread.java:745)
  312. Caused by: java.lang.OutOfMemoryError: Java heap space
  313. [2016-06-14 09:23:06,930][DEBUG][action.bulk ] [Infant Terrible] failed to execute [BulkShardRequest to [address-reputation-data] containing [212] requests] on [[address-reputation-data][0]]
  314. [address-reputation-data][[address-reputation-data][0]] IllegalIndexShardStateException[CurrentState[CLOSED] operation only allowed when started/recovering, origin [PRIMARY]]
  315. at org.elasticsearch.index.shard.IndexShard.ensureWriteAllowed(IndexShard.java:1062)
  316. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:593)
  317. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  318. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  319. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  320. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  321. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  322. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  323. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  324. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  325. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  326. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  327. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  328. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  329. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  330. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  331. at java.lang.Thread.run(Thread.java:745)
  332. [2016-06-14 09:23:06,926][DEBUG][action.bulk ] [Infant Terrible] failed to execute [BulkShardRequest to [address-reputation-data] containing [195] requests] on [[address-reputation-data][0]]
  333. [address-reputation-data][[address-reputation-data][0]] ShardNotFoundException[no such shard]
  334. at org.elasticsearch.index.IndexService.shardSafe(IndexService.java:197)
  335. at org.elasticsearch.action.support.replication.TransportReplicationAction.getIndexShardOperationsCounter(TransportReplicationAction.java:746)
  336. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:638)
  337. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  338. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  339. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  340. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  341. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  342. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  343. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  344. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  345. at java.lang.Thread.run(Thread.java:745)
  346. [2016-06-14 09:23:06,937][INFO ][cluster.routing.allocation] [Infant Terrible] Cluster health status changed from [YELLOW] to [RED] (reason: [shards failed [[address-reputation-data][0]] ...]).
  347. [2016-06-14 09:23:07,644][DEBUG][action.bulk ] [Infant Terrible] failed to execute [BulkShardRequest to [address-reputation-data] containing [235] requests] on [[address-reputation-data][0]]
  348. [address-reputation-data][[address-reputation-data][0]] ShardNotFoundException[no such shard]
  349. at org.elasticsearch.index.IndexService.shardSafe(IndexService.java:197)
  350. at org.elasticsearch.action.support.replication.TransportReplicationAction.getIndexShardOperationsCounter(TransportReplicationAction.java:746)
  351. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:638)
  352. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  353. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  354. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  355. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  356. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  357. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  358. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  359. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  360. at java.lang.Thread.run(Thread.java:745)
  361. [2016-06-14 09:23:06,932][DEBUG][action.bulk ] [Infant Terrible] failed to execute [BulkShardRequest to [address-reputation-data] containing [245] requests] on [[address-reputation-data][0]]
  362. [address-reputation-data][[address-reputation-data][0]] IllegalIndexShardStateException[CurrentState[CLOSED] operation only allowed when started/recovering, origin [PRIMARY]]
  363. at org.elasticsearch.index.shard.IndexShard.ensureWriteAllowed(IndexShard.java:1062)
  364. at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:593)
  365. at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836)
  366. at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:237)
  367. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
  368. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
  369. at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
  370. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
  371. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  372. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
  373. at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
  374. at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
  375. at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
  376. at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  377. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  378. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  379. at java.lang.Thread.run(Thread.java:745)
  380. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  381. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  382. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  383. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  384. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  385. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  386. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  387. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  388. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  389. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  390. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  391. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  392. [2016-06-14 09:23:15,496][INFO ][node ] [Infant Terrible] stopping ...
  393. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  394. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  395. [2016-06-14 09:23:17,041][WARN ][transport.netty ] [Infant Terrible] exception caught on transport layer [[id: 0x4ad0c069, /127.0.0.1:50553 => /127.0.0.1:9300]], closing connection
  396. java.io.IOException: Protocol wrong type for socket
  397. at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
  398. at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
  399. at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
  400. at sun.nio.ch.IOUtil.write(IOUtil.java:65)
  401. at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
  402. at org.jboss.netty.channel.socket.nio.SocketSendBufferPool$UnpooledSendBuffer.transferTo(SocketSendBufferPool.java:203)
  403. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.write0(AbstractNioWorker.java:201)
  404. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.writeFromSelectorLoop(AbstractNioWorker.java:158)
  405. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:114)
  406. at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstException in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  407. ractNioSelector.java:337)
  408. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
  409. at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
  410. at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
  411. at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
  412. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  413. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  414. at java.lang.Thread.run(Thread.java:745)
  415. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  416. [2016-06-14 09:23:17,465][WARN ][transport.netty ] [Infant Terrible] exception caught on transport layer [[id: 0xd2df1f91, /127.0.0.1:50552 => /127.0.0.1:9300]], closing connection
  417. java.io.IOException: Protocol wrong type for socket
  418. at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
  419. at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
  420. at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
  421. at sun.nio.ch.IOUtil.write(IOUtil.java:65)
  422. at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
  423. at org.jboss.netty.channel.socket.nio.SocketSendBufferPool$UnpooledSendBuffer.transferTo(SocketSendBufferPool.java:203)
  424. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.write0(AbstractNioWorker.java:201)
  425. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.writeFromSelectorLoop(AbstractNioWorker.java:158)
  426. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:114)
  427. at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
  428. at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
  429. at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
  430. at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
  431. at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
  432. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  433. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  434. at java.lang.Thread.run(Thread.java:745)
  435. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  436. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  437. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  438. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  439. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  440. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  441. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  442. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  443. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  444. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  445. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  446. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  447. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  448. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  449. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  450. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  451. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  452. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  453. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  454. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  455. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  456. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  457. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  458. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  459. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  460. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  461. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  462. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  463. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  464. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  465. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  466. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  467. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  468. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  469. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  470. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  471. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  472. Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
  473. [1] 82244 killed bin/elasticsearch:
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!