Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- Mar 22 18:51:26 XXXXXXXXX systemd: Started Kibana.
- Mar 22 18:51:34 XXXXXXXXX systemd: Created slice User Slice of root.
- Mar 22 18:51:34 XXXXXXXXX systemd: Started Session 1 of user root.
- Mar 22 18:51:34 XXXXXXXXX systemd-logind: New session 1 of user root.
- Mar 22 18:51:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:34Z","tags":["plugin","warning"],"pid":7355,"path":"/usr/share/kibana/src/legacy/core_plugins/ems_util","message":"Skipping non-plugin directory at /usr/share/kibana/src/legacy/core_plugins/ems_util"}
- Mar 22 18:51:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:36Z","tags":["status","plugin:kibana@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:36Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:36Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:36Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:36Z","tags":["status","plugin:monitoring@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:36Z","tags":["status","plugin:spaces@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:36Z","tags":["security","warning"],"pid":7355,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml"}
- Mar 22 18:51:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:36Z","tags":["security","warning"],"pid":7355,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."}
- Mar 22 18:51:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:36Z","tags":["status","plugin:security@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:36Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:36Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:36Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:36Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:dashboard_mode@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:apm@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:interpreter@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:canvas@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:license_management@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:console@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:console_extensions@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:notifications@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:infra@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:upgrade_assistant@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:metrics@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:37Z","tags":["status","plugin:timelion@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["reporting","warning"],"pid":7355,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7355,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["license","info","xpack"],"pid":7355,"message":"Imported license information from Elasticsearch for the [data] cluster: mode: basic | status: active"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["info","monitoring-ui","kibana-monitoring"],"pid":7355,"message":"Starting monitoring stats collection"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["status","plugin:security@6.6.2","info"],"pid":7355,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:51:38Z","tags":["warning","stats-collection"],"pid":7355,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]] :: {\"path\":\"/.kibana/doc/kql-telemetry%3Akql-telemetry\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["warning","stats-collection"],"pid":7355,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:51:38Z","tags":["warning","stats-collection"],"pid":7355,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["warning","stats-collection"],"pid":7355,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:51:38Z","tags":["warning","stats-collection"],"pid":7355,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["warning","stats-collection"],"pid":7355,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:51:38Z","tags":["warning","stats-collection"],"pid":7355,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["warning","stats-collection"],"pid":7355,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:51:38Z","tags":["warning","stats-collection"],"pid":7355,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["warning","stats-collection"],"pid":7355,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:51:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:38Z","tags":["license","info","xpack"],"pid":7355,"message":"Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic | status: active"}
- Mar 22 18:51:39 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:39Z","tags":["reporting","browser-driver","warning"],"pid":7355,"message":"Enabling the Chromium sandbox provides an additional layer of protection."}
- Mar 22 18:51:48 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:51:48Z","tags":["warning","stats-collection"],"pid":7355,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:51:48 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:48Z","tags":["warning","stats-collection"],"pid":7355,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:51:48 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:51:48Z","tags":["warning","stats-collection"],"pid":7355,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:51:48 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:48Z","tags":["warning","stats-collection"],"pid":7355,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:51:48 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:51:48Z","tags":["warning","stats-collection"],"pid":7355,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]] :: {\"path\":\"/.kibana/doc/kql-telemetry%3Akql-telemetry\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]"}
- Mar 22 18:51:48 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:48Z","tags":["warning","stats-collection"],"pid":7355,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:51:48 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:51:48Z","tags":["warning","stats-collection"],"pid":7355,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:51:48 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:48Z","tags":["warning","stats-collection"],"pid":7355,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:51:48 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:51:48Z","tags":["warning","stats-collection"],"pid":7355,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:51:48 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:48Z","tags":["warning","stats-collection"],"pid":7355,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:51:50 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:50Z","tags":["status","plugin:spaces@6.6.2","error"],"pid":7355,"state":"red","message":"Status changed from yellow to red - all shards failed: [search_phase_execution_exception] all shards failed","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:50 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:50Z","tags":["fatal","root"],"pid":7355,"message":"{ [search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/doc/_count\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"should\\\":[{\\\"bool\\\":{\\\"must\\\":[{\\\"exists\\\":{\\\"field\\\":\\\"index-pattern\\\"}},{\\\"bool\\\":{\\\"must_not\\\":{\\\"term\\\":{\\\"migrationVersion.index-pattern\\\":\\\"6.5.0\\\"}}}}]}}]}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)\n status: 503,\n displayName: 'ServiceUnavailable',\n message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n path: '/.kibana/doc/_count',\n query: {},\n body:\n { error:\n { root_cause: [],\n type: 'search_phase_execution_exception',\n reason: 'all shards failed',\n phase: 'query',\n grouped: true,\n failed_shards: [] },\n status: 503 },\n statusCode: 503,\n response:\n '{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}',\n toString: [Function],\n toJSON: [Function],\n isBoom: true,\n isServer: true,\n data: null,\n output:\n { statusCode: 503,\n payload:\n { message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n statusCode: 503,\n error: 'Service Unavailable' },\n headers: {} },\n reformat: [Function],\n [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/esUnavailable' }"}
- Mar 22 18:51:50 XXXXXXXXX kibana: FATAL [search_phase_execution_exception] all shards failed :: {"path":"/.kibana/doc/_count","query":{},"body":"{\"query\":{\"bool\":{\"should\":[{\"bool\":{\"must\":[{\"exists\":{\"field\":\"index-pattern\"}},{\"bool\":{\"must_not\":{\"term\":{\"migrationVersion.index-pattern\":\"6.5.0\"}}}}]}}]}}}","statusCode":503,"response":"{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}"}
- Mar 22 18:51:50 XXXXXXXXX systemd: kibana.service: main process exited, code=exited, status=1/FAILURE
- Mar 22 18:51:50 XXXXXXXXX systemd: Unit kibana.service entered failed state.
- Mar 22 18:51:50 XXXXXXXXX systemd: kibana.service failed.
- Mar 22 18:51:50 XXXXXXXXX systemd: kibana.service holdoff time over, scheduling restart.
- Mar 22 18:51:50 XXXXXXXXX systemd: Stopped Kibana.
- Mar 22 18:51:50 XXXXXXXXX systemd: Started Kibana.
- Mar 22 18:51:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:57Z","tags":["plugin","warning"],"pid":7395,"path":"/usr/share/kibana/src/legacy/core_plugins/ems_util","message":"Skipping non-plugin directory at /usr/share/kibana/src/legacy/core_plugins/ems_util"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:kibana@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7395,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7395,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7395,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:monitoring@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:spaces@6.6.2","info"],"pid":7395,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["security","warning"],"pid":7395,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["security","warning"],"pid":7395,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:security@6.6.2","info"],"pid":7395,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7395,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7395,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7395,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7395,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7395,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:dashboard_mode@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7395,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7395,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:apm@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:interpreter@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:canvas@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:license_management@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7395,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:console@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:console_extensions@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:notifications@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7395,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:infra@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7395,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7395,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7395,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:upgrade_assistant@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:metrics@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:timelion@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["license","info","xpack"],"pid":7395,"message":"Imported license information from Elasticsearch for the [data] cluster: mode: basic | status: active"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["info","monitoring-ui","kibana-monitoring"],"pid":7395,"message":"Starting monitoring stats collection"}
- Mar 22 18:51:59 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:51:59Z","tags":["status","plugin:security@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:00 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:00Z","tags":["reporting","warning"],"pid":7395,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml"}
- Mar 22 18:52:00 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:00Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7395,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:00 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:00Z","tags":["warning","stats-collection"],"pid":7395,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:00 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:00Z","tags":["warning","stats-collection"],"pid":7395,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:52:00 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:00Z","tags":["warning","stats-collection"],"pid":7395,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]] :: {\"path\":\"/.kibana/doc/kql-telemetry%3Akql-telemetry\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]"}
- Mar 22 18:52:00 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:00Z","tags":["warning","stats-collection"],"pid":7395,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:52:00 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:00Z","tags":["warning","stats-collection"],"pid":7395,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:52:00 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:00Z","tags":["warning","stats-collection"],"pid":7395,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:52:00 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:00Z","tags":["license","info","xpack"],"pid":7395,"message":"Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic | status: active"}
- Mar 22 18:52:00 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:00Z","tags":["warning","stats-collection"],"pid":7395,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:00 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:00Z","tags":["warning","stats-collection"],"pid":7395,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:52:00 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:00Z","tags":["warning","stats-collection"],"pid":7395,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:00 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:00Z","tags":["warning","stats-collection"],"pid":7395,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:52:01 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:01Z","tags":["reporting","browser-driver","warning"],"pid":7395,"message":"Enabling the Chromium sandbox provides an additional layer of protection."}
- Mar 22 18:52:09 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:09Z","tags":["warning","stats-collection"],"pid":7395,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:09 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:09Z","tags":["warning","stats-collection"],"pid":7395,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:52:09 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:09Z","tags":["warning","stats-collection"],"pid":7395,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]] :: {\"path\":\"/.kibana/doc/kql-telemetry%3Akql-telemetry\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]"}
- Mar 22 18:52:09 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:09Z","tags":["warning","stats-collection"],"pid":7395,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:52:09 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:09Z","tags":["warning","stats-collection"],"pid":7395,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:52:09 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:09Z","tags":["warning","stats-collection"],"pid":7395,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:52:09 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:09Z","tags":["warning","stats-collection"],"pid":7395,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:09 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:09Z","tags":["warning","stats-collection"],"pid":7395,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:52:09 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:09Z","tags":["warning","stats-collection"],"pid":7395,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:09 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:09Z","tags":["warning","stats-collection"],"pid":7395,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:52:11 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:11Z","tags":["status","plugin:spaces@6.6.2","error"],"pid":7395,"state":"red","message":"Status changed from yellow to red - all shards failed: [search_phase_execution_exception] all shards failed","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:11 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:11Z","tags":["fatal","root"],"pid":7395,"message":"{ [search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/doc/_count\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"should\\\":[{\\\"bool\\\":{\\\"must\\\":[{\\\"exists\\\":{\\\"field\\\":\\\"index-pattern\\\"}},{\\\"bool\\\":{\\\"must_not\\\":{\\\"term\\\":{\\\"migrationVersion.index-pattern\\\":\\\"6.5.0\\\"}}}}]}}]}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)\n status: 503,\n displayName: 'ServiceUnavailable',\n message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n path: '/.kibana/doc/_count',\n query: {},\n body:\n { error:\n { root_cause: [],\n type: 'search_phase_execution_exception',\n reason: 'all shards failed',\n phase: 'query',\n grouped: true,\n failed_shards: [] },\n status: 503 },\n statusCode: 503,\n response:\n '{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}',\n toString: [Function],\n toJSON: [Function],\n isBoom: true,\n isServer: true,\n data: null,\n output:\n { statusCode: 503,\n payload:\n { message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n statusCode: 503,\n error: 'Service Unavailable' },\n headers: {} },\n reformat: [Function],\n [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/esUnavailable' }"}
- Mar 22 18:52:11 XXXXXXXXX kibana: FATAL [search_phase_execution_exception] all shards failed :: {"path":"/.kibana/doc/_count","query":{},"body":"{\"query\":{\"bool\":{\"should\":[{\"bool\":{\"must\":[{\"exists\":{\"field\":\"index-pattern\"}},{\"bool\":{\"must_not\":{\"term\":{\"migrationVersion.index-pattern\":\"6.5.0\"}}}}]}}]}}}","statusCode":503,"response":"{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}"}
- Mar 22 18:52:11 XXXXXXXXX systemd: kibana.service: main process exited, code=exited, status=1/FAILURE
- Mar 22 18:52:11 XXXXXXXXX systemd: Unit kibana.service entered failed state.
- Mar 22 18:52:11 XXXXXXXXX systemd: kibana.service failed.
- Mar 22 18:52:11 XXXXXXXXX systemd: kibana.service holdoff time over, scheduling restart.
- Mar 22 18:52:11 XXXXXXXXX systemd: Stopped Kibana.
- Mar 22 18:52:11 XXXXXXXXX systemd: Started Kibana.
- Mar 22 18:52:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:17Z","tags":["plugin","warning"],"pid":7419,"path":"/usr/share/kibana/src/legacy/core_plugins/ems_util","message":"Skipping non-plugin directory at /usr/share/kibana/src/legacy/core_plugins/ems_util"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:kibana@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:monitoring@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:spaces@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["security","warning"],"pid":7419,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["security","warning"],"pid":7419,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:security@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:dashboard_mode@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:apm@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:interpreter@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:canvas@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:license_management@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:console@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:console_extensions@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:notifications@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:infra@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:upgrade_assistant@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:metrics@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:18Z","tags":["status","plugin:timelion@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["reporting","warning"],"pid":7419,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7419,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["license","info","xpack"],"pid":7419,"message":"Imported license information from Elasticsearch for the [data] cluster: mode: basic | status: active"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["info","monitoring-ui","kibana-monitoring"],"pid":7419,"message":"Starting monitoring stats collection"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["status","plugin:security@6.6.2","info"],"pid":7419,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:19Z","tags":["warning","stats-collection"],"pid":7419,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]] :: {\"path\":\"/.kibana/doc/kql-telemetry%3Akql-telemetry\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["warning","stats-collection"],"pid":7419,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:19Z","tags":["warning","stats-collection"],"pid":7419,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["warning","stats-collection"],"pid":7419,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:19Z","tags":["warning","stats-collection"],"pid":7419,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["warning","stats-collection"],"pid":7419,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:19Z","tags":["warning","stats-collection"],"pid":7419,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["warning","stats-collection"],"pid":7419,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:19Z","tags":["warning","stats-collection"],"pid":7419,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["warning","stats-collection"],"pid":7419,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:52:19 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:19Z","tags":["license","info","xpack"],"pid":7419,"message":"Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic | status: active"}
- Mar 22 18:52:20 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:20Z","tags":["reporting","browser-driver","warning"],"pid":7419,"message":"Enabling the Chromium sandbox provides an additional layer of protection."}
- Mar 22 18:52:29 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:29Z","tags":["warning","stats-collection"],"pid":7419,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:29 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:29Z","tags":["warning","stats-collection"],"pid":7419,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:52:29 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:29Z","tags":["warning","stats-collection"],"pid":7419,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"term\\\":{\\\"type\\\":\\\"config\\\"}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:29 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:29Z","tags":["warning","stats-collection"],"pid":7419,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:52:29 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:29Z","tags":["warning","stats-collection"],"pid":7419,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:29 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:29Z","tags":["warning","stats-collection"],"pid":7419,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:52:29 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:29Z","tags":["warning","stats-collection"],"pid":7419,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:52:29 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:29Z","tags":["warning","stats-collection"],"pid":7419,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:52:29 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:29Z","tags":["warning","stats-collection"],"pid":7419,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:29 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:29Z","tags":["warning","stats-collection"],"pid":7419,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:52:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:30Z","tags":["status","plugin:spaces@6.6.2","error"],"pid":7419,"state":"red","message":"Status changed from yellow to red - all shards failed: [search_phase_execution_exception] all shards failed","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:30Z","tags":["fatal","root"],"pid":7419,"message":"{ [search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/doc/_count\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"should\\\":[{\\\"bool\\\":{\\\"must\\\":[{\\\"exists\\\":{\\\"field\\\":\\\"index-pattern\\\"}},{\\\"bool\\\":{\\\"must_not\\\":{\\\"term\\\":{\\\"migrationVersion.index-pattern\\\":\\\"6.5.0\\\"}}}}]}}]}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)\n status: 503,\n displayName: 'ServiceUnavailable',\n message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n path: '/.kibana/doc/_count',\n query: {},\n body:\n { error:\n { root_cause: [],\n type: 'search_phase_execution_exception',\n reason: 'all shards failed',\n phase: 'query',\n grouped: true,\n failed_shards: [] },\n status: 503 },\n statusCode: 503,\n response:\n '{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}',\n toString: [Function],\n toJSON: [Function],\n isBoom: true,\n isServer: true,\n data: null,\n output:\n { statusCode: 503,\n payload:\n { message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n statusCode: 503,\n error: 'Service Unavailable' },\n headers: {} },\n reformat: [Function],\n [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/esUnavailable' }"}
- Mar 22 18:52:30 XXXXXXXXX kibana: FATAL [search_phase_execution_exception] all shards failed :: {"path":"/.kibana/doc/_count","query":{},"body":"{\"query\":{\"bool\":{\"should\":[{\"bool\":{\"must\":[{\"exists\":{\"field\":\"index-pattern\"}},{\"bool\":{\"must_not\":{\"term\":{\"migrationVersion.index-pattern\":\"6.5.0\"}}}}]}}]}}}","statusCode":503,"response":"{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}"}
- Mar 22 18:52:30 XXXXXXXXX systemd: kibana.service: main process exited, code=exited, status=1/FAILURE
- Mar 22 18:52:30 XXXXXXXXX systemd: Unit kibana.service entered failed state.
- Mar 22 18:52:30 XXXXXXXXX systemd: kibana.service failed.
- Mar 22 18:52:30 XXXXXXXXX systemd: kibana.service holdoff time over, scheduling restart.
- Mar 22 18:52:30 XXXXXXXXX systemd: Stopped Kibana.
- Mar 22 18:52:30 XXXXXXXXX systemd: Started Kibana.
- Mar 22 18:52:35 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:35Z","tags":["plugin","warning"],"pid":7442,"path":"/usr/share/kibana/src/legacy/core_plugins/ems_util","message":"Skipping non-plugin directory at /usr/share/kibana/src/legacy/core_plugins/ems_util"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:kibana@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:monitoring@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:spaces@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["security","warning"],"pid":7442,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["security","warning"],"pid":7442,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:security@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:dashboard_mode@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:apm@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:interpreter@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:canvas@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:license_management@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:console@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:console_extensions@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:notifications@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:infra@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:upgrade_assistant@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:metrics@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:36Z","tags":["status","plugin:timelion@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["reporting","warning"],"pid":7442,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7442,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["license","info","xpack"],"pid":7442,"message":"Imported license information from Elasticsearch for the [data] cluster: mode: basic | status: active"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["info","monitoring-ui","kibana-monitoring"],"pid":7442,"message":"Starting monitoring stats collection"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["status","plugin:security@6.6.2","info"],"pid":7442,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:37Z","tags":["warning","stats-collection"],"pid":7442,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]] :: {\"path\":\"/.kibana/doc/kql-telemetry%3Akql-telemetry\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["warning","stats-collection"],"pid":7442,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:37Z","tags":["warning","stats-collection"],"pid":7442,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["warning","stats-collection"],"pid":7442,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:37Z","tags":["warning","stats-collection"],"pid":7442,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["warning","stats-collection"],"pid":7442,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:37Z","tags":["warning","stats-collection"],"pid":7442,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["warning","stats-collection"],"pid":7442,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:37Z","tags":["warning","stats-collection"],"pid":7442,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["warning","stats-collection"],"pid":7442,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:52:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:37Z","tags":["license","info","xpack"],"pid":7442,"message":"Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic | status: active"}
- Mar 22 18:52:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:38Z","tags":["reporting","browser-driver","warning"],"pid":7442,"message":"Enabling the Chromium sandbox provides an additional layer of protection."}
- Mar 22 18:52:47 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:47Z","tags":["warning","stats-collection"],"pid":7442,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:47 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:47Z","tags":["warning","stats-collection"],"pid":7442,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:52:47 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:47Z","tags":["warning","stats-collection"],"pid":7442,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:47 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:47Z","tags":["warning","stats-collection"],"pid":7442,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:52:47 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:47Z","tags":["warning","stats-collection"],"pid":7442,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:47 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:47Z","tags":["warning","stats-collection"],"pid":7442,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:52:47 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:47Z","tags":["warning","stats-collection"],"pid":7442,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]] :: {\"path\":\"/.kibana/doc/kql-telemetry%3Akql-telemetry\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]"}
- Mar 22 18:52:47 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:47Z","tags":["warning","stats-collection"],"pid":7442,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:52:47 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:47Z","tags":["warning","stats-collection"],"pid":7442,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:52:47 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:47Z","tags":["warning","stats-collection"],"pid":7442,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:52:49 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:49Z","tags":["status","plugin:spaces@6.6.2","error"],"pid":7442,"state":"red","message":"Status changed from yellow to red - all shards failed: [search_phase_execution_exception] all shards failed","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:49 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:49Z","tags":["fatal","root"],"pid":7442,"message":"{ [search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/doc/_count\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"should\\\":[{\\\"bool\\\":{\\\"must\\\":[{\\\"exists\\\":{\\\"field\\\":\\\"index-pattern\\\"}},{\\\"bool\\\":{\\\"must_not\\\":{\\\"term\\\":{\\\"migrationVersion.index-pattern\\\":\\\"6.5.0\\\"}}}}]}}]}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)\n status: 503,\n displayName: 'ServiceUnavailable',\n message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n path: '/.kibana/doc/_count',\n query: {},\n body:\n { error:\n { root_cause: [],\n type: 'search_phase_execution_exception',\n reason: 'all shards failed',\n phase: 'query',\n grouped: true,\n failed_shards: [] },\n status: 503 },\n statusCode: 503,\n response:\n '{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}',\n toString: [Function],\n toJSON: [Function],\n isBoom: true,\n isServer: true,\n data: null,\n output:\n { statusCode: 503,\n payload:\n { message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n statusCode: 503,\n error: 'Service Unavailable' },\n headers: {} },\n reformat: [Function],\n [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/esUnavailable' }"}
- Mar 22 18:52:49 XXXXXXXXX kibana: FATAL [search_phase_execution_exception] all shards failed :: {"path":"/.kibana/doc/_count","query":{},"body":"{\"query\":{\"bool\":{\"should\":[{\"bool\":{\"must\":[{\"exists\":{\"field\":\"index-pattern\"}},{\"bool\":{\"must_not\":{\"term\":{\"migrationVersion.index-pattern\":\"6.5.0\"}}}}]}}]}}}","statusCode":503,"response":"{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}"}
- Mar 22 18:52:49 XXXXXXXXX systemd: kibana.service: main process exited, code=exited, status=1/FAILURE
- Mar 22 18:52:49 XXXXXXXXX systemd: Unit kibana.service entered failed state.
- Mar 22 18:52:49 XXXXXXXXX systemd: kibana.service failed.
- Mar 22 18:52:49 XXXXXXXXX systemd: kibana.service holdoff time over, scheduling restart.
- Mar 22 18:52:49 XXXXXXXXX systemd: Stopped Kibana.
- Mar 22 18:52:49 XXXXXXXXX systemd: Started Kibana.
- Mar 22 18:52:55 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:55Z","tags":["plugin","warning"],"pid":7466,"path":"/usr/share/kibana/src/legacy/core_plugins/ems_util","message":"Skipping non-plugin directory at /usr/share/kibana/src/legacy/core_plugins/ems_util"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:kibana@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:monitoring@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:spaces@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["security","warning"],"pid":7466,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["security","warning"],"pid":7466,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:security@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:dashboard_mode@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:apm@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:interpreter@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:canvas@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:license_management@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:console@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:console_extensions@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:notifications@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:infra@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:upgrade_assistant@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:metrics@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:56Z","tags":["status","plugin:timelion@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["reporting","warning"],"pid":7466,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7466,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["license","info","xpack"],"pid":7466,"message":"Imported license information from Elasticsearch for the [data] cluster: mode: basic | status: active"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["info","monitoring-ui","kibana-monitoring"],"pid":7466,"message":"Starting monitoring stats collection"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["status","plugin:security@6.6.2","info"],"pid":7466,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:57Z","tags":["warning","stats-collection"],"pid":7466,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]] :: {\"path\":\"/.kibana/doc/kql-telemetry%3Akql-telemetry\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["warning","stats-collection"],"pid":7466,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:57Z","tags":["warning","stats-collection"],"pid":7466,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["warning","stats-collection"],"pid":7466,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:57Z","tags":["warning","stats-collection"],"pid":7466,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["warning","stats-collection"],"pid":7466,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:57Z","tags":["warning","stats-collection"],"pid":7466,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["warning","stats-collection"],"pid":7466,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:52:57Z","tags":["warning","stats-collection"],"pid":7466,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["warning","stats-collection"],"pid":7466,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["license","info","xpack"],"pid":7466,"message":"Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic | status: active"}
- Mar 22 18:52:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:52:57Z","tags":["reporting","browser-driver","warning"],"pid":7466,"message":"Enabling the Chromium sandbox provides an additional layer of protection."}
- Mar 22 18:53:07 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:07Z","tags":["warning","stats-collection"],"pid":7466,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"term\\\":{\\\"type\\\":\\\"config\\\"}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:07 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:07Z","tags":["warning","stats-collection"],"pid":7466,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:53:07 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:07Z","tags":["warning","stats-collection"],"pid":7466,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:07 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:07Z","tags":["warning","stats-collection"],"pid":7466,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:53:07 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:07Z","tags":["warning","stats-collection"],"pid":7466,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:07 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:07Z","tags":["warning","stats-collection"],"pid":7466,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:53:07 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:07Z","tags":["warning","stats-collection"],"pid":7466,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:53:07 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:07Z","tags":["warning","stats-collection"],"pid":7466,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:53:07 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:07Z","tags":["warning","stats-collection"],"pid":7466,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:07 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:07Z","tags":["warning","stats-collection"],"pid":7466,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:53:08 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:08Z","tags":["status","plugin:spaces@6.6.2","error"],"pid":7466,"state":"red","message":"Status changed from yellow to red - all shards failed: [search_phase_execution_exception] all shards failed","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:08 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:08Z","tags":["fatal","root"],"pid":7466,"message":"{ [search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/doc/_count\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"should\\\":[{\\\"bool\\\":{\\\"must\\\":[{\\\"exists\\\":{\\\"field\\\":\\\"index-pattern\\\"}},{\\\"bool\\\":{\\\"must_not\\\":{\\\"term\\\":{\\\"migrationVersion.index-pattern\\\":\\\"6.5.0\\\"}}}}]}}]}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)\n status: 503,\n displayName: 'ServiceUnavailable',\n message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n path: '/.kibana/doc/_count',\n query: {},\n body:\n { error:\n { root_cause: [],\n type: 'search_phase_execution_exception',\n reason: 'all shards failed',\n phase: 'query',\n grouped: true,\n failed_shards: [] },\n status: 503 },\n statusCode: 503,\n response:\n '{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}',\n toString: [Function],\n toJSON: [Function],\n isBoom: true,\n isServer: true,\n data: null,\n output:\n { statusCode: 503,\n payload:\n { message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n statusCode: 503,\n error: 'Service Unavailable' },\n headers: {} },\n reformat: [Function],\n [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/esUnavailable' }"}
- Mar 22 18:53:08 XXXXXXXXX kibana: FATAL [search_phase_execution_exception] all shards failed :: {"path":"/.kibana/doc/_count","query":{},"body":"{\"query\":{\"bool\":{\"should\":[{\"bool\":{\"must\":[{\"exists\":{\"field\":\"index-pattern\"}},{\"bool\":{\"must_not\":{\"term\":{\"migrationVersion.index-pattern\":\"6.5.0\"}}}}]}}]}}}","statusCode":503,"response":"{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}"}
- Mar 22 18:53:08 XXXXXXXXX systemd: kibana.service: main process exited, code=exited, status=1/FAILURE
- Mar 22 18:53:08 XXXXXXXXX systemd: Unit kibana.service entered failed state.
- Mar 22 18:53:08 XXXXXXXXX systemd: kibana.service failed.
- Mar 22 18:53:08 XXXXXXXXX systemd: kibana.service holdoff time over, scheduling restart.
- Mar 22 18:53:08 XXXXXXXXX systemd: Stopped Kibana.
- Mar 22 18:53:08 XXXXXXXXX systemd: Started Kibana.
- Mar 22 18:53:14 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:14Z","tags":["plugin","warning"],"pid":7492,"path":"/usr/share/kibana/src/legacy/core_plugins/ems_util","message":"Skipping non-plugin directory at /usr/share/kibana/src/legacy/core_plugins/ems_util"}
- Mar 22 18:53:15 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:15Z","tags":["status","plugin:kibana@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:15 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:15Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:15 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:15Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:15 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:15Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:15 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:15Z","tags":["status","plugin:monitoring@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:15 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:15Z","tags":["status","plugin:spaces@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:15 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:15Z","tags":["security","warning"],"pid":7492,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml"}
- Mar 22 18:53:15 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:15Z","tags":["security","warning"],"pid":7492,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."}
- Mar 22 18:53:15 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:15Z","tags":["status","plugin:security@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:15 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:15Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:15 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:15Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:dashboard_mode@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:apm@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:interpreter@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:canvas@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:license_management@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:console@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:console_extensions@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:notifications@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:infra@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:upgrade_assistant@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:metrics@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:timelion@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["reporting","warning"],"pid":7492,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7492,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["license","info","xpack"],"pid":7492,"message":"Imported license information from Elasticsearch for the [data] cluster: mode: basic | status: active"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["info","monitoring-ui","kibana-monitoring"],"pid":7492,"message":"Starting monitoring stats collection"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["status","plugin:security@6.6.2","info"],"pid":7492,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:16Z","tags":["warning","stats-collection"],"pid":7492,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]] :: {\"path\":\"/.kibana/doc/kql-telemetry%3Akql-telemetry\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]"}
- Mar 22 18:53:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:16Z","tags":["warning","stats-collection"],"pid":7492,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:53:17 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:17Z","tags":["warning","stats-collection"],"pid":7492,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:17Z","tags":["warning","stats-collection"],"pid":7492,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:53:17 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:17Z","tags":["warning","stats-collection"],"pid":7492,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:17Z","tags":["warning","stats-collection"],"pid":7492,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:53:17 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:17Z","tags":["warning","stats-collection"],"pid":7492,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:53:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:17Z","tags":["warning","stats-collection"],"pid":7492,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:53:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:17Z","tags":["license","info","xpack"],"pid":7492,"message":"Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic | status: active"}
- Mar 22 18:53:17 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:17Z","tags":["warning","stats-collection"],"pid":7492,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:17Z","tags":["warning","stats-collection"],"pid":7492,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:53:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:17Z","tags":["reporting","browser-driver","warning"],"pid":7492,"message":"Enabling the Chromium sandbox provides an additional layer of protection."}
- Mar 22 18:53:27 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:27Z","tags":["warning","stats-collection"],"pid":7492,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:27 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:27Z","tags":["warning","stats-collection"],"pid":7492,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:53:27 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:27Z","tags":["warning","stats-collection"],"pid":7492,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]] :: {\"path\":\"/.kibana/doc/kql-telemetry%3Akql-telemetry\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]"}
- Mar 22 18:53:27 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:27Z","tags":["warning","stats-collection"],"pid":7492,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:53:27 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:27Z","tags":["warning","stats-collection"],"pid":7492,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:53:27 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:27Z","tags":["warning","stats-collection"],"pid":7492,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:53:27 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:27Z","tags":["warning","stats-collection"],"pid":7492,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:27 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:27Z","tags":["warning","stats-collection"],"pid":7492,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:53:27 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:27Z","tags":["warning","stats-collection"],"pid":7492,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:27 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:27Z","tags":["warning","stats-collection"],"pid":7492,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:53:29 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:29Z","tags":["status","plugin:spaces@6.6.2","error"],"pid":7492,"state":"red","message":"Status changed from yellow to red - all shards failed: [search_phase_execution_exception] all shards failed","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:29 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:29Z","tags":["fatal","root"],"pid":7492,"message":"{ [search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/doc/_count\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"should\\\":[{\\\"bool\\\":{\\\"must\\\":[{\\\"exists\\\":{\\\"field\\\":\\\"index-pattern\\\"}},{\\\"bool\\\":{\\\"must_not\\\":{\\\"term\\\":{\\\"migrationVersion.index-pattern\\\":\\\"6.5.0\\\"}}}}]}}]}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)\n status: 503,\n displayName: 'ServiceUnavailable',\n message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n path: '/.kibana/doc/_count',\n query: {},\n body:\n { error:\n { root_cause: [],\n type: 'search_phase_execution_exception',\n reason: 'all shards failed',\n phase: 'query',\n grouped: true,\n failed_shards: [] },\n status: 503 },\n statusCode: 503,\n response:\n '{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}',\n toString: [Function],\n toJSON: [Function],\n isBoom: true,\n isServer: true,\n data: null,\n output:\n { statusCode: 503,\n payload:\n { message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n statusCode: 503,\n error: 'Service Unavailable' },\n headers: {} },\n reformat: [Function],\n [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/esUnavailable' }"}
- Mar 22 18:53:29 XXXXXXXXX kibana: FATAL [search_phase_execution_exception] all shards failed :: {"path":"/.kibana/doc/_count","query":{},"body":"{\"query\":{\"bool\":{\"should\":[{\"bool\":{\"must\":[{\"exists\":{\"field\":\"index-pattern\"}},{\"bool\":{\"must_not\":{\"term\":{\"migrationVersion.index-pattern\":\"6.5.0\"}}}}]}}]}}}","statusCode":503,"response":"{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}"}
- Mar 22 18:53:29 XXXXXXXXX systemd: kibana.service: main process exited, code=exited, status=1/FAILURE
- Mar 22 18:53:29 XXXXXXXXX systemd: Unit kibana.service entered failed state.
- Mar 22 18:53:29 XXXXXXXXX systemd: kibana.service failed.
- Mar 22 18:53:29 XXXXXXXXX systemd: kibana.service holdoff time over, scheduling restart.
- Mar 22 18:53:29 XXXXXXXXX systemd: Stopped Kibana.
- Mar 22 18:53:29 XXXXXXXXX systemd: Started Kibana.
- Mar 22 18:53:35 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:35Z","tags":["plugin","warning"],"pid":7513,"path":"/usr/share/kibana/src/legacy/core_plugins/ems_util","message":"Skipping non-plugin directory at /usr/share/kibana/src/legacy/core_plugins/ems_util"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:kibana@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:monitoring@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:spaces@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["security","warning"],"pid":7513,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["security","warning"],"pid":7513,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:security@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:dashboard_mode@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:apm@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:interpreter@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:canvas@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:license_management@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:console@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:console_extensions@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:notifications@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:infra@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:upgrade_assistant@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:metrics@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:timelion@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["reporting","warning"],"pid":7513,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml"}
- Mar 22 18:53:36 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:36Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7513,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["license","info","xpack"],"pid":7513,"message":"Imported license information from Elasticsearch for the [data] cluster: mode: basic | status: active"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["info","monitoring-ui","kibana-monitoring"],"pid":7513,"message":"Starting monitoring stats collection"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["status","plugin:security@6.6.2","info"],"pid":7513,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:37Z","tags":["warning","stats-collection"],"pid":7513,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["warning","stats-collection"],"pid":7513,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:37Z","tags":["warning","stats-collection"],"pid":7513,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"term\\\":{\\\"type\\\":\\\"config\\\"}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["warning","stats-collection"],"pid":7513,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:37Z","tags":["warning","stats-collection"],"pid":7513,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["warning","stats-collection"],"pid":7513,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:37Z","tags":["warning","stats-collection"],"pid":7513,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["warning","stats-collection"],"pid":7513,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:37Z","tags":["warning","stats-collection"],"pid":7513,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["warning","stats-collection"],"pid":7513,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["license","info","xpack"],"pid":7513,"message":"Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic | status: active"}
- Mar 22 18:53:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:37Z","tags":["reporting","browser-driver","warning"],"pid":7513,"message":"Enabling the Chromium sandbox provides an additional layer of protection."}
- Mar 22 18:53:47 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:47Z","tags":["warning","stats-collection"],"pid":7513,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"term\\\":{\\\"type\\\":\\\"config\\\"}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:47 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:47Z","tags":["warning","stats-collection"],"pid":7513,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:53:47 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:47Z","tags":["warning","stats-collection"],"pid":7513,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:47 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:47Z","tags":["warning","stats-collection"],"pid":7513,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:53:47 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:47Z","tags":["warning","stats-collection"],"pid":7513,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:47 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:47Z","tags":["warning","stats-collection"],"pid":7513,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:53:47 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:47Z","tags":["warning","stats-collection"],"pid":7513,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:53:47 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:47Z","tags":["warning","stats-collection"],"pid":7513,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:53:47 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:47Z","tags":["warning","stats-collection"],"pid":7513,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:47 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:47Z","tags":["warning","stats-collection"],"pid":7513,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:53:49 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:49Z","tags":["status","plugin:spaces@6.6.2","error"],"pid":7513,"state":"red","message":"Status changed from yellow to red - all shards failed: [search_phase_execution_exception] all shards failed","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:49 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:49Z","tags":["fatal","root"],"pid":7513,"message":"{ [search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/doc/_count\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"should\\\":[{\\\"bool\\\":{\\\"must\\\":[{\\\"exists\\\":{\\\"field\\\":\\\"index-pattern\\\"}},{\\\"bool\\\":{\\\"must_not\\\":{\\\"term\\\":{\\\"migrationVersion.index-pattern\\\":\\\"6.5.0\\\"}}}}]}}]}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)\n status: 503,\n displayName: 'ServiceUnavailable',\n message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n path: '/.kibana/doc/_count',\n query: {},\n body:\n { error:\n { root_cause: [],\n type: 'search_phase_execution_exception',\n reason: 'all shards failed',\n phase: 'query',\n grouped: true,\n failed_shards: [] },\n status: 503 },\n statusCode: 503,\n response:\n '{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}',\n toString: [Function],\n toJSON: [Function],\n isBoom: true,\n isServer: true,\n data: null,\n output:\n { statusCode: 503,\n payload:\n { message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n statusCode: 503,\n error: 'Service Unavailable' },\n headers: {} },\n reformat: [Function],\n [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/esUnavailable' }"}
- Mar 22 18:53:49 XXXXXXXXX kibana: FATAL [search_phase_execution_exception] all shards failed :: {"path":"/.kibana/doc/_count","query":{},"body":"{\"query\":{\"bool\":{\"should\":[{\"bool\":{\"must\":[{\"exists\":{\"field\":\"index-pattern\"}},{\"bool\":{\"must_not\":{\"term\":{\"migrationVersion.index-pattern\":\"6.5.0\"}}}}]}}]}}}","statusCode":503,"response":"{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}"}
- Mar 22 18:53:50 XXXXXXXXX systemd: kibana.service: main process exited, code=exited, status=1/FAILURE
- Mar 22 18:53:50 XXXXXXXXX systemd: Unit kibana.service entered failed state.
- Mar 22 18:53:50 XXXXXXXXX systemd: kibana.service failed.
- Mar 22 18:53:50 XXXXXXXXX systemd: kibana.service holdoff time over, scheduling restart.
- Mar 22 18:53:50 XXXXXXXXX systemd: Stopped Kibana.
- Mar 22 18:53:50 XXXXXXXXX systemd: Started Kibana.
- Mar 22 18:53:55 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:55Z","tags":["plugin","warning"],"pid":7540,"path":"/usr/share/kibana/src/legacy/core_plugins/ems_util","message":"Skipping non-plugin directory at /usr/share/kibana/src/legacy/core_plugins/ems_util"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:kibana@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:monitoring@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:spaces@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["security","warning"],"pid":7540,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["security","warning"],"pid":7540,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:security@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:dashboard_mode@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:apm@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:interpreter@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:canvas@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:license_management@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:console@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:console_extensions@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:notifications@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:infra@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:upgrade_assistant@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:metrics@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:56 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:56Z","tags":["status","plugin:timelion@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:57Z","tags":["reporting","warning"],"pid":7540,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml"}
- Mar 22 18:53:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:57Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7540,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:53:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:57Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:57 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:57Z","tags":["reporting","browser-driver","warning"],"pid":7540,"message":"Enabling the Chromium sandbox provides an additional layer of protection."}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["license","info","xpack"],"pid":7540,"message":"Imported license information from Elasticsearch for the [data] cluster: mode: basic | status: active"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["info","monitoring-ui","kibana-monitoring"],"pid":7540,"message":"Starting monitoring stats collection"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["status","plugin:security@6.6.2","info"],"pid":7540,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:58Z","tags":["warning","stats-collection"],"pid":7540,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]] :: {\"path\":\"/.kibana/doc/kql-telemetry%3Akql-telemetry\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["warning","stats-collection"],"pid":7540,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:58Z","tags":["warning","stats-collection"],"pid":7540,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["warning","stats-collection"],"pid":7540,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:58Z","tags":["warning","stats-collection"],"pid":7540,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["warning","stats-collection"],"pid":7540,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:58Z","tags":["warning","stats-collection"],"pid":7540,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["warning","stats-collection"],"pid":7540,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:53:58Z","tags":["warning","stats-collection"],"pid":7540,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["warning","stats-collection"],"pid":7540,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:53:58 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:53:58Z","tags":["license","info","xpack"],"pid":7540,"message":"Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic | status: active"}
- Mar 22 18:54:08 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:08Z","tags":["warning","stats-collection"],"pid":7540,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:54:08 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:08Z","tags":["warning","stats-collection"],"pid":7540,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:54:08 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:08Z","tags":["warning","stats-collection"],"pid":7540,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:54:08 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:08Z","tags":["warning","stats-collection"],"pid":7540,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:54:08 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:08Z","tags":["warning","stats-collection"],"pid":7540,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]] :: {\"path\":\"/.kibana/doc/kql-telemetry%3Akql-telemetry\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]"}
- Mar 22 18:54:08 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:08Z","tags":["warning","stats-collection"],"pid":7540,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:54:08 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:08Z","tags":["warning","stats-collection"],"pid":7540,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:54:08 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:08Z","tags":["warning","stats-collection"],"pid":7540,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:54:08 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:08Z","tags":["warning","stats-collection"],"pid":7540,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:54:08 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:08Z","tags":["warning","stats-collection"],"pid":7540,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:54:09 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:09Z","tags":["status","plugin:spaces@6.6.2","error"],"pid":7540,"state":"red","message":"Status changed from yellow to red - all shards failed: [search_phase_execution_exception] all shards failed","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:09 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:09Z","tags":["fatal","root"],"pid":7540,"message":"{ [search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/doc/_count\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"should\\\":[{\\\"bool\\\":{\\\"must\\\":[{\\\"exists\\\":{\\\"field\\\":\\\"index-pattern\\\"}},{\\\"bool\\\":{\\\"must_not\\\":{\\\"term\\\":{\\\"migrationVersion.index-pattern\\\":\\\"6.5.0\\\"}}}}]}}]}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)\n status: 503,\n displayName: 'ServiceUnavailable',\n message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n path: '/.kibana/doc/_count',\n query: {},\n body:\n { error:\n { root_cause: [],\n type: 'search_phase_execution_exception',\n reason: 'all shards failed',\n phase: 'query',\n grouped: true,\n failed_shards: [] },\n status: 503 },\n statusCode: 503,\n response:\n '{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}',\n toString: [Function],\n toJSON: [Function],\n isBoom: true,\n isServer: true,\n data: null,\n output:\n { statusCode: 503,\n payload:\n { message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n statusCode: 503,\n error: 'Service Unavailable' },\n headers: {} },\n reformat: [Function],\n [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/esUnavailable' }"}
- Mar 22 18:54:09 XXXXXXXXX kibana: FATAL [search_phase_execution_exception] all shards failed :: {"path":"/.kibana/doc/_count","query":{},"body":"{\"query\":{\"bool\":{\"should\":[{\"bool\":{\"must\":[{\"exists\":{\"field\":\"index-pattern\"}},{\"bool\":{\"must_not\":{\"term\":{\"migrationVersion.index-pattern\":\"6.5.0\"}}}}]}}]}}}","statusCode":503,"response":"{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}"}
- Mar 22 18:54:10 XXXXXXXXX systemd: kibana.service: main process exited, code=exited, status=1/FAILURE
- Mar 22 18:54:10 XXXXXXXXX systemd: Unit kibana.service entered failed state.
- Mar 22 18:54:10 XXXXXXXXX systemd: kibana.service failed.
- Mar 22 18:54:10 XXXXXXXXX systemd: kibana.service holdoff time over, scheduling restart.
- Mar 22 18:54:10 XXXXXXXXX systemd: Stopped Kibana.
- Mar 22 18:54:10 XXXXXXXXX systemd: Started Kibana.
- Mar 22 18:54:15 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:15Z","tags":["plugin","warning"],"pid":7560,"path":"/usr/share/kibana/src/legacy/core_plugins/ems_util","message":"Skipping non-plugin directory at /usr/share/kibana/src/legacy/core_plugins/ems_util"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:kibana@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:monitoring@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:spaces@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["security","warning"],"pid":7560,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["security","warning"],"pid":7560,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:security@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:dashboard_mode@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:apm@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:interpreter@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:canvas@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:license_management@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:console@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:console_extensions@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:notifications@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:infra@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:upgrade_assistant@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:16Z","tags":["status","plugin:metrics@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:timelion@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["reporting","warning"],"pid":7560,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7560,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["license","info","xpack"],"pid":7560,"message":"Imported license information from Elasticsearch for the [data] cluster: mode: basic | status: active"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["info","monitoring-ui","kibana-monitoring"],"pid":7560,"message":"Starting monitoring stats collection"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["status","plugin:security@6.6.2","info"],"pid":7560,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:17 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:17Z","tags":["reporting","browser-driver","warning"],"pid":7560,"message":"Enabling the Chromium sandbox provides an additional layer of protection."}
- Mar 22 18:54:18 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:18Z","tags":["warning","stats-collection"],"pid":7560,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:54:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:18Z","tags":["warning","stats-collection"],"pid":7560,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:54:18 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:18Z","tags":["warning","stats-collection"],"pid":7560,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]] :: {\"path\":\"/.kibana/doc/kql-telemetry%3Akql-telemetry\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]"}
- Mar 22 18:54:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:18Z","tags":["warning","stats-collection"],"pid":7560,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:54:18 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:18Z","tags":["warning","stats-collection"],"pid":7560,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:54:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:18Z","tags":["warning","stats-collection"],"pid":7560,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:54:18 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:18Z","tags":["warning","stats-collection"],"pid":7560,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:54:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:18Z","tags":["warning","stats-collection"],"pid":7560,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:54:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:18Z","tags":["license","info","xpack"],"pid":7560,"message":"Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic | status: active"}
- Mar 22 18:54:18 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:18Z","tags":["warning","stats-collection"],"pid":7560,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:54:18 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:18Z","tags":["warning","stats-collection"],"pid":7560,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:54:27 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:27Z","tags":["warning","stats-collection"],"pid":7560,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"term\\\":{\\\"type\\\":\\\"config\\\"}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:54:27 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:27Z","tags":["warning","stats-collection"],"pid":7560,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:54:27 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:27Z","tags":["warning","stats-collection"],"pid":7560,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:54:27 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:27Z","tags":["warning","stats-collection"],"pid":7560,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:54:27 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:27Z","tags":["warning","stats-collection"],"pid":7560,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:54:27 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:27Z","tags":["warning","stats-collection"],"pid":7560,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:54:27 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:27Z","tags":["warning","stats-collection"],"pid":7560,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:54:27 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:27Z","tags":["warning","stats-collection"],"pid":7560,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:54:27 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:27Z","tags":["warning","stats-collection"],"pid":7560,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:54:27 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:27Z","tags":["warning","stats-collection"],"pid":7560,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:54:32 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:32Z","tags":["status","plugin:spaces@6.6.2","error"],"pid":7560,"state":"red","message":"Status changed from yellow to red - all shards failed: [search_phase_execution_exception] all shards failed","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:32 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:32Z","tags":["fatal","root"],"pid":7560,"message":"{ [search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/doc/_count\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"should\\\":[{\\\"bool\\\":{\\\"must\\\":[{\\\"exists\\\":{\\\"field\\\":\\\"index-pattern\\\"}},{\\\"bool\\\":{\\\"must_not\\\":{\\\"term\\\":{\\\"migrationVersion.index-pattern\\\":\\\"6.5.0\\\"}}}}]}}]}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)\n status: 503,\n displayName: 'ServiceUnavailable',\n message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n path: '/.kibana/doc/_count',\n query: {},\n body:\n { error:\n { root_cause: [],\n type: 'search_phase_execution_exception',\n reason: 'all shards failed',\n phase: 'query',\n grouped: true,\n failed_shards: [] },\n status: 503 },\n statusCode: 503,\n response:\n '{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}',\n toString: [Function],\n toJSON: [Function],\n isBoom: true,\n isServer: true,\n data: null,\n output:\n { statusCode: 503,\n payload:\n { message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n statusCode: 503,\n error: 'Service Unavailable' },\n headers: {} },\n reformat: [Function],\n [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/esUnavailable' }"}
- Mar 22 18:54:32 XXXXXXXXX kibana: FATAL [search_phase_execution_exception] all shards failed :: {"path":"/.kibana/doc/_count","query":{},"body":"{\"query\":{\"bool\":{\"should\":[{\"bool\":{\"must\":[{\"exists\":{\"field\":\"index-pattern\"}},{\"bool\":{\"must_not\":{\"term\":{\"migrationVersion.index-pattern\":\"6.5.0\"}}}}]}}]}}}","statusCode":503,"response":"{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}"}
- Mar 22 18:54:32 XXXXXXXXX systemd: kibana.service: main process exited, code=exited, status=1/FAILURE
- Mar 22 18:54:32 XXXXXXXXX systemd: Unit kibana.service entered failed state.
- Mar 22 18:54:32 XXXXXXXXX systemd: kibana.service failed.
- Mar 22 18:54:32 XXXXXXXXX systemd: kibana.service holdoff time over, scheduling restart.
- Mar 22 18:54:32 XXXXXXXXX systemd: Stopped Kibana.
- Mar 22 18:54:32 XXXXXXXXX systemd: Started Kibana.
- Mar 22 18:54:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:37Z","tags":["plugin","warning"],"pid":7580,"path":"/usr/share/kibana/src/legacy/core_plugins/ems_util","message":"Skipping non-plugin directory at /usr/share/kibana/src/legacy/core_plugins/ems_util"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:kibana@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:monitoring@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:spaces@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["security","warning"],"pid":7580,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["security","warning"],"pid":7580,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:security@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:dashboard_mode@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:apm@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:interpreter@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:canvas@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:license_management@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:console@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:console_extensions@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:notifications@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:38 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:38Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:39 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:39Z","tags":["status","plugin:infra@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:39 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:39Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:39 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:39Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:39 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:39Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:39 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:39Z","tags":["status","plugin:upgrade_assistant@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:39 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:39Z","tags":["status","plugin:metrics@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:39 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:39Z","tags":["status","plugin:timelion@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:39 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:39Z","tags":["reporting","warning"],"pid":7580,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml"}
- Mar 22 18:54:39 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:39Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7580,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:54:39 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:39Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:39 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:39Z","tags":["reporting","browser-driver","warning"],"pid":7580,"message":"Enabling the Chromium sandbox provides an additional layer of protection."}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["license","info","xpack"],"pid":7580,"message":"Imported license information from Elasticsearch for the [data] cluster: mode: basic | status: active"}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["info","monitoring-ui","kibana-monitoring"],"pid":7580,"message":"Starting monitoring stats collection"}
- Mar 22 18:54:41 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:41Z","tags":["status","plugin:security@6.6.2","info"],"pid":7580,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:42 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:42Z","tags":["warning","stats-collection"],"pid":7580,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:54:42 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:42Z","tags":["warning","stats-collection"],"pid":7580,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:54:42 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:42Z","tags":["warning","stats-collection"],"pid":7580,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]] :: {\"path\":\"/.kibana/doc/kql-telemetry%3Akql-telemetry\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]"}
- Mar 22 18:54:42 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:42Z","tags":["warning","stats-collection"],"pid":7580,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:54:42 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:42Z","tags":["warning","stats-collection"],"pid":7580,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:54:42 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:42Z","tags":["warning","stats-collection"],"pid":7580,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:54:42 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:42Z","tags":["warning","stats-collection"],"pid":7580,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:54:42 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:42Z","tags":["warning","stats-collection"],"pid":7580,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:54:42 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:42Z","tags":["warning","stats-collection"],"pid":7580,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:54:42 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:42Z","tags":["warning","stats-collection"],"pid":7580,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:54:44 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:44Z","tags":["license","info","xpack"],"pid":7580,"message":"Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic | status: active"}
- Mar 22 18:54:52 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:52Z","tags":["warning","stats-collection"],"pid":7580,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:54:52 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:52Z","tags":["warning","stats-collection"],"pid":7580,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:54:52 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:52Z","tags":["warning","stats-collection"],"pid":7580,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"term\\\":{\\\"type\\\":\\\"config\\\"}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:54:52 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:52Z","tags":["warning","stats-collection"],"pid":7580,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:54:52 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:52Z","tags":["warning","stats-collection"],"pid":7580,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:54:52 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:52Z","tags":["warning","stats-collection"],"pid":7580,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:54:52 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:52Z","tags":["warning","stats-collection"],"pid":7580,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:54:52 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:52Z","tags":["warning","stats-collection"],"pid":7580,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:54:52 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:54:52Z","tags":["warning","stats-collection"],"pid":7580,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:54:52 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:52Z","tags":["warning","stats-collection"],"pid":7580,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:54:55 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:55Z","tags":["status","plugin:spaces@6.6.2","error"],"pid":7580,"state":"red","message":"Status changed from yellow to red - all shards failed: [search_phase_execution_exception] all shards failed","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:54:55 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:54:55Z","tags":["fatal","root"],"pid":7580,"message":"{ [search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/doc/_count\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"should\\\":[{\\\"bool\\\":{\\\"must\\\":[{\\\"exists\\\":{\\\"field\\\":\\\"index-pattern\\\"}},{\\\"bool\\\":{\\\"must_not\\\":{\\\"term\\\":{\\\"migrationVersion.index-pattern\\\":\\\"6.5.0\\\"}}}}]}}]}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)\n status: 503,\n displayName: 'ServiceUnavailable',\n message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n path: '/.kibana/doc/_count',\n query: {},\n body:\n { error:\n { root_cause: [],\n type: 'search_phase_execution_exception',\n reason: 'all shards failed',\n phase: 'query',\n grouped: true,\n failed_shards: [] },\n status: 503 },\n statusCode: 503,\n response:\n '{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}',\n toString: [Function],\n toJSON: [Function],\n isBoom: true,\n isServer: true,\n data: null,\n output:\n { statusCode: 503,\n payload:\n { message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n statusCode: 503,\n error: 'Service Unavailable' },\n headers: {} },\n reformat: [Function],\n [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/esUnavailable' }"}
- Mar 22 18:54:55 XXXXXXXXX kibana: FATAL [search_phase_execution_exception] all shards failed :: {"path":"/.kibana/doc/_count","query":{},"body":"{\"query\":{\"bool\":{\"should\":[{\"bool\":{\"must\":[{\"exists\":{\"field\":\"index-pattern\"}},{\"bool\":{\"must_not\":{\"term\":{\"migrationVersion.index-pattern\":\"6.5.0\"}}}}]}}]}}}","statusCode":503,"response":"{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}"}
- Mar 22 18:54:55 XXXXXXXXX systemd: kibana.service: main process exited, code=exited, status=1/FAILURE
- Mar 22 18:54:55 XXXXXXXXX systemd: Unit kibana.service entered failed state.
- Mar 22 18:54:55 XXXXXXXXX systemd: kibana.service failed.
- Mar 22 18:54:56 XXXXXXXXX systemd: kibana.service holdoff time over, scheduling restart.
- Mar 22 18:54:56 XXXXXXXXX systemd: Stopped Kibana.
- Mar 22 18:54:56 XXXXXXXXX systemd: Started Kibana.
- Mar 22 18:55:01 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:01Z","tags":["plugin","warning"],"pid":7594,"path":"/usr/share/kibana/src/legacy/core_plugins/ems_util","message":"Skipping non-plugin directory at /usr/share/kibana/src/legacy/core_plugins/ems_util"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:kibana@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:monitoring@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:spaces@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["security","warning"],"pid":7594,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["security","warning"],"pid":7594,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:security@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:dashboard_mode@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:apm@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:interpreter@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:canvas@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:license_management@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:console@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:console_extensions@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:notifications@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:infra@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:upgrade_assistant@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:metrics@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:timelion@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["reporting","warning"],"pid":7594,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7594,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:02 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:02Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:03 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:03Z","tags":["reporting","browser-driver","warning"],"pid":7594,"message":"Enabling the Chromium sandbox provides an additional layer of protection."}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["license","info","xpack"],"pid":7594,"message":"Imported license information from Elasticsearch for the [data] cluster: mode: basic | status: active"}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["info","monitoring-ui","kibana-monitoring"],"pid":7594,"message":"Starting monitoring stats collection"}
- Mar 22 18:55:05 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:05Z","tags":["status","plugin:security@6.6.2","info"],"pid":7594,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:06 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:06Z","tags":["warning","stats-collection"],"pid":7594,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"term\\\":{\\\"type\\\":\\\"config\\\"}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:55:06 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:06Z","tags":["warning","stats-collection"],"pid":7594,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:55:06 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:06Z","tags":["warning","stats-collection"],"pid":7594,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:55:06 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:06Z","tags":["warning","stats-collection"],"pid":7594,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:55:07 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:07Z","tags":["warning","stats-collection"],"pid":7594,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:55:07 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:07Z","tags":["warning","stats-collection"],"pid":7594,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:55:07 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:07Z","tags":["warning","stats-collection"],"pid":7594,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:55:07 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:07Z","tags":["warning","stats-collection"],"pid":7594,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:55:07 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:07Z","tags":["warning","stats-collection"],"pid":7594,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:55:07 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:07Z","tags":["warning","stats-collection"],"pid":7594,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:55:10 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:10Z","tags":["license","info","xpack"],"pid":7594,"message":"Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic | status: active"}
- Mar 22 18:55:15 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:15Z","tags":["warning","stats-collection"],"pid":7594,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:55:15 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:15Z","tags":["warning","stats-collection"],"pid":7594,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:55:15 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:15Z","tags":["warning","stats-collection"],"pid":7594,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:55:15 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:15Z","tags":["warning","stats-collection"],"pid":7594,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:55:15 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:15Z","tags":["warning","stats-collection"],"pid":7594,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]] :: {\"path\":\"/.kibana/doc/kql-telemetry%3Akql-telemetry\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]"}
- Mar 22 18:55:15 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:15Z","tags":["warning","stats-collection"],"pid":7594,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:55:15 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:15Z","tags":["warning","stats-collection"],"pid":7594,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:55:15 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:15Z","tags":["warning","stats-collection"],"pid":7594,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:55:16 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:16Z","tags":["warning","stats-collection"],"pid":7594,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:55:16 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:16Z","tags":["warning","stats-collection"],"pid":7594,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:55:23 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:23Z","tags":["status","plugin:spaces@6.6.2","error"],"pid":7594,"state":"red","message":"Status changed from yellow to red - all shards failed: [search_phase_execution_exception] all shards failed","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:23 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:23Z","tags":["fatal","root"],"pid":7594,"message":"{ [search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/doc/_count\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"should\\\":[{\\\"bool\\\":{\\\"must\\\":[{\\\"exists\\\":{\\\"field\\\":\\\"index-pattern\\\"}},{\\\"bool\\\":{\\\"must_not\\\":{\\\"term\\\":{\\\"migrationVersion.index-pattern\\\":\\\"6.5.0\\\"}}}}]}}]}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)\n status: 503,\n displayName: 'ServiceUnavailable',\n message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n path: '/.kibana/doc/_count',\n query: {},\n body:\n { error:\n { root_cause: [],\n type: 'search_phase_execution_exception',\n reason: 'all shards failed',\n phase: 'query',\n grouped: true,\n failed_shards: [] },\n status: 503 },\n statusCode: 503,\n response:\n '{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}',\n toString: [Function],\n toJSON: [Function],\n isBoom: true,\n isServer: true,\n data: null,\n output:\n { statusCode: 503,\n payload:\n { message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n statusCode: 503,\n error: 'Service Unavailable' },\n headers: {} },\n reformat: [Function],\n [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/esUnavailable' }"}
- Mar 22 18:55:23 XXXXXXXXX kibana: FATAL [search_phase_execution_exception] all shards failed :: {"path":"/.kibana/doc/_count","query":{},"body":"{\"query\":{\"bool\":{\"should\":[{\"bool\":{\"must\":[{\"exists\":{\"field\":\"index-pattern\"}},{\"bool\":{\"must_not\":{\"term\":{\"migrationVersion.index-pattern\":\"6.5.0\"}}}}]}}]}}}","statusCode":503,"response":"{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}"}
- Mar 22 18:55:24 XXXXXXXXX systemd: kibana.service: main process exited, code=exited, status=1/FAILURE
- Mar 22 18:55:24 XXXXXXXXX systemd: Unit kibana.service entered failed state.
- Mar 22 18:55:24 XXXXXXXXX systemd: kibana.service failed.
- Mar 22 18:55:24 XXXXXXXXX systemd: kibana.service holdoff time over, scheduling restart.
- Mar 22 18:55:24 XXXXXXXXX systemd: Stopped Kibana.
- Mar 22 18:55:24 XXXXXXXXX systemd: Started Kibana.
- Mar 22 18:55:29 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:29Z","tags":["plugin","warning"],"pid":7614,"path":"/usr/share/kibana/src/legacy/core_plugins/ems_util","message":"Skipping non-plugin directory at /usr/share/kibana/src/legacy/core_plugins/ems_util"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:kibana@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:monitoring@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:spaces@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["security","warning"],"pid":7614,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["security","warning"],"pid":7614,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:security@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:dashboard_mode@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:apm@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:interpreter@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:canvas@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:license_management@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:console@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:console_extensions@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:notifications@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:infra@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:upgrade_assistant@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:metrics@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:30 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:30Z","tags":["status","plugin:timelion@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:31 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:31Z","tags":["reporting","warning"],"pid":7614,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml"}
- Mar 22 18:55:31 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:31Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7614,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
- Mar 22 18:55:31 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:31Z","tags":["reporting","browser-driver","warning"],"pid":7614,"message":"Enabling the Chromium sandbox provides an additional layer of protection."}
- Mar 22 18:55:32 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:32Z","tags":["status","plugin:elasticsearch@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["license","info","xpack"],"pid":7614,"message":"Imported license information from Elasticsearch for the [data] cluster: mode: basic | status: active"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["status","plugin:xpack_main@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["status","plugin:graph@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["status","plugin:searchprofiler@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["status","plugin:ml@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["status","plugin:tilemap@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["status","plugin:watcher@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["status","plugin:grokdebugger@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["status","plugin:logstash@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["status","plugin:beats_management@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["status","plugin:index_management@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["status","plugin:index_lifecycle_management@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["status","plugin:rollup@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["status","plugin:remote_clusters@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["status","plugin:cross_cluster_replication@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["status","plugin:reporting@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["info","monitoring-ui","kibana-monitoring"],"pid":7614,"message":"Starting monitoring stats collection"}
- Mar 22 18:55:34 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:34Z","tags":["status","plugin:security@6.6.2","info"],"pid":7614,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:35 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:35Z","tags":["warning","stats-collection"],"pid":7614,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"term\\\":{\\\"type\\\":\\\"config\\\"}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:55:35 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:35Z","tags":["warning","stats-collection"],"pid":7614,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:55:35 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:35Z","tags":["warning","stats-collection"],"pid":7614,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:55:35 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:35Z","tags":["warning","stats-collection"],"pid":7614,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:55:35 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:35Z","tags":["warning","stats-collection"],"pid":7614,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:55:35 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:35Z","tags":["warning","stats-collection"],"pid":7614,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:55:35 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:35Z","tags":["warning","stats-collection"],"pid":7614,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:55:35 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:35Z","tags":["warning","stats-collection"],"pid":7614,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:55:35 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:35Z","tags":["warning","stats-collection"],"pid":7614,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:55:35 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:35Z","tags":["warning","stats-collection"],"pid":7614,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:55:37 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:37Z","tags":["license","info","xpack"],"pid":7614,"message":"Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic | status: active"}
- Mar 22 18:55:45 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:45Z","tags":["warning","stats-collection"],"pid":7614,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]] :: {\"path\":\"/.kibana/doc/kql-telemetry%3Akql-telemetry\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][kql-telemetry:kql-telemetry]: routing [null]]"}
- Mar 22 18:55:45 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:45Z","tags":["warning","stats-collection"],"pid":7614,"message":"Unable to fetch data from kql collector"}
- Mar 22 18:55:45 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:45Z","tags":["warning","stats-collection"],"pid":7614,"level":"error","error":{"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]","name":"Error","stack":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]] :: {\"path\":\"/.kibana/doc/config%3A6.6.2\",\"query\":{},\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"}],\\\"type\\\":\\\"no_shard_available_action_exception\\\",\\\"reason\\\":\\\"No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]\\\"},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[no_shard_available_action_exception] No shard available for [get [.kibana][doc][config:6.6.2]: routing [null]]"}
- Mar 22 18:55:45 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:45Z","tags":["warning","stats-collection"],"pid":7614,"message":"Unable to fetch data from kibana_settings collector"}
- Mar 22 18:55:45 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:45Z","tags":["warning","stats-collection"],"pid":7614,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:55:45 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:45Z","tags":["warning","stats-collection"],"pid":7614,"message":"Unable to fetch data from kibana collector"}
- Mar 22 18:55:45 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:45Z","tags":["warning","stats-collection"],"pid":7614,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":1000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._id\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"index-pattern.type\\\":\\\"rollup\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:55:45 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:45Z","tags":["warning","stats-collection"],"pid":7614,"message":"Unable to fetch data from rollups collector"}
- Mar 22 18:55:45 XXXXXXXXX kibana: {"type":"error","@timestamp":"2022-03-22T17:55:45Z","tags":["warning","stats-collection"],"pid":7614,"level":"error","error":{"message":"[search_phase_execution_exception] all shards failed","name":"Error","stack":"[search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_search\",\"query\":{\"size\":10000,\"ignore_unavailable\":true,\"filter_path\":\"hits.hits._source.canvas-workpad\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"filter\\\":{\\\"term\\\":{\\\"type\\\":\\\"canvas-workpad\\\"}}}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)"},"message":"[search_phase_execution_exception] all shards failed"}
- Mar 22 18:55:45 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:45Z","tags":["warning","stats-collection"],"pid":7614,"message":"Unable to fetch data from canvas collector"}
- Mar 22 18:55:49 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:49Z","tags":["status","plugin:spaces@6.6.2","error"],"pid":7614,"state":"red","message":"Status changed from yellow to red - all shards failed: [search_phase_execution_exception] all shards failed","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
- Mar 22 18:55:49 XXXXXXXXX kibana: {"type":"log","@timestamp":"2022-03-22T17:55:49Z","tags":["fatal","root"],"pid":7614,"message":"{ [search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/doc/_count\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"should\\\":[{\\\"bool\\\":{\\\"must\\\":[{\\\"exists\\\":{\\\"field\\\":\\\"index-pattern\\\"}},{\\\"bool\\\":{\\\"must_not\\\":{\\\"term\\\":{\\\"migrationVersion.index-pattern\\\":\\\"6.5.0\\\"}}}}]}}]}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n at IncomingMessage.emit (events.js:194:15)\n at endReadableNT (_stream_readable.js:1103:12)\n at process._tickCallback (internal/process/next_tick.js:63:19)\n status: 503,\n displayName: 'ServiceUnavailable',\n message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n path: '/.kibana/doc/_count',\n query: {},\n body:\n { error:\n { root_cause: [],\n type: 'search_phase_execution_exception',\n reason: 'all shards failed',\n phase: 'query',\n grouped: true,\n failed_shards: [] },\n status: 503 },\n statusCode: 503,\n response:\n '{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}',\n toString: [Function],\n toJSON: [Function],\n isBoom: true,\n isServer: true,\n data: null,\n output:\n { statusCode: 503,\n payload:\n { message:\n 'all shards failed: [search_phase_execution_exception] all shards failed',\n statusCode: 503,\n error: 'Service Unavailable' },\n headers: {} },\n reformat: [Function],\n [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/esUnavailable' }"}
- Mar 22 18:55:49 XXXXXXXXX kibana: FATAL [search_phase_execution_exception] all shards failed :: {"path":"/.kibana/doc/_count","query":{},"body":"{\"query\":{\"bool\":{\"should\":[{\"bool\":{\"must\":[{\"exists\":{\"field\":\"index-pattern\"}},{\"bool\":{\"must_not\":{\"term\":{\"migrationVersion.index-pattern\":\"6.5.0\"}}}}]}}]}}}","statusCode":503,"response":"{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}"}
- Mar 22 18:55:49 XXXXXXXXX systemd: kibana.service: main process exited, code=exited, status=1/FAILURE
- Mar 22 18:55:49 XXXXXXXXX systemd: Unit kibana.service entered failed state.
- Mar 22 18:55:49 XXXXXXXXX systemd: kibana.service failed.
- Mar 22 18:55:49 XXXXXXXXX systemd: kibana.service holdoff time over, scheduling restart.
- Mar 22 18:55:49 XXXXXXXXX systemd: Stopped Kibana.
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement