Guest User

Untitled

a guest
Dec 19th, 2019
1,077
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 91.55 KB | None | 0 0
  1. * ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
  2. * ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
  3. * directory[/opt/gitlab/sv/postgres-exporter] action create
  4. - create new directory /opt/gitlab/sv/postgres-exporter
  5. - change mode from '' to '0755'
  6. - change owner from '' to 'root'
  7. - change group from '' to 'root'
  8. * template[/opt/gitlab/sv/postgres-exporter/run] action create
  9. - create new file /opt/gitlab/sv/postgres-exporter/run
  10. - update content in file /opt/gitlab/sv/postgres-exporter/run from none to b40d34
  11. --- /opt/gitlab/sv/postgres-exporter/run 2019-12-19 13:16:15.959435227 +0000
  12. +++ /opt/gitlab/sv/postgres-exporter/.chef-run20191219-25-1lmm8q4 2019-12-19 13:16:15.959435227 +0000
  13. @@ -1 +1,7 @@
  14. +#!/bin/sh
  15. +exec 2>&1
  16. +
  17. +umask 077
  18. +exec chpst -e /opt/gitlab/etc/postgres-exporter/env -P -U gitlab-psql:git -u gitlab-psql:git /opt/gitlab/embedded/bin/postgres_exporter --web.listen-address=localhost:9187 --extend.query-path=/var/opt/gitlab/postgres-exporter/queries.yaml
  19. +
  20. - change mode from '' to '0755'
  21. - change owner from '' to 'root'
  22. - change group from '' to 'root'
  23. * directory[/opt/gitlab/sv/postgres-exporter/log] action create
  24. - create new directory /opt/gitlab/sv/postgres-exporter/log
  25. - change mode from '' to '0755'
  26. - change owner from '' to 'root'
  27. - change group from '' to 'root'
  28. * directory[/opt/gitlab/sv/postgres-exporter/log/main] action create
  29. - create new directory /opt/gitlab/sv/postgres-exporter/log/main
  30. - change mode from '' to '0755'
  31. - change owner from '' to 'root'
  32. - change group from '' to 'root'
  33. * template[/opt/gitlab/sv/postgres-exporter/log/run] action create
  34. - create new file /opt/gitlab/sv/postgres-exporter/log/run
  35. - update content in file /opt/gitlab/sv/postgres-exporter/log/run from none to b971c9
  36. --- /opt/gitlab/sv/postgres-exporter/log/run 2019-12-19 13:16:15.971435350 +0000
  37. +++ /opt/gitlab/sv/postgres-exporter/log/.chef-run20191219-25-1qwyar6 2019-12-19 13:16:15.971435350 +0000
  38. @@ -1 +1,3 @@
  39. +#!/bin/sh
  40. +exec svlogd -tt /var/log/gitlab/postgres-exporter
  41. - change mode from '' to '0755'
  42. - change owner from '' to 'root'
  43. - change group from '' to 'root'
  44. * template[/var/log/gitlab/postgres-exporter/config] action create (up to date)
  45. * ruby_block[verify_chown_persisted_on_postgres-exporter] action nothing (skipped due to action :nothing)
  46. * directory[/opt/gitlab/sv/postgres-exporter/env] action create
  47. - create new directory /opt/gitlab/sv/postgres-exporter/env
  48. - change mode from '' to '0755'
  49. - change owner from '' to 'root'
  50. - change group from '' to 'root'
  51. * ruby_block[Delete unmanaged env files for postgres-exporter service] action run (skipped due to only_if)
  52. * template[/opt/gitlab/sv/postgres-exporter/check] action create (skipped due to only_if)
  53. * template[/opt/gitlab/sv/postgres-exporter/finish] action create (skipped due to only_if)
  54. * directory[/opt/gitlab/sv/postgres-exporter/control] action create
  55. - create new directory /opt/gitlab/sv/postgres-exporter/control
  56. - change mode from '' to '0755'
  57. - change owner from '' to 'root'
  58. - change group from '' to 'root'
  59. * link[/opt/gitlab/init/postgres-exporter] action create
  60. - create symlink at /opt/gitlab/init/postgres-exporter to /opt/gitlab/embedded/bin/sv
  61. * file[/opt/gitlab/sv/postgres-exporter/down] action delete (up to date)
  62. * ruby_block[restart_service] action run (skipped due to only_if)
  63. * ruby_block[restart_log_service] action create
  64. * ruby_block[restart_service] action nothing (skipped due to action :nothing)
  65. * ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
  66. * ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
  67. * directory[/opt/gitlab/sv/postgres-exporter] action create (up to date)
  68. * template[/opt/gitlab/sv/postgres-exporter/run] action create (up to date)
  69. * directory[/opt/gitlab/sv/postgres-exporter/log] action create (up to date)
  70. * directory[/opt/gitlab/sv/postgres-exporter/log/main] action create (up to date)
  71. * template[/opt/gitlab/sv/postgres-exporter/log/run] action create (up to date)
  72. * template[/var/log/gitlab/postgres-exporter/config] action create (up to date)
  73. * ruby_block[verify_chown_persisted_on_postgres-exporter] action nothing (skipped due to action :nothing)
  74. * directory[/opt/gitlab/sv/postgres-exporter/env] action create (up to date)
  75. * ruby_block[Delete unmanaged env files for postgres-exporter service] action run (skipped due to only_if)
  76. * template[/opt/gitlab/sv/postgres-exporter/check] action create (skipped due to only_if)
  77. * template[/opt/gitlab/sv/postgres-exporter/finish] action create (skipped due to only_if)
  78. * directory[/opt/gitlab/sv/postgres-exporter/control] action create (up to date)
  79. * link[/opt/gitlab/init/postgres-exporter] action create (up to date)
  80. * file[/opt/gitlab/sv/postgres-exporter/down] action delete (up to date)
  81. * directory[/opt/gitlab/service] action create (up to date)
  82. * link[/opt/gitlab/service/postgres-exporter] action create
  83. - create symlink at /opt/gitlab/service/postgres-exporter to /opt/gitlab/sv/postgres-exporter
  84. * ruby_block[wait for postgres-exporter service socket] action run
  85. - execute the ruby block wait for postgres-exporter service socket
  86. - execute the ruby block restart_log_service
  87. * directory[/opt/gitlab/service] action create (up to date)
  88. * link[/opt/gitlab/service/postgres-exporter] action create (up to date)
  89. * ruby_block[wait for postgres-exporter service socket] action run (skipped due to not_if)
  90.  
  91. * template[/var/opt/gitlab/postgres-exporter/queries.yaml] action create (up to date)
  92. * consul_service[postgres-exporter] action delete
  93. * file[/var/opt/gitlab/consul/config.d/postgres-exporter-service.json] action delete (up to date)
  94. (up to date)
  95. Recipe: monitoring::grafana
  96. * directory[/var/log/gitlab/grafana] action create (up to date)
  97. * directory[/var/opt/gitlab/grafana] action create (up to date)
  98. * directory[/var/opt/gitlab/grafana/provisioning] action create (up to date)
  99. * directory[/var/opt/gitlab/grafana/provisioning/dashboards] action create (up to date)
  100. * directory[/var/opt/gitlab/grafana/provisioning/datasources] action create (up to date)
  101. * directory[/var/opt/gitlab/grafana/provisioning/notifiers] action create (up to date)
  102. * link[/var/opt/gitlab/grafana/conf] action create (up to date)
  103. * link[/var/opt/gitlab/grafana/public] action create (up to date)
  104. * directory[/opt/gitlab/etc/grafana/env] action create
  105. - create new directory /opt/gitlab/etc/grafana/env
  106. - change mode from '' to '0700'
  107. - change owner from '' to 'gitlab-prometheus'
  108. * ruby_block[populate Grafana configuration options] action run
  109. - execute the ruby block populate Grafana configuration options
  110. * env_dir[/opt/gitlab/etc/grafana/env] action create
  111. * directory[/opt/gitlab/etc/grafana/env] action create (up to date)
  112. * file[/opt/gitlab/etc/grafana/env/SSL_CERT_DIR] action create
  113. - create new file /opt/gitlab/etc/grafana/env/SSL_CERT_DIR
  114. - update content in file /opt/gitlab/etc/grafana/env/SSL_CERT_DIR from none to 4f45cf
  115. --- /opt/gitlab/etc/grafana/env/SSL_CERT_DIR 2019-12-19 13:16:19.355469984 +0000
  116. +++ /opt/gitlab/etc/grafana/env/.chef-SSL_CERT_DIR20191219-25-4dg1ip 2019-12-19 13:16:19.355469984 +0000
  117. @@ -1 +1,2 @@
  118. +/opt/gitlab/embedded/ssl/certs/
  119.  
  120. * template[/var/opt/gitlab/grafana/grafana.ini] action create (up to date)
  121. * file[/var/opt/gitlab/grafana/provisioning/dashboards/gitlab_dashboards.yml] action create (up to date)
  122. * file[/var/opt/gitlab/grafana/provisioning/datasources/gitlab_datasources.yml] action create (up to date)
  123. Recipe: <Dynamically Defined Resource>
  124. * service[grafana] action nothing (skipped due to action :nothing)
  125. Recipe: monitoring::grafana
  126. * runit_service[grafana] action enable
  127. * ruby_block[restart_service] action nothing (skipped due to action :nothing)
  128. * ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
  129. * ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
  130. * directory[/opt/gitlab/sv/grafana] action create
  131. - create new directory /opt/gitlab/sv/grafana
  132. - change mode from '' to '0755'
  133. - change owner from '' to 'root'
  134. - change group from '' to 'root'
  135. * template[/opt/gitlab/sv/grafana/run] action create
  136. - create new file /opt/gitlab/sv/grafana/run
  137. - update content in file /opt/gitlab/sv/grafana/run from none to b54d7b
  138. --- /opt/gitlab/sv/grafana/run 2019-12-19 13:16:19.383470271 +0000
  139. +++ /opt/gitlab/sv/grafana/.chef-run20191219-25-jyu2ux 2019-12-19 13:16:19.383470271 +0000
  140. @@ -1 +1,12 @@
  141. +#!/bin/sh
  142. +exec 2>&1
  143. +
  144. +
  145. +cd '/var/opt/gitlab/grafana'
  146. +
  147. +umask 077
  148. +exec chpst -P -e /opt/gitlab/etc/grafana/env \
  149. + -U gitlab-prometheus:gitlab-prometheus \
  150. + -u gitlab-prometheus:gitlab-prometheus \
  151. + /opt/gitlab/embedded/bin/grafana-server -config '/var/opt/gitlab/grafana/grafana.ini'
  152. - change mode from '' to '0755'
  153. - change owner from '' to 'root'
  154. - change group from '' to 'root'
  155. * directory[/opt/gitlab/sv/grafana/log] action create
  156. - create new directory /opt/gitlab/sv/grafana/log
  157. - change mode from '' to '0755'
  158. - change owner from '' to 'root'
  159. - change group from '' to 'root'
  160. * directory[/opt/gitlab/sv/grafana/log/main] action create
  161. - create new directory /opt/gitlab/sv/grafana/log/main
  162. - change mode from '' to '0755'
  163. - change owner from '' to 'root'
  164. - change group from '' to 'root'
  165. * template[/opt/gitlab/sv/grafana/log/run] action create
  166. - create new file /opt/gitlab/sv/grafana/log/run
  167. - update content in file /opt/gitlab/sv/grafana/log/run from none to 49180c
  168. --- /opt/gitlab/sv/grafana/log/run 2019-12-19 13:16:19.395470393 +0000
  169. +++ /opt/gitlab/sv/grafana/log/.chef-run20191219-25-1lgg54o 2019-12-19 13:16:19.395470393 +0000
  170. @@ -1 +1,3 @@
  171. +#!/bin/sh
  172. +exec svlogd -tt /var/log/gitlab/grafana
  173. - change mode from '' to '0755'
  174. - change owner from '' to 'root'
  175. - change group from '' to 'root'
  176. * template[/var/log/gitlab/grafana/config] action create (up to date)
  177. * ruby_block[verify_chown_persisted_on_grafana] action nothing (skipped due to action :nothing)
  178. * directory[/opt/gitlab/sv/grafana/env] action create
  179. - create new directory /opt/gitlab/sv/grafana/env
  180. - change mode from '' to '0755'
  181. - change owner from '' to 'root'
  182. - change group from '' to 'root'
  183. * ruby_block[Delete unmanaged env files for grafana service] action run (skipped due to only_if)
  184. * template[/opt/gitlab/sv/grafana/check] action create (skipped due to only_if)
  185. * template[/opt/gitlab/sv/grafana/finish] action create (skipped due to only_if)
  186. * directory[/opt/gitlab/sv/grafana/control] action create
  187. - create new directory /opt/gitlab/sv/grafana/control
  188. - change mode from '' to '0755'
  189. - change owner from '' to 'root'
  190. - change group from '' to 'root'
  191. * link[/opt/gitlab/init/grafana] action create
  192. - create symlink at /opt/gitlab/init/grafana to /opt/gitlab/embedded/bin/sv
  193. * file[/opt/gitlab/sv/grafana/down] action delete (up to date)
  194. * ruby_block[restart_service] action run (skipped due to only_if)
  195. * ruby_block[restart_log_service] action create
  196. * ruby_block[restart_service] action nothing (skipped due to action :nothing)
  197. * ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
  198. * ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
  199. * directory[/opt/gitlab/sv/grafana] action create (up to date)
  200. * template[/opt/gitlab/sv/grafana/run] action create (up to date)
  201. * directory[/opt/gitlab/sv/grafana/log] action create (up to date)
  202. * directory[/opt/gitlab/sv/grafana/log/main] action create (up to date)
  203. * template[/opt/gitlab/sv/grafana/log/run] action create (up to date)
  204. * template[/var/log/gitlab/grafana/config] action create (up to date)
  205. * ruby_block[verify_chown_persisted_on_grafana] action nothing (skipped due to action :nothing)
  206. * directory[/opt/gitlab/sv/grafana/env] action create (up to date)
  207. * ruby_block[Delete unmanaged env files for grafana service] action run (skipped due to only_if)
  208. * template[/opt/gitlab/sv/grafana/check] action create (skipped due to only_if)
  209. * template[/opt/gitlab/sv/grafana/finish] action create (skipped due to only_if)
  210. * directory[/opt/gitlab/sv/grafana/control] action create (up to date)
  211. * link[/opt/gitlab/init/grafana] action create (up to date)
  212. * file[/opt/gitlab/sv/grafana/down] action delete (up to date)
  213. * directory[/opt/gitlab/service] action create (up to date)
  214. * link[/opt/gitlab/service/grafana] action create
  215. - create symlink at /opt/gitlab/service/grafana to /opt/gitlab/sv/grafana
  216. * ruby_block[wait for grafana service socket] action run
  217. - execute the ruby block wait for grafana service socket
  218. - execute the ruby block restart_log_service
  219. * directory[/opt/gitlab/service] action create (up to date)
  220. * link[/opt/gitlab/service/grafana] action create (up to date)
  221. * ruby_block[wait for grafana service socket] action run (skipped due to not_if)
  222.  
  223. Recipe: gitlab::gitlab-rails
  224. * execute[clear the gitlab-rails cache] action run
  225. - execute /opt/gitlab/bin/gitlab-rake cache:clear
  226. Recipe: <Dynamically Defined Resource>
  227. * service[gitaly] action restart
  228. - restart service service[gitaly]
  229. * service[gitlab-workhorse] action restart
  230. - restart service service[gitlab-workhorse]
  231. * service[redis-exporter] action restart
  232. - restart service service[redis-exporter]
  233. * service[prometheus] action restart
  234. - restart service service[prometheus]
  235. * service[alertmanager] action restart
  236. - restart service service[alertmanager]
  237. * service[postgres-exporter] action restart
  238. - restart service service[postgres-exporter]
  239. * service[grafana] action restart
  240. - restart service service[grafana]
  241.  
  242. Running handlers:
  243. Running handlers complete
  244. Chef Client finished, 284/993 resources updated in 01 minutes 13 seconds
  245. gitlab Reconfigured!
  246. Checking for an omnibus managed postgresql: OK
  247. Checking for a newer version of PostgreSQL to install
  248. Upgrading PostgreSQL to 10.9
  249. Checking if we already upgraded: OK
  250. The latest version 10.9 is already running, nothing to do
  251.  
  252. Grafana was reset during a previous upgrade. Skipping.
  253. ==> /var/log/gitlab/gitlab-workhorse/current <==
  254. {"level":"info","msg":"Starting","time":"2019-12-19T13:16:03Z","version":"gitlab-workhorse v8.14.0-20191118.175033"}
  255. {"level":"info","msg":"keywatcher: starting process loop","time":"2019-12-19T13:16:03Z"}
  256. {"address":"/var/opt/gitlab/redis/redis.socket","level":"info","msg":"redis: dialing","network":"unix","time":"2019-12-19T13:16:03Z"}
  257. {"correlation_id":"PuFwbneOZ91","error":"badgateway: failed after 0s: dial unix /var/opt/gitlab/gitlab-rails/sockets/gitlab.socket: connect: connection refused","level":"error","method":"GET","msg":"error","time":"2019-12-19T13:16:24Z","uri":"/help"}
  258. {"correlation_id":"PuFwbneOZ91","duration_ms":1,"host":"localhost:8929","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","status":502,"system":"http","time":"2019-12-19T13:16:24Z","uri":"/help","user_agent":"curl/7.59.0-DEV","written_bytes":2940}
  259. {"level":"info","msg":"Starting","time":"2019-12-19T13:16:41Z","version":"gitlab-workhorse v8.14.0-20191118.175033"}
  260. {"level":"info","msg":"keywatcher: starting process loop","time":"2019-12-19T13:16:41Z"}
  261. {"address":"/var/opt/gitlab/redis/redis.socket","level":"info","msg":"redis: dialing","network":"unix","time":"2019-12-19T13:16:41Z"}
  262.  
  263. ==> /var/log/gitlab/gitlab-rails/sidekiq.log <==
  264. {"severity":"INFO","time":"2019-12-19T13:16:16.990Z","message":"Cron Jobs - add job with name: stuck_merge_jobs_worker"}
  265. {"severity":"INFO","time":"2019-12-19T13:16:16.992Z","message":"Cron Jobs - add job with name: issue_due_scheduler_worker"}
  266. {"severity":"INFO","time":"2019-12-19T13:16:16.993Z","message":"Cron Jobs - add job with name: prune_web_hook_logs_worker"}
  267. {"severity":"INFO","time":"2019-12-19T13:16:16.994Z","message":"Cron Jobs - add job with name: namespaces_prune_aggregation_schedules_worker"}
  268. {"severity":"INFO","time":"2019-12-19T13:16:30.581Z","message":"Running in ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]"}
  269. {"severity":"INFO","time":"2019-12-19T13:16:30.581Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}
  270. {"severity":"INFO","time":"2019-12-19T13:16:30.581Z","message":"Upgrade to Sidekiq Pro for more features and support: http://sidekiq.org"}
  271. {"severity":"WARN","time":"2019-12-19T13:16:30.581Z","message":"Sidekiq 6.0 will require Redis 4.0+, you are using Redis v3.2.12"}
  272. {"severity":"INFO","time":"2019-12-19T13:16:30.779Z","message":"Starting processing, hit Ctrl-C to stop"}
  273. {"severity":"INFO","time":"2019-12-19T13:16:30.786Z","message":"Listening on queues [\"admin_emails\", \"authorized_projects\", \"auto_devops\", \"auto_devops:auto_devops_disable\", \"auto_merge\", \"auto_merge:auto_merge_process\", \"background_migration\", \"chaos\", \"chaos:chaos_cpu_spin\", \"chaos:chaos_db_spin\", \"chaos:chaos_kill\", \"chaos:chaos_leak_mem\", \"chaos:chaos_sleep\", \"chat_notification\", \"container_repository\", \"container_repository:cleanup_container_repository\", \"container_repository:delete_container_repository\", \"create_evidence\", \"create_github_webhook\", \"create_gpg_signature\", \"create_note_diff_file\", \"cronjob\", \"cronjob:admin_email\", \"cronjob:ci_archive_traces_cron\", \"cronjob:expire_build_artifacts\", \"cronjob:gitlab_usage_ping\", \"cronjob:import_export_project_cleanup\", \"cronjob:issue_due_scheduler\", \"cronjob:namespaces_prune_aggregation_schedules\", \"cronjob:pages_domain_removal_cron\", \"cronjob:pages_domain_ssl_renewal_cron\", \"cronjob:pages_domain_verification_cron\", \"cronjob:pipeline_schedule\", \"cronjob:prune_old_events\", \"cronjob:prune_web_hook_logs\", \"cronjob:remove_expired_group_links\", \"cronjob:remove_expired_members\", \"cronjob:remove_unreferenced_lfs_objects\", \"cronjob:repository_archive_cache\", \"cronjob:repository_check_dispatch\", \"cronjob:requests_profiles\", \"cronjob:schedule_migrate_external_diffs\", \"cronjob:stuck_ci_jobs\", \"cronjob:stuck_import_jobs\", \"cronjob:stuck_merge_jobs\", \"cronjob:trending_projects\", \"default\", \"delete_diff_files\", \"delete_merged_branches\", \"delete_stored_files\", \"delete_user\", \"deployment\", \"deployment:deployments_finished\", \"deployment:deployments_success\", \"design_management_new_version\", \"detect_repository_languages\", \"elastic_batch_project_indexer\", \"elastic_commit_indexer\", \"elastic_full_index\", \"elastic_indexer\", \"elastic_namespace_indexer\", \"email_receiver\", \"emails_on_push\", \"expire_build_instance_artifacts\", \"export_csv\", \"gcp_cluster\", \"gcp_cluster:cluster_configure\", \"gcp_cluster:cluster_install_app\", \"gcp_cluster:cluster_patch_app\", \"gcp_cluster:cluster_project_configure\", \"gcp_cluster:cluster_provision\", \"gcp_cluster:cluster_upgrade_app\", \"gcp_cluster:cluster_wait_for_app_installation\", \"gcp_cluster:cluster_wait_for_ingress_ip_address\", \"gcp_cluster:clusters_applications_uninstall\", \"gcp_cluster:clusters_applications_wait_for_uninstall_app\", \"gcp_cluster:wait_for_cluster_creation\", \"geo\", \"git_garbage_collect\", \"github_import_advance_stage\", \"github_importer\", \"github_importer:github_import_import_diff_note\", \"github_importer:github_import_import_issue\", \"github_importer:github_import_import_lfs_object\", \"github_importer:github_import_import_note\", \"github_importer:github_import_import_pull_request\", \"github_importer:github_import_refresh_import_jid\", \"github_importer:github_import_stage_finish_import\", \"github_importer:github_import_stage_import_base_data\", \"github_importer:github_import_stage_import_issues_and_diff_notes\", \"github_importer:github_import_stage_import_lfs_objects\", \"github_importer:github_import_stage_import_notes\", \"github_importer:github_import_stage_import_pull_requests\", \"github_importer:github_import_stage_import_repository\", \"gitlab_shell\", \"group_destroy\", \"hashed_storage\", \"hashed_storage:hashed_storage_migrator\", \"hashed_storage:hashed_storage_project_migrate\", \"hashed_storage:hashed_storage_project_rollback\", \"hashed_storage:hashed_storage_rollbacker\", \"import_issues_csv\", \"incident_management\", \"invalid_gpg_signature_update\", \"irker\", \"jira_connect\", \"ldap_group_sync\", \"mail_scheduler\", \"mail_scheduler:mail_scheduler_issue_due\", \"mail_scheduler:mail_scheduler_notification_service\", \"mailers\", \"merge\", \"migrate_external_diffs\", \"namespaceless_project_destroy\", \"new_epic\", \"new_issue\", \"new_merge_request\", \"new_note\", \"notifications\", \"notifications:new_release\", \"object_pool\", \"object_pool:object_pool_create\", \"object_pool:object_pool_destroy\", \"object_pool:object_pool_join\", \"object_pool:object_pool_schedule_join\", \"object_storage\", \"object_storage:object_storage_background_move\", \"object_storage:object_storage_migrate_uploads\", \"object_storage_upload\", \"pages\", \"pages_domain_ssl_renewal\", \"pages_domain_verification\", \"phabricator_import_import_tasks\", \"pipeline_background\", \"pipeline_background:archive_trace\", \"pipeline_background:ci_build_trace_chunk_flush\", \"pipeline_cache\", \"pipeline_cache:expire_job_cache\", \"pipeline_cache:expire_pipeline_cache\", \"pipeline_creation\", \"pipeline_creation:create_pipeline\", \"pipeline_creation:run_pipeline_schedule\", \"pipeline_default\", \"pipeline_default:build_coverage\", \"pipeline_default:build_trace_sections\", \"pipeline_default:pipeline_metrics\", \"pipeline_default:pipeline_notification\", \"pipeline_hooks\", \"pipeline_hooks:build_hooks\", \"pipeline_hooks:pipeline_hooks\", \"pipeline_processing\", \"pipeline_processing:build_finished\", \"pipeline_processing:build_queue\", \"pipeline_processing:build_success\", \"pipeline_processing:ci_build_prepare\", \"pipeline_processing:ci_build_schedule\", \"pipeline_processing:pipeline_process\", \"pipeline_processing:pipeline_success\", \"pipeline_processing:pipeline_update\", \"pipeline_processing:stage_update\", \"pipeline_processing:update_head_pipeline_for_merge_request\", \"plugin\", \"post_receive\", \"process_commit\", \"project_cache\", \"project_daily_statistics\", \"project_destroy\", \"project_export\", \"project_import_schedule\", \"project_migrate_hashed_storage\", \"project_rollback_hashed_storage\", \"project_service\", \"project_update_repository_storage\", \"propagate_service_template\", \"reactive_caching\", \"rebase\", \"refresh_license_compliance_checks\", \"remote_mirror_notification\", \"repository_check\", \"repository_check:repository_check_batch\", \"repository_check:repository_check_clear\", \"repository_check:repository_check_single_repository\", \"repository_cleanup\", \"repository_fork\", \"repository_import\", \"repository_push_audit_event\", \"repository_remove_remote\", \"repository_update_mirror\", \"repository_update_remote_mirror\", \"system_hook_push\", \"todos_destroyer\", \"todos_destroyer:todos_destroyer_confidential_issue\", \"todos_destroyer:todos_destroyer_entity_leave\", \"todos_destroyer:todos_destroyer_group_private\", \"todos_destroyer:todos_destroyer_private_features\", \"todos_destroyer:todos_destroyer_project_private\", \"update_external_pull_requests\", \"update_merge_requests\", \"update_namespace_statistics\", \"update_namespace_statistics:namespaces_root_statistics\", \"update_namespace_statistics:namespaces_schedule_aggregation\", \"update_project_statistics\", \"upload_checksum\", \"web_hook\"]"}
  274.  
  275. ==> /var/log/gitlab/gitlab-rails/production.log <==
  276. Filter chain halted as :redirect_unlogged_user rendered or redirected
  277. Completed 302 Found in 46ms (ActiveRecord: 14.2ms | Elasticsearch: 0.0ms)
  278. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:16:35 +0000
  279. Processing by MetricsController#index as HTML
  280. Completed 200 OK in 34ms (Views: 21.2ms | ActiveRecord: 7.6ms | Elasticsearch: 0.0ms)
  281. Creating scope :of_projects. Overwriting existing method MergeRequest.of_projects.
  282. Creating scope :join_project. Overwriting existing method MergeRequest.join_project.
  283. Creating scope :references_project. Overwriting existing method MergeRequest.references_project.
  284. Creating scope :system. Overwriting existing method Note.system.
  285. Creating scope :without_statuses. Overwriting existing method CommitStatus.without_statuses.
  286.  
  287. ==> /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2019-12-10-11-15-39.log <==
  288.  
  289. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  290. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":32.17,"view":0.38,"db":0.0,"time":"2019-12-19T13:13:21.695Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"87fa355f-1db4-4583-9746-d9b7ca3ccc53","cpu_s":0.04131165100000089}
  291. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":20.48,"view":0.34,"db":0.0,"time":"2019-12-19T13:13:36.699Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"0424c558-3cea-48a1-a5c8-dff75c03170a","cpu_s":0.041431185000000426}
  292. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":24.0,"view":0.39,"db":0.0,"time":"2019-12-19T13:13:51.750Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"b45b424f-3198-4e58-8206-ed1d5e7289f8","cpu_s":0.061558795999999916}
  293. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":23.22,"view":0.35,"db":0.0,"time":"2019-12-19T13:14:06.715Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"d676fe17-500c-44a2-ac93-dcdc60f8fd13","cpu_s":0.05692712599999972}
  294. {"method":"GET","path":"/help","format":"*/*","controller":"HelpController","action":"index","status":200,"duration":345.03,"view":337.68,"db":0.75,"time":"2019-12-19T13:14:10.323Z","params":[],"remote_ip":"127.0.0.1","user_id":null,"username":null,"ua":"curl/7.59.0-DEV","queue_duration":11.04,"correlation_id":"glzJlgzbLC2","cpu_s":0.35617558800000104}
  295. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":23.03,"view":0.35,"db":0.0,"time":"2019-12-19T13:14:21.710Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"03670bcb-2543-4517-acd8-343c492e31a1","cpu_s":0.052628594999999834}
  296. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":25.99,"view":0.34,"db":0.0,"time":"2019-12-19T13:14:36.721Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"88b8d0b5-750e-482f-a46e-05f925c91cc2","cpu_s":0.06255033799999943}
  297. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":28.06,"view":0.35,"db":0.0,"time":"2019-12-19T13:14:51.721Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"ef4e7d85-0f33-4d98-b496-a5a9935ebe2f","cpu_s":0.06137530700000049}
  298. {"method":"GET","path":"/","format":"html","controller":"RootController","action":"index","status":302,"duration":46.56,"view":0.0,"db":14.19,"location":"http://example.org/users/sign_in","time":"2019-12-19T13:16:32.420Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"acb2737f-9978-4102-96fd-975baf8baa22","cpu_s":2.198681292}
  299. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":34.26,"view":21.24,"db":7.64,"time":"2019-12-19T13:16:35.104Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"cd97a2d6-5505-4421-9759-ff241f544927","cpu_s":0.030166639999999995}
  300.  
  301. ==> /var/log/gitlab/gitlab-rails/grpc.log <==
  302.  
  303. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  304. [2019-12-19T13:13:48.024+0000] 127.0.0.1 - - [19/Dec/2019:13:13:47 UTC] "GET /metrics HTTP/1.1" 200 10935 "-" "Prometheus/2.12.0"
  305. [2019-12-19T13:14:03.024+0000] 127.0.0.1 - - [19/Dec/2019:13:14:02 UTC] "GET /metrics HTTP/1.1" 200 10935 "-" "Prometheus/2.12.0"
  306. [2019-12-19T13:14:18.006+0000] 127.0.0.1 - - [19/Dec/2019:13:14:17 UTC] "GET /metrics HTTP/1.1" 200 10933 "-" "Prometheus/2.12.0"
  307. [2019-12-19T13:14:33.023+0000] 127.0.0.1 - - [19/Dec/2019:13:14:32 UTC] "GET /metrics HTTP/1.1" 200 10933 "-" "Prometheus/2.12.0"
  308. [2019-12-19T13:14:48.020+0000] 127.0.0.1 - - [19/Dec/2019:13:14:47 UTC] "GET /metrics HTTP/1.1" 200 10933 "-" "Prometheus/2.12.0"
  309. [2019-12-19T13:15:03.025+0000] 127.0.0.1 - - [19/Dec/2019:13:15:02 UTC] "GET /metrics HTTP/1.1" 200 10933 "-" "Prometheus/2.12.0"
  310. [2019-12-19T13:16:30.752+0000] INFO WEBrick 1.4.2
  311. [2019-12-19T13:16:30.752+0000] INFO ruby 2.6.3 (2019-04-16) [x86_64-linux]
  312. [2019-12-19T13:16:30.780+0000] INFO WEBrick::HTTPServer#start: pid=494 port=8082
  313. [2019-12-19T13:16:37.084+0000] 127.0.0.1 - - [19/Dec/2019:13:16:37 UTC] "GET /metrics HTTP/1.1" 200 4145 "-" "Prometheus/2.12.0"
  314.  
  315. ==> /var/log/gitlab/gitlab-rails/api_json.log <==
  316.  
  317. ==> /var/log/gitlab/gitlab-rails/application.log <==
  318.  
  319. ==> /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2019-10-21-12-40-00.log <==
  320.  
  321. ==> /var/log/gitlab/gitlab-rails/audit_json.log <==
  322.  
  323. ==> /var/log/gitlab/gitlab-exporter/current <==
  324. 2019-12-19_13:16:13.49477 == Sinatra (v2.0.7) has taken the stage on 9168 for development with backup from Puma
  325. 2019-12-19_13:16:13.49480 Puma starting in single mode...
  326. 2019-12-19_13:16:13.49481 * Version 3.12.0 (ruby 2.6.3-p62), codename: Llamas in Pajamas
  327. 2019-12-19_13:16:13.49481 * Min threads: 0, max threads: 16
  328. 2019-12-19_13:16:13.49481 * Environment: development
  329. 2019-12-19_13:16:13.49481 * Listening on tcp://localhost:9168
  330. 2019-12-19_13:16:13.49481 Use Ctrl-C to stop
  331.  
  332. ==> /var/log/gitlab/gitaly/current <==
  333. time="2019-12-19T13:16:41Z" level=info msg="Starting file walker for /var/opt/gitlab/git-data/repositories/+gitaly/state" storage=default
  334. {"address":"/var/opt/gitlab/gitaly/gitaly.socket","level":"info","msg":"listening at unix address","time":"2019-12-19T13:16:41Z"}
  335. {"level":"info","msg":"finished tempdir cleaner walk","storage":"default","time":"2019-12-19T13:16:41Z","time_ms":0}
  336. {"address":"localhost:9236","level":"info","msg":"starting prometheus listener","time":"2019-12-19T13:16:41Z"}
  337. {"level":"info","msg":"starting RSS monitor","supervisor.name":"gitaly-ruby.1","supervisor.rss_threshold":209715200,"time":"2019-12-19T13:16:41Z"}
  338. {"level":"info","msg":"starting RSS monitor","supervisor.name":"gitaly-ruby.0","supervisor.rss_threshold":209715200,"time":"2019-12-19T13:16:41Z"}
  339. {"level":"warning","msg":"spawned","supervisor.args":["bundle","exec","bin/ruby-cd","/var/opt/gitlab/gitaly","/opt/gitlab/embedded/service/gitaly-ruby/bin/gitaly-ruby","817","/tmp/gitaly-ruby388487961/socket.1"],"supervisor.name":"gitaly-ruby.1","supervisor.pid":852,"time":"2019-12-19T13:16:41Z"}
  340. {"level":"warning","msg":"spawned","supervisor.args":["bundle","exec","bin/ruby-cd","/var/opt/gitlab/gitaly","/opt/gitlab/embedded/service/gitaly-ruby/bin/gitaly-ruby","817","/tmp/gitaly-ruby388487961/socket.0"],"supervisor.name":"gitaly-ruby.0","supervisor.pid":853,"time":"2019-12-19T13:16:41Z"}
  341. {"level":"info","msg":"PID 853 BUNDLE_GEMFILE=/opt/gitlab/embedded/service/gitaly-ruby/Gemfile","supervisor.args":["bundle","exec","bin/ruby-cd","/var/opt/gitlab/gitaly","/opt/gitlab/embedded/service/gitaly-ruby/bin/gitaly-ruby","817","/tmp/gitaly-ruby388487961/socket.0"],"supervisor.name":"gitaly-ruby.0","time":"2019-12-19T13:16:41Z"}
  342. {"level":"info","msg":"PID 852 BUNDLE_GEMFILE=/opt/gitlab/embedded/service/gitaly-ruby/Gemfile","supervisor.args":["bundle","exec","bin/ruby-cd","/var/opt/gitlab/gitaly","/opt/gitlab/embedded/service/gitaly-ruby/bin/gitaly-ruby","817","/tmp/gitaly-ruby388487961/socket.1"],"supervisor.name":"gitaly-ruby.1","time":"2019-12-19T13:16:41Z"}
  343.  
  344. ==> /var/log/gitlab/gitlab-shell/gitlab-shell.log <==
  345.  
  346. ==> /var/log/gitlab/unicorn/unicorn_stdout.log <==
  347.  
  348. ==> /var/log/gitlab/unicorn/current <==
  349. 2019-12-19_13:16:01.07753 starting new unicorn master
  350. 2019-12-19_13:16:33.76269 adopted new unicorn master 491
  351.  
  352. ==> /var/log/gitlab/unicorn/unicorn_stderr.log <==
  353. I, [2019-12-19T13:16:33.023365 #736] INFO -- : worker=9 ready
  354. I, [2019-12-19T13:16:33.032191 #729] INFO -- : worker=2 ready
  355. I, [2019-12-19T13:16:33.067786 #732] INFO -- : worker=5 ready
  356. I, [2019-12-19T13:16:33.071158 #728] INFO -- : worker=1 ready
  357. I, [2019-12-19T13:16:33.077421 #735] INFO -- : worker=8 ready
  358. I, [2019-12-19T13:16:33.080329 #733] INFO -- : worker=6 ready
  359. I, [2019-12-19T13:16:33.107293 #737] INFO -- : worker=10 ready
  360. I, [2019-12-19T13:16:33.110220 #734] INFO -- : worker=7 ready
  361. I, [2019-12-19T13:16:33.134138 #738] INFO -- : worker=11 ready
  362. I, [2019-12-19T13:16:33.149413 #739] INFO -- : worker=12 ready
  363.  
  364. ==> /var/log/gitlab/redis/current <==
  365. 2019-12-19_13:15:42.93566 `-._ `-.__.-' _.-'
  366. 2019-12-19_13:15:42.93566 `-._ _.-'
  367. 2019-12-19_13:15:42.93568 `-.__.-'
  368. 2019-12-19_13:15:42.93568
  369. 2019-12-19_13:15:42.93580 243:M 19 Dec 13:15:42.935 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
  370. 2019-12-19_13:15:42.93586 243:M 19 Dec 13:15:42.935 # Server started, Redis version 3.2.12
  371. 2019-12-19_13:15:42.93593 243:M 19 Dec 13:15:42.935 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
  372. 2019-12-19_13:15:42.93601 243:M 19 Dec 13:15:42.935 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
  373. 2019-12-19_13:15:42.93721 243:M 19 Dec 13:15:42.937 * DB loaded from disk: 0.001 seconds
  374. 2019-12-19_13:15:42.93727 243:M 19 Dec 13:15:42.937 * The server is now ready to accept connections at /var/opt/gitlab/redis/redis.socket
  375.  
  376. ==> /var/log/gitlab/logrotate/current <==
  377.  
  378. ==> /var/log/gitlab/postgresql/current <==
  379. 2019-12-19_13:15:55.05322 LOG: listening on Unix socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"
  380. 2019-12-19_13:15:55.07119 LOG: database system was shut down at 2019-12-19 13:15:03 GMT
  381. 2019-12-19_13:15:55.07512 LOG: database system is ready to accept connections
  382.  
  383. ==> /var/log/gitlab/grafana/current <==
  384. 2019-12-19_13:16:43.42039 t=2019-12-19T13:16:43+0000 lvl=info msg="Initializing SearchService" logger=server
  385. 2019-12-19_13:16:43.42039 t=2019-12-19T13:16:43+0000 lvl=info msg="Initializing ServerLockService" logger=server
  386. 2019-12-19_13:16:43.42040 t=2019-12-19T13:16:43+0000 lvl=info msg="Initializing TracingService" logger=server
  387. 2019-12-19_13:16:43.42044 t=2019-12-19T13:16:43+0000 lvl=info msg="Initializing UsageStatsService" logger=server
  388. 2019-12-19_13:16:43.42045 t=2019-12-19T13:16:43+0000 lvl=info msg="Initializing UserAuthTokenService" logger=server
  389. 2019-12-19_13:16:43.42047 t=2019-12-19T13:16:43+0000 lvl=info msg="Initializing CleanUpService" logger=server
  390. 2019-12-19_13:16:43.42048 t=2019-12-19T13:16:43+0000 lvl=info msg="Initializing NotificationService" logger=server
  391. 2019-12-19_13:16:43.42122 t=2019-12-19T13:16:43+0000 lvl=info msg="Initializing provisioningServiceImpl" logger=server
  392. 2019-12-19_13:16:43.44459 t=2019-12-19T13:16:43+0000 lvl=info msg="Initializing Stream Manager"
  393. 2019-12-19_13:16:43.44462 t=2019-12-19T13:16:43+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=localhost:3000 protocol=http subUrl=/-/grafana socket=
  394.  
  395. ==> /var/log/gitlab/postgres-exporter/current <==
  396. 2019-12-19_13:16:17.27105 time="2019-12-19T13:16:17Z" level=info msg="Established new database connection to \"/var/opt/gitlab/postgresql:5432\"." source="postgres_exporter.go:777"
  397. 2019-12-19_13:16:17.27601 time="2019-12-19T13:16:17Z" level=info msg="Semantic Version Changed on \"/var/opt/gitlab/postgresql:5432\": 0.0.0 -> 10.9.0" source="postgres_exporter.go:1229"
  398. 2019-12-19_13:16:17.30851 time="2019-12-19T13:16:17Z" level=info msg="Starting Server: localhost:9187" source="postgres_exporter.go:1437"
  399. 2019-12-19_13:16:42.83713 time="2019-12-19T13:16:42Z" level=info msg="Established new database connection to \"/var/opt/gitlab/postgresql:5432\"." source="postgres_exporter.go:777"
  400. 2019-12-19_13:16:42.84177 time="2019-12-19T13:16:42Z" level=info msg="Semantic Version Changed on \"/var/opt/gitlab/postgresql:5432\": 0.0.0 -> 10.9.0" source="postgres_exporter.go:1229"
  401. 2019-12-19_13:16:42.88894 time="2019-12-19T13:16:42Z" level=info msg="Starting Server: localhost:9187" source="postgres_exporter.go:1437"
  402.  
  403. ==> /var/log/gitlab/sidekiq/current <==
  404. {"severity":"INFO","time":"2019-12-19T13:16:16.990Z","message":"Cron Jobs - add job with name: stuck_merge_jobs_worker"}
  405. {"severity":"INFO","time":"2019-12-19T13:16:16.992Z","message":"Cron Jobs - add job with name: issue_due_scheduler_worker"}
  406. {"severity":"INFO","time":"2019-12-19T13:16:16.993Z","message":"Cron Jobs - add job with name: prune_web_hook_logs_worker"}
  407. {"severity":"INFO","time":"2019-12-19T13:16:16.994Z","message":"Cron Jobs - add job with name: namespaces_prune_aggregation_schedules_worker"}
  408. {"severity":"INFO","time":"2019-12-19T13:16:30.581Z","message":"Running in ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]"}
  409. {"severity":"INFO","time":"2019-12-19T13:16:30.581Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}
  410. {"severity":"INFO","time":"2019-12-19T13:16:30.581Z","message":"Upgrade to Sidekiq Pro for more features and support: http://sidekiq.org"}
  411. {"severity":"WARN","time":"2019-12-19T13:16:30.581Z","message":"Sidekiq 6.0 will require Redis 4.0+, you are using Redis v3.2.12"}
  412. {"severity":"INFO","time":"2019-12-19T13:16:30.779Z","message":"Starting processing, hit Ctrl-C to stop"}
  413. {"severity":"INFO","time":"2019-12-19T13:16:30.786Z","message":"Listening on queues [\"admin_emails\", \"authorized_projects\", \"auto_devops\", \"auto_devops:auto_devops_disable\", \"auto_merge\", \"auto_merge:auto_merge_process\", \"background_migration\", \"chaos\", \"chaos:chaos_cpu_spin\", \"chaos:chaos_db_spin\", \"chaos:chaos_kill\", \"chaos:chaos_leak_mem\", \"chaos:chaos_sleep\", \"chat_notification\", \"container_repository\", \"container_repository:cleanup_container_repository\", \"container_repository:delete_container_repository\", \"create_evidence\", \"create_github_webhook\", \"create_gpg_signature\", \"create_note_diff_file\", \"cronjob\", \"cronjob:admin_email\", \"cronjob:ci_archive_traces_cron\", \"cronjob:expire_build_artifacts\", \"cronjob:gitlab_usage_ping\", \"cronjob:import_export_project_cleanup\", \"cronjob:issue_due_scheduler\", \"cronjob:namespaces_prune_aggregation_schedules\", \"cronjob:pages_domain_removal_cron\", \"cronjob:pages_domain_ssl_renewal_cron\", \"cronjob:pages_domain_verification_cron\", \"cronjob:pipeline_schedule\", \"cronjob:prune_old_events\", \"cronjob:prune_web_hook_logs\", \"cronjob:remove_expired_group_links\", \"cronjob:remove_expired_members\", \"cronjob:remove_unreferenced_lfs_objects\", \"cronjob:repository_archive_cache\", \"cronjob:repository_check_dispatch\", \"cronjob:requests_profiles\", \"cronjob:schedule_migrate_external_diffs\", \"cronjob:stuck_ci_jobs\", \"cronjob:stuck_import_jobs\", \"cronjob:stuck_merge_jobs\", \"cronjob:trending_projects\", \"default\", \"delete_diff_files\", \"delete_merged_branches\", \"delete_stored_files\", \"delete_user\", \"deployment\", \"deployment:deployments_finished\", \"deployment:deployments_success\", \"design_management_new_version\", \"detect_repository_languages\", \"elastic_batch_project_indexer\", \"elastic_commit_indexer\", \"elastic_full_index\", \"elastic_indexer\", \"elastic_namespace_indexer\", \"email_receiver\", \"emails_on_push\", \"expire_build_instance_artifacts\", \"export_csv\", \"gcp_cluster\", \"gcp_cluster:cluster_configure\", \"gcp_cluster:cluster_install_app\", \"gcp_cluster:cluster_patch_app\", \"gcp_cluster:cluster_project_configure\", \"gcp_cluster:cluster_provision\", \"gcp_cluster:cluster_upgrade_app\", \"gcp_cluster:cluster_wait_for_app_installation\", \"gcp_cluster:cluster_wait_for_ingress_ip_address\", \"gcp_cluster:clusters_applications_uninstall\", \"gcp_cluster:clusters_applications_wait_for_uninstall_app\", \"gcp_cluster:wait_for_cluster_creation\", \"geo\", \"git_garbage_collect\", \"github_import_advance_stage\", \"github_importer\", \"github_importer:github_import_import_diff_note\", \"github_importer:github_import_import_issue\", \"github_importer:github_import_import_lfs_object\", \"github_importer:github_import_import_note\", \"github_importer:github_import_import_pull_request\", \"github_importer:github_import_refresh_import_jid\", \"github_importer:github_import_stage_finish_import\", \"github_importer:github_import_stage_import_base_data\", \"github_importer:github_import_stage_import_issues_and_diff_notes\", \"github_importer:github_import_stage_import_lfs_objects\", \"github_importer:github_import_stage_import_notes\", \"github_importer:github_import_stage_import_pull_requests\", \"github_importer:github_import_stage_import_repository\", \"gitlab_shell\", \"group_destroy\", \"hashed_storage\", \"hashed_storage:hashed_storage_migrator\", \"hashed_storage:hashed_storage_project_migrate\", \"hashed_storage:hashed_storage_project_rollback\", \"hashed_storage:hashed_storage_rollbacker\", \"import_issues_csv\", \"incident_management\", \"invalid_gpg_signature_update\", \"irker\", \"jira_connect\", \"ldap_group_sync\", \"mail_scheduler\", \"mail_scheduler:mail_scheduler_issue_due\", \"mail_scheduler:mail_scheduler_notification_service\", \"mailers\", \"merge\", \"migrate_external_diffs\", \"namespaceless_project_destroy\", \"new_epic\", \"new_issue\", \"new_merge_request\", \"new_note\", \"notifications\", \"notifications:new_release\", \"object_pool\", \"object_pool:object_pool_create\", \"object_pool:object_pool_destroy\", \"object_pool:object_pool_join\", \"object_pool:object_pool_schedule_join\", \"object_storage\", \"object_storage:object_storage_background_move\", \"object_storage:object_storage_migrate_uploads\", \"object_storage_upload\", \"pages\", \"pages_domain_ssl_renewal\", \"pages_domain_verification\", \"phabricator_import_import_tasks\", \"pipeline_background\", \"pipeline_background:archive_trace\", \"pipeline_background:ci_build_trace_chunk_flush\", \"pipeline_cache\", \"pipeline_cache:expire_job_cache\", \"pipeline_cache:expire_pipeline_cache\", \"pipeline_creation\", \"pipeline_creation:create_pipeline\", \"pipeline_creation:run_pipeline_schedule\", \"pipeline_default\", \"pipeline_default:build_coverage\", \"pipeline_default:build_trace_sections\", \"pipeline_default:pipeline_metrics\", \"pipeline_default:pipeline_notification\", \"pipeline_hooks\", \"pipeline_hooks:build_hooks\", \"pipeline_hooks:pipeline_hooks\", \"pipeline_processing\", \"pipeline_processing:build_finished\", \"pipeline_processing:build_queue\", \"pipeline_processing:build_success\", \"pipeline_processing:ci_build_prepare\", \"pipeline_processing:ci_build_schedule\", \"pipeline_processing:pipeline_process\", \"pipeline_processing:pipeline_success\", \"pipeline_processing:pipeline_update\", \"pipeline_processing:stage_update\", \"pipeline_processing:update_head_pipeline_for_merge_request\", \"plugin\", \"post_receive\", \"process_commit\", \"project_cache\", \"project_daily_statistics\", \"project_destroy\", \"project_export\", \"project_import_schedule\", \"project_migrate_hashed_storage\", \"project_rollback_hashed_storage\", \"project_service\", \"project_update_repository_storage\", \"propagate_service_template\", \"reactive_caching\", \"rebase\", \"refresh_license_compliance_checks\", \"remote_mirror_notification\", \"repository_check\", \"repository_check:repository_check_batch\", \"repository_check:repository_check_clear\", \"repository_check:repository_check_single_repository\", \"repository_cleanup\", \"repository_fork\", \"repository_import\", \"repository_push_audit_event\", \"repository_remove_remote\", \"repository_update_mirror\", \"repository_update_remote_mirror\", \"system_hook_push\", \"todos_destroyer\", \"todos_destroyer:todos_destroyer_confidential_issue\", \"todos_destroyer:todos_destroyer_entity_leave\", \"todos_destroyer:todos_destroyer_group_private\", \"todos_destroyer:todos_destroyer_private_features\", \"todos_destroyer:todos_destroyer_project_private\", \"update_external_pull_requests\", \"update_merge_requests\", \"update_namespace_statistics\", \"update_namespace_statistics:namespaces_root_statistics\", \"update_namespace_statistics:namespaces_schedule_aggregation\", \"update_project_statistics\", \"upload_checksum\", \"web_hook\"]"}
  414.  
  415. ==> /var/log/gitlab/nginx/current <==
  416.  
  417. ==> /var/log/gitlab/nginx/gitlab_error.log <==
  418.  
  419. ==> /var/log/gitlab/nginx/access.log <==
  420.  
  421. ==> /var/log/gitlab/nginx/error.log <==
  422.  
  423. ==> /var/log/gitlab/nginx/gitlab_access.log <==
  424. 127.0.0.1 - - [19/Dec/2019:13:06:05 +0000] "GET /help HTTP/1.1" 200 83495 "" "curl/7.59.0-DEV"
  425. 127.0.0.1 - - [19/Dec/2019:13:07:05 +0000] "GET /help HTTP/1.1" 200 83495 "" "curl/7.59.0-DEV"
  426. 127.0.0.1 - - [19/Dec/2019:13:08:06 +0000] "GET /help HTTP/1.1" 200 83495 "" "curl/7.59.0-DEV"
  427. 127.0.0.1 - - [19/Dec/2019:13:09:07 +0000] "GET /help HTTP/1.1" 200 83495 "" "curl/7.59.0-DEV"
  428. 127.0.0.1 - - [19/Dec/2019:13:10:07 +0000] "GET /help HTTP/1.1" 200 83502 "" "curl/7.59.0-DEV"
  429. 127.0.0.1 - - [19/Dec/2019:13:11:08 +0000] "GET /help HTTP/1.1" 200 83502 "" "curl/7.59.0-DEV"
  430. 127.0.0.1 - - [19/Dec/2019:13:12:08 +0000] "GET /help HTTP/1.1" 200 83495 "" "curl/7.59.0-DEV"
  431. 127.0.0.1 - - [19/Dec/2019:13:13:09 +0000] "GET /help HTTP/1.1" 200 83495 "" "curl/7.59.0-DEV"
  432. 127.0.0.1 - - [19/Dec/2019:13:14:10 +0000] "GET /help HTTP/1.1" 200 83502 "" "curl/7.59.0-DEV"
  433. 127.0.0.1 - - [19/Dec/2019:13:16:24 +0000] "GET /help HTTP/1.1" 502 2940 "" "curl/7.59.0-DEV"
  434.  
  435. ==> /var/log/gitlab/sshd/current <==
  436. 2019-12-19_13:15:27.85959 Server listening on 0.0.0.0 port 22.
  437. 2019-12-19_13:15:27.86025 Server listening on :: port 22.
  438.  
  439. ==> /var/log/gitlab/prometheus/current <==
  440. 2019-12-19_13:16:43.16472 level=info ts=2019-12-19T13:16:43.164Z caller=head.go:557 component=tsdb msg="WAL segment loaded" segment=2859 maxSegment=2860
  441. 2019-12-19_13:16:43.16491 level=info ts=2019-12-19T13:16:43.164Z caller=head.go:557 component=tsdb msg="WAL segment loaded" segment=2860 maxSegment=2860
  442. 2019-12-19_13:16:43.18149 level=info ts=2019-12-19T13:16:43.181Z caller=main.go:669 fs_type=EXT4_SUPER_MAGIC
  443. 2019-12-19_13:16:43.18151 level=info ts=2019-12-19T13:16:43.181Z caller=main.go:670 msg="TSDB started"
  444. 2019-12-19_13:16:43.18154 level=info ts=2019-12-19T13:16:43.181Z caller=main.go:740 msg="Loading configuration file" filename=/var/opt/gitlab/prometheus/prometheus.yml
  445. 2019-12-19_13:16:43.18355 level=error ts=2019-12-19T13:16:43.183Z caller=manager.go:334 component="discovery manager scrape" msg="Cannot create service discovery" err="unable to load specified CA cert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory" type=*kubernetes.SDConfig
  446. 2019-12-19_13:16:43.18360 level=error ts=2019-12-19T13:16:43.183Z caller=manager.go:334 component="discovery manager scrape" msg="Cannot create service discovery" err="unable to load specified CA cert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory" type=*kubernetes.SDConfig
  447. 2019-12-19_13:16:43.18366 level=error ts=2019-12-19T13:16:43.183Z caller=manager.go:334 component="discovery manager scrape" msg="Cannot create service discovery" err="unable to load specified CA cert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory" type=*kubernetes.SDConfig
  448. 2019-12-19_13:16:43.19109 level=info ts=2019-12-19T13:16:43.191Z caller=main.go:768 msg="Completed loading of configuration file" filename=/var/opt/gitlab/prometheus/prometheus.yml
  449. 2019-12-19_13:16:43.19112 level=info ts=2019-12-19T13:16:43.191Z caller=main.go:623 msg="Server is ready to receive web requests."
  450.  
  451. ==> /var/log/gitlab/alertmanager/current <==
  452. 2019-12-19_13:16:25.33112 level=info ts=2019-12-19T13:16:25.331Z caller=cluster.go:640 component=cluster msg="gossip settled; proceeding" elapsed=10.000565232s
  453. 2019-12-19_13:16:42.35685 level=info ts=2019-12-19T13:16:42.356Z caller=main.go:468 msg="Received SIGTERM, exiting gracefully..."
  454. 2019-12-19_13:16:42.41737 level=info ts=2019-12-19T13:16:42.417Z caller=main.go:197 msg="Starting Alertmanager" version="(version=0.18.0, branch=master, revision=)"
  455. 2019-12-19_13:16:42.41741 level=info ts=2019-12-19T13:16:42.417Z caller=main.go:198 build_context="(go=go1.12.7, user=GitLab-Omnibus, date=)"
  456. 2019-12-19_13:16:42.41799 level=info ts=2019-12-19T13:16:42.417Z caller=cluster.go:161 component=cluster msg="setting advertise address explicitly" addr=172.18.0.5 port=9094
  457. 2019-12-19_13:16:42.41888 level=info ts=2019-12-19T13:16:42.418Z caller=cluster.go:623 component=cluster msg="Waiting for gossip to settle..." interval=2s
  458. 2019-12-19_13:16:42.46448 level=info ts=2019-12-19T13:16:42.464Z caller=coordinator.go:119 component=configuration msg="Loading configuration file" file=/var/opt/gitlab/alertmanager/alertmanager.yml
  459. 2019-12-19_13:16:42.46680 level=info ts=2019-12-19T13:16:42.466Z caller=coordinator.go:131 component=configuration msg="Completed loading of configuration file" file=/var/opt/gitlab/alertmanager/alertmanager.yml
  460. 2019-12-19_13:16:42.47006 level=info ts=2019-12-19T13:16:42.470Z caller=main.go:429 msg=Listening address=localhost:9093
  461. 2019-12-19_13:16:44.41907 level=info ts=2019-12-19T13:16:44.419Z caller=cluster.go:648 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000129028s
  462.  
  463. ==> /var/log/gitlab/redis-exporter/current <==
  464. 2019-12-19_13:16:13.20707 time="2019-12-19T13:16:13Z" level=info msg="Redis Metrics Exporter 1.1.0 build date: sha1: Go: go1.12.7"
  465. 2019-12-19_13:16:13.20750 time="2019-12-19T13:16:13Z" level=info msg="Providing metrics at localhost:9121/metrics"
  466. 2019-12-19_13:16:41.46898 time="2019-12-19T13:16:41Z" level=info msg="Redis Metrics Exporter 1.1.0 build date: sha1: Go: go1.12.7"
  467. 2019-12-19_13:16:41.46948 time="2019-12-19T13:16:41Z" level=info msg="Providing metrics at localhost:9121/metrics"
  468.  
  469. ==> /var/log/gitlab/prometheus/current <==
  470. 2019-12-19_13:16:48.19190 level=error ts=2019-12-19T13:16:48.191Z caller=manager.go:123 component="scrape manager" msg="error creating new scrape pool" err="error creating HTTP client: unable to load specified CA cert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory" scrape_pool=kubernetes-nodes
  471. 2019-12-19_13:16:48.19206 level=error ts=2019-12-19T13:16:48.191Z caller=manager.go:123 component="scrape manager" msg="error creating new scrape pool" err="error creating HTTP client: unable to load specified CA cert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory" scrape_pool=kubernetes-cadvisor
  472. 2019-12-19_13:16:48.19259 level=error ts=2019-12-19T13:16:48.192Z caller=manager.go:123 component="scrape manager" msg="error creating new scrape pool" err="error creating HTTP client: unable to load specified CA cert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory" scrape_pool=kubernetes-pods
  473.  
  474. ==> /var/log/gitlab/gitlab-rails/production.log <==
  475. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:16:50 +0000
  476. Processing by MetricsController#index as HTML
  477. Completed 200 OK in 44ms (Views: 21.0ms | ActiveRecord: 8.1ms | Elasticsearch: 0.0ms)
  478.  
  479. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  480. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":43.62,"view":20.99,"db":8.05,"time":"2019-12-19T13:16:50.131Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"4efe818e-d3e0-49da-be9a-ba5699e35ddf","cpu_s":0.05500549499999999}
  481.  
  482. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  483. [2019-12-19T13:16:52.083+0000] 127.0.0.1 - - [19/Dec/2019:13:16:52 UTC] "GET /metrics HTTP/1.1" 200 4145 "-" "Prometheus/2.12.0"
  484.  
  485. ==> /var/log/gitlab/alertmanager/current <==
  486. 2019-12-19_13:16:52.42000 level=info ts=2019-12-19T13:16:52.419Z caller=cluster.go:640 component=cluster msg="gossip settled; proceeding" elapsed=10.001039397s
  487.  
  488. ==> /var/log/gitlab/gitlab-rails/production.log <==
  489. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:17:05 +0000
  490. Processing by MetricsController#index as HTML
  491. Completed 200 OK in 24ms (Views: 4.0ms | Elasticsearch: 0.0ms)
  492.  
  493. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  494. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":24.56,"view":4.0,"time":"2019-12-19T13:17:05.099Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"0c8393b2-20b5-4271-a5d8-227bbadd2fe2","cpu_s":0.03415199799999999}
  495.  
  496. ==> /var/log/gitlab/prometheus/current <==
  497. 2019-12-19_13:17:05.41864 level=warn ts=2019-12-19T13:17:05.418Z caller=manager.go:567 component="rule manager" group="GitLab Saturation Ratios" msg="Error on ingesting results from rule evaluation with different value but same timestamp" numDropped=1
  498.  
  499. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  500. [2019-12-19T13:17:07.083+0000] 127.0.0.1 - - [19/Dec/2019:13:17:07 UTC] "GET /metrics HTTP/1.1" 200 4145 "-" "Prometheus/2.12.0"
  501.  
  502. ==> /var/log/gitlab/gitlab-rails/sidekiq.log <==
  503. {"severity":"INFO","time":"2019-12-19T13:17:10.797Z","queue":"cronjob:ci_archive_traces_cron","class":"Ci::ArchiveTracesCronWorker","retry":false,"queue_namespace":"cronjob","jid":"17d3ac28bc66385898163685","created_at":"2019-12-19T13:17:10.790676Z","correlation_id":"df116634-7412-4093-85b0-cb8100a209f1","enqueued_at":"2019-12-19T13:17:10.795917Z","pid":494,"message":"Ci::ArchiveTracesCronWorker JID-17d3ac28bc66385898163685: start","job_status":"start","scheduling_latency_s":0.001224}
  504. {"severity":"INFO","time":"2019-12-19T13:17:10.955Z","queue":"cronjob:ci_archive_traces_cron","class":"Ci::ArchiveTracesCronWorker","retry":false,"queue_namespace":"cronjob","jid":"17d3ac28bc66385898163685","created_at":"2019-12-19T13:17:10.790676Z","correlation_id":"df116634-7412-4093-85b0-cb8100a209f1","enqueued_at":"2019-12-19T13:17:10.795917Z","pid":494,"message":"Ci::ArchiveTracesCronWorker JID-17d3ac28bc66385898163685: done: 0.158577 sec","job_status":"done","scheduling_latency_s":0.001224,"duration":0.158577,"cpu_s":0.033521,"completed_at":"2019-12-19T13:17:10.955573Z"}
  505.  
  506. ==> /var/log/gitlab/sidekiq/current <==
  507. {"severity":"INFO","time":"2019-12-19T13:17:10.797Z","queue":"cronjob:ci_archive_traces_cron","class":"Ci::ArchiveTracesCronWorker","retry":false,"queue_namespace":"cronjob","jid":"17d3ac28bc66385898163685","created_at":"2019-12-19T13:17:10.790676Z","correlation_id":"df116634-7412-4093-85b0-cb8100a209f1","enqueued_at":"2019-12-19T13:17:10.795917Z","pid":494,"message":"Ci::ArchiveTracesCronWorker JID-17d3ac28bc66385898163685: start","job_status":"start","scheduling_latency_s":0.001224}
  508. {"severity":"INFO","time":"2019-12-19T13:17:10.955Z","queue":"cronjob:ci_archive_traces_cron","class":"Ci::ArchiveTracesCronWorker","retry":false,"queue_namespace":"cronjob","jid":"17d3ac28bc66385898163685","created_at":"2019-12-19T13:17:10.790676Z","correlation_id":"df116634-7412-4093-85b0-cb8100a209f1","enqueued_at":"2019-12-19T13:17:10.795917Z","pid":494,"message":"Ci::ArchiveTracesCronWorker JID-17d3ac28bc66385898163685: done: 0.158577 sec","job_status":"done","scheduling_latency_s":0.001224,"duration":0.158577,"cpu_s":0.033521,"completed_at":"2019-12-19T13:17:10.955573Z"}
  509.  
  510. ==> /var/log/gitlab/gitlab-rails/production.log <==
  511. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:17:20 +0000
  512. Processing by MetricsController#index as HTML
  513. Completed 200 OK in 23ms (Views: 0.5ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  514.  
  515. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  516. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":23.07,"view":0.53,"db":0.0,"time":"2019-12-19T13:17:20.138Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"b78235e7-61f5-4217-ab1f-92b6c5964ec7","cpu_s":0.05342825499999998}
  517.  
  518. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  519. [2019-12-19T13:17:22.098+0000] 127.0.0.1 - - [19/Dec/2019:13:17:22 UTC] "GET /metrics HTTP/1.1" 200 5023 "-" "Prometheus/2.12.0"
  520.  
  521. ==> /var/log/gitlab/gitlab-rails/production.log <==
  522. Started GET "/help" for 127.0.0.1 at 2019-12-19 13:17:24 +0000
  523. Processing by HelpController#index as */*
  524.  
  525. ==> /var/log/gitlab/gitlab-workhorse/current <==
  526. {"correlation_id":"KS1so6Uu2Y5","duration_ms":733,"host":"localhost:8929","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","status":200,"system":"http","time":"2019-12-19T13:17:25Z","uri":"/help","user_agent":"curl/7.59.0-DEV","written_bytes":83403}
  527.  
  528. ==> /var/log/gitlab/gitlab-rails/production.log <==
  529. Completed 200 OK in 691ms (Views: 662.5ms | ActiveRecord: 16.7ms | Elasticsearch: 0.0ms)
  530.  
  531. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  532. {"method":"GET","path":"/help","format":"*/*","controller":"HelpController","action":"index","status":200,"duration":690.8,"view":662.5,"db":16.74,"time":"2019-12-19T13:17:25.097Z","params":[],"remote_ip":"127.0.0.1","user_id":null,"username":null,"ua":"curl/7.59.0-DEV","queue_duration":31.92,"correlation_id":"KS1so6Uu2Y5","cpu_s":0.696594749}
  533.  
  534. ==> /var/log/gitlab/nginx/gitlab_access.log <==
  535. 127.0.0.1 - - [19/Dec/2019:13:17:25 +0000] "GET /help HTTP/1.1" 200 83495 "" "curl/7.59.0-DEV"
  536.  
  537. ==> /var/log/gitlab/gitlab-rails/production.log <==
  538. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:17:35 +0000
  539. Processing by MetricsController#index as HTML
  540. Completed 200 OK in 24ms (Views: 3.5ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  541.  
  542. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  543. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":24.2,"view":3.47,"db":0.0,"time":"2019-12-19T13:17:35.128Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"7fefdeab-7c55-4b40-bf8a-2d3fb6fcf554","cpu_s":0.049953886}
  544.  
  545. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  546. [2019-12-19T13:17:37.095+0000] 127.0.0.1 - - [19/Dec/2019:13:17:37 UTC] "GET /metrics HTTP/1.1" 200 5023 "-" "Prometheus/2.12.0"
  547.  
  548. ==> /var/log/gitlab/gitlab-rails/production.log <==
  549. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:17:50 +0000
  550. Processing by MetricsController#index as HTML
  551. Completed 200 OK in 38ms (Views: 0.9ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  552.  
  553. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  554. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":37.94,"view":0.89,"db":0.0,"time":"2019-12-19T13:17:50.113Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"e8b34b3d-f112-4746-9673-54e9a96b27b9","cpu_s":0.04680259099999995}
  555.  
  556. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  557. [2019-12-19T13:17:52.104+0000] 127.0.0.1 - - [19/Dec/2019:13:17:52 UTC] "GET /metrics HTTP/1.1" 200 5813 "-" "Prometheus/2.12.0"
  558.  
  559. ==> /var/log/gitlab/gitlab-rails/production.log <==
  560. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:18:05 +0000
  561. Processing by MetricsController#index as HTML
  562. Completed 200 OK in 23ms (Views: 0.4ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  563.  
  564. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  565. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":22.77,"view":0.42,"db":0.0,"time":"2019-12-19T13:18:05.111Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"07b3f2ae-724a-450c-986b-8d3276478786","cpu_s":0.040888641}
  566.  
  567. ==> /var/log/gitlab/prometheus/current <==
  568. 2019-12-19_13:18:05.41867 level=warn ts=2019-12-19T13:18:05.418Z caller=manager.go:567 component="rule manager" group="GitLab Saturation Ratios" msg="Error on ingesting results from rule evaluation with different value but same timestamp" numDropped=1
  569.  
  570. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  571. [2019-12-19T13:18:07.084+0000] 127.0.0.1 - - [19/Dec/2019:13:18:07 UTC] "GET /metrics HTTP/1.1" 200 5813 "-" "Prometheus/2.12.0"
  572.  
  573. ==> /var/log/gitlab/gitlab-rails/production.log <==
  574. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:18:20 +0000
  575. Processing by MetricsController#index as HTML
  576. Completed 200 OK in 36ms (Views: 0.4ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  577.  
  578. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  579. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":35.79,"view":0.36,"db":0.0,"time":"2019-12-19T13:18:20.109Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"16379c86-a84f-4812-a3d5-f772ddf4a125","cpu_s":0.043736881000000005}
  580.  
  581. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  582. [2019-12-19T13:18:22.098+0000] 127.0.0.1 - - [19/Dec/2019:13:18:22 UTC] "GET /metrics HTTP/1.1" 200 5813 "-" "Prometheus/2.12.0"
  583.  
  584. ==> /var/log/gitlab/gitlab-rails/production.log <==
  585. Started GET "/help" for 127.0.0.1 at 2019-12-19 13:18:25 +0000
  586. Processing by HelpController#index as */*
  587.  
  588. ==> /var/log/gitlab/gitlab-workhorse/current <==
  589. {"correlation_id":"xMkPJC8KGi1","duration_ms":699,"host":"localhost:8929","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","status":200,"system":"http","time":"2019-12-19T13:18:26Z","uri":"/help","user_agent":"curl/7.59.0-DEV","written_bytes":83403}
  590.  
  591. ==> /var/log/gitlab/gitlab-rails/production.log <==
  592. Completed 200 OK in 655ms (Views: 626.9ms | ActiveRecord: 11.3ms | Elasticsearch: 0.0ms)
  593.  
  594. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  595. {"method":"GET","path":"/help","format":"*/*","controller":"HelpController","action":"index","status":200,"duration":655.33,"view":626.86,"db":11.3,"time":"2019-12-19T13:18:26.117Z","params":[],"remote_ip":"127.0.0.1","user_id":null,"username":null,"ua":"curl/7.59.0-DEV","queue_duration":32.59,"correlation_id":"xMkPJC8KGi1","cpu_s":0.6714060369999999}
  596.  
  597. ==> /var/log/gitlab/nginx/gitlab_access.log <==
  598. 127.0.0.1 - - [19/Dec/2019:13:18:26 +0000] "GET /help HTTP/1.1" 200 83495 "" "curl/7.59.0-DEV"
  599.  
  600. ==> /var/log/gitlab/gitlab-rails/production.log <==
  601. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:18:35 +0000
  602. Processing by MetricsController#index as HTML
  603. Completed 200 OK in 35ms (Views: 0.4ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  604.  
  605. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  606. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":34.92,"view":0.37,"db":0.0,"time":"2019-12-19T13:18:35.109Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"8551aa4e-e69d-44ae-bb99-8900d6e22b18","cpu_s":0.04399539499999999}
  607.  
  608. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  609. [2019-12-19T13:18:37.097+0000] 127.0.0.1 - - [19/Dec/2019:13:18:37 UTC] "GET /metrics HTTP/1.1" 200 5813 "-" "Prometheus/2.12.0"
  610.  
  611. ==> /var/log/gitlab/gitlab-rails/production.log <==
  612. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:18:50 +0000
  613. Processing by MetricsController#index as HTML
  614. Completed 200 OK in 25ms (Views: 3.7ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  615.  
  616. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  617. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":25.48,"view":3.67,"db":0.0,"time":"2019-12-19T13:18:50.162Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"a07ad15a-7a1d-4bf9-8209-b07e794edfc5","cpu_s":0.06143918699999998}
  618.  
  619. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  620. [2019-12-19T13:18:52.099+0000] 127.0.0.1 - - [19/Dec/2019:13:18:52 UTC] "GET /metrics HTTP/1.1" 200 5813 "-" "Prometheus/2.12.0"
  621.  
  622. ==> /var/log/gitlab/gitlab-rails/production.log <==
  623. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:19:05 +0000
  624. Processing by MetricsController#index as HTML
  625. Completed 200 OK in 36ms (Views: 0.9ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  626.  
  627. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  628. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":36.12,"view":0.91,"db":0.0,"time":"2019-12-19T13:19:05.110Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"b3ee8118-9eda-4476-9d21-3d0b28640ec3","cpu_s":0.04542176099999995}
  629.  
  630. ==> /var/log/gitlab/prometheus/current <==
  631. 2019-12-19_13:19:05.41697 level=warn ts=2019-12-19T13:19:05.416Z caller=manager.go:567 component="rule manager" group="GitLab Saturation Ratios" msg="Error on ingesting results from rule evaluation with different value but same timestamp" numDropped=1
  632.  
  633. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  634. [2019-12-19T13:19:07.098+0000] 127.0.0.1 - - [19/Dec/2019:13:19:07 UTC] "GET /metrics HTTP/1.1" 200 5813 "-" "Prometheus/2.12.0"
  635.  
  636. ==> /var/log/gitlab/gitlab-rails/production.log <==
  637. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:19:20 +0000
  638. Processing by MetricsController#index as HTML
  639. Completed 200 OK in 38ms (Views: 0.4ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  640.  
  641. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  642. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":37.82,"view":0.37,"db":0.0,"time":"2019-12-19T13:19:20.112Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"a62c1374-645d-42e9-b29b-67e7b87c0f06","cpu_s":0.047344330999999906}
  643.  
  644. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  645. [2019-12-19T13:19:22.088+0000] 127.0.0.1 - - [19/Dec/2019:13:19:22 UTC] "GET /metrics HTTP/1.1" 200 5817 "-" "Prometheus/2.12.0"
  646.  
  647. ==> /var/log/gitlab/gitlab-rails/production.log <==
  648. Started GET "/help" for 127.0.0.1 at 2019-12-19 13:19:26 +0000
  649. Processing by HelpController#index as */*
  650.  
  651. ==> /var/log/gitlab/gitlab-workhorse/current <==
  652. {"correlation_id":"IPK32zQ60u9","duration_ms":663,"host":"localhost:8929","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","status":200,"system":"http","time":"2019-12-19T13:19:27Z","uri":"/help","user_agent":"curl/7.59.0-DEV","written_bytes":83403}
  653.  
  654. ==> /var/log/gitlab/gitlab-rails/production.log <==
  655. Completed 200 OK in 639ms (Views: 615.9ms | ActiveRecord: 10.4ms | Elasticsearch: 0.0ms)
  656.  
  657. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  658. {"method":"GET","path":"/help","format":"*/*","controller":"HelpController","action":"index","status":200,"duration":639.58,"view":615.89,"db":10.37,"time":"2019-12-19T13:19:27.093Z","params":[],"remote_ip":"127.0.0.1","user_id":null,"username":null,"ua":"curl/7.59.0-DEV","queue_duration":15.14,"correlation_id":"IPK32zQ60u9","cpu_s":0.6475237859999999}
  659.  
  660. ==> /var/log/gitlab/nginx/gitlab_access.log <==
  661. 127.0.0.1 - - [19/Dec/2019:13:19:27 +0000] "GET /help HTTP/1.1" 200 83495 "" "curl/7.59.0-DEV"
  662.  
  663. ==> /var/log/gitlab/gitlab-rails/production.log <==
  664. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:19:35 +0000
  665. Processing by MetricsController#index as HTML
  666. Completed 200 OK in 26ms (Views: 3.6ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  667.  
  668. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  669. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":26.06,"view":3.64,"db":0.0,"time":"2019-12-19T13:19:35.191Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"9e898237-f7df-4247-8751-f8c0c2a65220","cpu_s":0.07976827300000003}
  670.  
  671. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  672. [2019-12-19T13:19:37.085+0000] 127.0.0.1 - - [19/Dec/2019:13:19:37 UTC] "GET /metrics HTTP/1.1" 200 5817 "-" "Prometheus/2.12.0"
  673.  
  674. ==> /var/log/gitlab/gitlab-rails/sidekiq.log <==
  675. {"severity":"INFO","time":"2019-12-19T13:19:39.131Z","queue":"cronjob:pipeline_schedule","class":"PipelineScheduleWorker","retry":false,"queue_namespace":"cronjob","jid":"11ea92e487422e11c613a0c8","created_at":"2019-12-19T13:19:39.129857Z","correlation_id":"be215804-c738-49bb-9d6d-81d0d0e7e3f0","enqueued_at":"2019-12-19T13:19:39.130150Z","pid":494,"message":"PipelineScheduleWorker JID-11ea92e487422e11c613a0c8: start","job_status":"start","scheduling_latency_s":0.00099}
  676. {"severity":"INFO","time":"2019-12-19T13:19:39.181Z","queue":"cronjob:pipeline_schedule","class":"PipelineScheduleWorker","retry":false,"queue_namespace":"cronjob","jid":"11ea92e487422e11c613a0c8","created_at":"2019-12-19T13:19:39.129857Z","correlation_id":"be215804-c738-49bb-9d6d-81d0d0e7e3f0","enqueued_at":"2019-12-19T13:19:39.130150Z","pid":494,"message":"PipelineScheduleWorker JID-11ea92e487422e11c613a0c8: done: 0.050226 sec","job_status":"done","scheduling_latency_s":0.00099,"duration":0.050226,"cpu_s":0.008083,"completed_at":"2019-12-19T13:19:39.181211Z"}
  677.  
  678. ==> /var/log/gitlab/sidekiq/current <==
  679. {"severity":"INFO","time":"2019-12-19T13:19:39.131Z","queue":"cronjob:pipeline_schedule","class":"PipelineScheduleWorker","retry":false,"queue_namespace":"cronjob","jid":"11ea92e487422e11c613a0c8","created_at":"2019-12-19T13:19:39.129857Z","correlation_id":"be215804-c738-49bb-9d6d-81d0d0e7e3f0","enqueued_at":"2019-12-19T13:19:39.130150Z","pid":494,"message":"PipelineScheduleWorker JID-11ea92e487422e11c613a0c8: start","job_status":"start","scheduling_latency_s":0.00099}
  680. {"severity":"INFO","time":"2019-12-19T13:19:39.181Z","queue":"cronjob:pipeline_schedule","class":"PipelineScheduleWorker","retry":false,"queue_namespace":"cronjob","jid":"11ea92e487422e11c613a0c8","created_at":"2019-12-19T13:19:39.129857Z","correlation_id":"be215804-c738-49bb-9d6d-81d0d0e7e3f0","enqueued_at":"2019-12-19T13:19:39.130150Z","pid":494,"message":"PipelineScheduleWorker JID-11ea92e487422e11c613a0c8: done: 0.050226 sec","job_status":"done","scheduling_latency_s":0.00099,"duration":0.050226,"cpu_s":0.008083,"completed_at":"2019-12-19T13:19:39.181211Z"}
  681.  
  682. ==> /var/log/gitlab/gitlab-rails/production.log <==
  683. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:19:50 +0000
  684. Processing by MetricsController#index as HTML
  685. Completed 200 OK in 20ms (Views: 0.4ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  686.  
  687. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  688. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":19.69,"view":0.39,"db":0.0,"time":"2019-12-19T13:19:50.137Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"7c94bcb7-6ad7-4333-ba4b-8f2cfbfffd28","cpu_s":0.058902224999999975}
  689.  
  690. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  691. [2019-12-19T13:19:52.085+0000] 127.0.0.1 - - [19/Dec/2019:13:19:52 UTC] "GET /metrics HTTP/1.1" 200 6145 "-" "Prometheus/2.12.0"
  692.  
  693. ==> /var/log/gitlab/gitlab-rails/production.log <==
  694. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:20:05 +0000
  695. Processing by MetricsController#index as HTML
  696. Completed 200 OK in 23ms (Views: 0.4ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  697.  
  698. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  699. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":23.23,"view":0.38,"db":0.0,"time":"2019-12-19T13:20:05.131Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"ddbec117-d2db-4730-827c-e7f249d0fa60","cpu_s":0.06074286899999998}
  700.  
  701. ==> /var/log/gitlab/prometheus/current <==
  702. 2019-12-19_13:20:05.41956 level=warn ts=2019-12-19T13:20:05.419Z caller=manager.go:567 component="rule manager" group="GitLab Saturation Ratios" msg="Error on ingesting results from rule evaluation with different value but same timestamp" numDropped=1
  703.  
  704. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  705. [2019-12-19T13:20:07.092+0000] 127.0.0.1 - - [19/Dec/2019:13:20:07 UTC] "GET /metrics HTTP/1.1" 200 6145 "-" "Prometheus/2.12.0"
  706.  
  707. ==> /var/log/gitlab/gitlab-rails/sidekiq.log <==
  708. {"severity":"INFO","time":"2019-12-19T13:20:19.380Z","queue":"cronjob:pages_domain_ssl_renewal_cron","class":"PagesDomainSslRenewalCronWorker","retry":false,"queue_namespace":"cronjob","jid":"c0cbf6bf366703c14640af60","created_at":"2019-12-19T13:20:19.379003Z","correlation_id":"ca4642f4-4260-4252-a16e-adb30d0f0a8d","enqueued_at":"2019-12-19T13:20:19.379196Z","pid":494,"message":"PagesDomainSslRenewalCronWorker JID-c0cbf6bf366703c14640af60: start","job_status":"start","scheduling_latency_s":0.001044}
  709. {"severity":"INFO","time":"2019-12-19T13:20:19.400Z","queue":"cronjob:repository_check_dispatch","class":"RepositoryCheck::DispatchWorker","retry":false,"queue_namespace":"cronjob","jid":"2047829cf4c9be03a1f71010","created_at":"2019-12-19T13:20:19.398524Z","correlation_id":"a74508ba-1cc1-46be-8c7b-88c9861b38f2","enqueued_at":"2019-12-19T13:20:19.399033Z","pid":494,"message":"RepositoryCheck::DispatchWorker JID-2047829cf4c9be03a1f71010: start","job_status":"start","scheduling_latency_s":0.001273}
  710. {"severity":"INFO","time":"2019-12-19T13:20:19.426Z","queue":"cronjob:pages_domain_ssl_renewal_cron","class":"PagesDomainSslRenewalCronWorker","retry":false,"queue_namespace":"cronjob","jid":"c0cbf6bf366703c14640af60","created_at":"2019-12-19T13:20:19.379003Z","correlation_id":"ca4642f4-4260-4252-a16e-adb30d0f0a8d","enqueued_at":"2019-12-19T13:20:19.379196Z","pid":494,"message":"PagesDomainSslRenewalCronWorker JID-c0cbf6bf366703c14640af60: done: 0.045992 sec","job_status":"done","scheduling_latency_s":0.001044,"duration":0.045992,"cpu_s":0.015429,"completed_at":"2019-12-19T13:20:19.426062Z"}
  711. {"severity":"INFO","time":"2019-12-19T13:20:19.452Z","class":"RepositoryCheck::BatchWorker","retry":false,"queue":"repository_check:repository_check_batch","queue_namespace":"repository_check","jid":"0a11dd072db21e4e129dfc5d","created_at":"2019-12-19T13:20:19.451463Z","correlation_id":"a74508ba-1cc1-46be-8c7b-88c9861b38f2","enqueued_at":"2019-12-19T13:20:19.451658Z","pid":494,"message":"RepositoryCheck::BatchWorker JID-0a11dd072db21e4e129dfc5d: start","job_status":"start","scheduling_latency_s":0.001167}
  712. {"severity":"INFO","time":"2019-12-19T13:20:19.467Z","queue":"cronjob:repository_check_dispatch","class":"RepositoryCheck::DispatchWorker","retry":false,"queue_namespace":"cronjob","jid":"2047829cf4c9be03a1f71010","created_at":"2019-12-19T13:20:19.398524Z","correlation_id":"a74508ba-1cc1-46be-8c7b-88c9861b38f2","enqueued_at":"2019-12-19T13:20:19.399033Z","pid":494,"message":"RepositoryCheck::DispatchWorker JID-2047829cf4c9be03a1f71010: done: 0.066848 sec","job_status":"done","scheduling_latency_s":0.001273,"gitaly_calls":1,"gitaly_duration":10.33,"duration":0.066848,"cpu_s":0.0295,"completed_at":"2019-12-19T13:20:19.467056Z"}
  713. {"severity":"INFO","time":"2019-12-19T13:20:19.490Z","class":"RepositoryCheck::BatchWorker","retry":false,"queue":"repository_check:repository_check_batch","queue_namespace":"repository_check","jid":"0a11dd072db21e4e129dfc5d","created_at":"2019-12-19T13:20:19.451463Z","correlation_id":"a74508ba-1cc1-46be-8c7b-88c9861b38f2","enqueued_at":"2019-12-19T13:20:19.451658Z","pid":494,"message":"RepositoryCheck::BatchWorker JID-0a11dd072db21e4e129dfc5d: done: 0.038098 sec","job_status":"done","scheduling_latency_s":0.001167,"duration":0.038098,"cpu_s":0.021527,"completed_at":"2019-12-19T13:20:19.490712Z"}
  714.  
  715. ==> /var/log/gitlab/gitaly/current <==
  716. {"correlation_id":"a74508ba-1cc1-46be-8c7b-88c9861b38f2","grpc.code":"OK","grpc.meta.auth_version":"v2","grpc.meta.client_name":"gitlab-sidekiq","grpc.method":"Check","grpc.request.deadline":"2019-12-19T13:20:29Z","grpc.request.fullMethod":"/grpc.health.v1.Health/Check","grpc.service":"grpc.health.v1.Health","grpc.start_time":"2019-12-19T13:20:19Z","grpc.time_ms":0.037,"level":"info","msg":"finished unary call with code OK","peer.address":"@","pid":817,"span.kind":"server","system":"grpc","time":"2019-12-19T13:20:19Z"}
  717.  
  718. ==> /var/log/gitlab/sidekiq/current <==
  719. {"severity":"INFO","time":"2019-12-19T13:20:19.380Z","queue":"cronjob:pages_domain_ssl_renewal_cron","class":"PagesDomainSslRenewalCronWorker","retry":false,"queue_namespace":"cronjob","jid":"c0cbf6bf366703c14640af60","created_at":"2019-12-19T13:20:19.379003Z","correlation_id":"ca4642f4-4260-4252-a16e-adb30d0f0a8d","enqueued_at":"2019-12-19T13:20:19.379196Z","pid":494,"message":"PagesDomainSslRenewalCronWorker JID-c0cbf6bf366703c14640af60: start","job_status":"start","scheduling_latency_s":0.001044}
  720. {"severity":"INFO","time":"2019-12-19T13:20:19.400Z","queue":"cronjob:repository_check_dispatch","class":"RepositoryCheck::DispatchWorker","retry":false,"queue_namespace":"cronjob","jid":"2047829cf4c9be03a1f71010","created_at":"2019-12-19T13:20:19.398524Z","correlation_id":"a74508ba-1cc1-46be-8c7b-88c9861b38f2","enqueued_at":"2019-12-19T13:20:19.399033Z","pid":494,"message":"RepositoryCheck::DispatchWorker JID-2047829cf4c9be03a1f71010: start","job_status":"start","scheduling_latency_s":0.001273}
  721. {"severity":"INFO","time":"2019-12-19T13:20:19.426Z","queue":"cronjob:pages_domain_ssl_renewal_cron","class":"PagesDomainSslRenewalCronWorker","retry":false,"queue_namespace":"cronjob","jid":"c0cbf6bf366703c14640af60","created_at":"2019-12-19T13:20:19.379003Z","correlation_id":"ca4642f4-4260-4252-a16e-adb30d0f0a8d","enqueued_at":"2019-12-19T13:20:19.379196Z","pid":494,"message":"PagesDomainSslRenewalCronWorker JID-c0cbf6bf366703c14640af60: done: 0.045992 sec","job_status":"done","scheduling_latency_s":0.001044,"duration":0.045992,"cpu_s":0.015429,"completed_at":"2019-12-19T13:20:19.426062Z"}
  722. {"severity":"INFO","time":"2019-12-19T13:20:19.452Z","class":"RepositoryCheck::BatchWorker","retry":false,"queue":"repository_check:repository_check_batch","queue_namespace":"repository_check","jid":"0a11dd072db21e4e129dfc5d","created_at":"2019-12-19T13:20:19.451463Z","correlation_id":"a74508ba-1cc1-46be-8c7b-88c9861b38f2","enqueued_at":"2019-12-19T13:20:19.451658Z","pid":494,"message":"RepositoryCheck::BatchWorker JID-0a11dd072db21e4e129dfc5d: start","job_status":"start","scheduling_latency_s":0.001167}
  723. {"severity":"INFO","time":"2019-12-19T13:20:19.467Z","queue":"cronjob:repository_check_dispatch","class":"RepositoryCheck::DispatchWorker","retry":false,"queue_namespace":"cronjob","jid":"2047829cf4c9be03a1f71010","created_at":"2019-12-19T13:20:19.398524Z","correlation_id":"a74508ba-1cc1-46be-8c7b-88c9861b38f2","enqueued_at":"2019-12-19T13:20:19.399033Z","pid":494,"message":"RepositoryCheck::DispatchWorker JID-2047829cf4c9be03a1f71010: done: 0.066848 sec","job_status":"done","scheduling_latency_s":0.001273,"gitaly_calls":1,"gitaly_duration":10.33,"duration":0.066848,"cpu_s":0.0295,"completed_at":"2019-12-19T13:20:19.467056Z"}
  724. {"severity":"INFO","time":"2019-12-19T13:20:19.490Z","class":"RepositoryCheck::BatchWorker","retry":false,"queue":"repository_check:repository_check_batch","queue_namespace":"repository_check","jid":"0a11dd072db21e4e129dfc5d","created_at":"2019-12-19T13:20:19.451463Z","correlation_id":"a74508ba-1cc1-46be-8c7b-88c9861b38f2","enqueued_at":"2019-12-19T13:20:19.451658Z","pid":494,"message":"RepositoryCheck::BatchWorker JID-0a11dd072db21e4e129dfc5d: done: 0.038098 sec","job_status":"done","scheduling_latency_s":0.001167,"duration":0.038098,"cpu_s":0.021527,"completed_at":"2019-12-19T13:20:19.490712Z"}
  725.  
  726. ==> /var/log/gitlab/gitlab-rails/production.log <==
  727. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:20:20 +0000
  728. Processing by MetricsController#index as HTML
  729. Completed 200 OK in 38ms (Views: 0.5ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  730.  
  731. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  732. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":38.11,"view":0.46,"db":0.0,"time":"2019-12-19T13:20:20.112Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"ebfd4bfa-0f1f-464b-8b02-bbc6cfe9caed","cpu_s":0.04630984599999999}
  733.  
  734. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  735. [2019-12-19T13:20:22.101+0000] 127.0.0.1 - - [19/Dec/2019:13:20:22 UTC] "GET /metrics HTTP/1.1" 200 7185 "-" "Prometheus/2.12.0"
  736.  
  737. ==> /var/log/gitlab/gitlab-rails/production.log <==
  738. Started GET "/help" for 127.0.0.1 at 2019-12-19 13:20:27 +0000
  739. Processing by HelpController#index as */*
  740.  
  741. ==> /var/log/gitlab/gitlab-workhorse/current <==
  742. {"correlation_id":"8Sb3V0NQgK","duration_ms":670,"host":"localhost:8929","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","status":200,"system":"http","time":"2019-12-19T13:20:28Z","uri":"/help","user_agent":"curl/7.59.0-DEV","written_bytes":83403}
  743.  
  744. ==> /var/log/gitlab/gitlab-rails/production.log <==
  745. Completed 200 OK in 661ms (Views: 635.1ms | ActiveRecord: 10.7ms | Elasticsearch: 0.0ms)
  746.  
  747. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  748. {"method":"GET","path":"/help","format":"*/*","controller":"HelpController","action":"index","status":200,"duration":660.7,"view":635.14,"db":10.72,"time":"2019-12-19T13:20:28.085Z","params":[],"remote_ip":"127.0.0.1","user_id":null,"username":null,"ua":"curl/7.59.0-DEV","queue_duration":1.9,"correlation_id":"8Sb3V0NQgK","cpu_s":0.6565452759999999}
  749.  
  750. ==> /var/log/gitlab/nginx/gitlab_access.log <==
  751. 127.0.0.1 - - [19/Dec/2019:13:20:28 +0000] "GET /help HTTP/1.1" 200 83495 "" "curl/7.59.0-DEV"
  752.  
  753. ==> /var/log/gitlab/gitlab-rails/production.log <==
  754. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:20:35 +0000
  755. Processing by MetricsController#index as HTML
  756. Completed 200 OK in 26ms (Views: 3.7ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  757.  
  758. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  759. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":26.64,"view":3.72,"db":0.0,"time":"2019-12-19T13:20:35.134Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"c1fa400b-0c27-42e6-b763-4c4db3e1184b","cpu_s":0.053445577999999966}
  760.  
  761. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  762. [2019-12-19T13:20:37.102+0000] 127.0.0.1 - - [19/Dec/2019:13:20:37 UTC] "GET /metrics HTTP/1.1" 200 7185 "-" "Prometheus/2.12.0"
  763.  
  764. ==> /var/log/gitlab/redis/current <==
  765. 2019-12-19_13:20:43.08771 243:M 19 Dec 13:20:43.087 * 10 changes in 300 seconds. Saving...
  766. 2019-12-19_13:20:43.08799 243:M 19 Dec 13:20:43.087 * Background saving started by pid 1547
  767. 2019-12-19_13:20:43.10016 1547:C 19 Dec 13:20:43.099 * DB saved on disk
  768. 2019-12-19_13:20:43.10103 1547:C 19 Dec 13:20:43.100 * RDB: 0 MB of memory used by copy-on-write
  769. 2019-12-19_13:20:43.18971 243:M 19 Dec 13:20:43.189 * Background saving terminated with success
  770.  
  771. ==> /var/log/gitlab/gitlab-rails/production.log <==
  772. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:20:50 +0000
  773. Processing by MetricsController#index as HTML
  774. Completed 200 OK in 20ms (Views: 0.4ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  775.  
  776. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  777. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":20.04,"view":0.37,"db":0.0,"time":"2019-12-19T13:20:50.116Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"8025d1c9-2b08-4f4e-87a5-f9d0fdcd722f","cpu_s":0.047332780999999935}
  778.  
  779. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  780. [2019-12-19T13:20:52.086+0000] 127.0.0.1 - - [19/Dec/2019:13:20:52 UTC] "GET /metrics HTTP/1.1" 200 7185 "-" "Prometheus/2.12.0"
  781.  
  782. ==> /var/log/gitlab/gitlab-rails/production.log <==
  783. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:21:05 +0000
  784. Processing by MetricsController#index as HTML
  785. Completed 200 OK in 33ms (Views: 0.4ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  786.  
  787. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  788. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":33.52,"view":0.37,"db":0.0,"time":"2019-12-19T13:21:05.107Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"90fbd2f1-d63a-4237-832d-2163084d99bc","cpu_s":0.04180506499999992}
  789.  
  790. ==> /var/log/gitlab/prometheus/current <==
  791. 2019-12-19_13:21:05.41671 level=warn ts=2019-12-19T13:21:05.416Z caller=manager.go:567 component="rule manager" group="GitLab Saturation Ratios" msg="Error on ingesting results from rule evaluation with different value but same timestamp" numDropped=1
  792.  
  793. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  794. [2019-12-19T13:21:07.161+0000] 127.0.0.1 - - [19/Dec/2019:13:21:07 UTC] "GET /metrics HTTP/1.1" 200 7185 "-" "Prometheus/2.12.0"
  795.  
  796. ==> /var/log/gitlab/gitlab-rails/production.log <==
  797. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:21:20 +0000
  798. Processing by MetricsController#index as HTML
  799. Completed 200 OK in 28ms (Views: 0.4ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  800.  
  801. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  802. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":27.95,"view":0.37,"db":0.0,"time":"2019-12-19T13:21:20.135Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"3d870f24-1eeb-470b-80f2-33f7a721ff33","cpu_s":0.06415036699999999}
  803.  
  804. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  805. [2019-12-19T13:21:22.094+0000] 127.0.0.1 - - [19/Dec/2019:13:21:22 UTC] "GET /metrics HTTP/1.1" 200 7185 "-" "Prometheus/2.12.0"
  806.  
  807. ==> /var/log/gitlab/gitlab-workhorse/current <==
  808. {"correlation_id":"Bn2HukVHnE6","duration_ms":330,"host":"localhost:8929","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","status":200,"system":"http","time":"2019-12-19T13:21:28Z","uri":"/help","user_agent":"curl/7.59.0-DEV","written_bytes":83403}
  809.  
  810. ==> /var/log/gitlab/gitlab-rails/production.log <==
  811. Started GET "/help" for 127.0.0.1 at 2019-12-19 13:21:28 +0000
  812. Processing by HelpController#index as */*
  813. Completed 200 OK in 312ms (Views: 303.5ms | ActiveRecord: 0.7ms | Elasticsearch: 0.0ms)
  814.  
  815. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  816. {"method":"GET","path":"/help","format":"*/*","controller":"HelpController","action":"index","status":200,"duration":312.37,"view":303.45,"db":0.69,"time":"2019-12-19T13:21:28.733Z","params":[],"remote_ip":"127.0.0.1","user_id":null,"username":null,"ua":"curl/7.59.0-DEV","queue_duration":12.63,"correlation_id":"Bn2HukVHnE6","cpu_s":0.32593416399999997}
  817.  
  818. ==> /var/log/gitlab/nginx/gitlab_access.log <==
  819. 127.0.0.1 - - [19/Dec/2019:13:21:28 +0000] "GET /help HTTP/1.1" 200 83495 "" "curl/7.59.0-DEV"
  820.  
  821. ==> /var/log/gitlab/gitlab-rails/production.log <==
  822. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:21:35 +0000
  823. Processing by MetricsController#index as HTML
  824. Completed 200 OK in 37ms (Views: 0.4ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  825.  
  826. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  827. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":37.17,"view":0.36,"db":0.0,"time":"2019-12-19T13:21:35.111Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"01252ed1-e7e3-44af-81c5-5a4088d575d7","cpu_s":0.04605181300000005}
  828.  
  829. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  830. [2019-12-19T13:21:37.105+0000] 127.0.0.1 - - [19/Dec/2019:13:21:37 UTC] "GET /metrics HTTP/1.1" 200 7197 "-" "Prometheus/2.12.0"
  831.  
  832. ==> /var/log/gitlab/gitlab-rails/production.log <==
  833. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:21:50 +0000
  834. Processing by MetricsController#index as HTML
  835. Completed 200 OK in 27ms (Views: 0.4ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  836.  
  837. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  838. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":26.65,"view":0.39,"db":0.0,"time":"2019-12-19T13:21:50.132Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"110d1f4b-5252-4f81-9e1b-638a5690e4b2","cpu_s":0.06118913300000001}
  839.  
  840. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  841. [2019-12-19T13:21:52.095+0000] 127.0.0.1 - - [19/Dec/2019:13:21:52 UTC] "GET /metrics HTTP/1.1" 200 7197 "-" "Prometheus/2.12.0"
  842.  
  843. ==> /var/log/gitlab/gitlab-rails/production.log <==
  844. Started GET "/-/metrics" for 127.0.0.1 at 2019-12-19 13:22:05 +0000
  845. Processing by MetricsController#index as HTML
  846. Completed 200 OK in 38ms (Views: 0.5ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms)
  847.  
  848. ==> /var/log/gitlab/gitlab-rails/production_json.log <==
  849. {"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":38.17,"view":0.5,"db":0.0,"time":"2019-12-19T13:22:05.113Z","params":[],"remote_ip":null,"user_id":null,"username":null,"ua":null,"queue_duration":null,"correlation_id":"174d6a35-3188-4cec-9394-7518acee311d","cpu_s":0.04794500600000001}
  850.  
  851. ==> /var/log/gitlab/prometheus/current <==
  852. 2019-12-19_13:22:05.41349 level=warn ts=2019-12-19T13:22:05.413Z caller=manager.go:567 component="rule manager" group="GitLab Saturation Ratios" msg="Error on ingesting results from rule evaluation with different value but same timestamp" numDropped=1
  853.  
  854. ==> /var/log/gitlab/gitlab-rails/sidekiq_exporter.log <==
  855. [2019-12-19T13:22:07.103+0000] 127.0.0.1 - - [19/Dec/2019:13:22:07 UTC] "GET /metrics HTTP/1.1" 200 7197 "-" "Prometheus/2.12.0"
Advertisement
Add Comment
Please, Sign In to add comment