API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Mar 25th, 2019
138
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 53.88 KB | None | 0 0
raw download clone embed print report
  1. 2019-03-20T14:46:03.956Z 1 TID-oswagtopc CreateServiceTokenWorker JID-2771b7bc65f90bbacd5e6edb INFO: start
  2. 2019-03-20T14:46:03.975Z 1 TID-1vrn7g BackendMetricWorker JID-eb31aa43e170ea9e692c5902 INFO: start
  3. 2019-03-20T14:46:03.977Z 1 TID-175mgo AuditedWorker JID-49d4772dbf6c86ef5a091fe7 INFO: start
  4. 2019-03-20T14:46:03.981Z 1 TID-1vmxik AuditedWorker JID-a35b98520d5b210aaaedca9a INFO: start
  5. 2019-03-20T14:46:03.987Z 1 TID-osvyuixvg AuditedWorker JID-8eb91a57787633d3bbb9a104 INFO: start
  6. 2019-03-20T14:46:03.987Z 1 TID-22sz4o ZyncWorker JID-8bffb17a457e334913b39345 INFO: start
  7. 2019-03-20T14:46:04.014Z 1 TID-1vrn7g BackendMetricWorker JID-eb31aa43e170ea9e692c5902 INFO: done: 0.039 sec
  8. 2019-03-20T14:46:04.039Z 1 TID-175mgo AuditedWorker JID-49d4772dbf6c86ef5a091fe7 INFO: done: 0.062 sec
  9. 2019-03-20T14:46:04.044Z 1 TID-oswagtopc CreateServiceTokenWorker JID-2771b7bc65f90bbacd5e6edb INFO: done: 0.089 sec
  10. 2019-03-20T14:46:04.045Z 1 TID-osvyuixvg AuditedWorker JID-8eb91a57787633d3bbb9a104 INFO: done: 0.058 sec
  11. 2019-03-20T14:46:04.059Z 1 TID-1vmxik AuditedWorker JID-a35b98520d5b210aaaedca9a INFO: done: 0.078 sec
  12. 2019-03-20T14:46:04.086Z 1 TID-22sz4o ZyncWorker JID-8bffb17a457e334913b39345 INFO: done: 0.099 sec
  13. 2019-03-20T14:46:04.533Z 1 TID-osw89s50g AuditedWorker JID-ed75244f790d3659ad32c256 INFO: start
  14. 2019-03-20T14:46:04.578Z 1 TID-osw89s50g AuditedWorker JID-ed75244f790d3659ad32c256 INFO: done: 0.045 sec
  15. 2019-03-20T14:46:09.310Z 1 TID-16fkfs PublishEnabledChangedEventForProviderApplicationsWorker JID-7bede6bdba0cfd37cf4769b5 INFO: start
  16. PublishEnabledChangedEventForProviderApplicationsWorker#perform raised ActiveJob::DeserializationError with message Error while trying to deserialize arguments: Couldn't find Account with 'id'=130
  17. 2019-03-20T14:46:09.314Z 1 TID-16fkfs PublishEnabledChangedEventForProviderApplicationsWorker JID-7bede6bdba0cfd37cf4769b5 INFO: done: 0.004 sec
  18. 2019-03-20T14:46:09.397Z 1 TID-1w1hmc AuditedWorker JID-0219e8e9511013680e6a1d1d INFO: start
  19. 2019-03-20T14:46:09.404Z 1 TID-1vxods AuditedWorker JID-f20d7520ac349850fa43c69b INFO: start
  20. 2019-03-20T14:46:09.407Z 1 TID-1xtwiw AuditedWorker JID-20f9a9b32db578c4a014f560 INFO: start
  21. 2019-03-20T14:46:09.454Z 1 TID-1xtwiw AuditedWorker JID-20f9a9b32db578c4a014f560 INFO: done: 0.047 sec
  22. 2019-03-20T14:46:09.455Z 1 TID-1w1hmc AuditedWorker JID-0219e8e9511013680e6a1d1d INFO: done: 0.058 sec
  23. 2019-03-20T14:46:09.456Z 1 TID-1vxods AuditedWorker JID-f20d7520ac349850fa43c69b INFO: done: 0.052 sec
  24. 2019-03-20T14:46:09.477Z 1 TID-1wrvpo AuditedWorker JID-903ce8fc0b4403e00b97ed5b INFO: start
  25. 2019-03-20T14:46:09.480Z 1 TID-1y29mw ProcessNotificationEventWorker JID-8034fb69e573a07c44c61ddb BID--0ZBWn1jyj0dLw INFO: start
  26. 2019-03-20T14:46:09.494Z 1 TID-1tp08w ProcessNotificationEventWorker::UserNotificationWorker JID-93a6fc5847fee2760ff31431 BID--0ZBWn1jyj0dLw INFO: start
  27. 2019-03-20T14:46:09.503Z 1 TID-1wrvpo AuditedWorker JID-903ce8fc0b4403e00b97ed5b INFO: done: 0.026 sec
  28. 2019-03-20T14:46:09.503Z 1 TID-1y29mw ProcessNotificationEventWorker JID-8034fb69e573a07c44c61ddb BID--0ZBWn1jyj0dLw INFO: done: 8034fb69e573a07c44c61ddb in batch -0ZBWn1jyj0dLw
  29. 2019-03-20T14:46:09.503Z 1 TID-1y29mw ProcessNotificationEventWorker JID-8034fb69e573a07c44c61ddb BID--0ZBWn1jyj0dLw INFO: done: 0.023 sec
  30. 2019-03-20T14:46:09.568Z 1 TID-1wpgew AuditedWorker JID-cfc30fb881ae19adedfd1f83 INFO: start
  31. 2019-03-20T14:46:09.573Z 1 TID-osw274ww4 ZyncWorker JID-0ce88dcc94134ce2dea3d919 INFO: start
  32. 2019-03-20T14:46:09.575Z 1 TID-osw8svayk ProcessNotificationEventWorker JID-1a43eeb6c3c1f369bd78d425 BID-ELiEp3HT8TdR7w INFO: start
  33. 2019-03-20T14:46:09.576Z 1 TID-22sz4o ZyncWorker JID-587f02c16b2a954526bb3c22 INFO: start
  34. 2019-03-20T14:46:09.578Z 1 TID-1vmxik AuditedWorker JID-76445f97f70e9674983585cf INFO: start
  35. 2019-03-20T14:46:09.586Z 1 TID-osvyuixvg ProcessNotificationEventWorker JID-a5c0de6ff6713f6712376014 BID-o1f7b4rqApgenA INFO: start
  36. 2019-03-20T14:46:09.590Z 1 TID-osw89s50g AuditedWorker JID-e15b86b37d7670afedff9ad0 INFO: start
  37. Rendered notification_mailer/service_contract_created.html.slim within layouts/notification_email (0.7ms)
  38. 2019-03-20T14:46:09.591Z 1 TID-1vrn7g AuditedWorker JID-f9af15c6f33e8a96e1b8082d INFO: start
  39. 2019-03-20T14:46:09.594Z 1 TID-oswagtopc ProcessNotificationEventWorker JID-3393288d6a5fac61a3129f21 BID-rlzuddzE1RGygQ INFO: start
  40. 2019-03-20T14:46:09.597Z 1 TID-osw26s8wk AuditedWorker JID-c2ea57811ff826cca5edbfc0 INFO: start
  41. Rendered notification_mailer/_footer.html.slim (0.3ms)
  42. Rendered notification_mailer/service_contract_created.text.erb within layouts/notification_email (0.5ms)
  43. Rendered notification_mailer/_footer.text.erb (0.2ms)
  44. 2019-03-20T14:46:09.655Z 1 TID-24ok64 ProcessNotificationEventWorker::UserNotificationWorker JID-3bea65b20782ba14b8b053dc BID-ELiEp3HT8TdR7w INFO: start
  45. 2019-03-20T14:46:09.666Z 1 TID-15way8 ProcessNotificationEventWorker::UserNotificationWorker JID-c772b0700068b5c269ed63bc BID-o1f7b4rqApgenA INFO: start
  46. 2019-03-20T14:46:09.668Z 1 TID-22o2qo AuditedWorker JID-a23e7d26687d514eff6832a6 INFO: start
  47. 2019-03-20T14:46:09.681Z 1 TID-1wpgew AuditedWorker JID-cfc30fb881ae19adedfd1f83 INFO: done: 0.113 sec
  48. 2019-03-20T14:46:09.695Z 1 TID-osvyuixvg ProcessNotificationEventWorker JID-a5c0de6ff6713f6712376014 BID-o1f7b4rqApgenA INFO: done: a5c0de6ff6713f6712376014 in batch o1f7b4rqApgenA
  49. 2019-03-20T14:46:09.696Z 1 TID-osvyuixvg ProcessNotificationEventWorker JID-a5c0de6ff6713f6712376014 BID-o1f7b4rqApgenA INFO: done: 0.11 sec
  50.  
  51. Sent mail to admin@3scale.3scale.apps.ocp.api-qe.eng.rdu2.redhat.com (49.2ms)
  52. 2019-03-20T14:46:09.699Z 1 TID-osw8svayk ProcessNotificationEventWorker JID-1a43eeb6c3c1f369bd78d425 BID-ELiEp3HT8TdR7w INFO: done: 1a43eeb6c3c1f369bd78d425 in batch ELiEp3HT8TdR7w
  53. 2019-03-20T14:46:09.699Z 1 TID-1v5dl4 ProcessNotificationEventWorker::UserNotificationWorker JID-3af5153a0f00ca61af8851df BID-rlzuddzE1RGygQ INFO: start
  54. 2019-03-20T14:46:09.701Z 1 TID-osw8svayk ProcessNotificationEventWorker JID-1a43eeb6c3c1f369bd78d425 BID-ELiEp3HT8TdR7w INFO: done: 0.126 sec
  55. 2019-03-20T14:46:09.706Z 1 TID-1vmxik AuditedWorker JID-76445f97f70e9674983585cf INFO: done: 0.129 sec
  56. 2019-03-20T14:46:09.708Z 1 TID-osw89s50g AuditedWorker JID-e15b86b37d7670afedff9ad0 INFO: done: 0.118 sec
  57. 2019-03-20T14:46:09.712Z 1 TID-1vrn7g AuditedWorker JID-f9af15c6f33e8a96e1b8082d INFO: done: 0.121 sec
  58. 2019-03-20T14:46:09.729Z 1 TID-oswagtopc ProcessNotificationEventWorker JID-3393288d6a5fac61a3129f21 BID-rlzuddzE1RGygQ INFO: done: 3393288d6a5fac61a3129f21 in batch rlzuddzE1RGygQ
  59. 2019-03-20T14:46:09.730Z 1 TID-oswagtopc ProcessNotificationEventWorker JID-3393288d6a5fac61a3129f21 BID-rlzuddzE1RGygQ INFO: done: 0.136 sec
  60. 2019-03-20T14:46:09.729Z 1 TID-osw274ww4 ZyncWorker JID-0ce88dcc94134ce2dea3d919 INFO: done: 0.156 sec
  61. 2019-03-20T14:46:09.730Z 1 TID-osw26s8wk AuditedWorker JID-c2ea57811ff826cca5edbfc0 INFO: done: 0.133 sec
  62. 2019-03-20T14:46:09.733Z 1 TID-22sz4o ZyncWorker JID-587f02c16b2a954526bb3c22 INFO: done: 0.157 sec
  63. 2019-03-20T14:46:09.740Z 1 TID-1tp08w ProcessNotificationEventWorker::UserNotificationWorker JID-93a6fc5847fee2760ff31431 BID--0ZBWn1jyj0dLw INFO: done: 93a6fc5847fee2760ff31431 in batch -0ZBWn1jyj0dLw
  64. 2019-03-20T14:46:09.742Z 1 TID-22o2qo AuditedWorker JID-a23e7d26687d514eff6832a6 INFO: done: 0.075 sec
  65. 2019-03-20T14:46:09.763Z 1 TID-1tp08w ProcessNotificationEventWorker::UserNotificationWorker JID-93a6fc5847fee2760ff31431 BID--0ZBWn1jyj0dLw INFO: done: 0.269 sec
  66. 2019-03-20T14:46:09.792Z 1 TID-15way8 ProcessNotificationEventWorker::UserNotificationWorker JID-c772b0700068b5c269ed63bc BID-o1f7b4rqApgenA INFO: done: c772b0700068b5c269ed63bc in batch o1f7b4rqApgenA
  67. 2019-03-20T14:46:09.800Z 1 TID-15way8 ProcessNotificationEventWorker::UserNotificationWorker JID-c772b0700068b5c269ed63bc BID-o1f7b4rqApgenA INFO: done: 0.134 sec
  68. Rendered notification_mailer/account_created.html.slim within layouts/notification_email (27.1ms)
  69. Rendered notification_mailer/_footer.html.slim (0.2ms)
  70. Rendered notification_mailer/application_created.html.slim within layouts/notification_email (14.0ms)
  71. Rendered notification_mailer/_footer.html.slim (0.2ms)
  72. Rendered notification_mailer/application_created.text.erb within layouts/notification_email (0.4ms)
  73. Rendered notification_mailer/_footer.text.erb (0.2ms)
  74. Rendered notification_mailer/account_created.text.erb within layouts/notification_email (11.5ms)
  75. Rendered notification_mailer/_footer.text.erb (0.2ms)
  76.  
  77. Sent mail to admin@3scale.3scale.apps.ocp.api-qe.eng.rdu2.redhat.com (16.4ms)
  78.  
  79. Sent mail to admin@3scale.3scale.apps.ocp.api-qe.eng.rdu2.redhat.com (18.0ms)
  80. 2019-03-20T14:46:09.911Z 1 TID-24ok64 ProcessNotificationEventWorker::UserNotificationWorker JID-3bea65b20782ba14b8b053dc BID-ELiEp3HT8TdR7w INFO: done: 3bea65b20782ba14b8b053dc in batch ELiEp3HT8TdR7w
  81. 2019-03-20T14:46:09.913Z 1 TID-24ok64 ProcessNotificationEventWorker::UserNotificationWorker JID-3bea65b20782ba14b8b053dc BID-ELiEp3HT8TdR7w INFO: done: 0.258 sec
  82. 2019-03-20T14:46:09.923Z 1 TID-1v5dl4 ProcessNotificationEventWorker::UserNotificationWorker JID-3af5153a0f00ca61af8851df BID-rlzuddzE1RGygQ INFO: done: 3af5153a0f00ca61af8851df in batch rlzuddzE1RGygQ
  83. 2019-03-20T14:46:09.925Z 1 TID-1v5dl4 ProcessNotificationEventWorker::UserNotificationWorker JID-3af5153a0f00ca61af8851df BID-rlzuddzE1RGygQ INFO: done: 0.226 sec
  84. 2019-03-20T14:46:10.248Z 1 TID-175mgo AuditedWorker JID-1ae8cce9ada151493cacc531 INFO: start
  85. 2019-03-20T14:46:10.255Z 1 TID-1ve4go ProcessNotificationEventWorker JID-e3e7eeaa4e568eec92f83cb9 BID-BYGsRLP0gHch2g INFO: start
  86. 2019-03-20T14:46:10.267Z 1 TID-1zpvks ProcessNotificationEventWorker::UserNotificationWorker JID-58a83120be7986d21cbc2a86 BID-BYGsRLP0gHch2g INFO: start
  87. 2019-03-20T14:46:10.272Z 1 TID-1ve4go ProcessNotificationEventWorker JID-e3e7eeaa4e568eec92f83cb9 BID-BYGsRLP0gHch2g INFO: done: e3e7eeaa4e568eec92f83cb9 in batch BYGsRLP0gHch2g
  88. 2019-03-20T14:46:10.272Z 1 TID-1ve4go ProcessNotificationEventWorker JID-e3e7eeaa4e568eec92f83cb9 BID-BYGsRLP0gHch2g INFO: done: 0.017 sec
  89. 2019-03-20T14:46:10.274Z 1 TID-175mgo AuditedWorker JID-1ae8cce9ada151493cacc531 INFO: done: 0.026 sec
  90. Rendered notification_mailer/service_contract_created.html.slim within layouts/notification_email (0.5ms)
  91. Rendered notification_mailer/_footer.html.slim (0.2ms)
  92. Rendered notification_mailer/service_contract_created.text.erb within layouts/notification_email (0.3ms)
  93. Rendered notification_mailer/_footer.text.erb (0.2ms)
  94.  
  95. Sent mail to admin@3scale.3scale.apps.ocp.api-qe.eng.rdu2.redhat.com (10.4ms)
  96. 2019-03-20T14:46:10.379Z 1 TID-1zpvks ProcessNotificationEventWorker::UserNotificationWorker JID-58a83120be7986d21cbc2a86 BID-BYGsRLP0gHch2g INFO: done: 58a83120be7986d21cbc2a86 in batch BYGsRLP0gHch2g
  97. 2019-03-20T14:46:10.381Z 1 TID-1zpvks ProcessNotificationEventWorker::UserNotificationWorker JID-58a83120be7986d21cbc2a86 BID-BYGsRLP0gHch2g INFO: done: 0.114 sec
  98. 2019-03-20T14:46:10.487Z 1 TID-poq00 AuditedWorker JID-f68a3741ee73ce35ad2528b3 INFO: start
  99. 2019-03-20T14:46:10.491Z 1 TID-16fkfs ZyncWorker JID-46606bf666c7dee4b71bc223 INFO: start
  100. 2019-03-20T14:46:10.498Z 1 TID-1xtwiw ProcessNotificationEventWorker JID-5235419649902e38644654b6 BID-GxEUehHW7v02-w INFO: start
  101. 2019-03-20T14:46:10.499Z 1 TID-1w1hmc ZyncWorker JID-c6c043d18a5f4a39c881954e INFO: start
  102. 2019-03-20T14:46:10.508Z 1 TID-1vxods ZyncWorker JID-cb9a55d73868c87ea9924f31 INFO: start
  103. 2019-03-20T14:46:10.531Z 1 TID-poq00 AuditedWorker JID-f68a3741ee73ce35ad2528b3 INFO: done: 0.044 sec
  104. 2019-03-20T14:46:10.536Z 1 TID-1wrvpo ProcessNotificationEventWorker::UserNotificationWorker JID-42d7bac6e4e5c975559a2bdf BID-GxEUehHW7v02-w INFO: start
  105. 2019-03-20T14:46:10.563Z 1 TID-1xtwiw ProcessNotificationEventWorker JID-5235419649902e38644654b6 BID-GxEUehHW7v02-w INFO: done: 5235419649902e38644654b6 in batch GxEUehHW7v02-w
  106. 2019-03-20T14:46:10.566Z 1 TID-1xtwiw ProcessNotificationEventWorker JID-5235419649902e38644654b6 BID-GxEUehHW7v02-w INFO: done: 0.067 sec
  107. 2019-03-20T14:46:10.611Z 1 TID-1w1hmc ZyncWorker JID-c6c043d18a5f4a39c881954e INFO: done: 0.111 sec
  108. 2019-03-20T14:46:10.634Z 1 TID-16fkfs ZyncWorker JID-46606bf666c7dee4b71bc223 INFO: done: 0.143 sec
  109. 2019-03-20T14:46:10.636Z 1 TID-1vxods ZyncWorker JID-cb9a55d73868c87ea9924f31 INFO: done: 0.129 sec
  110. Rendered notification_mailer/application_created.html.slim within layouts/notification_email (8.4ms)
  111. Rendered notification_mailer/_footer.html.slim (0.2ms)
  112. Rendered notification_mailer/application_created.text.erb within layouts/notification_email (0.2ms)
  113. Rendered notification_mailer/_footer.text.erb (0.1ms)
  114.  
  115. Sent mail to admin@3scale.3scale.apps.ocp.api-qe.eng.rdu2.redhat.com (9.0ms)
  116. 2019-03-20T14:46:10.682Z 1 TID-1wrvpo ProcessNotificationEventWorker::UserNotificationWorker JID-42d7bac6e4e5c975559a2bdf BID-GxEUehHW7v02-w INFO: done: 42d7bac6e4e5c975559a2bdf in batch GxEUehHW7v02-w
  117. 2019-03-20T14:46:10.685Z 1 TID-1wrvpo ProcessNotificationEventWorker::UserNotificationWorker JID-42d7bac6e4e5c975559a2bdf BID-GxEUehHW7v02-w INFO: done: 0.149 sec
  118. 2019-03-20T14:46:16.897Z 1 TID-osw8svayk EventsFetchWorker JID-7542d61e2f7567df5a880666 INFO: start
  119. Started fetching events from backend
  120. 2019-03-20T14:46:16.905Z 1 TID-osw89s50g PersistEventWorker JID-f4aa9535f8035a9ca6059da6 INFO: start
  121. 2019-03-20T14:46:16.909Z 1 TID-osvyuixvg PersistEventWorker JID-bc19f099041b4673526f7499 INFO: start
  122. Finished fetching events from backend
  123. 2019-03-20T14:46:16.919Z 1 TID-osw8svayk EventsFetchWorker JID-7542d61e2f7567df5a880666 INFO: done: 0.021 sec
  124. 2019-03-20T14:46:16.934Z 1 TID-osw89s50g PersistEventWorker JID-f4aa9535f8035a9ca6059da6 INFO: done: 0.029 sec
  125. 2019-03-20T14:46:16.935Z 1 TID-1zpvks EventImportWorker JID-cde92d548626e16452bda979 INFO: start
  126. importing first_traffic event: #<Events::Event id=216, type="first_traffic", timestamp="2019-03-20 14:46:16 UTC", object={"service_id"=>"171", "application_id"=>"ip-check-app-pstanko-lbuzzhxl", "timestamp"=>"2019-03-20 14:46:16 UTC"}>
  127. 2019-03-20T14:46:16.950Z 1 TID-osvyuixvg PersistEventWorker JID-bc19f099041b4673526f7499 INFO: done: 0.041 sec
  128. 2019-03-20T14:46:16.951Z 1 TID-1wpgew EventImportWorker JID-7a03572ca2734c2bd2a95a0a INFO: start
  129. importing first_daily_traffic event: #<Events::Event id=217, type="first_daily_traffic", timestamp="2019-03-20 14:46:16 UTC", object={"service_id"=>"171", "application_id"=>"ip-check-app-pstanko-lbuzzhxl", "timestamp"=>"2019-03-20 14:46:16 UTC"}>
  130. [EventBroker] notifying subscribers of Applications::ApplicationUpdatedEvent 93d8ee6b-c8a4-4488-9a42-592908be3d04
  131. [EventBroker] notifying subscribers of ZyncEvent 7a7d94b1-fd2f-4097-b74f-417c129bc9a0
  132. [EventBroker] notifying subscribers of Applications::ApplicationUpdatedEvent 8d8d5d3c-c4f3-429c-b97a-31c01baf6b99
  133. [EventBroker] notifying subscribers of ZyncEvent cae393a1-01a9-4b35-85c0-bb2e8d1cd848
  134. 2019-03-20T14:46:17.013Z 1 TID-1v5dl4 AuditedWorker JID-9c069bcc958cf4aadcb70799 INFO: start
  135. 2019-03-20T14:46:17.016Z 1 TID-175mgo ZyncWorker JID-67bb08d59e130cd4d1f23dde INFO: start
  136. 2019-03-20T14:46:17.025Z 1 TID-1zpvks EventImportWorker JID-cde92d548626e16452bda979 INFO: done: 0.091 sec
  137. 2019-03-20T14:46:17.042Z 1 TID-1v5dl4 AuditedWorker JID-9c069bcc958cf4aadcb70799 INFO: done: 0.029 sec
  138. 2019-03-20T14:46:17.051Z 1 TID-1ve4go AuditedWorker JID-8fd4e0ebdbe3c28795f89b29 INFO: start
  139. 2019-03-20T14:46:17.054Z 1 TID-1w1hmc ZyncWorker JID-68b53a68c6a7dfb8442a2b90 INFO: start
  140. 2019-03-20T14:46:17.064Z 1 TID-1wpgew EventImportWorker JID-7a03572ca2734c2bd2a95a0a INFO: done: 0.113 sec
  141. 2019-03-20T14:46:17.077Z 1 TID-175mgo ZyncWorker JID-67bb08d59e130cd4d1f23dde INFO: done: 0.061 sec
  142. 2019-03-20T14:46:17.081Z 1 TID-1ve4go AuditedWorker JID-8fd4e0ebdbe3c28795f89b29 INFO: done: 0.03 sec
  143. 2019-03-20T14:46:17.104Z 1 TID-1w1hmc ZyncWorker JID-68b53a68c6a7dfb8442a2b90 INFO: done: 0.049 sec
  144. 2019-03-20T14:46:17.488Z 1 TID-16fkfs AuditedWorker JID-e5028a184de9f0aa27c6a690 INFO: start
  145. 2019-03-20T14:46:17.491Z 1 TID-1xtwiw ProcessNotificationEventWorker JID-4f38a4657a065e3ff426b31e BID-rvnvguZohT8sOg INFO: start
  146. 2019-03-20T14:46:17.494Z 1 TID-poq00 ZyncWorker JID-22e66c84920b27c215cc2594 INFO: start
  147. 2019-03-20T14:46:17.520Z 1 TID-16fkfs AuditedWorker JID-e5028a184de9f0aa27c6a690 INFO: done: 0.032 sec
  148. 2019-03-20T14:46:17.522Z 1 TID-1wrvpo ProcessNotificationEventWorker::UserNotificationWorker JID-40c0858534555cdd6914a2ac BID-rvnvguZohT8sOg INFO: start
  149. 2019-03-20T14:46:17.528Z 1 TID-1xtwiw ProcessNotificationEventWorker JID-4f38a4657a065e3ff426b31e BID-rvnvguZohT8sOg INFO: done: 4f38a4657a065e3ff426b31e in batch rvnvguZohT8sOg
  150. 2019-03-20T14:46:17.528Z 1 TID-1xtwiw ProcessNotificationEventWorker JID-4f38a4657a065e3ff426b31e BID-rvnvguZohT8sOg INFO: done: 0.036 sec
  151. 2019-03-20T14:46:17.554Z 1 TID-poq00 ZyncWorker JID-22e66c84920b27c215cc2594 INFO: done: 0.059 sec
  152. Rendered notification_mailer/cinstance_cancellation.html.slim within layouts/notification_email (0.1ms)
  153. Rendered notification_mailer/_footer.html.slim (0.4ms)
  154. Rendered notification_mailer/cinstance_cancellation.text.erb within layouts/notification_email (0.1ms)
  155. Rendered notification_mailer/_footer.text.erb (0.2ms)
  156.  
  157. Sent mail to admin@3scale.3scale.apps.ocp.api-qe.eng.rdu2.redhat.com (8.8ms)
  158. 2019-03-20T14:46:17.610Z 1 TID-1wrvpo ProcessNotificationEventWorker::UserNotificationWorker JID-40c0858534555cdd6914a2ac BID-rvnvguZohT8sOg INFO: done: 40c0858534555cdd6914a2ac in batch rvnvguZohT8sOg
  159. 2019-03-20T14:46:17.612Z 1 TID-1wrvpo ProcessNotificationEventWorker::UserNotificationWorker JID-40c0858534555cdd6914a2ac BID-rvnvguZohT8sOg INFO: done: 0.09 sec
  160. 2019-03-20T14:46:18.222Z 1 TID-1y29mw AuditedWorker JID-be14d2be13509312277be45b INFO: start
  161. 2019-03-20T14:46:18.236Z 1 TID-1tp08w AuditedWorker JID-15ea753233c529db9c6dfc6e INFO: start
  162. 2019-03-20T14:46:18.241Z 1 TID-22sz4o ZyncWorker JID-9ebd4f24e7b0afa09388e97e INFO: start
  163. 2019-03-20T14:46:18.248Z 1 TID-1vrn7g AuditedWorker JID-5ad266c1fa289b6a91411718 INFO: start
  164. 2019-03-20T14:46:18.249Z 1 TID-osw26s8wk AuditedWorker JID-990703f511de0f1ec263c536 INFO: start
  165. 2019-03-20T14:46:18.260Z 1 TID-oswagtopc AuditedWorker JID-ea3b867cbd76f3bc5c495b1b INFO: start
  166. 2019-03-20T14:46:18.260Z 1 TID-osw89s50g ProcessNotificationEventWorker JID-8a84e4223b3dc23db67708c5 BID-iKKsdyrPOkAgPg INFO: start
  167. 2019-03-20T14:46:18.261Z 1 TID-osw8svayk AuditedWorker JID-6cc27e8fd6f8214227405984 INFO: start
  168. 2019-03-20T14:46:18.262Z 1 TID-osw274ww4 AuditedWorker JID-a9b08b1d360c41f662a302d2 INFO: start
  169. 2019-03-20T14:46:18.280Z 1 TID-1y29mw AuditedWorker JID-be14d2be13509312277be45b INFO: done: 0.058 sec
  170. 2019-03-20T14:46:18.316Z 1 TID-1tp08w AuditedWorker JID-15ea753233c529db9c6dfc6e INFO: done: 0.08 sec
  171. 2019-03-20T14:46:18.321Z 1 TID-1vrn7g AuditedWorker JID-5ad266c1fa289b6a91411718 INFO: done: 0.073 sec
  172. 2019-03-20T14:46:18.326Z 1 TID-osw26s8wk AuditedWorker JID-990703f511de0f1ec263c536 INFO: done: 0.077 sec
  173. 2019-03-20T14:46:18.334Z 1 TID-osvyuixvg ProcessNotificationEventWorker::UserNotificationWorker JID-cbec68d0fd378f5eef0ee4d9 BID-iKKsdyrPOkAgPg INFO: start
  174. 2019-03-20T14:46:18.345Z 1 TID-osw89s50g ProcessNotificationEventWorker JID-8a84e4223b3dc23db67708c5 BID-iKKsdyrPOkAgPg INFO: done: 8a84e4223b3dc23db67708c5 in batch iKKsdyrPOkAgPg
  175. 2019-03-20T14:46:18.346Z 1 TID-osw89s50g ProcessNotificationEventWorker JID-8a84e4223b3dc23db67708c5 BID-iKKsdyrPOkAgPg INFO: done: 0.086 sec
  176. 2019-03-20T14:46:18.346Z 1 TID-oswagtopc AuditedWorker JID-ea3b867cbd76f3bc5c495b1b INFO: done: 0.086 sec
  177. 2019-03-20T14:46:18.358Z 1 TID-22sz4o ZyncWorker JID-9ebd4f24e7b0afa09388e97e INFO: done: 0.117 sec
  178. 2019-03-20T14:46:18.359Z 1 TID-osw274ww4 AuditedWorker JID-a9b08b1d360c41f662a302d2 INFO: done: 0.097 sec
  179. 2019-03-20T14:46:18.368Z 1 TID-osw8svayk AuditedWorker JID-6cc27e8fd6f8214227405984 INFO: done: 0.107 sec
  180. Rendered notification_mailer/account_deleted.html.slim within layouts/notification_email (0.3ms)
  181. Rendered notification_mailer/_footer.html.slim (0.3ms)
  182. Rendered notification_mailer/account_deleted.text.erb within layouts/notification_email (0.3ms)
  183. Rendered notification_mailer/_footer.text.erb (0.2ms)
  184.  
  185. Sent mail to admin@3scale.3scale.apps.ocp.api-qe.eng.rdu2.redhat.com (11.1ms)
  186. 2019-03-20T14:46:18.445Z 1 TID-osvyuixvg ProcessNotificationEventWorker::UserNotificationWorker JID-cbec68d0fd378f5eef0ee4d9 BID-iKKsdyrPOkAgPg INFO: done: cbec68d0fd378f5eef0ee4d9 in batch iKKsdyrPOkAgPg
  187. 2019-03-20T14:46:18.447Z 1 TID-osvyuixvg ProcessNotificationEventWorker::UserNotificationWorker JID-cbec68d0fd378f5eef0ee4d9 BID-iKKsdyrPOkAgPg INFO: done: 0.113 sec
  188. 2019-03-20T14:46:18.775Z 1 TID-1vmxik AuditedWorker JID-8f754bd434a0dcf07b5acc7e INFO: start
  189. 2019-03-20T14:46:18.804Z 1 TID-1vmxik AuditedWorker JID-8f754bd434a0dcf07b5acc7e INFO: done: 0.029 sec
  190. 2019-03-20T14:46:19.297Z 1 TID-1zpvks AuditedWorker JID-28a43deab2287ac95ac2dcd4 INFO: start
  191. 2019-03-20T14:46:19.299Z 1 TID-1v5dl4 ZyncWorker JID-5012846a61309d63cdf7b49e INFO: start
  192. 2019-03-20T14:46:19.303Z 1 TID-1wpgew DeleteObjectHierarchyWorker JID-4339631552b1941396c3dfe7 INFO: start
  193. [ActiveJob] [DeleteObjectHierarchyWorker] [806b2857-85ca-4a87-a074-fd2ac9c5b019] Performing DeleteObjectHierarchyWorker from Sidekiq(low) with arguments: gid://system/Service/171
  194. [ActiveJob] [DeleteObjectHierarchyWorker] [806b2857-85ca-4a87-a074-fd2ac9c5b019] Starting DeleteObjectHierarchyWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171"]
  195. 2019-03-20T14:46:19.330Z 1 TID-1zpvks AuditedWorker JID-28a43deab2287ac95ac2dcd4 INFO: done: 0.032 sec
  196. [ActiveJob] [DeleteObjectHierarchyWorker] [806b2857-85ca-4a87-a074-fd2ac9c5b019] Enqueued DeleteObjectHierarchyWorker (Job ID: 2dc33269-5197-4e60-9194-c67fd9dc5bfe) to Sidekiq(low) with arguments: gid://system/ServicePlan/371, ["Hierarchy-Service-171"]
  197. 2019-03-20T14:46:19.339Z 1 TID-175mgo DeleteObjectHierarchyWorker JID-b79cf51ed0e5b8df072b7f3f BID-5OKTPt9p-hNrWQ INFO: start
  198. [ActiveJob] [DeleteObjectHierarchyWorker] [2dc33269-5197-4e60-9194-c67fd9dc5bfe] Performing DeleteObjectHierarchyWorker from Sidekiq(low) with arguments: gid://system/ServicePlan/371, ["Hierarchy-Service-171"]
  199. [ActiveJob] [DeleteObjectHierarchyWorker] [2dc33269-5197-4e60-9194-c67fd9dc5bfe] Starting DeleteObjectHierarchyWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-ServicePlan-371"]
  200. [ActiveJob] [DeleteObjectHierarchyWorker] [806b2857-85ca-4a87-a074-fd2ac9c5b019] Enqueued DeleteObjectHierarchyWorker (Job ID: 79340104-f187-4a2a-a154-263738a0a974) to Sidekiq(low) with arguments: gid://system/Proxy/171, ["Hierarchy-Service-171"]
  201. 2019-03-20T14:46:19.357Z 1 TID-1ve4go DeleteObjectHierarchyWorker JID-919369fb1161141783c9a455 BID-5OKTPt9p-hNrWQ INFO: start
  202. 2019-03-20T14:46:19.364Z 1 TID-1v5dl4 ZyncWorker JID-5012846a61309d63cdf7b49e INFO: done: 0.065 sec
  203. [ActiveJob] [DeleteObjectHierarchyWorker] [79340104-f187-4a2a-a154-263738a0a974] Performing DeleteObjectHierarchyWorker from Sidekiq(low) with arguments: gid://system/Proxy/171, ["Hierarchy-Service-171"]
  204. [ActiveJob] [DeleteObjectHierarchyWorker] [79340104-f187-4a2a-a154-263738a0a974] Starting DeleteObjectHierarchyWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Proxy-171"]
  205. 2019-03-20T14:46:19.369Z 1 TID-1w1hmc DeleteObjectHierarchyWorker JID-a0b5ca51b682235b1351a71b BID-5OKTPt9p-hNrWQ INFO: start
  206. [ActiveJob] [DeleteObjectHierarchyWorker] [806b2857-85ca-4a87-a074-fd2ac9c5b019] Enqueued DeleteObjectHierarchyWorker (Job ID: 1bcbca32-7b41-42f4-8928-a326cea4a64a) to Sidekiq(low) with arguments: gid://system/Metric/199, ["Hierarchy-Service-171"]
  207. [ActiveJob] [DeleteObjectHierarchyWorker] [1bcbca32-7b41-42f4-8928-a326cea4a64a] Performing DeleteObjectHierarchyWorker from Sidekiq(low) with arguments: gid://system/Metric/199, ["Hierarchy-Service-171"]
  208. [ActiveJob] [DeleteObjectHierarchyWorker] [1bcbca32-7b41-42f4-8928-a326cea4a64a] Starting DeleteObjectHierarchyWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Metric-199"]
  209. [ActiveJob] [DeleteObjectHierarchyWorker] [806b2857-85ca-4a87-a074-fd2ac9c5b019] Enqueued DeleteObjectHierarchyWorker (Job ID: 3bc54b4e-6c98-411d-ac34-1ae39f697601) to Sidekiq(low) with arguments: gid://system/ServiceToken/171, ["Hierarchy-Service-171"]
  210. 2019-03-20T14:46:19.378Z 1 TID-1vxods DeleteObjectHierarchyWorker JID-18c99b8a34c663eac04a7e23 BID-5OKTPt9p-hNrWQ INFO: start
  211. [ActiveJob] [DeleteObjectHierarchyWorker] [2dc33269-5197-4e60-9194-c67fd9dc5bfe] Starting DeleteObjectHierarchyWorker#on_complete with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-ServicePlan-371"]
  212. [ActiveJob] [DeleteObjectHierarchyWorker] [806b2857-85ca-4a87-a074-fd2ac9c5b019] DeleteObjectHierarchyWorker#batch_success_callback retry job with the hierarchy of workers: ["Hierarchy-Service-171"]
  213. [ActiveJob] [DeleteObjectHierarchyWorker] [806b2857-85ca-4a87-a074-fd2ac9c5b019] Enqueued DeleteObjectHierarchyWorker (Job ID: 806b2857-85ca-4a87-a074-fd2ac9c5b019) to Sidekiq(low) at 2019-03-20 14:51:19 UTC with arguments: gid://system/Service/171
  214. [ActiveJob] [DeleteObjectHierarchyWorker] [806b2857-85ca-4a87-a074-fd2ac9c5b019] Finished DeleteObjectHierarchyWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171"]
  215. 2019-03-20T14:46:19.387Z 1 TID-24ok64 DeletePlainObjectWorker JID-9172b8ee076446d1e62340c0 BID-5OKTPt9p-hNrWQ INFO: start
  216. [ActiveJob] [DeleteObjectHierarchyWorker] [806b2857-85ca-4a87-a074-fd2ac9c5b019] Performed DeleteObjectHierarchyWorker from Sidekiq(low) in 60.99ms
  217. [ActiveJob] [DeleteObjectHierarchyWorker] [79340104-f187-4a2a-a154-263738a0a974] Enqueued DeleteObjectHierarchyWorker (Job ID: 27095549-7595-43b7-9c6a-fa8ca72abe16) to Sidekiq(low) with arguments: gid://system/ProxyRule/204, ["Hierarchy-Service-171", "Hierarchy-Proxy-171"]
  218. 2019-03-20T14:46:19.387Z 1 TID-1wpgew DeleteObjectHierarchyWorker JID-4339631552b1941396c3dfe7 INFO: done: 0.084 sec
  219. [ActiveJob] [DeleteObjectHierarchyWorker] [3bc54b4e-6c98-411d-ac34-1ae39f697601] Performing DeleteObjectHierarchyWorker from Sidekiq(low) with arguments: gid://system/ServiceToken/171, ["Hierarchy-Service-171"]
  220. [ActiveJob] [DeleteObjectHierarchyWorker] [2dc33269-5197-4e60-9194-c67fd9dc5bfe] Enqueued DeletePlainObjectWorker (Job ID: c9725c18-962f-4235-be92-157f5111eaa1) to Sidekiq(default) with arguments: gid://system/ServicePlan/371, ["Hierarchy-Service-171", "Hierarchy-ServicePlan-371"]
  221. 2019-03-20T14:46:19.388Z 1 TID-22o2qo DeleteObjectHierarchyWorker JID-2b5f5912c7ff283c6d505675 BID-U0vIZEgfT65bAg INFO: start
  222. [ActiveJob] [DeleteObjectHierarchyWorker] [3bc54b4e-6c98-411d-ac34-1ae39f697601] Starting DeleteObjectHierarchyWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-ServiceToken-171"]
  223. [ActiveJob] [DeleteObjectHierarchyWorker] [2dc33269-5197-4e60-9194-c67fd9dc5bfe] Finished DeleteObjectHierarchyWorker#on_complete with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-ServicePlan-371"]
  224. [ActiveJob] [DeleteObjectHierarchyWorker] [79340104-f187-4a2a-a154-263738a0a974] DeleteObjectHierarchyWorker#batch_success_callback retry job with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Proxy-171"]
  225. [ActiveJob] [DeleteObjectHierarchyWorker] [2dc33269-5197-4e60-9194-c67fd9dc5bfe] Finished DeleteObjectHierarchyWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-ServicePlan-371"]
  226. [ActiveJob] [DeletePlainObjectWorker] [c9725c18-962f-4235-be92-157f5111eaa1] Performing DeletePlainObjectWorker from Sidekiq(default) with arguments: gid://system/ServicePlan/371, ["Hierarchy-Service-171", "Hierarchy-ServicePlan-371"]
  227. [ActiveJob] [DeleteObjectHierarchyWorker] [2dc33269-5197-4e60-9194-c67fd9dc5bfe] Performed DeleteObjectHierarchyWorker from Sidekiq(low) in 51.59ms
  228. [ActiveJob] [DeletePlainObjectWorker] [c9725c18-962f-4235-be92-157f5111eaa1] Starting DeletePlainObjectWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-ServicePlan-371", "Plain-ServicePlan-371"]
  229. [ActiveJob] [DeleteObjectHierarchyWorker] [79340104-f187-4a2a-a154-263738a0a974] Enqueued DeleteObjectHierarchyWorker (Job ID: 79340104-f187-4a2a-a154-263738a0a974) to Sidekiq(low) at 2019-03-20 14:51:19 UTC with arguments: gid://system/Proxy/171, ["Hierarchy-Service-171"]
  230. [ActiveJob] [DeleteObjectHierarchyWorker] [79340104-f187-4a2a-a154-263738a0a974] Finished DeleteObjectHierarchyWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Proxy-171"]
  231. 2019-03-20T14:46:19.403Z 1 TID-175mgo DeleteObjectHierarchyWorker JID-b79cf51ed0e5b8df072b7f3f BID-5OKTPt9p-hNrWQ INFO: done: b79cf51ed0e5b8df072b7f3f in batch 5OKTPt9p-hNrWQ
  232. [ActiveJob] [DeleteObjectHierarchyWorker] [79340104-f187-4a2a-a154-263738a0a974] Performed DeleteObjectHierarchyWorker from Sidekiq(low) in 38.56ms
  233. 2019-03-20T14:46:19.405Z 1 TID-175mgo DeleteObjectHierarchyWorker JID-b79cf51ed0e5b8df072b7f3f BID-5OKTPt9p-hNrWQ INFO: done: 0.066 sec
  234. 2019-03-20T14:46:19.406Z 1 TID-1ve4go DeleteObjectHierarchyWorker JID-919369fb1161141783c9a455 BID-5OKTPt9p-hNrWQ INFO: done: 919369fb1161141783c9a455 in batch 5OKTPt9p-hNrWQ
  235. [ActiveJob] [DeleteObjectHierarchyWorker] [3bc54b4e-6c98-411d-ac34-1ae39f697601] Starting DeleteObjectHierarchyWorker#on_complete with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-ServiceToken-171"]
  236. 2019-03-20T14:46:19.406Z 1 TID-1ve4go DeleteObjectHierarchyWorker JID-919369fb1161141783c9a455 BID-5OKTPt9p-hNrWQ INFO: done: 0.049 sec
  237. [ActiveJob] [DeleteObjectHierarchyWorker] [27095549-7595-43b7-9c6a-fa8ca72abe16] Performing DeleteObjectHierarchyWorker from Sidekiq(low) with arguments: gid://system/ProxyRule/204, ["Hierarchy-Service-171", "Hierarchy-Proxy-171"]
  238. [ActiveJob] [DeleteObjectHierarchyWorker] [27095549-7595-43b7-9c6a-fa8ca72abe16] Starting DeleteObjectHierarchyWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Proxy-171", "Hierarchy-ProxyRule-204"]
  239. 2019-03-20T14:46:19.426Z 1 TID-15way8 DeletePlainObjectWorker JID-64828c7bf27350db1e123795 BID-5OKTPt9p-hNrWQ INFO: start
  240. 2019-03-20T14:46:19.427Z 1 TID-16fkfs DeleteObjectHierarchyWorker JID-9debe11f298f37c470ef9ea9 BID-Wj0UQneqewVWAA INFO: start
  241. [ActiveJob] [DeleteObjectHierarchyWorker] [3bc54b4e-6c98-411d-ac34-1ae39f697601] Enqueued DeletePlainObjectWorker (Job ID: 9c68bacd-5586-43be-a973-9d1a9d2cccc4) to Sidekiq(default) with arguments: gid://system/ServiceToken/171, ["Hierarchy-Service-171", "Hierarchy-ServiceToken-171"]
  242. [ActiveJob] [DeleteObjectHierarchyWorker] [3bc54b4e-6c98-411d-ac34-1ae39f697601] Finished DeleteObjectHierarchyWorker#on_complete with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-ServiceToken-171"]
  243. [ActiveJob] [DeleteObjectHierarchyWorker] [3bc54b4e-6c98-411d-ac34-1ae39f697601] Finished DeleteObjectHierarchyWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-ServiceToken-171"]
  244. [ActiveJob] [DeleteObjectHierarchyWorker] [1bcbca32-7b41-42f4-8928-a326cea4a64a] Enqueued DeleteObjectHierarchyWorker (Job ID: 1360f3a3-2ae3-4671-99a6-378ca7b1b209) to Sidekiq(low) with arguments: gid://system/ProxyRule/204, ["Hierarchy-Service-171", "Hierarchy-Metric-199"]
  245. [ActiveJob] [DeleteObjectHierarchyWorker] [3bc54b4e-6c98-411d-ac34-1ae39f697601] Performed DeleteObjectHierarchyWorker from Sidekiq(low) in 38.87ms
  246. [ActiveJob] [DeletePlainObjectWorker] [9c68bacd-5586-43be-a973-9d1a9d2cccc4] Performing DeletePlainObjectWorker from Sidekiq(default) with arguments: gid://system/ServiceToken/171, ["Hierarchy-Service-171", "Hierarchy-ServiceToken-171"]
  247. 2019-03-20T14:46:19.440Z 1 TID-1vxods DeleteObjectHierarchyWorker JID-18c99b8a34c663eac04a7e23 BID-5OKTPt9p-hNrWQ INFO: done: 18c99b8a34c663eac04a7e23 in batch 5OKTPt9p-hNrWQ
  248. 2019-03-20T14:46:19.440Z 1 TID-1vxods DeleteObjectHierarchyWorker JID-18c99b8a34c663eac04a7e23 BID-5OKTPt9p-hNrWQ INFO: done: 0.062 sec
  249. [ActiveJob] [DeleteObjectHierarchyWorker] [27095549-7595-43b7-9c6a-fa8ca72abe16] Starting DeleteObjectHierarchyWorker#on_complete with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Proxy-171", "Hierarchy-ProxyRule-204"]
  250. [ActiveJob] [DeleteObjectHierarchyWorker] [1360f3a3-2ae3-4671-99a6-378ca7b1b209] Performing DeleteObjectHierarchyWorker from Sidekiq(low) with arguments: gid://system/ProxyRule/204, ["Hierarchy-Service-171", "Hierarchy-Metric-199"]
  251. [ActiveJob] [DeleteObjectHierarchyWorker] [1bcbca32-7b41-42f4-8928-a326cea4a64a] DeleteObjectHierarchyWorker#batch_success_callback retry job with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Metric-199"]
  252. 2019-03-20T14:46:19.458Z 1 TID-1xtwiw DeletePlainObjectWorker JID-2912302dc1989b09549a10ed BID-U0vIZEgfT65bAg INFO: start
  253. [ActiveJob] [DeletePlainObjectWorker] [9c68bacd-5586-43be-a973-9d1a9d2cccc4] Starting DeletePlainObjectWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-ServiceToken-171", "Plain-ServiceToken-171"]
  254. [ActiveJob] [DeleteObjectHierarchyWorker] [1360f3a3-2ae3-4671-99a6-378ca7b1b209] Starting DeleteObjectHierarchyWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Metric-199", "Hierarchy-ProxyRule-204"]
  255. [ActiveJob] [DeleteObjectHierarchyWorker] [1bcbca32-7b41-42f4-8928-a326cea4a64a] Enqueued DeleteObjectHierarchyWorker (Job ID: 1bcbca32-7b41-42f4-8928-a326cea4a64a) to Sidekiq(low) at 2019-03-20 14:51:19 UTC with arguments: gid://system/Metric/199, ["Hierarchy-Service-171"]
  256. [ActiveJob] [DeleteObjectHierarchyWorker] [1bcbca32-7b41-42f4-8928-a326cea4a64a] Finished DeleteObjectHierarchyWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Metric-199"]
  257. [ActiveJob] [DeletePlainObjectWorker] [878916b4-1f03-4440-99f8-2fb0c5e18975] Performing DeletePlainObjectWorker from Sidekiq(default) with arguments: gid://system/ProxyRule/204, ["Hierarchy-Service-171", "Hierarchy-Proxy-171", "Hierarchy-ProxyRule-204"]
  258. [ActiveJob] [DeleteObjectHierarchyWorker] [27095549-7595-43b7-9c6a-fa8ca72abe16] Enqueued DeletePlainObjectWorker (Job ID: 878916b4-1f03-4440-99f8-2fb0c5e18975) to Sidekiq(default) with arguments: gid://system/ProxyRule/204, ["Hierarchy-Service-171", "Hierarchy-Proxy-171", "Hierarchy-ProxyRule-204"]
  259. [ActiveJob] [DeleteObjectHierarchyWorker] [1bcbca32-7b41-42f4-8928-a326cea4a64a] Performed DeleteObjectHierarchyWorker from Sidekiq(low) in 89.84ms
  260. [ActiveJob] [DeletePlainObjectWorker] [878916b4-1f03-4440-99f8-2fb0c5e18975] Starting DeletePlainObjectWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Proxy-171", "Hierarchy-ProxyRule-204", "Plain-ProxyRule-204"]
  261. [ActiveJob] [DeleteObjectHierarchyWorker] [27095549-7595-43b7-9c6a-fa8ca72abe16] Finished DeleteObjectHierarchyWorker#on_complete with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Proxy-171", "Hierarchy-ProxyRule-204"]
  262. [ActiveJob] [DeleteObjectHierarchyWorker] [27095549-7595-43b7-9c6a-fa8ca72abe16] Finished DeleteObjectHierarchyWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Proxy-171", "Hierarchy-ProxyRule-204"]
  263. 2019-03-20T14:46:19.478Z 1 TID-1w1hmc DeleteObjectHierarchyWorker JID-a0b5ca51b682235b1351a71b BID-5OKTPt9p-hNrWQ INFO: done: a0b5ca51b682235b1351a71b in batch 5OKTPt9p-hNrWQ
  264. 2019-03-20T14:46:19.478Z 1 TID-1w1hmc DeleteObjectHierarchyWorker JID-a0b5ca51b682235b1351a71b BID-5OKTPt9p-hNrWQ INFO: done: 0.109 sec
  265. [ActiveJob] [DeleteObjectHierarchyWorker] [27095549-7595-43b7-9c6a-fa8ca72abe16] Performed DeleteObjectHierarchyWorker from Sidekiq(low) in 70.63ms
  266. [ActiveJob] [DeletePlainObjectWorker] [9c68bacd-5586-43be-a973-9d1a9d2cccc4] [EventBroker] notifying subscribers of ServiceTokenDeletedEvent 6f688b71-b3a2-451e-9d53-788023a0b97d
  267. 2019-03-20T14:46:19.481Z 1 TID-22o2qo DeleteObjectHierarchyWorker JID-2b5f5912c7ff283c6d505675 BID-U0vIZEgfT65bAg INFO: done: 2b5f5912c7ff283c6d505675 in batch U0vIZEgfT65bAg
  268. [ActiveJob] [DeleteObjectHierarchyWorker] [1360f3a3-2ae3-4671-99a6-378ca7b1b209] Starting DeleteObjectHierarchyWorker#on_complete with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Metric-199", "Hierarchy-ProxyRule-204"]
  269. 2019-03-20T14:46:19.486Z 1 TID-poq00 Sidekiq::Batch::Callback::Worker JID-0e37251e8681c0ae632c8b0a INFO: start
  270. Starting DeleteObjectHierarchyWorker#on_complete with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Proxy-171"]
  271. 2019-03-20T14:46:19.491Z 1 TID-1wrvpo DeletePlainObjectWorker JID-0dd938413f3c19c3ebb0a87a BID-Wj0UQneqewVWAA INFO: start
  272. [ActiveJob] [DeleteObjectHierarchyWorker] [1360f3a3-2ae3-4671-99a6-378ca7b1b209] Enqueued DeletePlainObjectWorker (Job ID: acd086f9-1abf-4d1f-86f8-a949ca787c1a) to Sidekiq(default) with arguments: gid://system/ProxyRule/204, ["Hierarchy-Service-171", "Hierarchy-Metric-199", "Hierarchy-ProxyRule-204"]
  273. [ActiveJob] [DeleteObjectHierarchyWorker] [1360f3a3-2ae3-4671-99a6-378ca7b1b209] Finished DeleteObjectHierarchyWorker#on_complete with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Metric-199", "Hierarchy-ProxyRule-204"]
  274. [ActiveJob] [DeleteObjectHierarchyWorker] [1360f3a3-2ae3-4671-99a6-378ca7b1b209] Finished DeleteObjectHierarchyWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Metric-199", "Hierarchy-ProxyRule-204"]
  275. 2019-03-20T14:46:19.493Z 1 TID-1y29mw Sidekiq::Batch::Callback::Worker JID-dc0a1baeb116403e671365a0 INFO: start
  276. Starting DeleteObjectHierarchyWorker#on_success with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Proxy-171"]
  277. 2019-03-20T14:46:19.502Z 1 TID-1tp08w BackendDeleteServiceTokenWorker JID-aa359af0aa675526375c8a7a BID-5OKTPt9p-hNrWQ INFO: start
  278. [ActiveJob] [DeleteObjectHierarchyWorker] [1360f3a3-2ae3-4671-99a6-378ca7b1b209] Performed DeleteObjectHierarchyWorker from Sidekiq(low) in 43.39ms
  279. 2019-03-20T14:46:19.503Z 1 TID-1vrn7g DeletePlainObjectWorker JID-53e7b86584987cec862ef507 INFO: start
  280. [ActiveJob] [DeletePlainObjectWorker] [acd086f9-1abf-4d1f-86f8-a949ca787c1a] Performing DeletePlainObjectWorker from Sidekiq(default) with arguments: gid://system/ProxyRule/204, ["Hierarchy-Service-171", "Hierarchy-Metric-199", "Hierarchy-ProxyRule-204"]
  281. [ActiveJob] [DeletePlainObjectWorker] [9c68bacd-5586-43be-a973-9d1a9d2cccc4] Finished DeletePlainObjectWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-ServiceToken-171", "Plain-ServiceToken-171"]
  282. [ActiveJob] Enqueued DeletePlainObjectWorker (Job ID: 2cab8fd3-966e-4223-aaee-7680a4f774bb) to Sidekiq(default) with arguments: gid://system/Proxy/171, ["Hierarchy-Service-171", "Hierarchy-Proxy-171"]
  283. 2019-03-20T14:46:19.503Z 1 TID-22o2qo DeleteObjectHierarchyWorker JID-2b5f5912c7ff283c6d505675 BID-U0vIZEgfT65bAg INFO: done: 0.115 sec
  284. 2019-03-20T14:46:19.509Z 1 TID-osw26s8wk DeletePlainObjectWorker JID-9850a290230344d26defa350 INFO: start
  285. 2019-03-20T14:46:19.509Z 1 TID-16fkfs DeleteObjectHierarchyWorker JID-9debe11f298f37c470ef9ea9 BID-Wj0UQneqewVWAA INFO: done: 9debe11f298f37c470ef9ea9 in batch Wj0UQneqewVWAA
  286. [ActiveJob] [DeletePlainObjectWorker] [acd086f9-1abf-4d1f-86f8-a949ca787c1a] Starting DeletePlainObjectWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Metric-199", "Hierarchy-ProxyRule-204", "Plain-ProxyRule-204"]
  287. [ActiveJob] [DeletePlainObjectWorker] [9c68bacd-5586-43be-a973-9d1a9d2cccc4] Performed DeletePlainObjectWorker from Sidekiq(default) in 55.91ms
  288. 2019-03-20T14:46:19.515Z 1 TID-osw89s50g AuditedWorker JID-5f9b32e1bc78ce2cc142fa69 BID-5OKTPt9p-hNrWQ INFO: start
  289. Finished DeleteObjectHierarchyWorker#on_complete with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Proxy-171"]
  290. [ActiveJob] Enqueued DeletePlainObjectWorker (Job ID: b1a77b83-8242-470b-bc12-2a92e5be2de0) to Sidekiq(default) with arguments: gid://system/Proxy/171, ["Hierarchy-Service-171", "Hierarchy-Proxy-171"]
  291. 2019-03-20T14:46:19.527Z 1 TID-oswagtopc Sidekiq::Batch::Callback::Worker JID-0d447e022da3a16d117833e7 INFO: start
  292. 2019-03-20T14:46:19.534Z 1 TID-poq00 Sidekiq::Batch::Callback::Worker JID-0e37251e8681c0ae632c8b0a INFO: done: 0.047 sec
  293. [ActiveJob] [DeletePlainObjectWorker] [c9725c18-962f-4235-be92-157f5111eaa1] Finished DeletePlainObjectWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-ServicePlan-371", "Plain-ServicePlan-371"]
  294. 2019-03-20T14:46:19.534Z 1 TID-22sz4o Sidekiq::Batch::Callback::Worker JID-0270c276a5423fe404a745da INFO: start
  295. 2019-03-20T14:46:19.534Z 1 TID-16fkfs DeleteObjectHierarchyWorker JID-9debe11f298f37c470ef9ea9 BID-Wj0UQneqewVWAA INFO: done: 0.107 sec
  296. [ActiveJob] [DeletePlainObjectWorker] [2cab8fd3-966e-4223-aaee-7680a4f774bb] Performing DeletePlainObjectWorker from Sidekiq(default) with arguments: gid://system/Proxy/171, ["Hierarchy-Service-171", "Hierarchy-Proxy-171"]
  297. [ActiveJob] [DeletePlainObjectWorker] [b1a77b83-8242-470b-bc12-2a92e5be2de0] Performing DeletePlainObjectWorker from Sidekiq(default) with arguments: gid://system/Proxy/171, ["Hierarchy-Service-171", "Hierarchy-Proxy-171"]
  298. 2019-03-20T14:46:19.536Z 1 TID-15way8 DeletePlainObjectWorker JID-64828c7bf27350db1e123795 BID-5OKTPt9p-hNrWQ INFO: done: 64828c7bf27350db1e123795 in batch 5OKTPt9p-hNrWQ
  299. 2019-03-20T14:46:19.555Z 1 TID-15way8 DeletePlainObjectWorker JID-64828c7bf27350db1e123795 BID-5OKTPt9p-hNrWQ INFO: done: 0.129 sec
  300. 2019-03-20T14:46:19.558Z 1 TID-1tp08w BackendDeleteServiceTokenWorker JID-aa359af0aa675526375c8a7a BID-5OKTPt9p-hNrWQ INFO: done: aa359af0aa675526375c8a7a in batch 5OKTPt9p-hNrWQ
  301. [ActiveJob] [DeletePlainObjectWorker] [acd086f9-1abf-4d1f-86f8-a949ca787c1a] Performed DeletePlainObjectWorker from Sidekiq(default) in 20.74ms
  302. 2019-03-20T14:46:19.558Z 1 TID-1tp08w BackendDeleteServiceTokenWorker JID-aa359af0aa675526375c8a7a BID-5OKTPt9p-hNrWQ INFO: done: 0.057 sec
  303. Finished DeleteObjectHierarchyWorker#on_success with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Proxy-171"]
  304. Starting DeleteObjectHierarchyWorker#on_complete with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Metric-199"]
  305. [ActiveJob] [DeletePlainObjectWorker] [c9725c18-962f-4235-be92-157f5111eaa1] Performed DeletePlainObjectWorker from Sidekiq(default) in 137.18ms
  306. Starting DeleteObjectHierarchyWorker#on_success with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Metric-199"]
  307. [ActiveJob] [DeletePlainObjectWorker] [2cab8fd3-966e-4223-aaee-7680a4f774bb] Starting DeletePlainObjectWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Proxy-171", "Plain-Proxy-171"]
  308. [ActiveJob] [DeletePlainObjectWorker] [b1a77b83-8242-470b-bc12-2a92e5be2de0] Starting DeletePlainObjectWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Proxy-171", "Plain-Proxy-171"]
  309. DeletePlainObjectWorker#perform raised ActiveRecord::RecordNotFound with message Couldn't find ProxyRule with 'id'=204
  310. 2019-03-20T14:46:19.565Z 1 TID-24ok64 DeletePlainObjectWorker JID-9172b8ee076446d1e62340c0 BID-5OKTPt9p-hNrWQ INFO: done: 9172b8ee076446d1e62340c0 in batch 5OKTPt9p-hNrWQ
  311. 2019-03-20T14:46:19.565Z 1 TID-1y29mw Sidekiq::Batch::Callback::Worker JID-dc0a1baeb116403e671365a0 INFO: done: 0.072 sec
  312. 2019-03-20T14:46:19.565Z 1 TID-24ok64 DeletePlainObjectWorker JID-9172b8ee076446d1e62340c0 BID-5OKTPt9p-hNrWQ INFO: done: 0.179 sec
  313. [ActiveJob] [DeletePlainObjectWorker] [878916b4-1f03-4440-99f8-2fb0c5e18975] Finished DeletePlainObjectWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Proxy-171", "Hierarchy-ProxyRule-204", "Plain-ProxyRule-204"]
  314. [ActiveJob] [DeletePlainObjectWorker] [878916b4-1f03-4440-99f8-2fb0c5e18975] Performed DeletePlainObjectWorker from Sidekiq(default) in 103.08ms
  315. 2019-03-20T14:46:19.568Z 1 TID-osw274ww4 DeletePlainObjectWorker JID-5b485fbfe41aa655769bb306 INFO: start
  316. [ActiveJob] Enqueued DeletePlainObjectWorker (Job ID: 1ddd55e5-0488-4dce-b80e-9713a8d79c41) to Sidekiq(default) with arguments: gid://system/Metric/199, ["Hierarchy-Service-171", "Hierarchy-Metric-199"]
  317. [ActiveJob] Enqueued DeletePlainObjectWorker (Job ID: a0ef90fd-a605-4dd0-a3c7-37631e7dcdc9) to Sidekiq(default) with arguments: gid://system/Metric/199, ["Hierarchy-Service-171", "Hierarchy-Metric-199"]
  318. 2019-03-20T14:46:19.569Z 1 TID-1wrvpo DeletePlainObjectWorker JID-0dd938413f3c19c3ebb0a87a BID-Wj0UQneqewVWAA INFO: done: 0dd938413f3c19c3ebb0a87a in batch Wj0UQneqewVWAA
  319. Finished DeleteObjectHierarchyWorker#on_complete with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Metric-199"]
  320. 2019-03-20T14:46:19.575Z 1 TID-osw89s50g AuditedWorker JID-5f9b32e1bc78ce2cc142fa69 BID-5OKTPt9p-hNrWQ INFO: done: 5f9b32e1bc78ce2cc142fa69 in batch 5OKTPt9p-hNrWQ
  321. Finished DeleteObjectHierarchyWorker#on_success with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Metric-199"]
  322. 2019-03-20T14:46:19.575Z 1 TID-osw8svayk DeletePlainObjectWorker JID-f7840e9acfee04d94e139479 INFO: start
  323. [ActiveJob] [DeletePlainObjectWorker] [1ddd55e5-0488-4dce-b80e-9713a8d79c41] Performing DeletePlainObjectWorker from Sidekiq(default) with arguments: gid://system/Metric/199, ["Hierarchy-Service-171", "Hierarchy-Metric-199"]
  324. [ActiveJob] [DeletePlainObjectWorker] [1ddd55e5-0488-4dce-b80e-9713a8d79c41] Starting DeletePlainObjectWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Metric-199", "Plain-Metric-199"]
  325. 2019-03-20T14:46:19.577Z 1 TID-1xtwiw DeletePlainObjectWorker JID-2912302dc1989b09549a10ed BID-U0vIZEgfT65bAg INFO: done: 2912302dc1989b09549a10ed in batch U0vIZEgfT65bAg
  326. 2019-03-20T14:46:19.578Z 1 TID-1wrvpo DeletePlainObjectWorker JID-0dd938413f3c19c3ebb0a87a BID-Wj0UQneqewVWAA INFO: done: 0.086 sec
  327. 2019-03-20T14:46:19.578Z 1 TID-osw89s50g AuditedWorker JID-5f9b32e1bc78ce2cc142fa69 BID-5OKTPt9p-hNrWQ INFO: done: 0.063 sec
  328. [ActiveJob] [DeletePlainObjectWorker] [a0ef90fd-a605-4dd0-a3c7-37631e7dcdc9] Performing DeletePlainObjectWorker from Sidekiq(default) with arguments: gid://system/Metric/199, ["Hierarchy-Service-171", "Hierarchy-Metric-199"]
  329. 2019-03-20T14:46:19.580Z 1 TID-1xtwiw DeletePlainObjectWorker JID-2912302dc1989b09549a10ed BID-U0vIZEgfT65bAg INFO: done: 0.122 sec
  330. [ActiveJob] [DeletePlainObjectWorker] [a0ef90fd-a605-4dd0-a3c7-37631e7dcdc9] Starting DeletePlainObjectWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Metric-199", "Plain-Metric-199"]
  331. 2019-03-20T14:46:19.581Z 1 TID-22sz4o Sidekiq::Batch::Callback::Worker JID-0270c276a5423fe404a745da INFO: done: 0.047 sec
  332. 2019-03-20T14:46:19.584Z 1 TID-oswagtopc Sidekiq::Batch::Callback::Worker JID-0d447e022da3a16d117833e7 INFO: done: 0.057 sec
  333. [ActiveJob] [DeletePlainObjectWorker] [2cab8fd3-966e-4223-aaee-7680a4f774bb] Performed DeletePlainObjectWorker from Sidekiq(default) in 64.07ms
  334. DeletePlainObjectWorker#perform raised ActiveRecord::StaleObjectError with message Attempted to destroy a stale object: Proxy for the hierarchy ["Hierarchy-Service-171", "Hierarchy-Proxy-171", "Plain-Proxy-171"]
  335. 2019-03-20T14:46:19.621Z 1 TID-1vrn7g DeletePlainObjectWorker JID-53e7b86584987cec862ef507 INFO: done: 0.118 sec
  336. 2019-03-20T14:46:19.640Z 1 TID-osvyuixvg BackendMetricWorker JID-9e8edf507e552dac60dabfbf INFO: start
  337. 2019-03-20T14:46:19.642Z 1 TID-1vmxik BackendMetricWorker JID-ae47ea3dbfdd08e67f20abe7 INFO: start
  338. [ActiveJob] [DeletePlainObjectWorker] [b1a77b83-8242-470b-bc12-2a92e5be2de0] Finished DeletePlainObjectWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Proxy-171", "Plain-Proxy-171"]
  339. [ActiveJob] [DeletePlainObjectWorker] [b1a77b83-8242-470b-bc12-2a92e5be2de0] Performed DeletePlainObjectWorker from Sidekiq(default) in 85.36ms
  340. 2019-03-20T14:46:19.646Z 1 TID-1zpvks AuditedWorker JID-c15364217cebfa96f6fb40ec INFO: start
  341. 2019-03-20T14:46:19.647Z 1 TID-osw26s8wk DeletePlainObjectWorker JID-9850a290230344d26defa350 INFO: done: 0.139 sec
  342. 2019-03-20T14:46:19.650Z 1 TID-1v5dl4 AuditedWorker JID-d7ca321443fe8a79f0978970 INFO: start
  343. 2019-03-20T14:46:19.654Z 1 TID-1vmxik BackendMetricWorker JID-ae47ea3dbfdd08e67f20abe7 INFO: fail: 0.012 sec
  344. 2019-03-20T14:46:19.664Z 1 TID-1vmxik WARN: {"context":"Job raised exception","job":{"class":"BackendMetricWorker","args":["171",199,"hits"],"retry":3,"queue":"backend_sync","lock":{"timeout":3600000,"name":"#<Proc:0x000000000d4a77e0@/opt/system/app/workers/backend_metric_worker.rb:13>"},"jid":"ae47ea3dbfdd08e67f20abe7","created_at":1553093179.6397812,"enqueued_at":1553093179.6398344,"error_message":"BackendMetricWorker::LockError","error_class":"BackendMetricWorker::LockError","failed_at":1553093179.6632278,"retry_count":0},"jobstr":"{\"class\":\"BackendMetricWorker\",\"args\":[\"171\",199,\"hits\"],\"retry\":3,\"queue\":\"backend_sync\",\"lock\":{\"timeout\":3600000,\"name\":\"#<Proc:0x000000000d4a77e0@/opt/system/app/workers/backend_metric_worker.rb:13>\"},\"jid\":\"ae47ea3dbfdd08e67f20abe7\",\"created_at\":1553093179.6397812,\"enqueued_at\":1553093179.6398344}"}
  345. 2019-03-20T14:46:19.666Z 1 TID-1vmxik WARN: BackendMetricWorker::LockError: BackendMetricWorker::LockError
  346. 2019-03-20T14:46:19.668Z 1 TID-1vmxik WARN: /opt/system/app/workers/backend_metric_worker.rb:28:in `perform'
  347. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb:167:in `execute_job'
  348. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb:139:in `block (5 levels) in process'
  349. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq.rb:36:in `block in <module:Sidekiq>'
  350. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb:135:in `block (4 levels) in process'
  351. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
  352. /opt/system/app/lib/three_scale/sidekiq_retry_support.rb:54:in `call'
  353. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
  354. /opt/system/app/lib/three_scale/analytics/sidekiq_middleware.rb:5:in `call'
  355. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
  356. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/server/active_record.rb:6:in `call'
  357. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
  358. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-batch-0.1.5/lib/sidekiq/batch/middleware.rb:30:in `call'
  359. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
  360. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-lock-0.3.1/lib/sidekiq/lock/middleware.rb:8:in `call'
  361. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
  362. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/server/logging.rb:10:in `call'
  363. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
  364. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/server/retry_jobs.rb:74:in `call'
  365. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
  366. /opt/system/vendor/bundle/ruby/2.3.0/gems/bugsnag-6.6.4/lib/bugsnag/integrations/sidekiq.rb:23:in `call'
  367. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
  368. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb:133:in `invoke'
  369. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb:134:in `block (3 levels) in process'
  370. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/logging.rb:32:in `with_context'
  371. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb:132:in `block (2 levels) in process'
  372. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb:183:in `stats'
  373. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb:131:in `block in process'
  374. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq.rb:35:in `block in <module:Sidekiq>'
  375. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb:126:in `process'
  376. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb:82:in `process_one'
  377. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb:70:in `run'
  378. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/util.rb:17:in `watchdog'
  379. /opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/util.rb:26:in `block in safe_thread'
  380. [ActiveJob] [DeletePlainObjectWorker] [a0ef90fd-a605-4dd0-a3c7-37631e7dcdc9] Finished DeletePlainObjectWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Metric-199", "Plain-Metric-199"]
  381. Exception -- {:exception=>{:class=>BackendMetricWorker::LockError, :message=>"BackendMetricWorker::LockError", :backtrace=>["/opt/system/app/workers/backend_metric_worker.rb:28:in `perform'", "/opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb:167:in `execute_job'", "/opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb:139:in `block (5 levels) in process'", "/opt/system/vendor/bundle/ruby/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq.rb:36:in `block in <module:Sidekiq>'"]}, :parameters=>{:context=>"Job raised exception", :job=>{"class"=>"BackendMetricWorker", "args"=>["171", 199, "hits"], "retry"=>3, "queue"=>"backend_sync", "lock"=>{"timeout"=>3600000, "name"=>"#<Proc:0x000000000d4a77e0@/opt/system/app/workers/backend_metric_worker.rb:13>"}, "jid"=>"ae47ea3dbfdd08e67f20abe7", "created_at"=>1553093179.6397812, "enqueued_at"=>1553093179.6398344, "error_message"=>"BackendMetricWorker::LockError", "error_class"=>"BackendMetricWorker::LockError", "failed_at"=>1553093179.6632278, "retry_count"=>0}, :jobstr=>"{\"class\":\"BackendMetricWorker\",\"args\":[\"171\",199,\"hits\"],\"retry\":3,\"queue\":\"backend_sync\",\"lock\":{\"timeout\":3600000,\"name\":\"#<Proc:0x000000000d4a77e0@/opt/system/app/workers/backend_metric_worker.rb:13>\"},\"jid\":\"ae47ea3dbfdd08e67f20abe7\",\"created_at\":1553093179.6397812,\"enqueued_at\":1553093179.6398344}"}}
  382. [ActiveJob] [DeletePlainObjectWorker] [a0ef90fd-a605-4dd0-a3c7-37631e7dcdc9] Performed DeletePlainObjectWorker from Sidekiq(default) in 88.2ms
  383. 2019-03-20T14:46:19.673Z 1 TID-osw8svayk DeletePlainObjectWorker JID-f7840e9acfee04d94e139479 INFO: done: 0.099 sec
  384. 2019-03-20T14:46:19.681Z 1 TID-1zpvks AuditedWorker JID-c15364217cebfa96f6fb40ec INFO: done: 0.034 sec
  385. 2019-03-20T14:46:19.683Z 1 TID-osvyuixvg BackendMetricWorker JID-9e8edf507e552dac60dabfbf INFO: done: 0.043 sec
  386. [ActiveJob] [DeletePlainObjectWorker] [1ddd55e5-0488-4dce-b80e-9713a8d79c41] Finished DeletePlainObjectWorker#perform with the hierarchy of workers: ["Hierarchy-Service-171", "Hierarchy-Metric-199", "Plain-Metric-199"]
  387. [ActiveJob] [DeletePlainObjectWorker] [1ddd55e5-0488-4dce-b80e-9713a8d79c41] Performed DeletePlainObjectWorker from Sidekiq(default) in 108.14ms
  388. 2019-03-20T14:46:19.688Z 1 TID-osw274ww4 DeletePlainObjectWorker JID-5b485fbfe41aa655769bb306 INFO: done: 0.121 sec
  389. 2019-03-20T14:46:19.694Z 1 TID-1v5dl4 AuditedWorker JID-d7ca321443fe8a79f0978970 INFO: done: 0.045 sec
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!