API tools faq
paste
Advertisement
Guest User

ccc deploy

a guest
May 6th, 2021
68
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 25.12 KB | None | 0 0
raw download clone embed print report
  1. [197.x.x.x] sudo: /home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/bin/python manage.py migrate_multi --noinput
  2. [197.x.x.x] out: 2021-05-06 13:39:27,197 INFO synced 'auditcare' in couchdb
  3. [197.x.x.x] out: 2021-05-06 13:39:27,222 INFO synced 'case' in couchdb
  4. [197.x.x.x] out: 2021-05-06 13:39:27,244 INFO synced 'domain' in couchdb
  5. [197.x.x.x] out: 2021-05-06 13:39:27,266 INFO synced 'hqadmin' in couchdb
  6. [197.x.x.x] out: 2021-05-06 13:39:27,291 INFO synced 'hqmedia' in couchdb
  7. [197.x.x.x] out: 2021-05-06 13:39:27,312 INFO synced 'couchforms' in couchdb
  8. [197.x.x.x] out: 2021-05-06 13:39:27,333 INFO synced 'app_manager' in couchdb
  9. [197.x.x.x] out: 2021-05-06 13:39:27,351 INFO synced 'fixtures' in couchdb
  10. [197.x.x.x] out: 2021-05-06 13:39:27,374 INFO synced 'users' in couchdb
  11. [197.x.x.x] out: 2021-05-06 13:39:27,393 INFO synced 'groups' in couchdb
  12. [197.x.x.x] out: 2021-05-06 13:39:27,411 INFO synced 'mobile_auth' in couchdb
  13. [197.x.x.x] out: 2021-05-06 13:39:27,433 INFO synced 'userreports' in couchdb
  14. [197.x.x.x] out: 2021-05-06 13:39:27,449 INFO synced 'builds' in couchdb
  15. [197.x.x.x] out: 2021-05-06 13:39:27,760 INFO synced 'attachments' in couchdb
  16. [197.x.x.x] out: 2021-05-06 13:39:27,774 INFO synced 'case_types_by_domain' in couchdb
  17. [197.x.x.x] out: 2021-05-06 13:39:27,786 INFO synced 'reportconfig' in couchdb
  18. [197.x.x.x] out: 2021-05-06 13:39:27,798 INFO synced 'users_extra' in couchdb
  19. [197.x.x.x] out: 2021-05-06 13:39:27,810 INFO synced 'deleted_data' in couchdb
  20. [197.x.x.x] out: 2021-05-06 13:39:27,825 INFO synced 'case_indices' in couchdb
  21. [197.x.x.x] out: 2021-05-06 13:39:27,838 INFO synced 'export_instances_by_is_daily_saved' in couchdb
  22. [197.x.x.x] out: 2021-05-06 13:39:27,852 INFO synced 'cases_by_server_date' in couchdb
  23. [197.x.x.x] out: 2021-05-06 13:39:27,865 INFO synced 'deleted_users_by_username' in couchdb
  24. [197.x.x.x] out: 2021-05-06 13:39:27,879 INFO synced 'all_docs' in couchdb
  25. [197.x.x.x] out: 2021-05-06 13:39:27,891 INFO synced 'all_docs' in couchdb
  26. [197.x.x.x] out: 2021-05-06 13:39:27,902 INFO synced 'all_docs' in couchdb
  27. [197.x.x.x] out: 2021-05-06 13:39:27,913 INFO synced 'all_docs' in couchdb
  28. [197.x.x.x] out: 2021-05-06 13:39:27,924 INFO synced 'all_docs' in couchdb
  29. [197.x.x.x] out: 2021-05-06 13:39:27,936 INFO synced 'all_docs' in couchdb
  30. [197.x.x.x] out: 2021-05-06 13:39:27,949 INFO synced 'exports_forms_by_app' in couchdb
  31. [197.x.x.x] out: 2021-05-06 13:39:27,961 INFO synced 'supply_point_by_loc' in couchdb
  32. [197.x.x.x] out: 2021-05-06 13:39:27,972 INFO synced 'export_instances_by_domain' in couchdb
  33. [197.x.x.x] out: 2021-05-06 13:39:27,986 INFO synced 'case_by_domain_hq_user_id_type' in couchdb
  34. [197.x.x.x] out: 2021-05-06 13:39:27,999 INFO synced 'program_by_code' in couchdb
  35. [197.x.x.x] out: 2021-05-06 13:39:28,013 INFO synced 'tests' in couchdb
  36. [197.x.x.x] out: 2021-05-06 13:39:28,029 INFO synced 'forms_by_app_info' in couchdb
  37. [197.x.x.x] out: 2021-05-06 13:39:28,050 INFO synced 'open_cases' in couchdb
  38. [197.x.x.x] out: 2021-05-06 13:39:28,071 INFO synced 'cases_by_owner' in couchdb
  39. [197.x.x.x] out: 2021-05-06 13:39:28,092 INFO synced 'app_translations_by_popularity' in couchdb
  40. [197.x.x.x] out: 2021-05-06 13:39:28,109 INFO synced 'exports_forms_by_xform' in couchdb
  41. [197.x.x.x] out: 2021-05-06 13:39:28,121 INFO synced 'doc_conflicts' in couchdb
  42. [197.x.x.x] out: 2021-05-06 13:39:28,133 INFO synced 'schemas_by_xmlns_or_case_type' in couchdb
  43. [197.x.x.x] out: 2021-05-06 13:39:28,149 INFO synced 'apps_with_submissions' in couchdb
  44. [197.x.x.x] out: 2021-05-06 13:39:28,190 INFO synced 'cases_by_domain_external_id' in couchdb
  45. [197.x.x.x] out: 2021-05-06 13:39:28,205 INFO synced 'form_case_index' in couchdb
  46. [197.x.x.x] out: 2021-05-06 13:39:28,220 INFO synced 'all_forms' in couchdb
  47. [197.x.x.x] out: 2021-05-06 13:39:28,232 INFO synced 'saved_apps_auto_generated' in couchdb
  48. [197.x.x.x] out: 2021-05-06 13:39:28,245 INFO synced 'by_domain_doc_type_date' in couchdb
  49. [197.x.x.x] out: 2021-05-06 13:39:28,258 INFO synced 'by_domain_doc_type_date' in couchdb
  50. [197.x.x.x] out: 2021-05-06 13:39:28,270 INFO synced 'by_domain_doc_type_date' in couchdb
  51. [197.x.x.x] out: 2021-05-06 13:39:28,282 INFO synced 'by_domain_doc_type_date' in couchdb
  52. [197.x.x.x] out: 2021-05-06 13:39:28,301 INFO synced 'by_domain_doc_type_date' in couchdb
  53. [197.x.x.x] out: 2021-05-06 13:39:28,316 INFO synced 'by_domain_doc_type_date' in couchdb
  54. [197.x.x.x] out: 2021-05-06 13:39:28,328 INFO synced 'last_modified' in couchdb
  55. [197.x.x.x] out: 2021-05-06 13:39:28,346 INFO synced 'last_modified' in couchdb
  56. [197.x.x.x] out: 2021-05-06 13:39:28,358 INFO synced 'last_modified' in couchdb
  57. [197.x.x.x] out: 2021-05-06 13:39:28,384 INFO synced 'fluff' in couchdb
  58. [197.x.x.x] out: 2021-05-06 13:39:28,395 INFO synced 'fluff' in couchdb
  59. [197.x.x.x] out: 2021-05-06 13:39:28,415 INFO synced 'fluff_filter' in couchdb
  60. [197.x.x.x] out: 2021-05-06 13:39:28,440 INFO synced 'repeaters' in couchdb
  61. [197.x.x.x] out: 2021-05-06 13:39:28,469 INFO synced 'pact' in couchdb
  62. [197.x.x.x] out: Traceback (most recent call last):
  63. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/apps/config.py", line 178, in get_model
  64. [197.x.x.x] out: return self.models[model_name.lower()]
  65. [197.x.x.x] out: KeyError: 'apikeysettings'
  66. [197.x.x.x] out:
  67. [197.x.x.x] out: During handling of the above exception, another exception occurred:
  68. [197.x.x.x] out:
  69. [197.x.x.x] out: Traceback (most recent call last):
  70. [197.x.x.x] out: File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run
  71. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/corehq/sql_db/management/commands/migrate_multi.py", line 41, in migrate_db
  72. [197.x.x.x] out: call_command('migrate', *args, **call_options)
  73. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/core/management/__init__.py", line 148, in call_command
  74. [197.x.x.x] out: return command.execute(*args, **defaults)
  75. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/core/management/base.py", line 364, in execute
  76. [197.x.x.x] out: output = self.handle(*args, **options)
  77. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/core/management/base.py", line 83, in wrapped
  78. [197.x.x.x] out: res = handle_func(*args, **kwargs)
  79. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 234, in handle
  80. [197.x.x.x] out: fake_initial=fake_initial,
  81. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/db/migrations/executor.py", line 117, in migrate
  82. [197.x.x.x] out: state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  83. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
  84. [197.x.x.x] out: state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  85. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
  86. [197.x.x.x] out: state = migration.apply(state, schema_editor)
  87. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/db/migrations/migration.py", line 124, in apply
  88. [197.x.x.x] out: operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  89. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/db/migrations/operations/special.py", line 190, in database_forwards
  90. [197.x.x.x] out: self.code(from_state.apps, schema_editor)
  91. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/corehq/util/django_migrations.py", line 103, in _inner
  92. [197.x.x.x] out: return migration_fn(*args, **kwargs)
  93. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/corehq/apps/users/migrations/0016_hqapikey.py", line 16, in populate_api_keys
  94. [197.x.x.x] out: ApiKeySettings = apps.get_model('hqwebapp', 'ApiKeySettings')
  95. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/apps/registry.py", line 210, in get_model
  96. [197.x.x.x] out: return app_config.get_model(model_name, require_ready=require_ready)
  97. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/apps/config.py", line 181, in get_model
  98. [197.x.x.x] out: "App '%s' doesn't have a '%s' model." % (self.label, model_name))
  99. [197.x.x.x] out: LookupError: App 'hqwebapp' doesn't have a 'ApiKeySettings' model.
  100. [197.x.x.x] out: 2021-05-06T13:39:28Z <Greenlet at 0x7f483012f948: migrate_db('default')> failed with LookupError
  101. [197.x.x.x] out:
  102. [197.x.x.x] out:
  103. [197.x.x.x] out: The following databases will be migrated:
  104. [197.x.x.x] out: * default
  105. [197.x.x.x] out: * p1
  106. [197.x.x.x] out: * p2
  107. [197.x.x.x] out: * proxy
  108. [197.x.x.x] out: * synclogs
  109. [197.x.x.x] out:
  110. [197.x.x.x] out:
  111. [197.x.x.x] out: The following databases will be skipped:
  112. [197.x.x.x] out: * ucr
  113. [197.x.x.x] out:
  114. [197.x.x.x] out:
  115. [197.x.x.x] out: ======================= Error During Migration =======================
  116. [197.x.x.x] out: <Greenlet at 0x7f483012f948: migrate_db('default')>
  117. [197.x.x.x] out: Traceback (most recent call last):
  118. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/apps/config.py", line 178, in get_model
  119. [197.x.x.x] out: return self.models[model_name.lower()]
  120. [197.x.x.x] out: KeyError: 'apikeysettings'
  121. [197.x.x.x] out:
  122. [197.x.x.x] out: During handling of the above exception, another exception occurred:
  123. [197.x.x.x] out:
  124. [197.x.x.x] out: Traceback (most recent call last):
  125. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/corehq/sql_db/management/commands/migrate_multi.py", line 64, in handle
  126. [197.x.x.x] out: job.get()
  127. [197.x.x.x] out: File "src/gevent/greenlet.py", line 683, in gevent._greenlet.Greenlet.get
  128. [197.x.x.x] out: File "src/gevent/greenlet.py", line 317, in gevent._greenlet.Greenlet._raise_exception
  129. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/gevent/_compat.py", line 47, in reraise
  130. [197.x.x.x] out: raise value.with_traceback(tb)
  131. [197.x.x.x] out: File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run
  132. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/corehq/sql_db/management/commands/migrate_multi.py", line 41, in migrate_db
  133. [197.x.x.x] out: call_command('migrate', *args, **call_options)
  134. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/core/management/__init__.py", line 148, in call_command
  135. [197.x.x.x] out: return command.execute(*args, **defaults)
  136. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/core/management/base.py", line 364, in execute
  137. [197.x.x.x] out: output = self.handle(*args, **options)
  138. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/core/management/base.py", line 83, in wrapped
  139. [197.x.x.x] out: res = handle_func(*args, **kwargs)
  140. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 234, in handle
  141. [197.x.x.x] out: fake_initial=fake_initial,
  142. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/db/migrations/executor.py", line 117, in migrate
  143. [197.x.x.x] out: state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  144. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
  145. [197.x.x.x] out: state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  146. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
  147. [197.x.x.x] out: state = migration.apply(state, schema_editor)
  148. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/db/migrations/migration.py", line 124, in apply
  149. [197.x.x.x] out: operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  150. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/db/migrations/operations/special.py", line 190, in database_forwards
  151. [197.x.x.x] out: self.code(from_state.apps, schema_editor)
  152. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/corehq/util/django_migrations.py", line 103, in _inner
  153. [197.x.x.x] out: return migration_fn(*args, **kwargs)
  154. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/corehq/apps/users/migrations/0016_hqapikey.py", line 16, in populate_api_keys
  155. [197.x.x.x] out: ApiKeySettings = apps.get_model('hqwebapp', 'ApiKeySettings')
  156. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/apps/registry.py", line 210, in get_model
  157. [197.x.x.x] out: return app_config.get_model(model_name, require_ready=require_ready)
  158. [197.x.x.x] out: File "/home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/lib/python3.6/site-packages/django/apps/config.py", line 181, in get_model
  159. [197.x.x.x] out: "App '%s' doesn't have a '%s' model." % (self.label, model_name))
  160. [197.x.x.x] out: LookupError: App 'hqwebapp' doesn't have a 'ApiKeySettings' model.
  161. [197.x.x.x] out:
  162. [197.x.x.x] out:
  163.  
  164.  
  165. Fatal error: sudo() received nonzero return code 1 while executing!
  166.  
  167. Requested: /home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/bin/python manage.py migrate_multi --noinput
  168. Executed: sudo -S -p 'sudo password:' -H -u "cchq" /bin/bash -l -c "cd /home/cchq/www/monolith/releases/2021-05-06_13.26 >/dev/null && /home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/bin/python manage.py migrate_multi --noinput"
  169.  
  170. Aborting.
  171. [197.x.x.x] Executing task 'send_email'
  172. [197.x.x.x] sudo: /home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/bin/python manage.py send_email --to-admins --subject 'Deploy to monolith failed. Try resuming with fab monolith deploy:resume=yes.' 'Traceback:
  173. File "/home/ccc/commcare-cloud/src/commcare_cloud/fab/fabfile.py", line 406, in _deploy_without_asking
  174. deploy_checkpoint(index, command.__name__, execute_with_timing, command)
  175. File "/home/ccc/commcare-cloud/src/commcare_cloud/fab/fabfile.py", line 399, in deploy_checkpoint
  176. fn(*args, **kwargs)
  177. File "/home/ccc/commcare-cloud/src/commcare_cloud/fab/utils.py", line 25, in execute_with_timing
  178. execute(fn, *args, **kwargs)
  179. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/tasks.py", line 385, in execute
  180. multiprocessing
  181. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/tasks.py", line 275, in _execute
  182. return task.run(*args, **kwargs)
  183. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/tasks.py", line 172, in run
  184. return self.wrapped(*args, **kwargs)
  185. File "/home/ccc/commcare-cloud/src/commcare_cloud/fab/fabfile.py", line 389, in conditionally_stop_pillows_and_celery_during_migrate
  186. execute_with_timing(db.migrate)
  187. File "/home/ccc/commcare-cloud/src/commcare_cloud/fab/utils.py", line 25, in execute_with_timing
  188. execute(fn, *args, **kwargs)
  189. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/tasks.py", line 385, in execute
  190. multiprocessing
  191. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/tasks.py", line 275, in _execute
  192. return task.run(*args, **kwargs)
  193. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/tasks.py", line 172, in run
  194. return self.wrapped(*args, **kwargs)
  195. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/decorators.py", line 52, in inner_decorator
  196. return func(*args, **kwargs)
  197. File "/home/ccc/commcare-cloud/src/commcare_cloud/fab/operations/db.py", line 96, in migrate
  198. sudo('"'"'%(py3_virtualenv_root)s/bin/python manage.py migrate_multi --noinput'"'"' % env)
  199. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/network.py", line 654, in host_prompting_wrapper
  200. return func(*args, **kwargs)
  201. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/operations.py", line 1121, in sudo
  202. stderr=stderr, timeout=timeout, shell_escape=shell_escape,
  203. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/operations.py", line 952, in _run_command
  204. error(message=msg, stdout=out, stderr=err)
  205. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/utils.py", line 359, in error
  206. return func(message)
  207. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/utils.py", line 55, in abort
  208. raise env.abort_exception(msg)
  209. Exception: sudo() received nonzero return code 1 while executing!
  210.  
  211. Requested: /home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/bin/python manage.py migrate_multi --noinput
  212. Executed: sudo -S -p '"'"'sudo password:'"'"' -H -u "cchq" /bin/bash -l -c "cd /home/cchq/www/monolith/releases/2021-05-06_13.26 >/dev/null && /home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/bin/python manage.py migrate_multi --noinput"'
  213. [197.x.x.x] Executing task 'set_in_progress_flag'
  214. [197.x.x.x] sudo: /home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/bin/python manage.py deploy_in_progress
  215. [197.x.x.x] Executing task 'restart_webworkers'
  216. [197.x.x.x] sudo: supervisorctl stop all
  217. [197.x.x.x] out: commcare-hq-monolith-errand-boy: stopped
  218. [197.x.x.x] out: commcare-hq-monolith-form_feed: stopped
  219. [197.x.x.x] out: commcare-hq-monolith-django: stopped
  220. [197.x.x.x] out: commcare-hq-monolith-formsplayer-spring: stopped
  221. [197.x.x.x] out: commcare-hq-monolith-pillow_retry_queue: stopped
  222. [197.x.x.x] out:
  223.  
  224. [197.x.x.x] sudo: supervisorctl reread
  225. [197.x.x.x] out: No config updates to processes
  226. [197.x.x.x] out:
  227.  
  228. [197.x.x.x] sudo: supervisorctl update
  229. [197.x.x.x] sudo: supervisorctl start all
  230. [197.x.x.x] out: commcare-hq-monolith-pillow_retry_queue: started
  231. [197.x.x.x] out: commcare-hq-monolith-form_feed: started
  232. [197.x.x.x] out: commcare-hq-monolith-errand-boy: started
  233. [197.x.x.x] out: commcare-hq-monolith-django: started
  234. [197.x.x.x] out: commcare-hq-monolith-pillowtop-GroupPillow-0: ERROR (spawn error)
  235. [197.x.x.x] out: commcare-hq-monolith-pillowtop-UserGroupsDbKafkaPillow-0: ERROR (spawn error)
  236. [197.x.x.x] out: commcare-hq-monolith-pillowtop-KafkaDomainPillow-0: ERROR (spawn error)
  237. [197.x.x.x] out: commcare-hq-monolith-pillowtop-XFormToElasticsearchPillow-0: ERROR (spawn error)
  238. [197.x.x.x] out: commcare-hq-monolith-pillowtop-UserPillow-0: ERROR (spawn error)
  239. [197.x.x.x] out: commcare-hq-monolith-pillowtop-UnknownUsersPillow-0: ERROR (spawn error)
  240. [197.x.x.x] out: commcare-hq-monolith-pillowtop-kafka-ucr-static-0: ERROR (spawn error)
  241. [197.x.x.x] out: commcare-hq-monolith-pillowtop-UserCacheInvalidatePillow-0: ERROR (spawn error)
  242. [197.x.x.x] out: commcare-hq-monolith-pillowtop-CaseToElasticsearchPillow-0: ERROR (spawn error)
  243. [197.x.x.x] out: commcare-hq-monolith-pillowtop-GroupToUserPillow-0: ERROR (spawn error)
  244. [197.x.x.x] out: commcare-hq-monolith-pillowtop-UpdateUserSyncHistoryPillow-0: ERROR (spawn error)
  245. [197.x.x.x] out: commcare-hq-monolith-pillowtop-DefaultChangeFeedPillow-0: ERROR (spawn error)
  246. [197.x.x.x] out: commcare-hq-monolith-pillowtop-LedgerToElasticsearchPillow-0: ERROR (spawn error)
  247. [197.x.x.x] out: commcare-hq-monolith-pillowtop-ApplicationToElasticsearchPillow-0: ERROR (spawn error)
  248. [197.x.x.x] out: commcare-hq-monolith-pillowtop-AppDbChangeFeedPillow-0: ERROR (spawn error)
  249. [197.x.x.x] out: commcare-hq-monolith-pillowtop-FormSubmissionMetadataTrackerPillow-0: ERROR (spawn error)
  250. [197.x.x.x] out: commcare-hq-monolith-pillowtop-CacheInvalidatePillow-0: ERROR (spawn error)
  251. [197.x.x.x] out: commcare-hq-monolith-pillowtop-DomainDbKafkaPillow-0: ERROR (spawn error)
  252. [197.x.x.x] out: commcare-hq-monolith-pillowtop-SqlSMSPillow-0: ERROR (spawn error)
  253. [197.x.x.x] out: commcare-hq-monolith-celerybeat: ERROR (spawn error)
  254. [197.x.x.x] out: commcare-hq-monolith-celery_repeat_record_queue,celery,case_import_queue,background_queue,export_download_queue,saved_exports_queue,analytics_queue,ucr_queue,async_restore_queue,email_queue,case_rule_queue,celery_periodic_0: ERROR (spawn error)
  255. [197.x.x.x] out: commcare-hq-monolith-celery_flower: ERROR (spawn error)
  256. [197.x.x.x] out: commcare-hq-monolith-formsplayer-spring: started
  257. [197.x.x.x] out:
  258.  
  259. Traceback (most recent call last):
  260. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/main.py", line 749, in main
  261. *args, **kwargs
  262. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/tasks.py", line 385, in execute
  263. multiprocessing
  264. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/tasks.py", line 275, in _execute
  265. return task.run(*args, **kwargs)
  266. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/tasks.py", line 172, in run
  267. return self.wrapped(*args, **kwargs)
  268. File "/home/ccc/commcare-cloud/src/commcare_cloud/fab/fabfile.py", line 550, in deploy_commcare
  269. _deploy_without_asking(skip_record)
  270. File "/home/ccc/commcare-cloud/src/commcare_cloud/fab/fabfile.py", line 406, in _deploy_without_asking
  271. deploy_checkpoint(index, command.__name__, execute_with_timing, command)
  272. File "/home/ccc/commcare-cloud/src/commcare_cloud/fab/fabfile.py", line 399, in deploy_checkpoint
  273. fn(*args, **kwargs)
  274. File "/home/ccc/commcare-cloud/src/commcare_cloud/fab/utils.py", line 25, in execute_with_timing
  275. execute(fn, *args, **kwargs)
  276. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/tasks.py", line 385, in execute
  277. multiprocessing
  278. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/tasks.py", line 275, in _execute
  279. return task.run(*args, **kwargs)
  280. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/tasks.py", line 172, in run
  281. return self.wrapped(*args, **kwargs)
  282. File "/home/ccc/commcare-cloud/src/commcare_cloud/fab/fabfile.py", line 389, in conditionally_stop_pillows_and_celery_during_migrate
  283. execute_with_timing(db.migrate)
  284. File "/home/ccc/commcare-cloud/src/commcare_cloud/fab/utils.py", line 25, in execute_with_timing
  285. execute(fn, *args, **kwargs)
  286. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/tasks.py", line 385, in execute
  287. multiprocessing
  288. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/tasks.py", line 275, in _execute
  289. return task.run(*args, **kwargs)
  290. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/tasks.py", line 172, in run
  291. return self.wrapped(*args, **kwargs)
  292. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/decorators.py", line 52, in inner_decorator
  293. return func(*args, **kwargs)
  294. File "/home/ccc/commcare-cloud/src/commcare_cloud/fab/operations/db.py", line 96, in migrate
  295. sudo('%(py3_virtualenv_root)s/bin/python manage.py migrate_multi --noinput' % env)
  296. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/network.py", line 654, in host_prompting_wrapper
  297. return func(*args, **kwargs)
  298. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/operations.py", line 1121, in sudo
  299. stderr=stderr, timeout=timeout, shell_escape=shell_escape,
  300. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/operations.py", line 952, in _run_command
  301. error(message=msg, stdout=out, stderr=err)
  302. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/utils.py", line 359, in error
  303. return func(message)
  304. File "/home/ccc/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/utils.py", line 55, in abort
  305. raise env.abort_exception(msg)
  306. Exception: sudo() received nonzero return code 1 while executing!
  307.  
  308. Requested: /home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/bin/python manage.py migrate_multi --noinput
  309. Executed: sudo -S -p 'sudo password:' -H -u "cchq" /bin/bash -l -c "cd /home/cchq/www/monolith/releases/2021-05-06_13.26 >/dev/null && /home/cchq/www/monolith/releases/2021-05-06_13.26/python_env-3.6/bin/python manage.py migrate_multi --noinput"
  310. Disconnecting from 197.x.x.x... done.
  311. You have new mail in /var/mail/ccc
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!