API tools faq
paste
gvas

Untitled

gvas
Jun 18th, 2018
220
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 50.75 KB | None | 0 0
raw download clone embed print report
  1. MainThread::INFO::2018-06-18 13:07:26,810::state_machine::177::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Local (id 4): {'engine-health': {'health': 'good', 'vm': 'up', 'detail': 'Up'}, '
  2. bridge': True, 'mem-free': 26636.0, 'maintenance': False, 'cpu-load': 0.0164, 'gateway': 1.0, 'storage-domain': True}
  3. MainThread::INFO::2018-06-18 13:07:27,202::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  4. MainThread::INFO::2018-06-18 13:07:36,613::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  5. MainThread::INFO::2018-06-18 13:07:47,023::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  6. MainThread::INFO::2018-06-18 13:07:47,024::hosted_engine::499::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Best remote host ohost01.datacenter.uoc.gr (id: 1, score: 3400)
  7. MainThread::INFO::2018-06-18 13:07:56,436::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  8. MainThread::INFO::2018-06-18 13:08:06,862::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  9. MainThread::INFO::2018-06-18 13:08:16,884::states::442::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost
  10. MainThread::INFO::2018-06-18 13:08:17,273::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  11. MainThread::INFO::2018-06-18 13:08:26,686::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  12. MainThread::INFO::2018-06-18 13:08:37,100::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  13. MainThread::INFO::2018-06-18 13:08:46,507::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  14. MainThread::INFO::2018-06-18 13:08:56,923::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  15. MainThread::INFO::2018-06-18 13:09:06,329::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  16. MainThread::INFO::2018-06-18 13:09:16,739::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  17. MainThread::INFO::2018-06-18 13:09:26,761::states::442::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost
  18. MainThread::INFO::2018-06-18 13:09:27,156::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  19. MainThread::INFO::2018-06-18 13:09:36,571::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  20. MainThread::INFO::2018-06-18 13:09:46,987::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  21. MainThread::INFO::2018-06-18 13:09:56,400::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  22. MainThread::INFO::2018-06-18 13:09:56,400::hosted_engine::499::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Best remote host ohost02.datacenter.uoc.gr (id: 2, score: 3400)
  23. MainThread::INFO::2018-06-18 13:10:06,818::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  24. MainThread::INFO::2018-06-18 13:10:17,231::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  25. MainThread::INFO::2018-06-18 13:10:26,643::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  26. MainThread::INFO::2018-06-18 13:10:36,660::states::442::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost
  27. MainThread::INFO::2018-06-18 13:10:37,048::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  28. MainThread::INFO::2018-06-18 13:10:46,456::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  29. MainThread::INFO::2018-06-18 13:10:56,869::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  30. MainThread::INFO::2018-06-18 13:11:07,084::ovf_store::151::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /var/run/vdsm/storage/e3ed3ebb-4d98-45ce-ae6b-3d24a317c9ed/87
  31. 9b9aa4-7035-458f-80e6-d04156e53fb1/9f53f8b8-19dd-48da-bef7-8b3ee63fc98f
  32. MainThread::INFO::2018-06-18 13:11:07,383::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  33. MainThread::INFO::2018-06-18 13:11:16,794::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  34. MainThread::INFO::2018-06-18 13:11:27,211::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  35. MainThread::INFO::2018-06-18 13:11:36,622::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  36. MainThread::INFO::2018-06-18 13:11:46,644::states::442::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost
  37. MainThread::INFO::2018-06-18 13:11:47,044::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  38. MainThread::INFO::2018-06-18 13:11:56,472::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  39. MainThread::INFO::2018-06-18 13:12:06,897::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  40. MainThread::INFO::2018-06-18 13:09:56,400::hosted_engine::499::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Best remote host ohost02.datacenter.uoc.gr (id: 2, score: 3400)
  41. MainThread::INFO::2018-06-18 13:10:06,818::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  42. MainThread::INFO::2018-06-18 13:10:17,231::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  43. MainThread::INFO::2018-06-18 13:10:26,643::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  44. MainThread::INFO::2018-06-18 13:10:36,660::states::442::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost
  45. MainThread::INFO::2018-06-18 13:10:37,048::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  46. MainThread::INFO::2018-06-18 13:10:46,456::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  47. MainThread::INFO::2018-06-18 13:10:56,869::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  48. MainThread::INFO::2018-06-18 13:11:07,084::ovf_store::151::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /var/run/vdsm/storage/e3ed3ebb-4d98-45ce-ae6b-3d24a317c9ed/87
  49. 9b9aa4-7035-458f-80e6-d04156e53fb1/9f53f8b8-19dd-48da-bef7-8b3ee63fc98f
  50. MainThread::INFO::2018-06-18 13:11:07,383::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  51. MainThread::INFO::2018-06-18 13:11:16,794::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  52. MainThread::INFO::2018-06-18 13:11:27,211::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  53. MainThread::INFO::2018-06-18 13:11:36,622::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  54. MainThread::INFO::2018-06-18 13:11:46,644::states::442::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost
  55. MainThread::INFO::2018-06-18 13:11:47,044::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  56. MainThread::INFO::2018-06-18 13:11:56,472::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  57. MainThread::INFO::2018-06-18 13:12:06,897::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  58. MainThread::INFO::2018-06-18 13:12:56,619::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  59. MainThread::INFO::2018-06-18 13:12:57,025::brokerlink::68::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Success, was notification of state_transition (EngineUp-EngineUpBadHealth) sent? sent
  60. MainThread::INFO::2018-06-18 13:12:57,401::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  61. MainThread::ERROR::2018-06-18 13:13:06,420::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 300 seconds
  62. MainThread::INFO::2018-06-18 13:13:06,440::state_decorators::99::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(check) Timeout set to Mon Jun 18 13:18:06 2018 while transitioning <class 'ovirt_hosted_engine_ha.agent.states.EngineUpBadHealth'> -> <class 'ovirt_hosted_engine_ha.agent.states.EngineUpBadHealth'>
  63. MainThread::INFO::2018-06-18 13:13:06,831::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  64. MainThread::ERROR::2018-06-18 13:13:16,853::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 290 seconds
  65. MainThread::INFO::2018-06-18 13:13:17,252::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  66. MainThread::ERROR::2018-06-18 13:13:26,270::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 280 seconds
  67. MainThread::INFO::2018-06-18 13:13:26,665::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  68. MainThread::ERROR::2018-06-18 13:13:36,688::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 270 seconds
  69. MainThread::INFO::2018-06-18 13:13:37,084::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  70. MainThread::ERROR::2018-06-18 13:13:46,106::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 260 seconds
  71. MainThread::INFO::2018-06-18 13:13:46,495::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  72. MainThread::ERROR::2018-06-18 13:13:56,518::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 250 seconds
  73. MainThread::INFO::2018-06-18 13:13:56,908::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  74. MainThread::ERROR::2018-06-18 13:14:06,930::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 239 seconds
  75. MainThread::INFO::2018-06-18 13:14:07,324::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  76. MainThread::ERROR::2018-06-18 13:14:17,346::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 229 seconds
  77. MainThread::INFO::2018-06-18 13:14:17,742::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  78. MainThread::ERROR::2018-06-18 13:14:27,764::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 219 seconds
  79. MainThread::INFO::2018-06-18 13:14:28,162::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  80. MainThread::ERROR::2018-06-18 13:14:37,183::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 209 seconds
  81. MainThread::INFO::2018-06-18 13:14:37,576::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  82. MainThread::ERROR::2018-06-18 13:14:47,599::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 199 seconds
  83. MainThread::INFO::2018-06-18 13:14:47,991::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  84. MainThread::ERROR::2018-06-18 13:14:57,012::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 189 seconds
  85. MainThread::INFO::2018-06-18 13:14:57,403::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  86. MainThread::INFO::2018-06-18 13:14:57,404::hosted_engine::499::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Best remote host ohost02.datacenter.uoc.gr (id: 2, score: 3400)
  87. MainThread::ERROR::2018-06-18 13:15:07,426::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 179 seconds
  88. MainThread::INFO::2018-06-18 13:16:48,033::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  89. MainThread::ERROR::2018-06-18 13:16:57,055::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 69 seconds
  90. MainThread::INFO::2018-06-18 13:16:57,447::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  91. MainThread::ERROR::2018-06-18 13:17:07,470::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 59 seconds
  92. MainThread::INFO::2018-06-18 13:17:07,866::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  93. MainThread::ERROR::2018-06-18 13:17:17,888::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 49 seconds
  94. MainThread::INFO::2018-06-18 13:17:18,276::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  95. MainThread::INFO::2018-06-18 13:17:27,298::state_machine::169::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Global metadata: {'maintenance': False}
  96. MainThread::INFO::2018-06-18 13:17:27,298::state_machine::174::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Host ohost01.datacenter.uoc.gr (id 1): {'conf_on_shared_storage': True, 'extra': 'metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=242403 (Mon Jun 18 13:11:50 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=242397 (Mon Jun 18 13:11:43 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=AgentStopped\nstopped=True\n', 'hostname': 'ohost01.datacenter.uoc.gr', 'alive': False, 'host-id': 1, 'engine-status': {'reason': 'vm not running on this host', 'health': 'bad', 'vm': 'down', 'detail': 'unknown'}, 'score': 0, 'stopped': True, 'maintenance': False, 'crc32': '84f77f3a', 'local_conf_timestamp': 242397, 'host-ts': 242403}
  97. MainThread::INFO::2018-06-18 13:17:27,298::state_machine::174::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Host ohost02.datacenter.uoc.gr (id 2): {'conf_on_shared_storage': True, 'extra': 'metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4067 (Mon Jun 18 13:17:11 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4067 (Mon Jun 18 13:17:11 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n', 'hostname': 'ohost02.datacenter.uoc.gr', 'alive': True, 'host-id': 2, 'engine-status': {'reason': 'vm not running on this host', 'health': 'bad', 'vm': 'down', 'detail': 'unknown'}, 'score': 3400, 'stopped': False, 'maintenance': False, 'crc32': '5299ef3f', 'local_conf_timestamp': 4067, 'host-ts': 4067}
  98. MainThread::INFO::2018-06-18 13:17:27,298::state_machine::174::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Host ohost04.datacenter.uoc.gr (id 3): {'conf_on_shared_storage': True, 'extra': 'metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=7790 (Mon Jun 18 13:17:12 2018)\nhost-id=3\nscore=3400\nvm_conf_refresh_time=7791 (Mon Jun 18 13:17:12 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n', 'hostname': 'ohost04.datacenter.uoc.gr', 'alive': True, 'host-id': 3, 'engine-status': {'reason': 'vm not running on this host', 'health': 'bad', 'vm': 'down', 'detail': 'unknown'}, 'score': 3400, 'stopped': False, 'maintenance': False, 'crc32': '1f3fb56c', 'local_conf_timestamp': 7791, 'host-ts': 7790}
  99. MainThread::INFO::2018-06-18 13:17:27,299::state_machine::177::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Local (id 4): {'engine-health': {'reason': 'bad vm status', 'health': 'bad', 'vm': 'up', 'detail': 'Paused'}, 'bridge': True, 'mem-free': 26634.0, 'maintenance': False, 'cpu-load': 0.0097, 'gateway': 1.0, 'storage-domain': True}
  100. MainThread::ERROR::2018-06-18 13:17:27,299::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 39 seconds
  101. MainThread::INFO::2018-06-18 13:17:27,686::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  102. MainThread::ERROR::2018-06-18 13:17:37,709::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 29 seconds
  103. MainThread::INFO::2018-06-18 13:17:38,102::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  104. MainThread::ERROR::2018-06-18 13:17:47,124::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 19 seconds
  105. MainThread::INFO::2018-06-18 13:17:47,512::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  106. MainThread::ERROR::2018-06-18 13:17:57,528::states::617::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 9 seconds
  107. MainThread::INFO::2018-06-18 13:17:57,921::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUpBadHealth (score: 3400)
  108. MainThread::INFO::2018-06-18 13:18:07,248::brokerlink::68::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Success, was notification of state_transition (EngineUpBadHealth-EngineStop) sent? sent
  109. MainThread::INFO::2018-06-18 13:18:07,628::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  110. MainThread::INFO::2018-06-18 13:18:17,649::hosted_engine::1007::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_engine_vm) Shutting down vm using `/usr/sbin/hosted-engine --vm-shutdown`
  111. MainThread::INFO::2018-06-18 13:18:17,990::hosted_engine::1012::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_engine_vm) stdout:
  112. MainThread::INFO::2018-06-18 13:18:17,990::hosted_engine::1013::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_engine_vm) stderr:
  113. MainThread::ERROR::2018-06-18 13:18:17,990::hosted_engine::1021::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_engine_vm) Engine VM stopped on localhost
  114. MainThread::INFO::2018-06-18 13:18:18,012::state_decorators::99::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(check) Timeout set to Mon Jun 18 13:23:17 2018 while transitioning <class 'ovirt_hosted_engine_ha.agent.states.EngineStop'> -> <class 'ovirt_hosted_engine_ha.agent.states.EngineStop'>
  115. MainThread::INFO::2018-06-18 13:18:18,404::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  116. MainThread::INFO::2018-06-18 13:18:28,427::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (11 of 300 seconds)
  117. MainThread::INFO::2018-06-18 13:18:28,821::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  118. MainThread::INFO::2018-06-18 13:18:38,844::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (21 of 300 seconds)
  119. MainThread::INFO::2018-06-18 13:18:39,256::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  120. MainThread::INFO::2018-06-18 13:18:48,276::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (31 of 300 seconds)
  121. MainThread::INFO::2018-06-18 13:18:48,675::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  122. MainThread::INFO::2018-06-18 13:18:58,690::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (41 of 300 seconds)
  123. MainThread::INFO::2018-06-18 13:18:59,084::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  124. MainThread::INFO::2018-06-18 13:19:08,106::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (51 of 300 seconds)
  125. MainThread::INFO::2018-06-18 13:19:08,497::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  126. MainThread::INFO::2018-06-18 13:19:08,106::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (51 of 300 seconds)
  127. MainThread::INFO::2018-06-18 13:19:08,497::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  128. MainThread::INFO::2018-06-18 13:19:18,519::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (61 of 300 seconds)
  129. MainThread::INFO::2018-06-18 13:19:18,912::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  130. MainThread::INFO::2018-06-18 13:19:28,934::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (72 of 300 seconds)
  131. MainThread::INFO::2018-06-18 13:19:29,331::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  132. MainThread::INFO::2018-06-18 13:19:39,349::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (82 of 300 seconds)
  133. MainThread::INFO::2018-06-18 13:19:39,752::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  134. MainThread::INFO::2018-06-18 13:19:49,769::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (92 of 300 seconds)
  135. MainThread::INFO::2018-06-18 13:19:50,161::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  136. MainThread::INFO::2018-06-18 13:19:59,182::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (102 of 300 seconds)
  137. MainThread::INFO::2018-06-18 13:19:59,581::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  138. MainThread::INFO::2018-06-18 13:19:59,581::hosted_engine::499::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Best remote host ohost02.datacenter.uoc.gr (id: 2, score: 3400)
  139. MainThread::INFO::2018-06-18 13:20:09,604::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (112 of 300 seconds)
  140. MainThread::INFO::2018-06-18 13:20:10,007::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  141. MainThread::INFO::2018-06-18 13:20:19,029::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (122 of 300 seconds)
  142. MainThread::INFO::2018-06-18 13:20:19,430::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  143. MainThread::INFO::2018-06-18 13:20:29,454::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (132 of 300 seconds)
  144. MainThread::INFO::2018-06-18 13:20:29,851::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  145. MainThread::INFO::2018-06-18 13:20:39,866::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (142 of 300 seconds)
  146. MainThread::INFO::2018-06-18 13:20:40,259::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  147. MainThread::INFO::2018-06-18 13:20:49,281::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (152 of 300 seconds)
  148. MainThread::INFO::2018-06-18 13:20:49,670::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  149. MainThread::INFO::2018-06-18 13:20:59,693::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (162 of 300 seconds)
  150. MainThread::INFO::2018-06-18 13:21:00,086::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  151. MainThread::INFO::2018-06-18 13:21:09,110::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (172 of 300 seconds)
  152. MainThread::INFO::2018-06-18 13:21:09,302::ovf_store::151::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /var/run/vdsm/storage/e3ed3ebb-4d98-45ce-ae6b-3d24a317c9ed/87
  153. 9b9aa4-7035-458f-80e6-d04156e53fb1/9f53f8b8-19dd-48da-bef7-8b3ee63fc98f
  154. MainThread::INFO::2018-06-18 13:21:09,591::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  155. MainThread::INFO::2018-06-18 13:21:19,612::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (182 of 300 seconds)
  156. MainThread::INFO::2018-06-18 13:21:20,004::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  157. MainThread::INFO::2018-06-18 13:21:29,025::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (192 of 300 seconds)
  158. MainThread::INFO::2018-06-18 13:21:29,413::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  159. MainThread::INFO::2018-06-18 13:21:39,436::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (202 of 300 seconds)
  160. MainThread::INFO::2018-06-18 13:21:39,829::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  161. MainThread::INFO::2018-06-18 13:21:49,851::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (212 of 300 seconds)
  162. MainThread::INFO::2018-06-18 13:21:50,244::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  163. MainThread::INFO::2018-06-18 13:21:59,264::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (222 of 300 seconds)
  164. MainThread::INFO::2018-06-18 13:21:59,659::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  165. MainThread::INFO::2018-06-18 13:22:09,681::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (232 of 300 seconds)
  166. MainThread::INFO::2018-06-18 13:22:10,072::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  167. MainThread::INFO::2018-06-18 13:22:19,094::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (242 of 300 seconds)
  168. MainThread::INFO::2018-06-18 13:22:19,485::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  169. MainThread::INFO::2018-06-18 13:22:29,507::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (252 of 300 seconds)
  170. MainThread::INFO::2018-06-18 13:22:29,904::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  171. MainThread::INFO::2018-06-18 13:22:39,923::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (263 of 300 seconds)
  172. MainThread::INFO::2018-06-18 13:22:40,310::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  173. MainThread::INFO::2018-06-18 13:22:50,331::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (273 of 300 seconds)
  174. MainThread::INFO::2018-06-18 13:21:09,302::ovf_store::151::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /var/run/vdsm/storage/e3ed3ebb-4d98-45ce-ae6b-3d24a317c9ed/87
  175. 9b9aa4-7035-458f-80e6-d04156e53fb1/9f53f8b8-19dd-48da-bef7-8b3ee63fc98f
  176. MainThread::INFO::2018-06-18 13:21:09,591::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  177. MainThread::INFO::2018-06-18 13:21:19,612::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (182 of 300 seconds)
  178. MainThread::INFO::2018-06-18 13:21:20,004::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  179. MainThread::INFO::2018-06-18 13:21:29,025::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (192 of 300 seconds)
  180. MainThread::INFO::2018-06-18 13:21:29,413::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  181. MainThread::INFO::2018-06-18 13:21:39,436::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (202 of 300 seconds)
  182. MainThread::INFO::2018-06-18 13:21:39,829::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  183. MainThread::INFO::2018-06-18 13:21:49,851::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (212 of 300 seconds)
  184. MainThread::INFO::2018-06-18 13:21:50,244::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  185. MainThread::INFO::2018-06-18 13:21:59,264::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (222 of 300 seconds)
  186. MainThread::INFO::2018-06-18 13:21:59,659::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  187. MainThread::INFO::2018-06-18 13:22:09,681::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (232 of 300 seconds)
  188. MainThread::INFO::2018-06-18 13:22:10,072::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  189. MainThread::INFO::2018-06-18 13:22:19,094::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (242 of 300 seconds)
  190. MainThread::INFO::2018-06-18 13:22:19,485::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  191. MainThread::INFO::2018-06-18 13:22:29,507::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (252 of 300 seconds)
  192. MainThread::INFO::2018-06-18 13:22:29,904::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  193. MainThread::INFO::2018-06-18 13:22:39,923::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (263 of 300 seconds)
  194. MainThread::INFO::2018-06-18 13:22:40,310::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  195. MainThread::INFO::2018-06-18 13:22:50,331::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (273 of 300 seconds)
  196. MainThread::INFO::2018-06-18 13:22:50,722::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  197. MainThread::INFO::2018-06-18 13:23:00,745::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (283 of 300 seconds)
  198. MainThread::INFO::2018-06-18 13:23:01,139::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  199. MainThread::INFO::2018-06-18 13:23:10,161::states::585::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Waiting on shutdown to complete (293 of 300 seconds)
  200. MainThread::INFO::2018-06-18 13:23:10,567::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStop (score: 3400)
  201. MainThread::INFO::2018-06-18 13:23:20,892::brokerlink::68::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Success, was notification of state_transition (EngineStop-EngineForceStop) sent? sent
  202. MainThread::INFO::2018-06-18 13:23:21,278::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineForceStop (score: 3400)
  203. MainThread::INFO::2018-06-18 13:23:30,301::hosted_engine::1007::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_engine_vm) Shutting down vm using `/usr/sbin/hosted-engine --vm-poweroff`
  204. MainThread::INFO::2018-06-18 13:23:57,451::hosted_engine::1012::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_engine_vm) stdout:
  205. MainThread::INFO::2018-06-18 13:23:57,452::hosted_engine::1013::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_engine_vm) stderr:
  206. MainThread::ERROR::2018-06-18 13:23:57,452::hosted_engine::1021::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_engine_vm) Engine VM stopped on localhost
  207. MainThread::INFO::2018-06-18 13:23:57,540::brokerlink::68::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Success, was notification of state_transition (EngineForceStop-EngineDown) sent? sent
  208. MainThread::INFO::2018-06-18 13:23:57,924::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineDown (score: 3400)
  209. MainThread::INFO::2018-06-18 13:24:06,939::states::508::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine down and local host has best score (3400), attempting to start engine VM
  210. MainThread::INFO::2018-06-18 13:24:07,025::brokerlink::68::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Success, was notification of state_transition (EngineDown-EngineStart) sent? sent
  211. MainThread::INFO::2018-06-18 13:24:07,403::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStart (score: 3400)
  212. MainThread::INFO::2018-06-18 13:24:07,423::hosted_engine::968::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_clean_vdsm_state) Ensuring VDSM state is clear for engine VM
  213. MainThread::INFO::2018-06-18 13:24:07,431::hosted_engine::980::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_clean_vdsm_state) Vdsm state for VM clean
  214. MainThread::INFO::2018-06-18 13:24:07,431::hosted_engine::929::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_start_engine_vm) Starting vm using `/usr/sbin/hosted-engine --vm-start`
  215. MainThread::INFO::2018-06-18 13:24:07,999::hosted_engine::935::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_start_engine_vm) stdout:
  216. MainThread::INFO::2018-06-18 13:24:07,999::hosted_engine::936::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_start_engine_vm) stderr: Command VM.getStats with args {'vmID': 'f041f267-04b0-4197-9a82-618997737e33'} failed:
  217. (code=1, message=Virtual machine does not exist: {'vmId': u'f041f267-04b0-4197-9a82-618997737e33'})
  218.  
  219. MainThread::INFO::2018-06-18 13:24:08,000::hosted_engine::948::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_start_engine_vm) Engine VM started on localhost
  220. MainThread::INFO::2018-06-18 13:24:08,090::brokerlink::68::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Success, was notification of state_transition (EngineStart-EngineStarting) sent? sent
  221. MainThread::INFO::2018-06-18 13:24:08,106::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400)
  222. MainThread::INFO::2018-06-18 13:24:18,123::states::772::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) VM is powering up..
  223. MainThread::INFO::2018-06-18 13:24:18,142::state_decorators::99::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(check) Timeout set to Mon Jun 18 13:34:18 2018 while transitioning <class 'ovirt_hosted_engine_ha.agent.states.EngineStarting'> -> <class 'ovirt_hosted_engine_ha.agent.states.EngineStarting'>
  224. MainThread::INFO::2018-06-18 13:24:18,550::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400)
  225. MainThread::INFO::2018-06-18 13:24:28,574::states::772::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) VM is powering up..
  226. MainThread::INFO::2018-06-18 13:24:28,997::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400)
  227. MainThread::INFO::2018-06-18 13:24:38,019::states::772::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) VM is powering up..
  228. MainThread::INFO::2018-06-18 13:24:38,418::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400)
  229. MainThread::INFO::2018-06-18 13:24:48,431::states::772::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) VM is powering up..
  230. MainThread::INFO::2018-06-18 13:24:48,835::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400)
  231. MainThread::INFO::2018-06-18 13:24:58,859::states::772::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) VM is powering up..
  232. MainThread::INFO::2018-06-18 13:24:59,285::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400)
  233. MainThread::INFO::2018-06-18 13:25:08,307::states::772::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) VM is powering up..
  234. MainThread::INFO::2018-06-18 13:25:08,744::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400)
  235. MainThread::INFO::2018-06-18 13:25:08,745::hosted_engine::499::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Best remote host ohost02.datacenter.uoc.gr (id: 2, score: 3400)
  236. MainThread::INFO::2018-06-18 13:25:18,788::state_decorators::92::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(check) Timeout cleared while transitioning <class 'ovirt_hosted_engine_ha.agent.states.EngineStarting'> -> <class 'ovirt_hosted_engine_ha.agent.states.EngineUp'>
  237. MainThread::INFO::2018-06-18 13:25:18,873::brokerlink::68::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Success, was notification of state_transition (EngineStarting-EngineUp) sent? sent
  238. MainThread::INFO::2018-06-18 13:25:19,253::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  239. MainThread::INFO::2018-06-18 13:25:28,275::states::442::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost
  240. MainThread::INFO::2018-06-18 13:25:28,672::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  241. MainThread::INFO::2018-06-18 13:25:39,092::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  242. MainThread::INFO::2018-06-18 13:25:48,494::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  243. MainThread::INFO::2018-06-18 13:25:58,913::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  244. MainThread::INFO::2018-06-18 13:26:09,331::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  245. MainThread::INFO::2018-06-18 13:26:19,550::ovf_store::151::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /var/run/vdsm/storage/e3ed3ebb-4d98-45ce-ae6b-3d24a317c9ed/879b9aa4-7035-458f-80e6-d04156e53fb1/9f53f8b8-19dd-48da-bef7-8b3ee63fc98f
  246. MainThread::INFO::2018-06-18 13:26:19,841::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  247. MainThread::INFO::2018-06-18 13:26:29,863::states::442::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost
  248. MainThread::INFO::2018-06-18 13:26:30,286::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  249. MainThread::INFO::2018-06-18 13:26:30,286::hosted_engine::499::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Best remote host ohost01.datacenter.uoc.gr (id: 1, score: 3400)
  250. MainThread::INFO::2018-06-18 13:26:39,687::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  251. MainThread::INFO::2018-06-18 13:26:50,101::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  252. MainThread::INFO::2018-06-18 13:26:59,528::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  253. MainThread::INFO::2018-06-18 13:27:09,949::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  254. MainThread::INFO::2018-06-18 13:27:19,361::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  255. MainThread::INFO::2018-06-18 13:27:29,384::state_machine::169::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Global metadata: {'maintenance': False}
  256. MainThread::INFO::2018-06-18 13:27:29,384::state_machine::174::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Host ohost01.datacenter.uoc.gr (id 1): {'conf_on_shared_storage': True, 'extra':
  257. 'metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=565 (Mon Jun 18 13:27:18 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=565 (Mon Jun 18 13:27:18 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n', 'hostname': 'ohost01.datacenter.uoc.gr', 'alive': True, 'host-id': 1, 'engine-status': {'reason': 'vm not running on this host', 'health': 'bad', 'vm': 'down', 'detail': 'unknown'}, 'score': 3400, 'stopped': False, 'maintenance': False, 'crc32': '2a272ac1', 'local_conf_timestamp': 565, 'host-ts': 565}
  258. MainThread::INFO::2018-06-18 13:27:29,385::state_machine::174::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Host ohost02.datacenter.uoc.gr (id 2): {'conf_on_shared_storage': True, 'extra': 'metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=4671 (Mon Jun 18 13:27:15 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=4671 (Mon Jun 18 13:27:15 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n', 'hostname': 'ohost02.datacenter.uoc.gr', 'alive': True, 'host-id': 2, 'engine-status': {'reason': 'vm not running on this host', 'health': 'bad', 'vm': 'down', 'detail': 'unknown'}, 'score': 3400, 'stopped': False, 'maintenance': False, 'crc32': 'a9ee4613', 'local_conf_timestamp': 4671, 'host-ts': 4671}
  259. MainThread::INFO::2018-06-18 13:27:29,385::state_machine::174::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Host ohost04.datacenter.uoc.gr (id 3): {'conf_on_shared_storage': True, 'extra': 'metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=8396 (Mon Jun 18 13:27:17 2018)\nhost-id=3\nscore=3400\nvm_conf_refresh_time=8396 (Mon Jun 18 13:27:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n', 'hostname': 'ohost04.datacenter.uoc.gr', 'alive': True, 'host-id': 3, 'engine-status': {'reason': 'vm not running on this host', 'health': 'bad', 'vm': 'down', 'detail': 'unknown'}, 'score': 3400, 'stopped': False, 'maintenance': False, 'crc32': 'f674e487', 'local_conf_timestamp': 8396, 'host-ts': 8396}
  260. MainThread::INFO::2018-06-18 13:27:29,385::state_machine::177::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(refresh) Local (id 4): {'engine-health': {'health': 'good', 'vm': 'up', 'detail': 'Up'}, 'bridge': True, 'mem-free': 30114.0, 'maintenance': False, 'cpu-load': 0.0525, 'gateway': 1.0, 'storage-domain': True}
  261. MainThread::INFO::2018-06-18 13:27:29,782::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  262. MainThread::INFO::2018-06-18 13:27:39,805::states::442::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost
  263. MainThread::INFO::2018-06-18 13:27:40,197::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  264. MainThread::INFO::2018-06-18 13:27:49,622::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  265. MainThread::INFO::2018-06-18 13:28:00,033::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  266. MainThread::INFO::2018-06-18 13:28:09,446::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  267. MainThread::INFO::2018-06-18 13:28:19,876::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  268. MainThread::INFO::2018-06-18 13:28:30,301::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  269. MainThread::INFO::2018-06-18 13:28:39,706::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
  270. MainThread::INFO::2018-06-18 13:28:49,728::states::442::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!