API tools faq
paste
digimer

Untitled

digimer
Jul 14th, 2020
150
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 214.76 KB | None | 0 0
raw download clone embed print report
  1. [root@el8-a01n01 ~]# cat /var/log/pacemaker/el8-a01n01/pacemaker.log.drbd-restart-to-crash
  2. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  3. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (get_cluster_type) info: Detected an active 'corosync' cluster
  4. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (mcp_read_config) info: Reading configure for stack: corosync
  5. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (main) notice: Starting Pacemaker 2.0.3-5.el8_2.1 | build=4b1f869f0f features: generated-manpages agent-manpages ncurses libqb-logging libqb-ipc systemd nagios corosync-native atomic-attrd acls
  6. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (main) info: Maximum core file size is: 18446744073709551615
  7. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (qb_ipcs_us_publish) info: server name: pacemakerd
  8. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (pcmk__ipc_is_authentic_process_active) info: Could not connect to lrmd IPC: Connection refused
  9. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (pcmk__ipc_is_authentic_process_active) info: Could not connect to cib_ro IPC: Connection refused
  10. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (pcmk__ipc_is_authentic_process_active) info: Could not connect to crmd IPC: Connection refused
  11. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (pcmk__ipc_is_authentic_process_active) info: Could not connect to attrd IPC: Connection refused
  12. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (pcmk__ipc_is_authentic_process_active) info: Could not connect to pengine IPC: Connection refused
  13. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (pcmk__ipc_is_authentic_process_active) info: Could not connect to stonith-ng IPC: Connection refused
  14. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (crm_get_peer) info: Created entry 81dfc4f1-02f0-435d-8154-a23bf123823b/0x558d0df928c0 for node el8-a01n01/1 (1 total)
  15. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  16. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (crm_get_peer) info: Node 1 has uuid 1
  17. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online
  18. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (cluster_connect_quorum) warning: Quorum lost
  19. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (crm_get_peer) info: Created entry c378e0a7-0e78-4209-a7c0-499064bd0716/0x558d0df94860 for node el8-a01n02/2 (2 total)
  20. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  21. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (crm_get_peer) info: Node 2 has uuid 2
  22. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (start_child) info: Using uid=189 and group=189 for process pacemaker-based
  23. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (start_child) info: Forked child 6051 for process pacemaker-based
  24. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (start_child) info: Forked child 6052 for process pacemaker-fenced
  25. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (start_child) info: Forked child 6053 for process pacemaker-execd
  26. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (start_child) info: Using uid=189 and group=189 for process pacemaker-attrd
  27. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (start_child) info: Forked child 6054 for process pacemaker-attrd
  28. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (start_child) info: Using uid=189 and group=189 for process pacemaker-schedulerd
  29. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (start_child) info: Forked child 6055 for process pacemaker-schedulerd
  30. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (start_child) info: Using uid=189 and group=189 for process pacemaker-controld
  31. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (start_child) info: Forked child 6056 for process pacemaker-controld
  32. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (main) notice: Pacemaker daemon successfully started and accepting connections
  33. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (pcmk_quorum_notification) info: Quorum still lost | membership=162 members=1
  34. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification
  35. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (crm_reap_unseen_nodes) info: State of node el8-a01n02[2] is still unknown
  36. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (pcmk_cpg_membership) info: Group pacemakerd event 0: node 1 pid 6050 joined via cpg_join
  37. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemakerd [6050] (pcmk_cpg_membership) info: Group pacemakerd event 0: el8-a01n01 (node 1 pid 6050) is member
  38. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-execd [6053] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  39. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-execd [6053] (main) notice: Starting Pacemaker local executor
  40. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-execd [6053] (qb_ipcs_us_publish) info: server name: lrmd
  41. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-execd [6053] (main) notice: Pacemaker local executor successfully started and accepting connections
  42. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  43. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (main) notice: Starting Pacemaker scheduler
  44. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (qb_ipcs_us_publish) info: server name: pengine
  45. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (main) notice: Pacemaker scheduler successfully started and accepting connections
  46. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-fenced [6052] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  47. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-fenced [6052] (main) notice: Starting Pacemaker fencer
  48. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-fenced [6052] (get_cluster_type) info: Verifying cluster type: 'corosync'
  49. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-fenced [6052] (get_cluster_type) info: Assuming an active 'corosync' cluster
  50. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-fenced [6052] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  51. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  52. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (main) notice: Starting Pacemaker CIB manager
  53. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (get_cluster_type) info: Verifying cluster type: 'corosync'
  54. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (get_cluster_type) info: Assuming an active 'corosync' cluster
  55. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-fenced [6052] (crm_get_peer) info: Created entry b880f033-a364-4fd5-b74b-07b7fa1c03c1/0x5653eed71d60 for node el8-a01n01/1 (1 total)
  56. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-fenced [6052] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  57. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-attrd [6054] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  58. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-attrd [6054] (main) notice: Starting Pacemaker node attribute manager
  59. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-fenced [6052] (crm_get_peer) info: Node 1 has uuid 1
  60. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-fenced [6052] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online
  61. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-fenced [6052] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  62. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-fenced [6052] (init_cs_connection_once) info: Connection to 'corosync': established
  63. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (retrieveCib) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.xml (digest: /var/lib/pacemaker/cib/cib.xml.sig)
  64. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  65. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-controld [6056] (main) notice: Starting Pacemaker controller
  66. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_log) info: Input I_STARTUP received in state S_STARTING from crmd_init
  67. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  68. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (crm_get_peer) info: Created entry 16e38ebd-b2a7-4fbd-b06e-4d33ecf0c8d6/0x563bd6586c00 for node el8-a01n01/1 (1 total)
  69. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  70. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (crm_get_peer) info: Node 1 has uuid 1
  71. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online
  72. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  73. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (init_cs_connection_once) info: Connection to 'corosync': established
  74. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (qb_ipcs_us_publish) info: server name: cib_ro
  75. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (qb_ipcs_us_publish) info: server name: cib_rw
  76. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (qb_ipcs_us_publish) info: server name: cib_shm
  77. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (main) notice: Pacemaker CIB manager successfully started and accepting connections
  78. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (pcmk_cpg_membership) info: Group cib event 0: node 1 pid 6051 joined via cpg_join
  79. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (pcmk_cpg_membership) info: Group cib event 0: el8-a01n01 (node 1 pid 6051) is member
  80. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-15.raw
  81. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_file_write_with_digest) info: Wrote version 0.13.0 of the CIB to disk (digest: 104d39c1b15a7a8d4fc8d317785f3f93)
  82. Jul 14 21:52:20 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.vVRNsY (digest: /var/lib/pacemaker/cib/cib.1pOqgd)
  83. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (main) info: CIB connection active
  84. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (get_cluster_type) info: Verifying cluster type: 'corosync'
  85. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (get_cluster_type) info: Assuming an active 'corosync' cluster
  86. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  87. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (crm_get_peer) info: Created entry ab98b779-8aea-4113-b448-95d76468bf29/0x55c354e20a20 for node el8-a01n01/1 (1 total)
  88. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  89. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (crm_get_peer) info: Node 1 has uuid 1
  90. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online
  91. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  92. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (init_cs_connection_once) info: Connection to 'corosync': established
  93. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (main) info: Cluster connection active
  94. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_erase_attrs) info: Clearing transient attributes from CIB | xpath=//node_state[@uname='el8-a01n01']/transient_attributes
  95. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (qb_ipcs_us_publish) info: server name: attrd
  96. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (main) notice: Pacemaker node attribute manager successfully started and accepting connections
  97. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n01']/transient_attributes to all (origin=local/attrd/2)
  98. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (setup_cib) info: Watching for stonith topology changes
  99. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (qb_ipcs_us_publish) info: server name: stonith-ng
  100. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (main) notice: Pacemaker fencer successfully started and accepting connections
  101. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (pcmk_cpg_membership) info: Group stonith-ng event 0: node 1 pid 6052 joined via cpg_join
  102. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (pcmk_cpg_membership) info: Group stonith-ng event 0: el8-a01n01 (node 1 pid 6052) is member
  103. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (init_cib_cache_cb) info: Updating device list from the cib: init
  104. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (cib_devices_update) info: Updating devices to version 0.13.0
  105. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (get_cluster_type) info: Verifying cluster type: 'corosync'
  106. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (get_cluster_type) info: Assuming an active 'corosync' cluster
  107. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  108. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_get_peer) info: Created entry e98b1b68-2110-443a-b851-43406298de36/0x55f25c909e20 for node el8-a01n01/1 (1 total)
  109. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  110. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (peer_update_callback) info: Cluster node el8-a01n01 is now in unknown state
  111. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_get_peer) info: Node 1 has uuid 1
  112. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online
  113. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (peer_update_callback) info: Node el8-a01n01 is now a peer | DC=<none> old=0x0000000 new=0x4000000
  114. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds
  115. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (init_cs_connection_once) info: Connection to 'corosync': established
  116. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (cluster_connect_quorum) warning: Quorum lost
  117. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_get_peer) info: Created entry 8bd55a82-ce1c-4ca8-8981-5bc419c15ae1/0x55f25c90c700 for node el8-a01n02/2 (2 total)
  118. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  119. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (peer_update_callback) info: Cluster node el8-a01n02 is now in unknown state
  120. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_get_peer) info: Node 2 has uuid 2
  121. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_ha_control) info: Connected to the cluster
  122. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (lrmd_ipc_connect) info: Connecting to executor
  123. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/4)
  124. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-execd [6053] (crm_client_new) debug: Connecting 0x565394bc4fb0 for uid=189 gid=189 pid=6056 id=29872d72-cf4e-4597-93c4-86f8b71374d4
  125. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-execd [6053] (handle_new_connection) debug: IPC credentials authenticated (/dev/shm/qb-6053-6056-9-WwUHL3/qb)
  126. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-execd [6053] (qb_ipcs_shm_connect) debug: connecting to client [6056]
  127. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-execd [6053] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  128. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-execd [6053] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  129. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-execd [6053] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  130. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed register operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=0
  131. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_lrm_control) info: Connection to the executor established
  132. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_started) info: Delaying start, no membership data (0000000000100000)
  133. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_started) info: Delaying start, no membership data (0000000000100000)
  134. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (pcmk_quorum_notification) info: Quorum still lost | membership=162 members=1
  135. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification
  136. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (peer_update_callback) info: Cluster node el8-a01n01 is now member (was in unknown state)
  137. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_reap_unseen_nodes) info: State of node el8-a01n02[2] is still unknown
  138. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_started) info: Delaying start, Config not read (0000000000000040)
  139. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (qb_ipcs_us_publish) info: server name: crmd
  140. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_started) notice: Pacemaker controller successfully started and accepting connections
  141. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_log) info: Input I_PENDING received in state S_STARTING from do_started
  142. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_state_transition) notice: State transition S_STARTING -> S_PENDING | input=I_PENDING cause=C_FSA_INTERNAL origin=do_started
  143. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemakerd [6050] (pcmk_cpg_membership) info: Group pacemakerd event 1: node 2 pid 1596 joined via cluster join
  144. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemakerd [6050] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n01 (node 1 pid 6050) is member
  145. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemakerd [6050] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n02 (node 2 pid 1596) is member
  146. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemakerd [6050] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online
  147. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemakerd [6050] (crm_cs_flush) info: Sent 0 CPG messages (1 remaining, last=7): Try again (6)
  148. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemakerd [6050] (pcmk_quorum_notification) notice: Quorum acquired | membership=166 members=2
  149. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemakerd [6050] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification
  150. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (pcmk_cpg_membership) info: Group attrd event 0: node 1 pid 6054 joined via cpg_join
  151. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-based [6051] (pcmk_cpg_membership) info: Group cib event 1: node 2 pid 1597 joined via cluster join
  152. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-based [6051] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n01 (node 1 pid 6051) is member
  153. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (pcmk_cpg_membership) info: Group attrd event 0: el8-a01n01 (node 1 pid 6054) is member
  154. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (crm_get_peer) info: Created entry ee035951-ee4d-40a4-a722-400d6905e854/0x55c354e27060 for node el8-a01n02/2 (2 total)
  155. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  156. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (crm_get_peer) info: Node 2 has uuid 2
  157. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (pcmk_cpg_membership) info: Group attrd event 0: el8-a01n02 (node 2 pid 1600) is member
  158. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online
  159. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  160. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-based [6051] (crm_get_peer) info: Created entry 7c226def-c52b-4818-8021-a87a2356d821/0x563bd6613490 for node el8-a01n02/2 (2 total)
  161. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-based [6051] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  162. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-based [6051] (crm_get_peer) info: Node 2 has uuid 2
  163. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-based [6051] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n02 (node 2 pid 1597) is member
  164. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-based [6051] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online
  165. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-based [6051] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  166. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/transient_attributes: OK (rc=0, origin=el8-a01n01/attrd/2, version=0.13.0)
  167. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/4, version=0.13.0)
  168. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/transient_attributes: OK (rc=0, origin=el8-a01n02/attrd/2, version=0.13.0)
  169. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n01]: (unset) -> 2 | from el8-a01n01
  170. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_start_election_if_needed) info: Starting an election to determine the writer
  171. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n02]: (unset) -> 2 | from el8-a01n02
  172. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (election_count_vote) info: election-attrd round 1 (owner node ID 2) pass: vote from el8-a01n02 (Uptime)
  173. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (election_check) info: election-attrd won by local node
  174. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_declare_winner) notice: Recorded local node as attribute writer (was unset)
  175. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-attrd [6054] (write_attribute) info: Processed 2 private changes for #attrd-protocol, id=n/a, set=n/a
  176. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/4, version=0.13.0)
  177. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 6096 exited with rc=0
  178. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (stonith_device_register) notice: Added 'virsh_node1' to the device list (1 active devices)
  179. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (stonith_device_register) notice: Added 'virsh_node2' to the device list (2 active devices)
  180. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (pcmk_cpg_membership) info: Group stonith-ng event 1: node 2 pid 1598 joined via cluster join
  181. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n01 (node 1 pid 6052) is member
  182. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (crm_get_peer) info: Created entry b9377532-f6fc-4e21-a8ea-32622712e98c/0x5653eee79000 for node el8-a01n02/2 (2 total)
  183. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  184. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (crm_get_peer) info: Node 2 has uuid 2
  185. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n02 (node 2 pid 1598) is member
  186. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online
  187. Jul 14 21:52:21 el8-a01n01.digimer.ca pacemaker-fenced [6052] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  188. Jul 14 21:52:22 el8-a01n01.digimer.ca pacemaker-controld [6056] (pcmk_quorum_notification) notice: Quorum acquired | membership=166 members=2
  189. Jul 14 21:52:22 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification
  190. Jul 14 21:52:22 el8-a01n01.digimer.ca pacemaker-controld [6056] (peer_update_callback) info: Cluster node el8-a01n02 is now member (was in unknown state)
  191. Jul 14 21:52:22 el8-a01n01.digimer.ca pacemaker-controld [6056] (pcmk_cpg_membership) info: Group crmd event 0: node 1 pid 6056 joined via cpg_join
  192. Jul 14 21:52:22 el8-a01n01.digimer.ca pacemaker-controld [6056] (pcmk_cpg_membership) info: Group crmd event 0: el8-a01n01 (node 1 pid 6056) is member
  193. Jul 14 21:52:22 el8-a01n01.digimer.ca pacemaker-controld [6056] (pcmk_cpg_membership) info: Group crmd event 0: el8-a01n02 (node 2 pid 1602) is member
  194. Jul 14 21:52:22 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online
  195. Jul 14 21:52:22 el8-a01n01.digimer.ca pacemaker-controld [6056] (peer_update_callback) info: Node el8-a01n02 is now a peer | DC=<none> old=0x0000000 new=0x4000000
  196. Jul 14 21:52:22 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds
  197. Jul 14 21:52:22 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 30 seconds
  198. Jul 14 21:52:22 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_connect_stonith) notice: Fencer successfully connected
  199. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_timer_popped) info: Election Trigger just popped | input=I_DC_TIMEOUT time=20000ms
  200. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_log) warning: Input I_DC_TIMEOUT received in state S_PENDING from crm_timer_popped
  201. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_state_transition) info: State transition S_PENDING -> S_ELECTION | input=I_DC_TIMEOUT cause=C_TIMER_POPPED origin=crm_timer_popped
  202. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (election_check) info: election-DC won by local node
  203. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_log) info: Input I_ELECTION_DC received in state S_ELECTION from election_win_cb
  204. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_state_transition) notice: State transition S_ELECTION -> S_INTEGRATION | input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=election_win_cb
  205. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_te_control) info: Registering TE UUID: 7fee6bf5-1c09-43c8-93d7-f6f42af0e11c
  206. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (set_graph_functions) info: Setting custom graph functions
  207. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_dc_takeover) info: Taking over DC status for this partition
  208. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_readwrite) info: We are now in R/W mode
  209. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_master operation for section 'all': OK (rc=0, origin=local/crmd/7, version=0.13.0)
  210. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/8)
  211. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/10)
  212. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/12)
  213. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n01/crmd/8, version=0.13.0)
  214. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n01/crmd/10, version=0.13.0)
  215. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n01/crmd/12, version=0.13.0)
  216. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (join_make_offer) info: Making join-1 offers based on membership event 166
  217. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (join_make_offer) info: Sending join-1 offer to el8-a01n01
  218. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/14)
  219. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (join_make_offer) info: Sending join-1 offer to el8-a01n02
  220. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n01/crmd/14, version=0.13.0)
  221. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_dc_join_offer_all) info: Waiting on join-1 requests from 2 outstanding nodes
  222. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (update_dc) info: Set DC to el8-a01n01 (3.3.0)
  223. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_update_peer_expected) info: update_dc: Node el8-a01n01[1] - expected state is now member (was (null))
  224. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_update_peer_expected) info: do_dc_join_filter_offer: Node el8-a01n02[2] - expected state is now member (was (null))
  225. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_state_transition) info: State transition S_INTEGRATION -> S_FINALIZE_JOIN | input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state
  226. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_replace) info: Digest matched on replace from el8-a01n01: e66cd993ae6725062fe95afbc6bd62d5
  227. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_replace) info: Replaced 0.13.0 with 0.13.0 from el8-a01n01
  228. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=el8-a01n01/crmd/17, version=0.13.0)
  229. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/18)
  230. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (controld_delete_node_state) info: Deleting resource history for node el8-a01n01 (via CIB call 20) | xpath=//node_state[@uname='el8-a01n01']/lrm
  231. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/18, version=0.13.0)
  232. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/19)
  233. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (controld_delete_node_state) info: Deleting resource history for node el8-a01n02 (via CIB call 22) | xpath=//node_state[@uname='el8-a01n02']/lrm
  234. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n01']/lrm to all (origin=local/crmd/20)
  235. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/21)
  236. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/19, version=0.13.0)
  237. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/lrm: OK (rc=0, origin=el8-a01n01/crmd/20, version=0.13.0)
  238. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.0 2
  239. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.1 (null)
  240. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=1
  241. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status: <node_state id="1" uname="el8-a01n01" in_ccm="true" crmd="online" crm-debug-origin="do_lrm_query_internal"/>
  242. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ <lrm id="1">
  243. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ <lrm_resources/>
  244. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ </lrm>
  245. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ </node_state>
  246. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/21, version=0.13.1)
  247. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_state_transition) info: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE | input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state
  248. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (abort_transition_graph) info: Transition -1 aborted: Peer Cancelled | source=do_te_invoke:129 complete=true
  249. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n02']/lrm to all (origin=local/crmd/22)
  250. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/23)
  251. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/lrm: OK (rc=0, origin=el8-a01n01/crmd/22, version=0.13.1)
  252. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.1 2
  253. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.2 (null)
  254. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=2
  255. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status: <node_state id="2" uname="el8-a01n02" in_ccm="true" crmd="online" crm-debug-origin="do_lrm_query_internal"/>
  256. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ <lrm id="2">
  257. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ <lrm_resources/>
  258. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ </lrm>
  259. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ </node_state>
  260. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/23, version=0.13.2)
  261. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/26)
  262. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/27)
  263. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/28)
  264. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/26, version=0.13.2)
  265. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.2 2
  266. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.3 (null)
  267. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=3
  268. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition, @join=member, @expected=member
  269. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition, @join=member, @expected=member
  270. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/27, version=0.13.3)
  271. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.3 2
  272. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.4 (null)
  273. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=4, @dc-uuid=1
  274. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n01/crmd/28, version=0.13.4)
  275. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (determine_online_status_fencing) info: Node el8-a01n01 is active
  276. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (determine_online_status) info: Node el8-a01n01 is online
  277. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (determine_online_status_fencing) info: Node el8-a01n02 is active
  278. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (determine_online_status) info: Node el8-a01n02 is online
  279. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 1 is already processed
  280. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 2 is already processed
  281. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 1 is already processed
  282. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 2 is already processed
  283. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (common_print) info: virsh_node1 (stonith:fence_virsh): Stopped
  284. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (common_print) info: virsh_node2 (stonith:fence_virsh): Stopped
  285. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  286. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (short_print) info: Stopped: [ el8-a01n01 el8-a01n02 ]
  287. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (clone_print) info: Clone Set: drbd-clone [drbd]
  288. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (short_print) info: Stopped: [ el8-a01n01 el8-a01n02 ]
  289. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (RecurringOp) info: Start recurring monitor (60s) for virsh_node1 on el8-a01n01
  290. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (RecurringOp) info: Start recurring monitor (60s) for virsh_node2 on el8-a01n02
  291. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (RecurringOp) info: Start recurring monitor (60s) for hypervisor:0 on el8-a01n01
  292. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (RecurringOp) info: Start recurring monitor (60s) for hypervisor:1 on el8-a01n02
  293. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (RecurringOp) info: Start recurring monitor (60s) for drbd:0 on el8-a01n01
  294. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (RecurringOp) info: Start recurring monitor (60s) for drbd:1 on el8-a01n02
  295. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogAction) notice: * Start virsh_node1 ( el8-a01n01 )
  296. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogAction) notice: * Start virsh_node2 ( el8-a01n02 )
  297. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogAction) notice: * Start hypervisor:0 ( el8-a01n01 )
  298. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogAction) notice: * Start hypervisor:1 ( el8-a01n02 )
  299. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogAction) notice: * Start drbd:0 ( el8-a01n01 )
  300. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogAction) notice: * Start drbd:1 ( el8-a01n02 )
  301. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (pcmk__log_transition_summary) notice: Calculated transition 0, saving inputs in /var/lib/pacemaker/pengine/pe-input-9.bz2
  302. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response
  303. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_te_invoke) info: Processing graph 0 (ref=pe_calc-dc-1594763562-10) derived from /var/lib/pacemaker/pengine/pe-input-9.bz2
  304. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating monitor operation virsh_node1_monitor_0 on el8-a01n02 | action 5
  305. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-16.raw
  306. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating monitor operation virsh_node1_monitor_0 locally on el8-a01n01 | action 1
  307. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node1' not in cache
  308. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=0
  309. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node1'
  310. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=1
  311. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=0
  312. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_lrm_rsc_op) info: Performing key=1:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c op=virsh_node1_monitor_0
  313. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_file_write_with_digest) info: Wrote version 0.13.0 of the CIB to disk (digest: fc0fc716ef8c227358159deeb0d8258c)
  314. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.rvY7DK (digest: /var/lib/pacemaker/cib/cib.mKtSV2)
  315. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.4 2
  316. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.5 (null)
  317. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=5
  318. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_update_resource
  319. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="virsh_node1" type="fence_virsh" class="stonith"/>
  320. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node1_last_0" operation_key="virsh_node1_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="5:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" transition-magic="-1:193;5:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-statu
  321. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ </lrm_resource>
  322. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/8, version=0.13.5)
  323. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.5 2
  324. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.6 (null)
  325. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=6
  326. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;5:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=5, @rc-code=7, @op-status=0, @exec-time=4, @queue-time=1
  327. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/9, version=0.13.6)
  328. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 7821 exited with rc=0
  329. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/30)
  330. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=5, reply=1, notify=0
  331. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:5
  332. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (stonith_api_signon) debug: Attempting fencer connection by pacemaker-execd with mainloop
  333. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.6 2
  334. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.7 (null)
  335. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=7
  336. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource
  337. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="virsh_node1" type="fence_virsh" class="stonith"/>
  338. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node1_last_0" operation_key="virsh_node1_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="1:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" transition-magic="-1:193;1:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-statu
  339. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ </lrm_resource>
  340. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/30, version=0.13.7)
  341. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating monitor operation virsh_node2_monitor_0 on el8-a01n02 | action 6
  342. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating monitor operation virsh_node2_monitor_0 locally on el8-a01n01 | action 2
  343. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  344. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  345. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  346. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (stonith_api_signon) debug: Connection to fencer by pacemaker-execd succeeded (registration token: 12bdea4b-c279-4414-9c2e-dcf1c44cde18)
  347. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:5 exit-code:7 exec-time:4ms queue-time:0ms
  348. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node2' not in cache
  349. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=0
  350. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.7 2
  351. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.8 (null)
  352. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=8
  353. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="virsh_node2" type="fence_virsh" class="stonith"/>
  354. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node2_last_0" operation_key="virsh_node2_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="6:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" transition-magic="-1:193;6:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-statu
  355. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ </lrm_resource>
  356. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/10, version=0.13.8)
  357. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node2'
  358. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=1
  359. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=0
  360. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_lrm_rsc_op) info: Performing key=2:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c op=virsh_node2_monitor_0
  361. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/31)
  362. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=9, reply=1, notify=0
  363. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:virsh_node2 action:monitor call_id:9
  364. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:virsh_node2 action:monitor call_id:9 exit-code:7 exec-time:0ms queue-time:0ms
  365. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.8 2
  366. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.9 (null)
  367. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=9
  368. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;6:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=9, @rc-code=7, @op-status=0
  369. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/11, version=0.13.9)
  370. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.9 2
  371. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.10 (null)
  372. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=10
  373. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="virsh_node2" type="fence_virsh" class="stonith"/>
  374. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node2_last_0" operation_key="virsh_node2_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="2:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" transition-magic="-1:193;2:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-statu
  375. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ </lrm_resource>
  376. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/31, version=0.13.10)
  377. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 5 (virsh_node1_monitor_0 on el8-a01n02) confirmed: not running | rc=7 call-id=5
  378. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 6 (virsh_node2_monitor_0 on el8-a01n02) confirmed: not running | rc=7 call-id=9
  379. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) notice: Result of probe operation for virsh_node1 on el8-a01n01: 7 (not running) | call=5 key=virsh_node1_monitor_0 confirmed=true cib-update=32
  380. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/32)
  381. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.10 2
  382. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.11 (null)
  383. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=11
  384. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;1:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=5, @rc-code=7, @op-status=0, @exec-time=4
  385. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/32, version=0.13.11)
  386. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/33)
  387. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.11 2
  388. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.12 (null)
  389. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=12
  390. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;2:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=9, @rc-code=7, @op-status=0
  391. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/33, version=0.13.12)
  392. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) notice: Result of probe operation for virsh_node2 on el8-a01n01: 7 (not running) | call=9 key=virsh_node2_monitor_0 confirmed=true cib-update=33
  393. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 1 (virsh_node1_monitor_0 on el8-a01n01) confirmed: not running | rc=7 call-id=5
  394. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 2 (virsh_node2_monitor_0 on el8-a01n01) confirmed: not running | rc=7 call-id=9
  395. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating start operation virsh_node1_start_0 locally on el8-a01n01 | action 9
  396. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_lrm_rsc_op) info: Performing key=9:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c op=virsh_node1_start_0
  397. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/34)
  398. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=10, reply=1, notify=0
  399. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) info: executing - rsc:virsh_node1 action:start call_id:10
  400. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.12 2
  401. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.13 (null)
  402. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=13
  403. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_start_0, @operation=start, @transition-key=9:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @transition-magic=-1:193;9:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0
  404. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/34, version=0.13.13)
  405. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating start operation virsh_node2_start_0 on el8-a01n02 | action 11
  406. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating monitor operation hypervisor:0_monitor_0 locally on el8-a01n01 | action 3
  407. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor' not in cache
  408. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=0
  409. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor:0' not in cache
  410. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=0
  411. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_rsc_register) info: Cached agent information for 'hypervisor'
  412. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=1
  413. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=0
  414. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_lrm_rsc_op) info: Performing key=3:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c op=hypervisor_monitor_0
  415. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/35)
  416. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=15, reply=1, notify=0
  417. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:15
  418. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.13 2
  419. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.14 (null)
  420. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=14
  421. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=11:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @transition-magic=-1:193;11:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=-1, @rc-code=193, @op-status=-1
  422. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/12, version=0.13.14)
  423. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.14 2
  424. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.15 (null)
  425. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=15
  426. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="hypervisor" type="libvirtd" class="systemd"/>
  427. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ <lrm_rsc_op id="hypervisor_last_0" operation_key="hypervisor_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="3:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" transition-magic="-1:193;3:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-status=
  428. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ </lrm_resource>
  429. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/35, version=0.13.15)
  430. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating monitor operation hypervisor:1_monitor_0 on el8-a01n02 | action 7
  431. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  432. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.15 2
  433. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.16 (null)
  434. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=16
  435. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="hypervisor" type="libvirtd" class="systemd"/>
  436. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ <lrm_rsc_op id="hypervisor_last_0" operation_key="hypervisor_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="7:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" transition-magic="-1:193;7:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-status=
  437. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ </lrm_resource>
  438. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/13, version=0.13.16)
  439. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  440. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:15 exit-code:7 exec-time:40ms queue-time:0ms
  441. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) notice: Result of probe operation for hypervisor on el8-a01n01: 7 (not running) | call=15 key=hypervisor_monitor_0 confirmed=true cib-update=36
  442. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/36)
  443. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.16 2
  444. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.17 (null)
  445. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=17
  446. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;3:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=15, @rc-code=7, @op-status=0, @exec-time=40
  447. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/36, version=0.13.17)
  448. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 3 (hypervisor_monitor_0 on el8-a01n01) confirmed: not running | rc=7 call-id=15
  449. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating monitor operation drbd:0_monitor_0 locally on el8-a01n01 | action 4
  450. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_get_rsc_info) info: Agent information for 'drbd' not in cache
  451. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=0
  452. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_get_rsc_info) info: Agent information for 'drbd:0' not in cache
  453. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=0
  454. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_rsc_register) info: Cached agent information for 'drbd'
  455. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=1
  456. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=0
  457. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_lrm_rsc_op) info: Performing key=4:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c op=drbd_monitor_0
  458. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/37)
  459. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=20, reply=1, notify=0
  460. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:20
  461. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  462. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.17 2
  463. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.18 (null)
  464. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=18
  465. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="drbd" type="drbd" class="systemd"/>
  466. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ <lrm_rsc_op id="drbd_last_0" operation_key="drbd_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="4:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" transition-magic="-1:193;4:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-status="-1" interva
  467. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ </lrm_resource>
  468. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  469. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/37, version=0.13.18)
  470. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:drbd action:monitor call_id:20 exit-code:7 exec-time:10ms queue-time:0ms
  471. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) notice: Result of probe operation for drbd on el8-a01n01: 7 (not running) | call=20 key=drbd_monitor_0 confirmed=true cib-update=38
  472. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/38)
  473. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.18 2
  474. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.19 (null)
  475. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=19
  476. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;4:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=20, @rc-code=7, @op-status=0, @exec-time=10
  477. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/38, version=0.13.19)
  478. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 4 (drbd_monitor_0 on el8-a01n01) confirmed: not running | rc=7 call-id=20
  479. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.19 2
  480. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.20 (null)
  481. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=20
  482. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;7:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=15, @rc-code=7, @op-status=0, @exec-time=52
  483. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 7 (hypervisor_monitor_0 on el8-a01n02) confirmed: not running | rc=7 call-id=15
  484. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating monitor operation drbd:1_monitor_0 on el8-a01n02 | action 8
  485. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating start operation hypervisor:0_start_0 locally on el8-a01n01 | action 13
  486. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_lrm_rsc_op) info: Performing key=13:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c op=hypervisor_start_0
  487. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/14, version=0.13.20)
  488. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/39)
  489. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=21, reply=1, notify=0
  490. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) info: executing - rsc:hypervisor action:start call_id:21
  491. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous start op on systemd unit libvirtd named 'hypervisor'
  492. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.20 2
  493. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.21 (null)
  494. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=21
  495. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=13:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @transition-magic=-1:193;13:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0
  496. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/39, version=0.13.21)
  497. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.21 2
  498. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.22 (null)
  499. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=22
  500. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="drbd" type="drbd" class="systemd"/>
  501. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ <lrm_rsc_op id="drbd_last_0" operation_key="drbd_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="8:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" transition-magic="-1:193;8:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-status="-1" interva
  502. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ </lrm_resource>
  503. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/15, version=0.13.22)
  504. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.22 2
  505. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.23 (null)
  506. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=23
  507. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;8:0:7:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=20, @rc-code=7, @op-status=0, @exec-time=12
  508. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/16, version=0.13.23)
  509. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 8 (drbd_monitor_0 on el8-a01n02) confirmed: not running | rc=7 call-id=20
  510. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating start operation hypervisor:1_start_0 on el8-a01n02 | action 15
  511. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec_with_unit) debug: Calling StartUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice
  512. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.23 2
  513. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.24 (null)
  514. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=24
  515. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=15:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @transition-magic=-1:193;15:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0
  516. Jul 14 21:52:42 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/17, version=0.13.24)
  517. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_exec_result) info: Call to start passed: /org/freedesktop/systemd1/job/982
  518. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: hypervisor start may still be in progress: re-scheduling (elapsed=222ms, remaining=99778ms, start_delay=2000ms)
  519. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) info: finished - rsc:virsh_node1 action:start call_id:10 exit-code:0 exec-time:784ms queue-time:0ms
  520. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) notice: Result of start operation for virsh_node1 on el8-a01n01: 0 (ok) | call=10 key=virsh_node1_start_0 confirmed=true cib-update=40
  521. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/40)
  522. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.24 2
  523. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.25 (null)
  524. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=25
  525. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:0;9:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=10, @rc-code=0, @op-status=0, @exec-time=784
  526. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/40, version=0.13.25)
  527. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 9 (virsh_node1_start_0 on el8-a01n01) confirmed: ok | rc=0 call-id=10
  528. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating monitor operation virsh_node1_monitor_60000 locally on el8-a01n01 | action 10
  529. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_lrm_rsc_op) info: Performing key=10:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c op=virsh_node1_monitor_60000
  530. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.25 2
  531. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.26 (null)
  532. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=26
  533. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;11:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=10, @rc-code=0, @op-status=0, @exec-time=782
  534. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=22, reply=1, notify=0
  535. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:22
  536. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/18, version=0.13.26)
  537. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/41)
  538. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 11 (virsh_node2_start_0 on el8-a01n02) confirmed: ok | rc=0 call-id=10
  539. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating monitor operation virsh_node2_monitor_60000 on el8-a01n02 | action 12
  540. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.26 2
  541. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.27 (null)
  542. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=27
  543. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']: <lrm_rsc_op id="virsh_node1_monitor_60000" operation_key="virsh_node1_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="10:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" transition-magic="-1:193;10:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" exit-reason="" on_node="el8-
  544. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/41, version=0.13.27)
  545. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.27 2
  546. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.28 (null)
  547. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=28
  548. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']: <lrm_rsc_op id="virsh_node2_monitor_60000" operation_key="virsh_node2_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="12:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" transition-magic="-1:193;12:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" exit-reason="" on_node="el8-
  549. Jul 14 21:52:43 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/19, version=0.13.28)
  550. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.28 2
  551. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.29 (null)
  552. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=29
  553. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_monitor_60000']: @transition-magic=0:0;12:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=22, @rc-code=0, @op-status=0, @exec-time=940
  554. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/20, version=0.13.29)
  555. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 12 (virsh_node2_monitor_60000 on el8-a01n02) confirmed: ok | rc=0 call-id=22
  556. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating start operation drbd:1_start_0 on el8-a01n02 | action 23
  557. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.29 2
  558. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.30 (null)
  559. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=30
  560. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_start_0, @operation=start, @transition-key=23:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @transition-magic=-1:193;23:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594763564, @last-run=1594763564, @exec-time=0
  561. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/21, version=0.13.30)
  562. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:22 exit-code:0 exec-time:972ms queue-time:0ms
  563. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) notice: Result of monitor operation for virsh_node1 on el8-a01n01: 0 (ok) | call=22 key=virsh_node1_monitor_60000 confirmed=false cib-update=42
  564. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/42)
  565. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.30 2
  566. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.31 (null)
  567. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=31
  568. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_monitor_60000']: @transition-magic=0:0;10:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=22, @rc-code=0, @op-status=0, @exec-time=972
  569. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/42, version=0.13.31)
  570. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 10 (virsh_node1_monitor_60000 on el8-a01n01) confirmed: ok | rc=0 call-id=22
  571. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating start operation drbd:0_start_0 locally on el8-a01n01 | action 21
  572. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_lrm_rsc_op) info: Performing key=21:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c op=drbd_start_0
  573. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=23, reply=1, notify=0
  574. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) info: executing - rsc:drbd action:start call_id:23
  575. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous start op on systemd unit drbd named 'drbd'
  576. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/43)
  577. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.31 2
  578. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.32 (null)
  579. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=32
  580. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_start_0, @operation=start, @transition-key=21:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @transition-magic=-1:193;21:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594763564, @last-run=1594763564, @exec-time=0
  581. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/43, version=0.13.32)
  582. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec_with_unit) debug: Calling StartUnit for drbd: /org/freedesktop/systemd1/unit/drbd_2eservice
  583. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_exec_result) info: Call to start passed: /org/freedesktop/systemd1/job/1066
  584. Jul 14 21:52:44 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd start may still be in progress: re-scheduling (elapsed=221ms, remaining=99779ms, start_delay=2000ms)
  585. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:21
  586. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  587. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  588. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: hypervisor start is now complete (elapsed=2230ms, remaining=97770ms): ok (0)
  589. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:21 exit-code:0 exec-time:2230ms queue-time:0ms
  590. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.32 2
  591. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.33 (null)
  592. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=33
  593. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;15:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=21, @rc-code=0, @op-status=0, @exec-time=2210
  594. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/22, version=0.13.33)
  595. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/44)
  596. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) notice: Result of start operation for hypervisor on el8-a01n01: 0 (ok) | call=21 key=hypervisor_start_0 confirmed=true cib-update=44
  597. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.33 2
  598. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.34 (null)
  599. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=34
  600. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;13:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=21, @rc-code=0, @op-status=0, @exec-time=2230
  601. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 15 (hypervisor_start_0 on el8-a01n02) confirmed: ok | rc=0 call-id=21
  602. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/44, version=0.13.34)
  603. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating monitor operation hypervisor:1_monitor_60000 on el8-a01n02 | action 16
  604. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 13 (hypervisor_start_0 on el8-a01n01) confirmed: ok | rc=0 call-id=21
  605. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating monitor operation hypervisor:0_monitor_60000 locally on el8-a01n01 | action 14
  606. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_lrm_rsc_op) info: Performing key=14:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c op=hypervisor_monitor_60000
  607. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=24, reply=1, notify=0
  608. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:24
  609. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  610. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/45)
  611. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.34 2
  612. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.35 (null)
  613. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=35
  614. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']: <lrm_rsc_op id="hypervisor_monitor_60000" operation_key="hypervisor_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="14:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" transition-magic="-1:193;14:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" exit-reason="" on_node="el8-a01
  615. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/45, version=0.13.35)
  616. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.35 2
  617. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.36 (null)
  618. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=36
  619. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']: <lrm_rsc_op id="hypervisor_monitor_60000" operation_key="hypervisor_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="16:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" transition-magic="-1:193;16:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" exit-reason="" on_node="el8-a01
  620. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/23, version=0.13.36)
  621. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  622. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:8ms queue-time:0ms
  623. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) notice: Result of monitor operation for hypervisor on el8-a01n01: 0 (ok) | call=24 key=hypervisor_monitor_60000 confirmed=false cib-update=46
  624. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.36 2
  625. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.37 (null)
  626. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=37
  627. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_monitor_60000']: @transition-magic=0:0;16:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=24, @rc-code=0, @op-status=0, @exec-time=5, @queue-time=1
  628. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/24, version=0.13.37)
  629. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/46)
  630. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 16 (hypervisor_monitor_60000 on el8-a01n02) confirmed: ok | rc=0 call-id=24
  631. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.37 2
  632. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.38 (null)
  633. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=38
  634. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_monitor_60000']: @transition-magic=0:0;14:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=24, @rc-code=0, @op-status=0, @exec-time=8
  635. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/46, version=0.13.38)
  636. Jul 14 21:52:45 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 14 (hypervisor_monitor_60000 on el8-a01n01) confirmed: ok | rc=0 call-id=24
  637. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.38 2
  638. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.39 (null)
  639. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=39
  640. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;23:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=23, @rc-code=0, @op-status=0, @exec-time=2213
  641. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/25, version=0.13.39)
  642. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 23 (drbd_start_0 on el8-a01n02) confirmed: ok | rc=0 call-id=23
  643. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating monitor operation drbd:1_monitor_60000 on el8-a01n02 | action 24
  644. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.39 2
  645. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.40 (null)
  646. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=40
  647. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']: <lrm_rsc_op id="drbd_monitor_60000" operation_key="drbd_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="24:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" transition-magic="-1:193;24:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" exit-reason="" on_node="el8-a01n02" call-id="-1"
  648. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/26, version=0.13.40)
  649. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.40 2
  650. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.41 (null)
  651. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=41
  652. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_monitor_60000']: @transition-magic=0:0;24:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=25, @rc-code=0, @op-status=0, @exec-time=5
  653. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/27, version=0.13.41)
  654. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 24 (drbd_monitor_60000 on el8-a01n02) confirmed: ok | rc=0 call-id=25
  655. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:23
  656. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  657. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  658. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd start is now complete (elapsed=2240ms, remaining=97760ms): ok (0)
  659. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:drbd action:monitor call_id:23 exit-code:0 exec-time:2240ms queue-time:0ms
  660. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) notice: Result of start operation for drbd on el8-a01n01: 0 (ok) | call=23 key=drbd_start_0 confirmed=true cib-update=47
  661. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/47)
  662. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.41 2
  663. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.42 (null)
  664. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=42
  665. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;21:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=23, @rc-code=0, @op-status=0, @exec-time=2240
  666. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 21 (drbd_start_0 on el8-a01n01) confirmed: ok | rc=0 call-id=23
  667. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating monitor operation drbd:0_monitor_60000 locally on el8-a01n01 | action 22
  668. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_lrm_rsc_op) info: Performing key=22:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c op=drbd_monitor_60000
  669. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=25, reply=1, notify=0
  670. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:25
  671. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/47, version=0.13.42)
  672. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  673. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/48)
  674. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.42 2
  675. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.43 (null)
  676. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=43
  677. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']: <lrm_rsc_op id="drbd_monitor_60000" operation_key="drbd_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="22:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" transition-magic="-1:193;22:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" exit-reason="" on_node="el8-a01n01" call-id="-1"
  678. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/48, version=0.13.43)
  679. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  680. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:drbd action:monitor call_id:25 exit-code:0 exec-time:11ms queue-time:0ms
  681. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) notice: Result of monitor operation for drbd on el8-a01n01: 0 (ok) | call=25 key=drbd_monitor_60000 confirmed=false cib-update=49
  682. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/49)
  683. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.43 2
  684. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.44 (null)
  685. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=44
  686. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_monitor_60000']: @transition-magic=0:0;22:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=25, @rc-code=0, @op-status=0, @exec-time=11
  687. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/49, version=0.13.44)
  688. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 0 action 22 (drbd_monitor_60000 on el8-a01n01) confirmed: ok | rc=0 call-id=25
  689. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (run_graph) notice: Transition 0 (Complete=24, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-9.bz2): Complete
  690. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_log) info: Input I_TE_SUCCESS received in state S_TRANSITION_ENGINE from notify_crmd
  691. Jul 14 21:52:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_state_transition) notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd
  692. Jul 14 21:52:51 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_procfs_pid_of) info: Found pacemaker-based active as process 6051
  693. Jul 14 21:52:51 el8-a01n01.digimer.ca pacemaker-controld [6056] (throttle_send_command) info: New throttle mode: negligible load (was undetermined)
  694. Jul 14 21:52:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_ping) info: Reporting our current digest to el8-a01n01: ede44ae72beac282a4507ae397160828 for 0.13.44 (0x563bd67251f0 0)
  695. Jul 14 21:53:44 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:22
  696. Jul 14 21:53:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (recurring_action_timer) debug: Scheduling another invocation of hypervisor_status_60000
  697. Jul 14 21:53:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  698. Jul 14 21:53:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  699. Jul 14 21:53:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  700. Jul 14 21:53:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:22 exit-code:0 exec-time:632ms queue-time:0ms
  701. Jul 14 21:53:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (recurring_action_timer) debug: Scheduling another invocation of drbd_status_60000
  702. Jul 14 21:53:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  703. Jul 14 21:53:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  704. Jul 14 21:53:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:drbd action:monitor call_id:25 exit-code:0 exec-time:0ms queue-time:0ms
  705. Jul 14 21:54:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (recurring_action_timer) debug: Scheduling another invocation of hypervisor_status_60000
  706. Jul 14 21:54:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  707. Jul 14 21:54:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  708. Jul 14 21:54:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  709. Jul 14 21:54:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:22
  710. Jul 14 21:54:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:22 exit-code:0 exec-time:643ms queue-time:0ms
  711. Jul 14 21:54:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (recurring_action_timer) debug: Scheduling another invocation of drbd_status_60000
  712. Jul 14 21:54:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  713. Jul 14 21:54:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  714. Jul 14 21:54:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:drbd action:monitor call_id:25 exit-code:0 exec-time:0ms queue-time:0ms
  715. Jul 14 21:55:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (recurring_action_timer) debug: Scheduling another invocation of hypervisor_status_60000
  716. Jul 14 21:55:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  717. Jul 14 21:55:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  718. Jul 14 21:55:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  719. Jul 14 21:55:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:22
  720. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:22 exit-code:0 exec-time:652ms queue-time:0ms
  721. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (recurring_action_timer) debug: Scheduling another invocation of drbd_status_60000
  722. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  723. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  724. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:drbd action:monitor call_id:25 exit-code:7 exec-time:0ms queue-time:0ms
  725. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) notice: Result of monitor operation for drbd on el8-a01n01: 7 (not running) | call=25 key=drbd_monitor_60000 confirmed=false cib-update=50
  726. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/50)
  727. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.44 2
  728. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.45 (null)
  729. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=45
  730. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']: <lrm_rsc_op id="drbd_last_failure_0" operation_key="drbd_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="22:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" transition-magic="0:7;22:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c" exit-reason="" on_node="el8-a01n01" call-id="25" rc
  731. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/50, version=0.13.45)
  732. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (abort_transition_graph) info: Transition 0 aborted by operation drbd_monitor_60000 'create' on el8-a01n01: Change in recurring result | magic=0:7;22:0:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c cib=0.13.45 source=process_graph_event:406 complete=true
  733. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (update_failcount) info: Updating failcount for drbd on el8-a01n01 after failed monitor: rc=7 (update=value++, time=1594763746)
  734. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) notice: Transition 0 action 22 (drbd_monitor_60000 on el8-a01n01): expected 'ok' but got 'not running' | target-rc=0 rc=7 call-id=25 event='arrived after initial scheduling'
  735. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_state_transition) notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph
  736. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_client_update) info: Expanded fail-count-drbd#monitor_60000=value++ to 1
  737. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_peer_update) notice: Setting fail-count-drbd#monitor_60000[el8-a01n01]: (unset) -> 1 | from el8-a01n01
  738. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-attrd [6054] (write_attribute) info: Sent CIB request 4 with 1 change for fail-count-drbd#monitor_60000 (id n/a, set n/a)
  739. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/4)
  740. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_peer_update) notice: Setting last-failure-drbd#monitor_60000[el8-a01n01]: (unset) -> 1594763746 | from el8-a01n01
  741. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-attrd [6054] (write_attribute) info: Sent CIB request 5 with 1 change for last-failure-drbd#monitor_60000 (id n/a, set n/a)
  742. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.45 2
  743. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.46 (null)
  744. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=46
  745. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']: <transient_attributes id="1"/>
  746. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ <instance_attributes id="status-1">
  747. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ <nvpair id="status-1-fail-count-drbd.monitor_60000" name="fail-count-drbd#monitor_60000" value="1"/>
  748. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ </instance_attributes>
  749. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ </transient_attributes>
  750. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/attrd/4, version=0.13.46)
  751. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_cib_callback) info: CIB update 4 result for fail-count-drbd#monitor_60000: OK | rc=0
  752. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_cib_callback) info: * fail-count-drbd#monitor_60000[el8-a01n01]=1
  753. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/5)
  754. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.46 2
  755. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.47 (null)
  756. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=47
  757. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']: <nvpair id="status-1-last-failure-drbd.monitor_60000" name="last-failure-drbd#monitor_60000" value="1594763746"/>
  758. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/attrd/5, version=0.13.47)
  759. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_cib_callback) info: CIB update 5 result for last-failure-drbd#monitor_60000: OK | rc=0
  760. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_cib_callback) info: * last-failure-drbd#monitor_60000[el8-a01n01]=1594763746
  761. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (abort_transition_graph) info: Transition 0 aborted by transient_attributes.1 'create': Transient attribute change | cib=0.13.46 source=abort_unless_down:329 path=/cib/status/node_state[@id='1'] complete=true
  762. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (abort_transition_graph) info: Transition 0 aborted by status-1-last-failure-drbd.monitor_60000 doing create last-failure-drbd#monitor_60000=1594763746: Transient attribute change | cib=0.13.47 source=abort_unless_down:329 path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1'] complete=true
  763. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (determine_online_status_fencing) info: Node el8-a01n01 is active
  764. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (determine_online_status) info: Node el8-a01n01 is online
  765. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (determine_online_status_fencing) info: Node el8-a01n02 is active
  766. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (determine_online_status) info: Node el8-a01n02 is online
  767. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_rsc_op_failure) warning: Unexpected result (not running) was recorded for monitor of drbd:0 on el8-a01n01 at Jul 14 21:55:46 2020 | rc=7 id=drbd_last_failure_0
  768. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 1 is already processed
  769. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 2 is already processed
  770. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 1 is already processed
  771. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 2 is already processed
  772. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (common_print) info: virsh_node1 (stonith:fence_virsh): Started el8-a01n01
  773. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (common_print) info: virsh_node2 (stonith:fence_virsh): Started el8-a01n02
  774. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  775. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (short_print) info: Started: [ el8-a01n01 el8-a01n02 ]
  776. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (clone_print) info: Clone Set: drbd-clone [drbd]
  777. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (common_print) info: drbd (systemd:drbd): FAILED el8-a01n01
  778. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (short_print) info: Started: [ el8-a01n02 ]
  779. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (RecurringOp) info: Start recurring monitor (60s) for drbd:0 on el8-a01n01
  780. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogActions) info: Leave virsh_node1 (Started el8-a01n01)
  781. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogActions) info: Leave virsh_node2 (Started el8-a01n02)
  782. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogActions) info: Leave hypervisor:0 (Started el8-a01n01)
  783. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogActions) info: Leave hypervisor:1 (Started el8-a01n02)
  784. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogAction) notice: * Recover drbd:0 ( el8-a01n01 )
  785. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogActions) info: Leave drbd:1 (Started el8-a01n02)
  786. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (pcmk__log_transition_summary) notice: Calculated transition 1, saving inputs in /var/lib/pacemaker/pengine/pe-input-10.bz2
  787. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (handle_response) info: pe_calc calculation pe_calc-dc-1594763746-32 is obsolete
  788. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (determine_online_status_fencing) info: Node el8-a01n01 is active
  789. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (determine_online_status) info: Node el8-a01n01 is online
  790. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (determine_online_status_fencing) info: Node el8-a01n02 is active
  791. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (determine_online_status) info: Node el8-a01n02 is online
  792. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_rsc_op_failure) warning: Unexpected result (not running) was recorded for monitor of drbd:0 on el8-a01n01 at Jul 14 21:55:46 2020 | rc=7 id=drbd_last_failure_0
  793. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 1 is already processed
  794. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 2 is already processed
  795. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 1 is already processed
  796. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 2 is already processed
  797. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (common_print) info: virsh_node1 (stonith:fence_virsh): Started el8-a01n01
  798. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (common_print) info: virsh_node2 (stonith:fence_virsh): Started el8-a01n02
  799. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  800. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (short_print) info: Started: [ el8-a01n01 el8-a01n02 ]
  801. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (clone_print) info: Clone Set: drbd-clone [drbd]
  802. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (common_print) info: drbd (systemd:drbd): FAILED el8-a01n01
  803. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (short_print) info: Started: [ el8-a01n02 ]
  804. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (pe_get_failcount) info: drbd:0 has failed 1 times on el8-a01n01
  805. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (check_migration_threshold) info: drbd-clone can fail 999999 more times on el8-a01n01 before being forced off
  806. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (pe_get_failcount) info: drbd:1 has failed 1 times on el8-a01n01
  807. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (check_migration_threshold) info: drbd-clone can fail 999999 more times on el8-a01n01 before being forced off
  808. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (RecurringOp) info: Start recurring monitor (60s) for drbd:0 on el8-a01n01
  809. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogActions) info: Leave virsh_node1 (Started el8-a01n01)
  810. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogActions) info: Leave virsh_node2 (Started el8-a01n02)
  811. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogActions) info: Leave hypervisor:0 (Started el8-a01n01)
  812. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogActions) info: Leave hypervisor:1 (Started el8-a01n02)
  813. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogAction) notice: * Recover drbd:0 ( el8-a01n01 )
  814. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogActions) info: Leave drbd:1 (Started el8-a01n02)
  815. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (pcmk__log_transition_summary) notice: Calculated transition 2, saving inputs in /var/lib/pacemaker/pengine/pe-input-11.bz2
  816. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response
  817. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_te_invoke) info: Processing graph 2 (ref=pe_calc-dc-1594763746-33) derived from /var/lib/pacemaker/pengine/pe-input-11.bz2
  818. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating stop operation drbd_stop_0 locally on el8-a01n01 | action 4
  819. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (cancel_recurring_action) info: Cancelling systemd operation drbd_status_60000
  820. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:drbd action:monitor call_id:25 exit-code:7 exec-time:0ms queue-time:0ms
  821. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=0
  822. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_lrm_rsc_op) info: Performing key=4:2:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c op=drbd_stop_0
  823. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/54)
  824. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=27, reply=1, notify=0
  825. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) info: executing - rsc:drbd action:stop call_id:27
  826. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit drbd named 'drbd'
  827. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) info: Result of monitor operation for drbd on el8-a01n01: Cancelled | call=25 key=drbd_monitor_60000 confirmed=true
  828. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.47 2
  829. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.48 (null)
  830. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=48
  831. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_stop_0, @operation=stop, @transition-key=4:2:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @transition-magic=-1:193;4:2:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594763746, @last-run=1594763746, @exec-time=0
  832. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/54, version=0.13.48)
  833. Jul 14 21:55:46 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec_with_unit) debug: Calling StopUnit for drbd: /org/freedesktop/systemd1/unit/drbd_2eservice
  834. Jul 14 21:55:47 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/1171
  835. Jul 14 21:55:47 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=232ms, remaining=99768ms, start_delay=2000ms)
  836. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:27
  837. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  838. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  839. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop is now complete (elapsed=2241ms, remaining=97759ms): not running (7)
  840. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:drbd action:monitor call_id:27 exit-code:0 exec-time:2241ms queue-time:0ms
  841. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) notice: Result of stop operation for drbd on el8-a01n01: 0 (ok) | call=27 key=drbd_stop_0 confirmed=true cib-update=55
  842. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/55)
  843. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.48 2
  844. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.49 (null)
  845. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=49
  846. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;4:2:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=27, @rc-code=0, @op-status=0, @exec-time=2241
  847. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/55, version=0.13.49)
  848. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 2 action 4 (drbd_stop_0 on el8-a01n01) confirmed: ok | rc=0 call-id=27
  849. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating start operation drbd_start_0 locally on el8-a01n01 | action 20
  850. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_lrm_rsc_op) info: Performing key=20:2:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c op=drbd_start_0
  851. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/56)
  852. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.49 2
  853. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.50 (null)
  854. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=50
  855. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_start_0, @operation=start, @transition-key=20:2:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @transition-magic=-1:193;20:2:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594763749, @last-run=1594763749, @exec-time=0
  856. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=28, reply=1, notify=0
  857. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) info: executing - rsc:drbd action:start call_id:28
  858. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous start op on systemd unit drbd named 'drbd'
  859. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/56, version=0.13.50)
  860. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec_with_unit) debug: Calling StartUnit for drbd: /org/freedesktop/systemd1/unit/drbd_2eservice
  861. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_exec_result) info: Call to start passed: /org/freedesktop/systemd1/job/1172
  862. Jul 14 21:55:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd start may still be in progress: re-scheduling (elapsed=179ms, remaining=99821ms, start_delay=2000ms)
  863. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:28
  864. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  865. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  866. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd start is now complete (elapsed=2187ms, remaining=97813ms): ok (0)
  867. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:drbd action:monitor call_id:28 exit-code:0 exec-time:2187ms queue-time:0ms
  868. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) notice: Result of start operation for drbd on el8-a01n01: 0 (ok) | call=28 key=drbd_start_0 confirmed=true cib-update=57
  869. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/57)
  870. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.50 2
  871. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.51 (null)
  872. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=51
  873. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;20:2:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=28, @rc-code=0, @op-status=0, @exec-time=2187
  874. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/57, version=0.13.51)
  875. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 2 action 20 (drbd_start_0 on el8-a01n01) confirmed: ok | rc=0 call-id=28
  876. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating monitor operation drbd_monitor_60000 locally on el8-a01n01 | action 3
  877. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_lrm_rsc_op) info: Performing key=3:2:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c op=drbd_monitor_60000
  878. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/58)
  879. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=29, reply=1, notify=0
  880. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:29
  881. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  882. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.51 2
  883. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.52 (null)
  884. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=52
  885. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_monitor_60000']: @transition-key=3:2:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @transition-magic=-1:193;3:2:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594763751, @exec-time=0
  886. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/58, version=0.13.52)
  887. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  888. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:drbd action:monitor call_id:29 exit-code:0 exec-time:4ms queue-time:0ms
  889. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) notice: Result of monitor operation for drbd on el8-a01n01: 0 (ok) | call=29 key=drbd_monitor_60000 confirmed=false cib-update=59
  890. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/59)
  891. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.52 2
  892. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.53 (null)
  893. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=53
  894. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_monitor_60000']: @transition-magic=0:0;3:2:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=29, @rc-code=0, @op-status=0, @exec-time=4
  895. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/59, version=0.13.53)
  896. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 2 action 3 (drbd_monitor_60000 on el8-a01n01) confirmed: ok | rc=0 call-id=29
  897. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-controld [6056] (run_graph) notice: Transition 2 (Complete=7, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-11.bz2): Complete
  898. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_log) info: Input I_TE_SUCCESS received in state S_TRANSITION_ENGINE from notify_crmd
  899. Jul 14 21:55:51 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_state_transition) notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd
  900. Jul 14 21:55:56 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_ping) info: Reporting our current digest to el8-a01n01: 6a01ad05ce19e14ae684dfabd34d038a for 0.13.53 (0x563bd67251f0 0)
  901. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemakerd [6050] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  902. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemakerd [6050] (pcmk_shutdown_worker) notice: Shutting down Pacemaker
  903. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemakerd [6050] (stop_child) notice: Stopping pacemaker-controld | sent signal 15 to process 6056
  904. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  905. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_shutdown) notice: Shutting down cluster resource manager | limit=1200000ms
  906. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_log) info: Input I_SHUTDOWN received in state S_IDLE from crm_shutdown
  907. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_state_transition) notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_SHUTDOWN cause=C_SHUTDOWN origin=crm_shutdown
  908. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_shutdown_req) info: Sending shutdown request to all peers (DC is el8-a01n01)
  909. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_update_peer_expected) info: handle_request: Node el8-a01n01[1] - expected state is now down (was member)
  910. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (handle_shutdown_request) info: Creating shutdown request for el8-a01n01 (state=S_POLICY_ENGINE)
  911. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_peer_update) notice: Setting shutdown[el8-a01n01]: (unset) -> 1594763793 | from el8-a01n01
  912. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-attrd [6054] (write_attribute) info: Sent CIB request 6 with 1 change for shutdown (id n/a, set n/a)
  913. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/6)
  914. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.53 2
  915. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.54 (null)
  916. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=54
  917. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']: <nvpair id="status-1-shutdown" name="shutdown" value="1594763793"/>
  918. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/attrd/6, version=0.13.54)
  919. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (abort_transition_graph) info: Transition 2 aborted by status-1-shutdown doing create shutdown=1594763793: Transient attribute change | cib=0.13.54 source=abort_unless_down:329 path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1'] complete=true
  920. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_cib_callback) info: CIB update 6 result for shutdown: OK | rc=0
  921. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_cib_callback) info: * shutdown[el8-a01n01]=1594763793
  922. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (determine_online_status) info: Node el8-a01n01 is shutting down
  923. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (determine_online_status_fencing) info: Node el8-a01n02 is active
  924. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (determine_online_status) info: Node el8-a01n02 is online
  925. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_rsc_op_failure) warning: Unexpected result (not running) was recorded for monitor of drbd:0 on el8-a01n01 at Jul 14 21:55:46 2020 | rc=7 id=drbd_last_failure_0
  926. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 1 is already processed
  927. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 2 is already processed
  928. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 1 is already processed
  929. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 2 is already processed
  930. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (common_print) info: virsh_node1 (stonith:fence_virsh): Started el8-a01n01
  931. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (common_print) info: virsh_node2 (stonith:fence_virsh): Started el8-a01n02
  932. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  933. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (short_print) info: Started: [ el8-a01n01 el8-a01n02 ]
  934. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (clone_print) info: Clone Set: drbd-clone [drbd]
  935. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (short_print) info: Started: [ el8-a01n01 el8-a01n02 ]
  936. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (pe_get_failcount) info: drbd:0 has failed 1 times on el8-a01n01
  937. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (check_migration_threshold) info: drbd-clone can fail 999999 more times on el8-a01n01 before being forced off
  938. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (pe_get_failcount) info: drbd:1 has failed 1 times on el8-a01n01
  939. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (check_migration_threshold) info: drbd-clone can fail 999999 more times on el8-a01n01 before being forced off
  940. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (native_color) info: Resource hypervisor:0 cannot run anywhere
  941. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (native_color) info: Resource drbd:0 cannot run anywhere
  942. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (RecurringOp) info: Start recurring monitor (60s) for virsh_node1 on el8-a01n02
  943. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (sched_shutdown_op) notice: Scheduling shutdown of node el8-a01n01
  944. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogNodeActions) notice: * Shutdown el8-a01n01
  945. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogAction) notice: * Move virsh_node1 ( el8-a01n01 -> el8-a01n02 )
  946. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogActions) info: Leave virsh_node2 (Started el8-a01n02)
  947. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogAction) notice: * Stop hypervisor:0 ( el8-a01n01 ) due to node availability
  948. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogActions) info: Leave hypervisor:1 (Started el8-a01n02)
  949. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogAction) notice: * Stop drbd:0 ( el8-a01n01 ) due to node availability
  950. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogActions) info: Leave drbd:1 (Started el8-a01n02)
  951. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (pcmk__log_transition_summary) notice: Calculated transition 3, saving inputs in /var/lib/pacemaker/pengine/pe-input-12.bz2
  952. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (crm_update_peer_expected) info: handle_request: Node el8-a01n02[2] - expected state is now down (was member)
  953. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (handle_shutdown_request) info: Creating shutdown request for el8-a01n02 (state=S_POLICY_ENGINE)
  954. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_peer_update) notice: Setting shutdown[el8-a01n02]: (unset) -> 1594763793 | from el8-a01n01
  955. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-attrd [6054] (write_attribute) info: Sent CIB request 7 with 2 changes for shutdown (id n/a, set n/a)
  956. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/7)
  957. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response
  958. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_te_invoke) info: Processing graph 3 (ref=pe_calc-dc-1594763793-38) derived from /var/lib/pacemaker/pengine/pe-input-12.bz2
  959. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating stop operation virsh_node1_stop_0 locally on el8-a01n01 | action 7
  960. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.54 2
  961. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.55 (null)
  962. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=55
  963. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']: <transient_attributes id="2"/>
  964. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ <instance_attributes id="status-2">
  965. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ <nvpair id="status-2-shutdown" name="shutdown" value="1594763793"/>
  966. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ </instance_attributes>
  967. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: ++ </transient_attributes>
  968. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:22 exit-code:0 exec-time:0ms queue-time:0ms
  969. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=0
  970. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_lrm_rsc_op) info: Performing key=7:3:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c op=virsh_node1_stop_0
  971. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=31, reply=1, notify=0
  972. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) info: executing - rsc:virsh_node1 action:stop call_id:31
  973. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/attrd/7, version=0.13.55)
  974. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/61)
  975. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) info: finished - rsc:virsh_node1 action:stop call_id:31 exit-code:0 exec-time:1ms queue-time:0ms
  976. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_cib_callback) info: CIB update 7 result for shutdown: OK | rc=0
  977. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_cib_callback) info: * shutdown[el8-a01n01]=1594763793
  978. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-attrd [6054] (attrd_cib_callback) info: * shutdown[el8-a01n02]=1594763793
  979. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (abort_transition_graph) notice: Transition 3 aborted by transient_attributes.2 'create': Transient attribute change | cib=0.13.55 source=abort_unless_down:329 path=/cib/status/node_state[@id='2'] complete=false
  980. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) info: Result of monitor operation for virsh_node1 on el8-a01n01: Cancelled | call=22 key=virsh_node1_monitor_60000 confirmed=true
  981. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.55 2
  982. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.56 (null)
  983. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=56
  984. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_stop_0, @operation=stop, @transition-key=7:3:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @transition-magic=-1:193;7:3:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594763793, @last-run=1594763793, @exec-time=0
  985. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) notice: Result of stop operation for virsh_node1 on el8-a01n01: 0 (ok) | call=31 key=virsh_node1_stop_0 confirmed=true cib-update=62
  986. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/61, version=0.13.56)
  987. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/62)
  988. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.56 2
  989. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.57 (null)
  990. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=57
  991. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:0;7:3:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=31, @rc-code=0, @op-status=0, @exec-time=1
  992. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 3 action 7 (virsh_node1_stop_0 on el8-a01n01) confirmed: ok | rc=0 call-id=31
  993. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (run_graph) notice: Transition 3 (Complete=3, Pending=0, Fired=0, Skipped=3, Incomplete=7, Source=/var/lib/pacemaker/pengine/pe-input-12.bz2): Stopped
  994. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_state_transition) info: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd
  995. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/62, version=0.13.57)
  996. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (determine_online_status) info: Node el8-a01n01 is shutting down
  997. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (determine_online_status) info: Node el8-a01n02 is shutting down
  998. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_rsc_op_failure) warning: Unexpected result (not running) was recorded for monitor of drbd:0 on el8-a01n01 at Jul 14 21:55:46 2020 | rc=7 id=drbd_last_failure_0
  999. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 1 is already processed
  1000. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 2 is already processed
  1001. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 1 is already processed
  1002. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (unpack_node_loop) info: Node 2 is already processed
  1003. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (common_print) info: virsh_node1 (stonith:fence_virsh): Stopped
  1004. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (common_print) info: virsh_node2 (stonith:fence_virsh): Started el8-a01n02
  1005. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  1006. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (short_print) info: Started: [ el8-a01n01 el8-a01n02 ]
  1007. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (clone_print) info: Clone Set: drbd-clone [drbd]
  1008. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (short_print) info: Started: [ el8-a01n01 el8-a01n02 ]
  1009. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (pe_get_failcount) info: drbd:0 has failed 1 times on el8-a01n01
  1010. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (check_migration_threshold) info: drbd-clone can fail 999999 more times on el8-a01n01 before being forced off
  1011. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (pe_get_failcount) info: drbd:1 has failed 1 times on el8-a01n01
  1012. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (check_migration_threshold) info: drbd-clone can fail 999999 more times on el8-a01n01 before being forced off
  1013. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (native_color) info: Resource virsh_node1 cannot run anywhere
  1014. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (native_color) info: Resource virsh_node2 cannot run anywhere
  1015. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (native_color) info: Resource hypervisor:0 cannot run anywhere
  1016. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (native_color) info: Resource hypervisor:1 cannot run anywhere
  1017. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (native_color) info: Resource drbd:0 cannot run anywhere
  1018. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (native_color) info: Resource drbd:1 cannot run anywhere
  1019. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (sched_shutdown_op) notice: Scheduling shutdown of node el8-a01n01
  1020. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (sched_shutdown_op) notice: Scheduling shutdown of node el8-a01n02
  1021. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogNodeActions) notice: * Shutdown el8-a01n02
  1022. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogNodeActions) notice: * Shutdown el8-a01n01
  1023. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogActions) info: Leave virsh_node1 (Stopped)
  1024. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogAction) notice: * Stop virsh_node2 ( el8-a01n02 ) due to node availability
  1025. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogAction) notice: * Stop hypervisor:0 ( el8-a01n01 ) due to node availability
  1026. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogAction) notice: * Stop hypervisor:1 ( el8-a01n02 ) due to node availability
  1027. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogAction) notice: * Stop drbd:0 ( el8-a01n01 ) due to node availability
  1028. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (LogAction) notice: * Stop drbd:1 ( el8-a01n02 ) due to node availability
  1029. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-schedulerd[6055] (pcmk__log_transition_summary) notice: Calculated transition 4, saving inputs in /var/lib/pacemaker/pengine/pe-input-13.bz2
  1030. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response
  1031. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_te_invoke) info: Processing graph 4 (ref=pe_calc-dc-1594763793-40) derived from /var/lib/pacemaker/pengine/pe-input-13.bz2
  1032. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating stop operation virsh_node2_stop_0 on el8-a01n02 | action 6
  1033. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating stop operation hypervisor_stop_0 locally on el8-a01n01 | action 7
  1034. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (cancel_recurring_action) info: Cancelling systemd operation hypervisor_status_60000
  1035. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  1036. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=0
  1037. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_lrm_rsc_op) info: Performing key=7:4:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c op=hypervisor_stop_0
  1038. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=33, reply=1, notify=0
  1039. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) info: executing - rsc:hypervisor action:stop call_id:33
  1040. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit libvirtd named 'hypervisor'
  1041. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/64)
  1042. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.57 2
  1043. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.58 (null)
  1044. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=58
  1045. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=7:4:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @transition-magic=-1:193;7:4:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594763793, @last-run=1594763793, @exec-time=0
  1046. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec_with_unit) debug: Calling StopUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice
  1047. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/64, version=0.13.58)
  1048. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.58 2
  1049. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.59 (null)
  1050. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=59
  1051. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=6:4:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @transition-magic=-1:193;6:4:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594763793, @last-run=1594763793, @exec-time=0
  1052. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/28, version=0.13.59)
  1053. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.59 2
  1054. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.60 (null)
  1055. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=60
  1056. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;6:4:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=27, @rc-code=0, @op-status=0, @exec-time=1
  1057. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/29, version=0.13.60)
  1058. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating stop operation hypervisor_stop_0 on el8-a01n02 | action 8
  1059. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating stop operation drbd_stop_0 locally on el8-a01n01 | action 13
  1060. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (cancel_recurring_action) info: Cancelling systemd operation drbd_status_60000
  1061. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:drbd action:monitor call_id:29 exit-code:0 exec-time:0ms queue-time:0ms
  1062. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=0, reply=1, notify=0
  1063. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (do_lrm_rsc_op) info: Performing key=13:4:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c op=drbd_stop_0
  1064. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 29872d72-cf4e-4597-93c4-86f8b71374d4: rc=35, reply=1, notify=0
  1065. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) info: executing - rsc:drbd action:stop call_id:35
  1066. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit drbd named 'drbd'
  1067. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/65)
  1068. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (te_rsc_command) notice: Initiating stop operation drbd_stop_0 on el8-a01n02 | action 14
  1069. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.60 2
  1070. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.61 (null)
  1071. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=61
  1072. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_stop_0, @operation=stop, @transition-key=13:4:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @transition-magic=-1:193;13:4:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594763793, @last-run=1594763793, @exec-time=0
  1073. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/65, version=0.13.61)
  1074. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 4 action 6 (virsh_node2_stop_0 on el8-a01n02) confirmed: ok | rc=0 call-id=27
  1075. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) info: Result of monitor operation for hypervisor on el8-a01n01: Cancelled | call=24 key=hypervisor_monitor_60000 confirmed=true
  1076. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) info: Result of monitor operation for drbd on el8-a01n01: Cancelled | call=29 key=drbd_monitor_60000 confirmed=true
  1077. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.61 2
  1078. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.62 (null)
  1079. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=62
  1080. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=8:4:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @transition-magic=-1:193;8:4:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594763793, @last-run=1594763793, @exec-time=0
  1081. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/30, version=0.13.62)
  1082. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.62 2
  1083. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.63 (null)
  1084. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=63
  1085. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_stop_0, @operation=stop, @transition-key=14:4:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @transition-magic=-1:193;14:4:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594763793, @last-run=1594763793, @exec-time=0
  1086. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/31, version=0.13.63)
  1087. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/1267
  1088. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: hypervisor stop may still be in progress: re-scheduling (elapsed=300ms, remaining=99700ms, start_delay=2000ms)
  1089. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec_with_unit) debug: Calling StopUnit for drbd: /org/freedesktop/systemd1/unit/drbd_2eservice
  1090. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/1268
  1091. Jul 14 21:56:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=497ms, remaining=99503ms, start_delay=2000ms)
  1092. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.63 2
  1093. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.64 (null)
  1094. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=64
  1095. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;8:4:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=29, @rc-code=0, @op-status=0, @exec-time=2194, @queue-time=1
  1096. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/32, version=0.13.64)
  1097. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 4 action 8 (hypervisor_stop_0 on el8-a01n02) confirmed: ok | rc=0 call-id=29
  1098. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:33
  1099. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  1100. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1101. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: hypervisor stop is now complete (elapsed=2306ms, remaining=97694ms): not running (7)
  1102. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:33 exit-code:0 exec-time:2307ms queue-time:0ms
  1103. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_lrm_event) notice: Result of stop operation for hypervisor on el8-a01n01: 0 (ok) | call=33 key=hypervisor_stop_0 confirmed=true cib-update=66
  1104. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/66)
  1105. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: --- 0.13.64 2
  1106. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: Diff: +++ 0.13.65 (null)
  1107. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib: @num_updates=65
  1108. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;7:4:0:7fee6bf5-1c09-43c8-93d7-f6f42af0e11c, @call-id=33, @rc-code=0, @op-status=0, @exec-time=2307
  1109. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/66, version=0.13.65)
  1110. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-controld [6056] (process_graph_event) info: Transition 4 action 7 (hypervisor_stop_0 on el8-a01n01) confirmed: ok | rc=0 call-id=33
  1111. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1112. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1113. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1114. Jul 14 21:56:35 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=2504ms, remaining=97496ms, start_delay=2000ms)
  1115. Jul 14 21:56:37 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1116. Jul 14 21:56:37 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1117. Jul 14 21:56:37 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1118. Jul 14 21:56:37 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=4507ms, remaining=95493ms, start_delay=2000ms)
  1119. Jul 14 21:56:39 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1120. Jul 14 21:56:39 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1121. Jul 14 21:56:39 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1122. Jul 14 21:56:39 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=6515ms, remaining=93485ms, start_delay=2000ms)
  1123. Jul 14 21:56:40 el8-a01n01.digimer.ca pacemaker-based [6051] (cib_process_ping) info: Reporting our current digest to el8-a01n01: 0265b0a70b084f185c477f8c2659b229 for 0.13.65 (0x563bd67251f0 0)
  1124. Jul 14 21:56:41 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1125. Jul 14 21:56:41 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1126. Jul 14 21:56:41 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1127. Jul 14 21:56:41 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=8528ms, remaining=91472ms, start_delay=2000ms)
  1128. Jul 14 21:56:43 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1129. Jul 14 21:56:43 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1130. Jul 14 21:56:43 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1131. Jul 14 21:56:43 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=10542ms, remaining=89458ms, start_delay=2000ms)
  1132. Jul 14 21:56:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1133. Jul 14 21:56:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1134. Jul 14 21:56:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1135. Jul 14 21:56:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=12546ms, remaining=87454ms, start_delay=2000ms)
  1136. Jul 14 21:56:47 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1137. Jul 14 21:56:47 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1138. Jul 14 21:56:47 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1139. Jul 14 21:56:47 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=14549ms, remaining=85451ms, start_delay=2000ms)
  1140. Jul 14 21:56:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1141. Jul 14 21:56:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1142. Jul 14 21:56:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1143. Jul 14 21:56:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=16553ms, remaining=83447ms, start_delay=2000ms)
  1144. Jul 14 21:56:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1145. Jul 14 21:56:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1146. Jul 14 21:56:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1147. Jul 14 21:56:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=18556ms, remaining=81444ms, start_delay=2000ms)
  1148. Jul 14 21:56:53 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1149. Jul 14 21:56:53 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1150. Jul 14 21:56:53 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1151. Jul 14 21:56:53 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=20560ms, remaining=79440ms, start_delay=2000ms)
  1152. Jul 14 21:56:55 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1153. Jul 14 21:56:55 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1154. Jul 14 21:56:55 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1155. Jul 14 21:56:55 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=22563ms, remaining=77437ms, start_delay=2000ms)
  1156. Jul 14 21:56:57 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1157. Jul 14 21:56:57 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1158. Jul 14 21:56:57 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1159. Jul 14 21:56:57 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=24570ms, remaining=75430ms, start_delay=2000ms)
  1160. Jul 14 21:56:59 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1161. Jul 14 21:56:59 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1162. Jul 14 21:56:59 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1163. Jul 14 21:56:59 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=26576ms, remaining=73424ms, start_delay=2000ms)
  1164. Jul 14 21:57:01 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1165. Jul 14 21:57:01 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1166. Jul 14 21:57:01 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1167. Jul 14 21:57:01 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=28592ms, remaining=71408ms, start_delay=2000ms)
  1168. Jul 14 21:57:03 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1169. Jul 14 21:57:03 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1170. Jul 14 21:57:03 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1171. Jul 14 21:57:03 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=30595ms, remaining=69405ms, start_delay=2000ms)
  1172. Jul 14 21:57:05 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1173. Jul 14 21:57:05 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1174. Jul 14 21:57:05 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1175. Jul 14 21:57:05 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=32598ms, remaining=67402ms, start_delay=2000ms)
  1176. Jul 14 21:57:07 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1177. Jul 14 21:57:07 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1178. Jul 14 21:57:07 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1179. Jul 14 21:57:07 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=34601ms, remaining=65399ms, start_delay=2000ms)
  1180. Jul 14 21:57:09 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1181. Jul 14 21:57:09 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1182. Jul 14 21:57:09 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1183. Jul 14 21:57:09 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=36605ms, remaining=63395ms, start_delay=2000ms)
  1184. Jul 14 21:57:11 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1185. Jul 14 21:57:11 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1186. Jul 14 21:57:11 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1187. Jul 14 21:57:11 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=38612ms, remaining=61388ms, start_delay=2000ms)
  1188. Jul 14 21:57:13 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1189. Jul 14 21:57:13 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1190. Jul 14 21:57:13 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1191. Jul 14 21:57:13 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=40615ms, remaining=59385ms, start_delay=2000ms)
  1192. Jul 14 21:57:15 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1193. Jul 14 21:57:15 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1194. Jul 14 21:57:15 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1195. Jul 14 21:57:15 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=42618ms, remaining=57382ms, start_delay=2000ms)
  1196. Jul 14 21:57:17 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1197. Jul 14 21:57:17 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1198. Jul 14 21:57:17 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1199. Jul 14 21:57:17 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=44622ms, remaining=55378ms, start_delay=2000ms)
  1200. Jul 14 21:57:19 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1201. Jul 14 21:57:19 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1202. Jul 14 21:57:19 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1203. Jul 14 21:57:19 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=46627ms, remaining=53373ms, start_delay=2000ms)
  1204. Jul 14 21:57:21 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1205. Jul 14 21:57:21 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1206. Jul 14 21:57:21 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1207. Jul 14 21:57:21 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=48630ms, remaining=51370ms, start_delay=2000ms)
  1208. Jul 14 21:57:23 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1209. Jul 14 21:57:23 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1210. Jul 14 21:57:23 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1211. Jul 14 21:57:23 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=50633ms, remaining=49367ms, start_delay=2000ms)
  1212. Jul 14 21:57:25 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1213. Jul 14 21:57:25 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1214. Jul 14 21:57:25 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1215. Jul 14 21:57:25 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=52638ms, remaining=47362ms, start_delay=2000ms)
  1216. Jul 14 21:57:27 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1217. Jul 14 21:57:27 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1218. Jul 14 21:57:27 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1219. Jul 14 21:57:27 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=54645ms, remaining=45355ms, start_delay=2000ms)
  1220. Jul 14 21:57:29 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1221. Jul 14 21:57:29 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1222. Jul 14 21:57:29 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1223. Jul 14 21:57:29 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=56649ms, remaining=43351ms, start_delay=2000ms)
  1224. Jul 14 21:57:31 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1225. Jul 14 21:57:31 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1226. Jul 14 21:57:31 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1227. Jul 14 21:57:31 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=58652ms, remaining=41348ms, start_delay=2000ms)
  1228. Jul 14 21:57:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1229. Jul 14 21:57:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1230. Jul 14 21:57:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1231. Jul 14 21:57:33 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=60654ms, remaining=39346ms, start_delay=2000ms)
  1232. Jul 14 21:57:35 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1233. Jul 14 21:57:35 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1234. Jul 14 21:57:35 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1235. Jul 14 21:57:35 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=62658ms, remaining=37342ms, start_delay=2000ms)
  1236. Jul 14 21:57:37 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1237. Jul 14 21:57:37 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1238. Jul 14 21:57:37 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1239. Jul 14 21:57:37 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=64661ms, remaining=35339ms, start_delay=2000ms)
  1240. Jul 14 21:57:39 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1241. Jul 14 21:57:39 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1242. Jul 14 21:57:39 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1243. Jul 14 21:57:39 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=66665ms, remaining=33335ms, start_delay=2000ms)
  1244. Jul 14 21:57:41 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1245. Jul 14 21:57:41 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1246. Jul 14 21:57:41 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1247. Jul 14 21:57:41 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=68669ms, remaining=31331ms, start_delay=2000ms)
  1248. Jul 14 21:57:43 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1249. Jul 14 21:57:43 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1250. Jul 14 21:57:43 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1251. Jul 14 21:57:43 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=70672ms, remaining=29328ms, start_delay=2000ms)
  1252. Jul 14 21:57:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1253. Jul 14 21:57:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1254. Jul 14 21:57:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1255. Jul 14 21:57:45 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=72676ms, remaining=27324ms, start_delay=2000ms)
  1256. Jul 14 21:57:47 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1257. Jul 14 21:57:47 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1258. Jul 14 21:57:47 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1259. Jul 14 21:57:47 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=74679ms, remaining=25321ms, start_delay=2000ms)
  1260. Jul 14 21:57:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1261. Jul 14 21:57:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1262. Jul 14 21:57:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1263. Jul 14 21:57:49 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=76683ms, remaining=23317ms, start_delay=2000ms)
  1264. Jul 14 21:57:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1265. Jul 14 21:57:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1266. Jul 14 21:57:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1267. Jul 14 21:57:51 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=78686ms, remaining=21314ms, start_delay=2000ms)
  1268. Jul 14 21:57:53 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1269. Jul 14 21:57:53 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1270. Jul 14 21:57:53 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1271. Jul 14 21:57:53 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=80688ms, remaining=19312ms, start_delay=2000ms)
  1272. Jul 14 21:57:55 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1273. Jul 14 21:57:55 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1274. Jul 14 21:57:55 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1275. Jul 14 21:57:55 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=82695ms, remaining=17305ms, start_delay=2000ms)
  1276. Jul 14 21:57:57 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1277. Jul 14 21:57:57 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1278. Jul 14 21:57:57 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1279. Jul 14 21:57:57 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=84698ms, remaining=15302ms, start_delay=2000ms)
  1280. Jul 14 21:57:59 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1281. Jul 14 21:57:59 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1282. Jul 14 21:57:59 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1283. Jul 14 21:57:59 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=86701ms, remaining=13299ms, start_delay=2000ms)
  1284. Jul 14 21:58:01 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1285. Jul 14 21:58:01 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1286. Jul 14 21:58:01 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1287. Jul 14 21:58:01 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=88705ms, remaining=11295ms, start_delay=2000ms)
  1288. Jul 14 21:58:03 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1289. Jul 14 21:58:03 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1290. Jul 14 21:58:03 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1291. Jul 14 21:58:03 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=90707ms, remaining=9293ms, start_delay=2000ms)
  1292. Jul 14 21:58:05 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1293. Jul 14 21:58:05 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1294. Jul 14 21:58:05 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1295. Jul 14 21:58:05 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=92710ms, remaining=7290ms, start_delay=2000ms)
  1296. Jul 14 21:58:07 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1297. Jul 14 21:58:07 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1298. Jul 14 21:58:07 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1299. Jul 14 21:58:07 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=94718ms, remaining=5282ms, start_delay=2000ms)
  1300. Jul 14 21:58:09 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1301. Jul 14 21:58:09 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1302. Jul 14 21:58:09 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1303. Jul 14 21:58:09 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=96725ms, remaining=3275ms, start_delay=1637ms)
  1304. Jul 14 21:58:11 el8-a01n01.digimer.ca pacemaker-execd [6053] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1305. Jul 14 21:58:11 el8-a01n01.digimer.ca pacemaker-execd [6053] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1306. Jul 14 21:58:11 el8-a01n01.digimer.ca pacemaker-execd [6053] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1307. Jul 14 21:58:11 el8-a01n01.digimer.ca pacemaker-execd [6053] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=98371ms, remaining=1629ms, start_delay=814ms)
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!