[root@el8-a01n01 ~]# cat /var/log/pacemaker/el8-a01n01/pacemaker.log.drbd-down-cluster-down-ok-after Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (get_cluster_type) info: Detected an active 'corosync' cluster Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (mcp_read_config) info: Reading configure for stack: corosync Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (main) notice: Starting Pacemaker 2.0.3-5.el8_2.1 | build=4b1f869f0f features: generated-manpages agent-manpages ncurses libqb-logging libqb-ipc systemd nagios corosync-native atomic-attrd acls Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (main) info: Maximum core file size is: 18446744073709551615 Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (qb_ipcs_us_publish) info: server name: pacemakerd Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk__ipc_is_authentic_process_active) info: Could not connect to lrmd IPC: Connection refused Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk__ipc_is_authentic_process_active) info: Could not connect to cib_ro IPC: Connection refused Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk__ipc_is_authentic_process_active) info: Could not connect to crmd IPC: Connection refused Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk__ipc_is_authentic_process_active) info: Could not connect to attrd IPC: Connection refused Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk__ipc_is_authentic_process_active) info: Could not connect to pengine IPC: Connection refused Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk__ipc_is_authentic_process_active) info: Could not connect to stonith-ng IPC: Connection refused Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (crm_get_peer) info: Created entry 43f2a803-da53-4279-8a7c-163841c74b7e/0x55f1e70978c0 for node el8-a01n01/1 (1 total) Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (crm_get_peer) info: Node 1 is now known as el8-a01n01 Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (crm_get_peer) info: Node 1 has uuid 1 Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (cluster_connect_quorum) warning: Quorum lost Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (crm_get_peer) info: Created entry fcfbdeb7-0310-4b7f-99c4-301d97d11dc4/0x55f1e7099860 for node el8-a01n02/2 (2 total) Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (crm_get_peer) info: Node 2 is now known as el8-a01n02 Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (crm_get_peer) info: Node 2 has uuid 2 Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (start_child) info: Using uid=189 and group=189 for process pacemaker-based Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (start_child) info: Forked child 7330 for process pacemaker-based Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (start_child) info: Forked child 7331 for process pacemaker-fenced Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (start_child) info: Forked child 7332 for process pacemaker-execd Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (start_child) info: Using uid=189 and group=189 for process pacemaker-attrd Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (start_child) info: Forked child 7333 for process pacemaker-attrd Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (start_child) info: Using uid=189 and group=189 for process pacemaker-schedulerd Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (start_child) info: Forked child 7334 for process pacemaker-schedulerd Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (start_child) info: Using uid=189 and group=189 for process pacemaker-controld Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (start_child) info: Forked child 7335 for process pacemaker-controld Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (main) notice: Pacemaker daemon successfully started and accepting connections Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk_quorum_notification) info: Quorum still lost | membership=184 members=1 Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (crm_reap_unseen_nodes) info: State of node el8-a01n02[2] is still unknown Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk_cpg_membership) info: Group pacemakerd event 0: node 1 pid 7329 joined via cpg_join Jul 14 22:18:33 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk_cpg_membership) info: Group pacemakerd event 0: el8-a01n01 (node 1 pid 7329) is member Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-based [7330] (main) notice: Starting Pacemaker CIB manager Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-based [7330] (get_cluster_type) info: Verifying cluster type: 'corosync' Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-based [7330] (get_cluster_type) info: Assuming an active 'corosync' cluster Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-based [7330] (retrieveCib) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.xml (digest: /var/lib/pacemaker/cib/cib.xml.sig) Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-fenced [7331] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-fenced [7331] (main) notice: Starting Pacemaker fencer Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-fenced [7331] (get_cluster_type) info: Verifying cluster type: 'corosync' Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-fenced [7331] (get_cluster_type) info: Assuming an active 'corosync' cluster Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-fenced [7331] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-schedulerd[7334] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-schedulerd[7334] (main) notice: Starting Pacemaker scheduler Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-schedulerd[7334] (qb_ipcs_us_publish) info: server name: pengine Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-schedulerd[7334] (main) notice: Pacemaker scheduler successfully started and accepting connections Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-execd [7332] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-execd [7332] (main) notice: Starting Pacemaker local executor Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_ipcs_us_publish) info: server name: lrmd Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-execd [7332] (main) notice: Pacemaker local executor successfully started and accepting connections Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-attrd [7333] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-attrd [7333] (main) notice: Starting Pacemaker node attribute manager Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-fenced [7331] (crm_get_peer) info: Created entry 1b38f2c4-10f8-45f3-8a05-497ea8d1af2f/0x5556c125ed60 for node el8-a01n01/1 (1 total) Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-fenced [7331] (crm_get_peer) info: Node 1 is now known as el8-a01n01 Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-fenced [7331] (crm_get_peer) info: Node 1 has uuid 1 Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-fenced [7331] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-fenced [7331] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-fenced [7331] (init_cs_connection_once) info: Connection to 'corosync': established Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-controld [7335] (main) notice: Starting Pacemaker controller Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_log) info: Input I_STARTUP received in state S_STARTING from crmd_init Jul 14 22:18:33 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_get_peer) info: Created entry 013eba3f-f8ee-4baf-81c2-de58b97d3cbe/0x55ac132a5ef0 for node el8-a01n01/1 (1 total) Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_get_peer) info: Node 1 is now known as el8-a01n01 Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_get_peer) info: Node 1 has uuid 1 Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-based [7330] (init_cs_connection_once) info: Connection to 'corosync': established Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-based [7330] (qb_ipcs_us_publish) info: server name: cib_ro Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-based [7330] (qb_ipcs_us_publish) info: server name: cib_rw Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-based [7330] (qb_ipcs_us_publish) info: server name: cib_shm Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-based [7330] (main) notice: Pacemaker CIB manager successfully started and accepting connections Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-based [7330] (pcmk_cpg_membership) info: Group cib event 0: node 1 pid 7330 joined via cpg_join Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-based [7330] (pcmk_cpg_membership) info: Group cib event 0: el8-a01n01 (node 1 pid 7330) is member Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-21.raw Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_file_write_with_digest) info: Wrote version 0.15.0 of the CIB to disk (digest: ebc4019e128491fe42d0179b85b82cf6) Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.UakxZA (digest: /var/lib/pacemaker/cib/cib.4HcrQX) Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-attrd [7333] (main) info: CIB connection active Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-attrd [7333] (get_cluster_type) info: Verifying cluster type: 'corosync' Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-attrd [7333] (get_cluster_type) info: Assuming an active 'corosync' cluster Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-attrd [7333] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-attrd [7333] (crm_get_peer) info: Created entry 0c28072e-97a8-4b55-a615-30854e38abb0/0x557da867ba20 for node el8-a01n01/1 (1 total) Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-attrd [7333] (crm_get_peer) info: Node 1 is now known as el8-a01n01 Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-attrd [7333] (crm_get_peer) info: Node 1 has uuid 1 Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-attrd [7333] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-attrd [7333] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-attrd [7333] (init_cs_connection_once) info: Connection to 'corosync': established Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-attrd [7333] (main) info: Cluster connection active Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-attrd [7333] (attrd_erase_attrs) info: Clearing transient attributes from CIB | xpath=//node_state[@uname='el8-a01n01']/transient_attributes Jul 14 22:18:34 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n01']/transient_attributes to all (origin=local/attrd/2) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (crm_cs_flush) info: Sent 0 CPG messages (1 remaining, last=0): Try again (6) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (qb_ipcs_us_publish) info: server name: attrd Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (main) notice: Pacemaker node attribute manager successfully started and accepting connections Jul 14 22:18:35 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk_cpg_membership) info: Group pacemakerd event 1: node 2 pid 1564 joined via cluster join Jul 14 22:18:35 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n01 (node 1 pid 7329) is member Jul 14 22:18:35 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n02 (node 2 pid 1564) is member Jul 14 22:18:35 el8-a01n01.digimer.ca pacemakerd [7329] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online Jul 14 22:18:35 el8-a01n01.digimer.ca pacemakerd [7329] (crm_cs_flush) info: Sent 0 CPG messages (1 remaining, last=7): Try again (6) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_cs_flush) info: Sent 0 CPG messages (1 remaining, last=0): Try again (6) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-based [7330] (pcmk_cpg_membership) info: Group cib event 1: node 2 pid 1565 joined via cluster join Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-based [7330] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n01 (node 1 pid 7330) is member Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (setup_cib) info: Watching for stonith topology changes Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (qb_ipcs_us_publish) info: server name: stonith-ng Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (main) notice: Pacemaker fencer successfully started and accepting connections Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (pcmk_cpg_membership) info: Group stonith-ng event 0: node 1 pid 7331 joined via cpg_join Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (pcmk_cpg_membership) info: Group stonith-ng event 0: el8-a01n01 (node 1 pid 7331) is member Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (pcmk_cpg_membership) info: Group attrd event 0: node 1 pid 7333 joined via cpg_join Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (pcmk_cpg_membership) info: Group attrd event 0: el8-a01n01 (node 1 pid 7333) is member Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (pcmk_cpg_membership) info: Group stonith-ng event 1: node 2 pid 1566 joined via cluster join Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n01 (node 1 pid 7331) is member Jul 14 22:18:35 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk_quorum_notification) notice: Quorum acquired | membership=188 members=2 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemakerd [7329] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_get_peer) info: Created entry f3a02ded-1c06-48fc-beaf-6325bca22b22/0x55ac132acbb0 for node el8-a01n02/2 (2 total) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_get_peer) info: Node 2 is now known as el8-a01n02 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_get_peer) info: Node 2 has uuid 2 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-based [7330] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n02 (node 2 pid 1565) is member Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (get_cluster_type) info: Verifying cluster type: 'corosync' Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (get_cluster_type) info: Assuming an active 'corosync' cluster Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_get_peer) info: Created entry 5b877f60-706c-4434-85b9-5de45f33f77d/0x55f1537e3e20 for node el8-a01n01/1 (1 total) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_get_peer) info: Node 1 is now known as el8-a01n01 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (peer_update_callback) info: Cluster node el8-a01n01 is now in unknown state Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (pcmk_cpg_membership) info: Group attrd event 1: node 2 pid 1568 joined via cpg_join Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (pcmk_cpg_membership) info: Group attrd event 1: el8-a01n01 (node 1 pid 7333) is member Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (crm_get_peer) info: Created entry 1962055c-18e3-4e30-9d64-bfc7054f0fa4/0x5556c1366990 for node el8-a01n02/2 (2 total) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (crm_get_peer) info: Node 2 is now known as el8-a01n02 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_get_peer) info: Node 1 has uuid 1 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (peer_update_callback) info: Node el8-a01n01 is now a peer | DC= old=0x0000000 new=0x4000000 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (init_cs_connection_once) info: Connection to 'corosync': established Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (crm_get_peer) info: Node 2 has uuid 2 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n02 (node 2 pid 1566) is member Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (crm_get_peer) info: Created entry b484872e-6f28-41f4-807d-41e1483cabda/0x557da85717b0 for node el8-a01n02/2 (2 total) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (crm_get_peer) info: Node 2 is now known as el8-a01n02 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (crm_get_peer) info: Node 2 has uuid 2 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (pcmk_cpg_membership) info: Group attrd event 1: el8-a01n02 (node 2 pid 1568) is member Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (init_cib_cache_cb) info: Updating device list from the cib: init Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (cib_devices_update) info: Updating devices to version 0.15.0 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (cluster_connect_quorum) notice: Quorum acquired Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n02]: (unset) -> 2 | from el8-a01n02 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (attrd_start_election_if_needed) info: Starting an election to determine the writer Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_get_peer) info: Created entry f4f8c8f4-9aa2-48a2-80f8-e11f4b5fc933/0x55f1537e6700 for node el8-a01n02/2 (2 total) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_get_peer) info: Node 2 is now known as el8-a01n02 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (peer_update_callback) info: Cluster node el8-a01n02 is now in unknown state Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_get_peer) info: Node 2 has uuid 2 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/transient_attributes: OK (rc=0, origin=el8-a01n02/attrd/2, version=0.15.0) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (election_count_vote) info: election-attrd round 1 (owner node ID 2) lost: vote from el8-a01n02 (Uptime) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_ha_control) info: Connected to the cluster Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (lrmd_ipc_connect) info: Connecting to executor Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-execd [7332] (crm_client_new) debug: Connecting 0x55732b4f1fb0 for uid=189 gid=189 pid=7335 id=7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-execd [7332] (handle_new_connection) debug: IPC credentials authenticated (/dev/shm/qb-7332-7335-9-NPbmWF/qb) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_ipcs_shm_connect) debug: connecting to client [7335] Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/4) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed register operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=0 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_lrm_control) info: Connection to the executor established Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_started) info: Delaying start, no membership data (0000000000100000) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_started) info: Delaying start, no membership data (0000000000100000) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (pcmk_quorum_notification) info: Quorum retained | membership=188 members=2 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (peer_update_callback) info: Cluster node el8-a01n01 is now member (was in unknown state) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (peer_update_callback) info: Cluster node el8-a01n02 is now member (was in unknown state) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_started) info: Delaying start, Config not read (0000000000000040) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (qb_ipcs_us_publish) info: server name: crmd Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_started) notice: Pacemaker controller successfully started and accepting connections Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_log) info: Input I_PENDING received in state S_STARTING from do_started Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_state_transition) notice: State transition S_STARTING -> S_PENDING | input=I_PENDING cause=C_FSA_INTERNAL origin=do_started Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (crm_cs_flush) info: Sent 3 CPG messages (0 remaining, last=3): OK (1) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n01]: (unset) -> 2 | from el8-a01n01 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (election_count_vote) info: election-attrd round 2 (owner node ID 2) lost: vote from el8-a01n02 (Uptime) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-attrd [7333] (attrd_peer_message) info: Processing sync-response from el8-a01n02 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_cs_flush) info: Sent 2 CPG messages (0 remaining, last=2): OK (1) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/transient_attributes: OK (rc=0, origin=el8-a01n01/attrd/2, version=0.15.0) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/4, version=0.15.0) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 7382 exited with rc=0 Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (stonith_device_register) notice: Added 'virsh_node1' to the device list (1 active devices) Jul 14 22:18:35 el8-a01n01.digimer.ca pacemaker-fenced [7331] (stonith_device_register) notice: Added 'virsh_node2' to the device list (2 active devices) Jul 14 22:18:36 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/4, version=0.15.0) Jul 14 22:18:36 el8-a01n01.digimer.ca pacemaker-controld [7335] (pcmk_cpg_membership) info: Group crmd event 0: node 1 pid 7335 joined via cpg_join Jul 14 22:18:36 el8-a01n01.digimer.ca pacemaker-controld [7335] (pcmk_cpg_membership) info: Group crmd event 0: el8-a01n01 (node 1 pid 7335) is member Jul 14 22:18:36 el8-a01n01.digimer.ca pacemaker-controld [7335] (pcmk_cpg_membership) info: Group crmd event 1: node 2 pid 1570 joined via cpg_join Jul 14 22:18:36 el8-a01n01.digimer.ca pacemaker-controld [7335] (pcmk_cpg_membership) info: Group crmd event 1: el8-a01n01 (node 1 pid 7335) is member Jul 14 22:18:36 el8-a01n01.digimer.ca pacemaker-controld [7335] (pcmk_cpg_membership) info: Group crmd event 1: el8-a01n02 (node 2 pid 1570) is member Jul 14 22:18:36 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online Jul 14 22:18:36 el8-a01n01.digimer.ca pacemaker-controld [7335] (peer_update_callback) info: Node el8-a01n02 is now a peer | DC= old=0x0000000 new=0x4000000 Jul 14 22:18:36 el8-a01n01.digimer.ca pacemaker-controld [7335] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds Jul 14 22:18:36 el8-a01n01.digimer.ca pacemaker-controld [7335] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 30 seconds Jul 14 22:18:36 el8-a01n01.digimer.ca pacemaker-controld [7335] (te_connect_stonith) notice: Fencer successfully connected Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_timer_popped) info: Election Trigger just popped | input=I_DC_TIMEOUT time=20000ms Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_log) warning: Input I_DC_TIMEOUT received in state S_PENDING from crm_timer_popped Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_state_transition) info: State transition S_PENDING -> S_ELECTION | input=I_DC_TIMEOUT cause=C_TIMER_POPPED origin=crm_timer_popped Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (election_count_vote) info: election-DC round 1 (owner node ID 2) lost: vote from el8-a01n02 (Uptime) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_state_transition) notice: State transition S_ELECTION -> S_PENDING | input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_dc_release) info: DC role released Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_te_control) info: Transitioner is now inactive Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_log) info: Input I_RELEASE_SUCCESS received in state S_PENDING from do_dc_release Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n02/crmd/8, version=0.15.0) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (update_dc) info: Set DC to el8-a01n02 (3.3.0) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_update_peer_expected) info: update_dc: Node el8-a01n02[2] - expected state is now member (was (null)) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n02/crmd/10, version=0.15.0) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n02/crmd/12, version=0.15.0) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n02/crmd/14, version=0.15.0) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_replace) info: Digest matched on replace from el8-a01n02: 2efe3adb921c1da251a57d1a819d3e7a Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_replace) info: Replaced 0.15.0 with 0.15.0 from el8-a01n02 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=el8-a01n02/crmd/17, version=0.15.0) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_log) info: Input I_NOT_DC received in state S_PENDING from do_cl_join_finalize_respond Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_state_transition) notice: State transition S_PENDING -> S_NOT_DC | input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/18, version=0.15.0) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/19, version=0.15.0) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/lrm: OK (rc=0, origin=el8-a01n02/crmd/20, version=0.15.0) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.0 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.1 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=1 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ /cib/status: Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/21, version=0.15.1) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/lrm: OK (rc=0, origin=el8-a01n02/crmd/22, version=0.15.1) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.1 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.2 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ /cib/status: Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/23, version=0.15.2) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-22.raw Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_file_write_with_digest) info: Wrote version 0.15.0 of the CIB to disk (digest: 82d7477d0ff671fbf6bcda5e74ef2f1d) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.uFpK26 (digest: /var/lib/pacemaker/cib/cib.EQQ3px) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/26, version=0.15.2) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.2 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.3 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=3 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition, @join=member, @expected=member Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition, @join=member, @expected=member Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/27, version=0.15.3) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.3 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.4 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=4, @dc-uuid=2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n02/crmd/28, version=0.15.4) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.4 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.5 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=5 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_update_resource Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/30, version=0.15.5) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node1' not in cache Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node1' Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=1 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_lrm_rsc_op) info: Performing key=1:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e op=virsh_node1_monitor_0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.5 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.6 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=6 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/31, version=0.15.6) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.6 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.7 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=7 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;5:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=5, @rc-code=7, @op-status=0, @exec-time=5 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/32, version=0.15.7) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.7 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.8 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=8 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;6:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=9, @rc-code=7, @op-status=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/33, version=0.15.8) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.8 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.9 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=9 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/34, version=0.15.9) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.9 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.10 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=10 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/35, version=0.15.10) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.10 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.11 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=11 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;7:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=14, @rc-code=7, @op-status=0, @exec-time=45 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/36, version=0.15.11) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.11 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.12 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=12 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;8:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=19, @rc-code=7, @op-status=0, @exec-time=37 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/37, version=0.15.12) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 8638 exited with rc=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/9) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=5, reply=1, notify=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:5 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (stonith_api_signon) debug: Attempting fencer connection by pacemaker-execd with mainloop Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.12 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.13 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=13 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/9, version=0.15.13) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (stonith_api_signon) debug: Connection to fencer by pacemaker-execd succeeded (registration token: 440e6f7f-2fe1-4a52-a5de-26ed4986c329) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:5 exit-code:7 exec-time:5ms queue-time:0ms Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node2' not in cache Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node2' Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=1 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_lrm_rsc_op) info: Performing key=2:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e op=virsh_node2_monitor_0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/10) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=9, reply=1, notify=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_execute) debug: executing - rsc:virsh_node2 action:monitor call_id:9 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_finished) debug: finished - rsc:virsh_node2 action:monitor call_id:9 exit-code:7 exec-time:0ms queue-time:0ms Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.13 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.14 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=14 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (process_lrm_event) notice: Result of probe operation for virsh_node1 on el8-a01n01: 7 (not running) | call=5 key=virsh_node1_monitor_0 confirmed=true cib-update=11 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/10, version=0.15.14) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/11) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.14 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.15 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=15 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;1:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=5, @rc-code=7, @op-status=0, @exec-time=5 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/11, version=0.15.15) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/12) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (process_lrm_event) notice: Result of probe operation for virsh_node2 on el8-a01n01: 7 (not running) | call=9 key=virsh_node2_monitor_0 confirmed=true cib-update=12 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.15 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.16 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=16 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;2:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=9, @rc-code=7, @op-status=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/12, version=0.15.16) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_lrm_rsc_op) info: Performing key=9:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e op=virsh_node1_start_0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/13) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=10, reply=1, notify=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_execute) info: executing - rsc:virsh_node1 action:start call_id:10 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.16 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.17 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=17 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_start_0, @operation=start, @transition-key=9:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;9:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/13, version=0.15.17) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.17 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.18 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=18 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=11:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;11:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/38, version=0.15.18) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor' not in cache Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor:0' not in cache Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_rsc_register) info: Cached agent information for 'hypervisor' Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=1 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_lrm_rsc_op) info: Performing key=3:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e op=hypervisor_monitor_0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/14) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=15, reply=1, notify=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:15 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.18 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.19 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=19 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/14, version=0.15.19) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor' Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:15 exit-code:7 exec-time:78ms queue-time:0ms Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (process_lrm_event) notice: Result of probe operation for hypervisor on el8-a01n01: 7 (not running) | call=15 key=hypervisor_monitor_0 confirmed=true cib-update=15 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/15) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.19 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.20 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=20 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;3:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=15, @rc-code=7, @op-status=0, @exec-time=78 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/15, version=0.15.20) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_get_rsc_info) info: Agent information for 'drbd' not in cache Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_get_rsc_info) info: Agent information for 'drbd:0' not in cache Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_rsc_register) info: Cached agent information for 'drbd' Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=1 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_lrm_rsc_op) info: Performing key=4:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e op=drbd_monitor_0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=20, reply=1, notify=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_execute) debug: executing - rsc:drbd action:monitor call_id:20 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/16) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.20 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.21 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=21 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/16, version=0.15.21) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.21 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.22 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=22 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=15:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;15:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/39, version=0.15.22) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_finished) debug: finished - rsc:drbd action:monitor call_id:20 exit-code:7 exec-time:16ms queue-time:0ms Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (process_lrm_event) notice: Result of probe operation for drbd on el8-a01n01: 7 (not running) | call=20 key=drbd_monitor_0 confirmed=true cib-update=17 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/17) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.22 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.23 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=23 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;4:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=20, @rc-code=7, @op-status=0, @exec-time=16 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/17, version=0.15.23) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_lrm_rsc_op) info: Performing key=13:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e op=hypervisor_start_0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/18) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=21, reply=1, notify=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_execute) info: executing - rsc:hypervisor action:start call_id:21 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (systemd_unit_exec) debug: Performing asynchronous start op on systemd unit libvirtd named 'hypervisor' Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.23 2 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.24 (null) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=24 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=13:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;13:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/18, version=0.15.24) Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (systemd_unit_exec_with_unit) debug: Calling StartUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (systemd_exec_result) info: Call to start passed: /org/freedesktop/systemd1/job/980 Jul 14 22:18:56 el8-a01n01.digimer.ca pacemaker-execd [7332] (action_complete) debug: hypervisor start may still be in progress: re-scheduling (elapsed=341ms, remaining=99659ms, start_delay=2000ms) Jul 14 22:18:57 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.24 2 Jul 14 22:18:57 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.25 (null) Jul 14 22:18:57 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=25 Jul 14 22:18:57 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;11:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=20, @rc-code=0, @op-status=0, @exec-time=810 Jul 14 22:18:57 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/40, version=0.15.25) Jul 14 22:18:57 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.15.25 2 Jul 14 22:18:57 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.15.26 (null) Jul 14 22:18:57 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=26 Jul 14 22:18:57 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']: Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.16.0) Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-fenced [7331] (update_cib_stonith_devices_v2) info: Updating device list from the cib: create meta_attributes[@id='drbd-meta_attributes'] Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-fenced [7331] (cib_devices_update) info: Updating devices to version 0.16.0 Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-23.raw Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_file_write_with_digest) info: Wrote version 0.16.0 of the CIB to disk (digest: 9ac5ee247228cbebb26d752a5874dee5) Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.7zU6I6 (digest: /var/lib/pacemaker/cib/cib.L3Y9CQ) Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-execd [7332] (cancel_recurring_action) info: Cancelling systemd operation drbd_status_60000 Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_finished) debug: finished - rsc:drbd action:monitor call_id:25 exit-code:0 exec-time:0ms queue-time:0ms Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=0 Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_lrm_rsc_op) info: Performing key=19:1:0:22298a5d-b80a-4cd7-adde-aba652405a4e op=drbd_stop_0 Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=27, reply=1, notify=0 Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_execute) info: executing - rsc:drbd action:stop call_id:27 Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-execd [7332] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit drbd named 'drbd' Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.0 2 Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.1 (null) Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=1 Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_stop_0, @operation=stop, @transition-key=20:1:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;20:1:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765184, @last-run=1594765184, @exec-time=0 Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/51, version=0.16.1) Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/29) Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.1 2 Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.2 (null) Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=2 Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_stop_0, @operation=stop, @transition-key=19:1:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;19:1:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765184, @last-run=1594765184, @exec-time=0, @queue-time=0 Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/29, version=0.16.2) Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-execd [7332] (systemd_unit_exec_with_unit) debug: Calling StopUnit for drbd: /org/freedesktop/systemd1/unit/drbd_2eservice Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-controld [7335] (process_lrm_event) info: Result of monitor operation for drbd on el8-a01n01: Cancelled | call=25 key=drbd_monitor_60000 confirmed=true Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-execd [7332] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/1158 Jul 14 22:19:44 el8-a01n01.digimer.ca pacemaker-execd [7332] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=236ms, remaining=99764ms, start_delay=2000ms) Jul 14 22:19:46 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.2 2 Jul 14 22:19:46 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.3 (null) Jul 14 22:19:46 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=3 Jul 14 22:19:46 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;20:1:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=27, @rc-code=0, @op-status=0, @exec-time=2205 Jul 14 22:19:46 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/52, version=0.16.3) Jul 14 22:19:46 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_execute) debug: executing - rsc:drbd action:monitor call_id:27 Jul 14 22:19:46 el8-a01n01.digimer.ca pacemaker-execd [7332] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 22:19:46 el8-a01n01.digimer.ca pacemaker-execd [7332] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 22:19:46 el8-a01n01.digimer.ca pacemaker-execd [7332] (action_complete) debug: drbd stop is now complete (elapsed=2244ms, remaining=97756ms): not running (7) Jul 14 22:19:46 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_finished) debug: finished - rsc:drbd action:monitor call_id:27 exit-code:0 exec-time:2244ms queue-time:0ms Jul 14 22:19:46 el8-a01n01.digimer.ca pacemaker-controld [7335] (process_lrm_event) notice: Result of stop operation for drbd on el8-a01n01: 0 (ok) | call=27 key=drbd_stop_0 confirmed=true cib-update=30 Jul 14 22:19:46 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/30) Jul 14 22:19:46 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.3 2 Jul 14 22:19:46 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.4 (null) Jul 14 22:19:46 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=4 Jul 14 22:19:46 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;19:1:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=27, @rc-code=0, @op-status=0, @exec-time=2244 Jul 14 22:19:46 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/30, version=0.16.4) Jul 14 22:19:51 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_ping) info: Reporting our current digest to el8-a01n02: c9fc5e41d0d91479ef3a8c683462f6ab for 0.16.4 (0x55ac13453a80 0) Jul 14 22:19:58 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:22 Jul 14 22:19:58 el8-a01n01.digimer.ca pacemaker-execd [7332] (recurring_action_timer) debug: Scheduling another invocation of hypervisor_status_60000 Jul 14 22:19:58 el8-a01n01.digimer.ca pacemaker-execd [7332] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor' Jul 14 22:19:58 el8-a01n01.digimer.ca pacemaker-execd [7332] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 22:19:58 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms Jul 14 22:19:59 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:22 exit-code:0 exec-time:706ms queue-time:0ms Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_update_peer_expected) info: handle_request: Node el8-a01n02[2] - expected state is now down (was member) Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-attrd [7333] (attrd_peer_update) notice: Setting shutdown[el8-a01n02]: (unset) -> 1594765206 | from el8-a01n02 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.4 2 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.5 (null) Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=5 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']: Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/4, version=0.16.5) Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.5 2 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.6 (null) Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=6 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=7:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;7:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765206, @last-run=1594765206, @exec-time=0 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/54, version=0.16.6) Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.6 2 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.7 (null) Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=7 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;7:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=29, @rc-code=0, @op-status=0 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/55, version=0.16.7) Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_lrm_rsc_op) info: Performing key=8:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e op=virsh_node2_start_0 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/31) Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.7 2 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.8 (null) Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=8 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=8:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;8:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765206, @last-run=1594765206 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/31, version=0.16.8) Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=28, reply=1, notify=0 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_execute) info: executing - rsc:virsh_node2 action:start call_id:28 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.8 2 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.9 (null) Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=9 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=12:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;12:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765206, @last-run=1594765206, @exec-time=0 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/56, version=0.16.9) Jul 14 22:20:06 el8-a01n01.digimer.ca pacemakerd [7329] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) Jul 14 22:20:06 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk_shutdown_worker) notice: Shutting down Pacemaker Jul 14 22:20:06 el8-a01n01.digimer.ca pacemakerd [7329] (stop_child) notice: Stopping pacemaker-controld | sent signal 15 to process 7335 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_shutdown) notice: Shutting down cluster resource manager | limit=1200000ms Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_shutdown_req) info: Sending shutdown request to all peers (DC is el8-a01n02) Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_update_peer_expected) info: handle_request: Node el8-a01n01[1] - expected state is now down (was (null)) Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-attrd [7333] (attrd_peer_update) notice: Setting shutdown[el8-a01n01]: (unset) -> 1594765206 | from el8-a01n02 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.9 2 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.10 (null) Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=10 Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']: Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: ++ Jul 14 22:20:06 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/5, version=0.16.10) Jul 14 22:20:07 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_finished) info: finished - rsc:virsh_node2 action:start call_id:28 exit-code:0 exec-time:697ms queue-time:0ms Jul 14 22:20:07 el8-a01n01.digimer.ca pacemaker-controld [7335] (process_lrm_event) notice: Result of start operation for virsh_node2 on el8-a01n01: 0 (ok) | call=28 key=virsh_node2_start_0 confirmed=true cib-update=32 Jul 14 22:20:07 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/32) Jul 14 22:20:07 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.10 2 Jul 14 22:20:07 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.11 (null) Jul 14 22:20:07 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=11 Jul 14 22:20:07 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;8:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=28, @rc-code=0, @op-status=0, @exec-time=697 Jul 14 22:20:07 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/32, version=0.16.11) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.11 2 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.12 (null) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=12 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;12:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=31, @rc-code=0, @op-status=0, @exec-time=2198 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/57, version=0.16.12) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:22 exit-code:0 exec-time:0ms queue-time:0ms Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=0 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_lrm_rsc_op) info: Performing key=3:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e op=virsh_node1_stop_0 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=30, reply=1, notify=0 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/33) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_execute) info: executing - rsc:virsh_node1 action:stop call_id:30 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-controld [7335] (process_lrm_event) info: Result of monitor operation for virsh_node1 on el8-a01n01: Cancelled | call=22 key=virsh_node1_monitor_60000 confirmed=true Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_finished) info: finished - rsc:virsh_node1 action:stop call_id:30 exit-code:0 exec-time:0ms queue-time:1ms Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_lrm_rsc_op) info: Performing key=4:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e op=virsh_node2_stop_0 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.12 2 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.13 (null) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=13 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=31, reply=1, notify=0 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_stop_0, @operation=stop, @transition-key=3:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;3:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765208, @last-run=1594765208, @exec-time=0, @q Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_execute) info: executing - rsc:virsh_node2 action:stop call_id:31 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-controld [7335] (process_lrm_event) notice: Result of stop operation for virsh_node1 on el8-a01n01: 0 (ok) | call=30 key=virsh_node1_stop_0 confirmed=true cib-update=35 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/33, version=0.16.13) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_finished) info: finished - rsc:virsh_node2 action:stop call_id:31 exit-code:0 exec-time:1ms queue-time:0ms Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/34) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/35) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-execd [7332] (cancel_recurring_action) info: Cancelling systemd operation hypervisor_status_60000 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=0, reply=1, notify=0 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_lrm_rsc_op) info: Performing key=5:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e op=hypervisor_stop_0 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-execd [7332] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 7a9d5d39-04c6-4f24-bb8c-bf00ea6fb29a: rc=33, reply=1, notify=0 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.13 2 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_execute) info: executing - rsc:hypervisor action:stop call_id:33 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.14 (null) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=14 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-execd [7332] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit libvirtd named 'hypervisor' Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=4:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;4:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765208, @last-run=1594765208, @exec-time=0 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/34, version=0.16.14) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.14 2 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.15 (null) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=15 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:0;3:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=30, @rc-code=0, @op-status=0, @queue-time=1 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/35, version=0.16.15) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/36) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.15 2 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.16 (null) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=16 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=5:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;5:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765208, @last-run=1594765208, @exec-time=0 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/36, version=0.16.16) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-execd [7332] (systemd_unit_exec_with_unit) debug: Calling StopUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/37) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.16 2 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.17 (null) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=17 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;4:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=31, @rc-code=0, @op-status=0, @exec-time=1 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/37, version=0.16.17) Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-controld [7335] (process_lrm_event) notice: Result of stop operation for virsh_node2 on el8-a01n01: 0 (ok) | call=31 key=virsh_node2_stop_0 confirmed=true cib-update=37 Jul 14 22:20:08 el8-a01n01.digimer.ca pacemaker-controld [7335] (process_lrm_event) info: Result of monitor operation for hypervisor on el8-a01n01: Cancelled | call=24 key=hypervisor_monitor_60000 confirmed=true Jul 14 22:20:09 el8-a01n01.digimer.ca pacemaker-execd [7332] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/1165 Jul 14 22:20:09 el8-a01n01.digimer.ca pacemaker-execd [7332] (action_complete) debug: hypervisor stop may still be in progress: re-scheduling (elapsed=256ms, remaining=99744ms, start_delay=2000ms) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:33 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor' Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (action_complete) debug: hypervisor stop is now complete (elapsed=2260ms, remaining=97740ms): not running (7) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:33 exit-code:0 exec-time:2260ms queue-time:0ms Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (process_lrm_event) notice: Result of stop operation for hypervisor on el8-a01n01: 0 (ok) | call=33 key=hypervisor_stop_0 confirmed=true cib-update=38 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/38) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.17 2 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.18 (null) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=18 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;5:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=33, @rc-code=0, @op-status=0, @exec-time=2260 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/38, version=0.16.18) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (handle_request) info: Shutting down Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_state_transition) notice: State transition S_NOT_DC -> S_STOPPING | input=I_STOP cause=C_HA_MESSAGE origin=route_message Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (tengine_stonith_connection_destroy) info: Fencing daemon disconnected Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_lrm_control) info: Disconnecting from the executor Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (lrmd_api_disconnect) info: Disconnecting IPC local executor connection Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_ipcs_dispatch_connection_request) debug: HUP conn (/dev/shm/qb-7332-7335-9-NPbmWF/qb) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_ipcs_disconnect) debug: qb_ipcs_disconnect(/dev/shm/qb-7332-7335-9-NPbmWF/qb) state:2 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (crm_client_destroy) debug: Destroying 0 events Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_rb_close_helper) debug: Free'ing ringbuffer: /dev/shm/qb-7332-7335-9-NPbmWF/qb-response-lrmd-header Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_rb_close_helper) debug: Free'ing ringbuffer: /dev/shm/qb-7332-7335-9-NPbmWF/qb-event-lrmd-header Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_rb_close_helper) debug: Free'ing ringbuffer: /dev/shm/qb-7332-7335-9-NPbmWF/qb-request-lrmd-header Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (lrmd_ipc_connection_destroy) info: IPC connection destroyed Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (lrm_connection_destroy) info: Disconnected from executor Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (lrmd_api_disconnect) info: Disconnecting IPC local executor connection Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_lrm_control) notice: Disconnected from the executor Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_cluster_disconnect) info: Disconnecting from cluster infrastructure: corosync Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (terminate_cs_connection) notice: Disconnected from Corosync Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_ha_control) info: Disconnected from the cluster Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_cib_control) info: Disconnecting from the CIB manager Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (crmd_cib_connection_destroy) info: Connection to the CIB manager terminated Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_cib_control) notice: Disconnected from the CIB manager Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_exit) info: Performing A_EXIT_0 - gracefully exiting the controller Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (do_exit) info: [pacemaker-controld] stopped (0) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (crmd_exit) info: Dropping I_TERMINATE: [ state=S_STOPPING cause=C_FSA_INTERNAL origin=do_stop ] Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (crmd_cs_destroy) info: Corosync connection closed Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (crmd_cs_destroy) info: Corosync connection closed Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (crmd_init) info: pacemaker-controld[7335] exiting with status 0 (OK) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-controld [7335] (crm_xml_cleanup) info: Cleaning up memory from libxml2 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk_child_exit) info: pacemaker-controld[7335] exited with status 0 (OK) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-schedulerd[7334] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-schedulerd[7334] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-schedulerd[7334] (crm_xml_cleanup) info: Cleaning up memory from libxml2 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemakerd [7329] (stop_child) notice: Stopping pacemaker-schedulerd | sent signal 15 to process 7334 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk_child_exit) info: pacemaker-schedulerd[7334] exited with status 0 (OK) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemakerd [7329] (stop_child) notice: Stopping pacemaker-attrd | sent signal 15 to process 7333 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: --- 0.16.18 2 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: Diff: +++ 0.16.19 (null) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib: @num_updates=19 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_dc_release, @expected=down Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-attrd [7333] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-attrd [7333] (main) info: Shutting down attribute manager Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-attrd [7333] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-attrd [7333] (attrd_cib_destroy_cb) info: Connection disconnection complete Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-attrd [7333] (crm_xml_cleanup) info: Cleaning up memory from libxml2 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (qb_ipcs_event_sendv) warning: new_event_notification (/dev/shm/qb-7330-7333-11-tDbJZG/qb): Broken pipe (32) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_notify_send_one) warning: Notification of client attrd/43138264-95a2-4437-86af-201b04e3f29b failed Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/59, version=0.16.19) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk_child_exit) info: pacemaker-attrd[7333] exited with status 0 (OK) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemakerd [7329] (stop_child) notice: Stopping pacemaker-execd | sent signal 15 to process 7332 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (lrmd_exit) info: Terminating with 0 clients Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (stonith_api_del_notification) debug: Removing callback for st_notify_disconnect events Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (stonith_api_signoff) debug: Disconnecting from the fencer Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect() Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-7331-7332-12-dwnItX/qb-request-stonith-ng-header Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-7331-7332-12-dwnItX/qb-response-stonith-ng-header Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-7331-7332-12-dwnItX/qb-event-stonith-ng-header Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (qb_ipcs_unref) debug: qb_ipcs_unref() - destroying Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-execd [7332] (crm_xml_cleanup) info: Cleaning up memory from libxml2 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk_child_exit) info: pacemaker-execd[7332] exited with status 0 (OK) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemakerd [7329] (stop_child) notice: Stopping pacemaker-fenced | sent signal 15 to process 7331 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-fenced [7331] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-fenced [7331] (stonith_shutdown) info: Terminating with 0 clients Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-fenced [7331] (cib_connection_destroy) info: Connection to the CIB manager closed Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-fenced [7331] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-fenced [7331] (crm_xml_cleanup) info: Cleaning up memory from libxml2 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk_child_exit) info: pacemaker-fenced[7331] exited with status 0 (OK) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemakerd [7329] (stop_child) notice: Stopping pacemaker-based | sent signal 15 to process 7330 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_shutdown) info: Disconnected 0 clients Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_shutdown) info: All clients disconnected (0) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (initiate_exit) info: Sending disconnect notification to 2 peers... Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_shutdown_req) info: Peer el8-a01n01 is requesting to shut down Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_process_shutdown_req) info: Peer el8-a01n01 has acknowledged our shutdown request Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (terminate_cib) info: cib_process_shutdown_req: Exiting from mainloop... Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_cluster_disconnect) info: Disconnecting from cluster infrastructure: corosync Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (terminate_cs_connection) notice: Disconnected from Corosync Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_get_peer) info: Created entry 847cbb6c-98da-4969-a88d-905f74eee34a/0x55ac132a6470 for node el8-a01n01/0 (1 total) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cib_peer_update_callback) info: No more peers Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (terminate_cib) info: cib_peer_update_callback: Exiting from mainloop... Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_cluster_disconnect) info: Disconnecting from cluster infrastructure: corosync Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (cluster_disconnect_cpg) info: No CPG connection Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (terminate_cs_connection) notice: Disconnected from Corosync Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:20:11 el8-a01n01.digimer.ca pacemaker-based [7330] (crm_xml_cleanup) info: Cleaning up memory from libxml2 Jul 14 22:20:11 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk_child_exit) info: pacemaker-based[7330] exited with status 0 (OK) Jul 14 22:20:11 el8-a01n01.digimer.ca pacemakerd [7329] (pcmk_shutdown_worker) notice: Shutdown complete Jul 14 22:20:11 el8-a01n01.digimer.ca pacemakerd [7329] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:20:11 el8-a01n01.digimer.ca pacemakerd [7329] (crm_xml_cleanup) info: Cleaning up memory from libxml2 Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (get_cluster_type) info: Detected an active 'corosync' cluster Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (mcp_read_config) info: Reading configure for stack: corosync Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (main) notice: Starting Pacemaker 2.0.3-5.el8_2.1 | build=4b1f869f0f features: generated-manpages agent-manpages ncurses libqb-logging libqb-ipc systemd nagios corosync-native atomic-attrd acls Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (main) info: Maximum core file size is: 18446744073709551615 Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (qb_ipcs_us_publish) info: server name: pacemakerd Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk__ipc_is_authentic_process_active) info: Could not connect to lrmd IPC: Connection refused Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk__ipc_is_authentic_process_active) info: Could not connect to cib_ro IPC: Connection refused Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk__ipc_is_authentic_process_active) info: Could not connect to crmd IPC: Connection refused Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk__ipc_is_authentic_process_active) info: Could not connect to attrd IPC: Connection refused Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk__ipc_is_authentic_process_active) info: Could not connect to pengine IPC: Connection refused Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk__ipc_is_authentic_process_active) info: Could not connect to stonith-ng IPC: Connection refused Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (crm_get_peer) info: Created entry a19fdd37-84d7-4118-a441-04031cc8cee3/0x55fc80c66860 for node el8-a01n01/1 (1 total) Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (crm_get_peer) info: Node 1 is now known as el8-a01n01 Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (crm_get_peer) info: Node 1 has uuid 1 Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (cluster_connect_quorum) warning: Quorum lost Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (crm_get_peer) info: Created entry e88f167c-3b4c-4309-a210-288f26823f9c/0x55fc80c68800 for node el8-a01n02/2 (2 total) Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (crm_get_peer) info: Node 2 is now known as el8-a01n02 Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (crm_get_peer) info: Node 2 has uuid 2 Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (start_child) info: Using uid=189 and group=189 for process pacemaker-based Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (start_child) info: Forked child 14953 for process pacemaker-based Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (start_child) info: Forked child 14954 for process pacemaker-fenced Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (start_child) info: Forked child 14955 for process pacemaker-execd Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (start_child) info: Using uid=189 and group=189 for process pacemaker-attrd Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (start_child) info: Forked child 14956 for process pacemaker-attrd Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (start_child) info: Using uid=189 and group=189 for process pacemaker-schedulerd Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (start_child) info: Forked child 14957 for process pacemaker-schedulerd Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (start_child) info: Using uid=189 and group=189 for process pacemaker-controld Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (start_child) info: Forked child 14958 for process pacemaker-controld Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (main) notice: Pacemaker daemon successfully started and accepting connections Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk_quorum_notification) info: Quorum still lost | membership=193 members=1 Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (crm_reap_unseen_nodes) info: State of node el8-a01n02[2] is still unknown Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk_cpg_membership) info: Group pacemakerd event 0: node 1 pid 14952 joined via cpg_join Jul 14 22:20:22 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk_cpg_membership) info: Group pacemakerd event 0: el8-a01n01 (node 1 pid 14952) is member Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-schedulerd[14957] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-schedulerd[14957] (main) notice: Starting Pacemaker scheduler Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-schedulerd[14957] (qb_ipcs_us_publish) info: server name: pengine Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-schedulerd[14957] (main) notice: Pacemaker scheduler successfully started and accepting connections Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-fenced [14954] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-fenced [14954] (main) notice: Starting Pacemaker fencer Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-fenced [14954] (get_cluster_type) info: Verifying cluster type: 'corosync' Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-fenced [14954] (get_cluster_type) info: Assuming an active 'corosync' cluster Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-fenced [14954] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (main) notice: Starting Pacemaker CIB manager Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (get_cluster_type) info: Verifying cluster type: 'corosync' Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (get_cluster_type) info: Assuming an active 'corosync' cluster Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (retrieveCib) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.xml (digest: /var/lib/pacemaker/cib/cib.xml.sig) Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-attrd [14956] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-attrd [14956] (main) notice: Starting Pacemaker node attribute manager Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-controld [14958] (main) notice: Starting Pacemaker controller Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_log) info: Input I_STARTUP received in state S_STARTING from crmd_init Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-execd [14955] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-execd [14955] (main) notice: Starting Pacemaker local executor Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_ipcs_us_publish) info: server name: lrmd Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-execd [14955] (main) notice: Pacemaker local executor successfully started and accepting connections Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-fenced [14954] (crm_get_peer) info: Created entry e9b081a8-013f-4438-a286-71a6a1aeef76/0x5579a28fcd60 for node el8-a01n01/1 (1 total) Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-fenced [14954] (crm_get_peer) info: Node 1 is now known as el8-a01n01 Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_get_peer) info: Created entry da54e080-ca2f-4a23-b8f2-8bf8445c53b8/0x561964eeb320 for node el8-a01n01/1 (1 total) Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_get_peer) info: Node 1 is now known as el8-a01n01 Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-fenced [14954] (crm_get_peer) info: Node 1 has uuid 1 Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-fenced [14954] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-fenced [14954] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-fenced [14954] (init_cs_connection_once) info: Connection to 'corosync': established Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_get_peer) info: Node 1 has uuid 1 Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (init_cs_connection_once) info: Connection to 'corosync': established Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (qb_ipcs_us_publish) info: server name: cib_ro Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (qb_ipcs_us_publish) info: server name: cib_rw Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (qb_ipcs_us_publish) info: server name: cib_shm Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (main) notice: Pacemaker CIB manager successfully started and accepting connections Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (pcmk_cpg_membership) info: Group cib event 0: node 1 pid 14953 joined via cpg_join Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (pcmk_cpg_membership) info: Group cib event 0: el8-a01n01 (node 1 pid 14953) is member Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-24.raw Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_file_write_with_digest) info: Wrote version 0.16.0 of the CIB to disk (digest: 71670f38eb62dc5320749c7be8e07fc8) Jul 14 22:20:22 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.JOtAEd (digest: /var/lib/pacemaker/cib/cib.IqkRpL) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk_cpg_membership) info: Group pacemakerd event 1: node 2 pid 2196 joined via cluster join Jul 14 22:20:23 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n01 (node 1 pid 14952) is member Jul 14 22:20:23 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n02 (node 2 pid 2196) is member Jul 14 22:20:23 el8-a01n01.digimer.ca pacemakerd [14952] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online Jul 14 22:20:23 el8-a01n01.digimer.ca pacemakerd [14952] (crm_cs_flush) info: Sent 0 CPG messages (1 remaining, last=7): Try again (6) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk_quorum_notification) notice: Quorum acquired | membership=197 members=2 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemakerd [14952] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-based [14953] (pcmk_cpg_membership) info: Group cib event 1: node 2 pid 2197 joined via cluster join Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-based [14953] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n01 (node 1 pid 14953) is member Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_get_peer) info: Created entry c8f2cda8-094b-4e09-b8c0-993444881423/0x561964ef0aa0 for node el8-a01n02/2 (2 total) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_get_peer) info: Node 2 is now known as el8-a01n02 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_get_peer) info: Node 2 has uuid 2 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-based [14953] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n02 (node 2 pid 2197) is member Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (get_cluster_type) info: Verifying cluster type: 'corosync' Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (get_cluster_type) info: Assuming an active 'corosync' cluster Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (main) info: CIB connection active Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (get_cluster_type) info: Verifying cluster type: 'corosync' Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (get_cluster_type) info: Assuming an active 'corosync' cluster Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_get_peer) info: Created entry c87d554a-70ed-47b3-a0a8-356e10aae50e/0x55b5f201deb0 for node el8-a01n01/1 (1 total) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_get_peer) info: Node 1 is now known as el8-a01n01 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (peer_update_callback) info: Cluster node el8-a01n01 is now in unknown state Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (crm_get_peer) info: Created entry 11f4468a-c4ad-4023-8fa4-45da18cf7a35/0x55cf12635a20 for node el8-a01n01/1 (1 total) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (crm_get_peer) info: Node 1 is now known as el8-a01n01 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_get_peer) info: Node 1 has uuid 1 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (peer_update_callback) info: Node el8-a01n01 is now a peer | DC= old=0x0000000 new=0x4000000 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (init_cs_connection_once) info: Connection to 'corosync': established Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (crm_get_peer) info: Node 1 has uuid 1 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (init_cs_connection_once) info: Connection to 'corosync': established Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/transient_attributes: OK (rc=0, origin=el8-a01n02/attrd/2, version=0.16.0) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (main) info: Cluster connection active Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (attrd_erase_attrs) info: Clearing transient attributes from CIB | xpath=//node_state[@uname='el8-a01n01']/transient_attributes Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n01']/transient_attributes to all (origin=local/attrd/2) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (cluster_connect_quorum) notice: Quorum acquired Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/transient_attributes: OK (rc=0, origin=el8-a01n01/attrd/2, version=0.16.0) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (setup_cib) info: Watching for stonith topology changes Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (qb_ipcs_us_publish) info: server name: stonith-ng Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (main) notice: Pacemaker fencer successfully started and accepting connections Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (pcmk_cpg_membership) info: Group stonith-ng event 0: node 1 pid 14954 joined via cpg_join Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (pcmk_cpg_membership) info: Group stonith-ng event 0: el8-a01n01 (node 1 pid 14954) is member Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (init_cib_cache_cb) info: Updating device list from the cib: init Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (qb_ipcs_us_publish) info: server name: attrd Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (main) notice: Pacemaker node attribute manager successfully started and accepting connections Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (pcmk_cpg_membership) info: Group attrd event 0: node 1 pid 14956 joined via cpg_join Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (pcmk_cpg_membership) info: Group attrd event 0: el8-a01n01 (node 1 pid 14956) is member Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (cib_devices_update) info: Updating devices to version 0.16.0 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_get_peer) info: Created entry 0f770a3c-3710-4a5e-82ad-12ab42870de9/0x55b5f2020790 for node el8-a01n02/2 (2 total) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_get_peer) info: Node 2 is now known as el8-a01n02 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (peer_update_callback) info: Cluster node el8-a01n02 is now in unknown state Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_get_peer) info: Node 2 has uuid 2 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/4, version=0.16.0) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (crm_get_peer) info: Created entry 6b9dbb5e-b9c5-450d-a9ba-525ef70debb9/0x55cf1263c060 for node el8-a01n02/2 (2 total) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (crm_get_peer) info: Node 2 is now known as el8-a01n02 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (crm_get_peer) info: Node 2 has uuid 2 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (pcmk_cpg_membership) info: Group attrd event 0: el8-a01n02 (node 2 pid 2200) is member Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n02]: (unset) -> 2 | from el8-a01n02 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (attrd_start_election_if_needed) info: Starting an election to determine the writer Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (election_count_vote) info: election-attrd round 1 (owner node ID 2) lost: vote from el8-a01n02 (Uptime) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n01]: (unset) -> 2 | from el8-a01n01 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (election_count_vote) info: election-attrd round 2 (owner node ID 2) lost: vote from el8-a01n02 (Uptime) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_ha_control) info: Connected to the cluster Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/4) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-attrd [14956] (attrd_peer_message) info: Processing sync-response from el8-a01n02 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (lrmd_ipc_connect) info: Connecting to executor Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-execd [14955] (crm_client_new) debug: Connecting 0x5559575bbfb0 for uid=189 gid=189 pid=14958 id=5125103c-578c-42d5-9219-4148b4fc6cd0 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-execd [14955] (handle_new_connection) debug: IPC credentials authenticated (/dev/shm/qb-14955-14958-9-DajKTi/qb) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_ipcs_shm_connect) debug: connecting to client [14958] Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/4, version=0.16.0) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed register operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=0, reply=1, notify=0 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_lrm_control) info: Connection to the executor established Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_started) info: Delaying start, no membership data (0000000000100000) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_started) info: Delaying start, no membership data (0000000000100000) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (pcmk_quorum_notification) info: Quorum retained | membership=197 members=2 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (peer_update_callback) info: Cluster node el8-a01n01 is now member (was in unknown state) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (peer_update_callback) info: Cluster node el8-a01n02 is now member (was in unknown state) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_started) info: Delaying start, Config not read (0000000000000040) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (qb_ipcs_us_publish) info: server name: crmd Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_started) notice: Pacemaker controller successfully started and accepting connections Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_log) info: Input I_PENDING received in state S_STARTING from do_started Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_state_transition) notice: State transition S_STARTING -> S_PENDING | input=I_PENDING cause=C_FSA_INTERNAL origin=do_started Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 15024 exited with rc=0 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (stonith_device_register) notice: Added 'virsh_node1' to the device list (1 active devices) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (stonith_device_register) notice: Added 'virsh_node2' to the device list (2 active devices) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (pcmk_cpg_membership) info: Group stonith-ng event 1: node 2 pid 2198 joined via cluster join Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n01 (node 1 pid 14954) is member Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (crm_get_peer) info: Created entry 89e9195b-a893-42a1-b09b-5163ad360fe4/0x5579a2a25af0 for node el8-a01n02/2 (2 total) Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (crm_get_peer) info: Node 2 is now known as el8-a01n02 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (crm_get_peer) info: Node 2 has uuid 2 Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n02 (node 2 pid 2198) is member Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online Jul 14 22:20:23 el8-a01n01.digimer.ca pacemaker-fenced [14954] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc Jul 14 22:20:24 el8-a01n01.digimer.ca pacemaker-controld [14958] (pcmk_cpg_membership) info: Group crmd event 0: node 1 pid 14958 joined via cpg_join Jul 14 22:20:24 el8-a01n01.digimer.ca pacemaker-controld [14958] (pcmk_cpg_membership) info: Group crmd event 0: el8-a01n01 (node 1 pid 14958) is member Jul 14 22:20:24 el8-a01n01.digimer.ca pacemaker-controld [14958] (pcmk_cpg_membership) info: Group crmd event 1: node 2 pid 2202 joined via cpg_join Jul 14 22:20:24 el8-a01n01.digimer.ca pacemaker-controld [14958] (pcmk_cpg_membership) info: Group crmd event 1: el8-a01n01 (node 1 pid 14958) is member Jul 14 22:20:24 el8-a01n01.digimer.ca pacemaker-controld [14958] (pcmk_cpg_membership) info: Group crmd event 1: el8-a01n02 (node 2 pid 2202) is member Jul 14 22:20:24 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online Jul 14 22:20:24 el8-a01n01.digimer.ca pacemaker-controld [14958] (peer_update_callback) info: Node el8-a01n02 is now a peer | DC= old=0x0000000 new=0x4000000 Jul 14 22:20:24 el8-a01n01.digimer.ca pacemaker-controld [14958] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds Jul 14 22:20:24 el8-a01n01.digimer.ca pacemaker-controld [14958] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 30 seconds Jul 14 22:20:24 el8-a01n01.digimer.ca pacemaker-controld [14958] (te_connect_stonith) notice: Fencer successfully connected Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-controld [14958] (election_count_vote) info: election-DC round 1 (owner node ID 2) lost: vote from el8-a01n02 (Uptime) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_log) info: Input I_PENDING received in state S_PENDING from do_election_count_vote Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-controld [14958] (update_dc) info: Set DC to el8-a01n02 (3.3.0) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_update_peer_expected) info: update_dc: Node el8-a01n02[2] - expected state is now member (was (null)) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n02/crmd/8, version=0.16.0) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n02/crmd/10, version=0.16.0) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n02/crmd/12, version=0.16.0) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n02/crmd/14, version=0.16.0) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_replace) info: Digest matched on replace from el8-a01n02: be52010831fd154fc90da9ca9708e392 Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_replace) info: Replaced 0.16.0 with 0.16.0 from el8-a01n02 Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=el8-a01n02/crmd/17, version=0.16.0) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_log) info: Input I_NOT_DC received in state S_PENDING from do_cl_join_finalize_respond Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_state_transition) notice: State transition S_PENDING -> S_NOT_DC | input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/18, version=0.16.0) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/19, version=0.16.0) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/lrm: OK (rc=0, origin=el8-a01n02/crmd/20, version=0.16.0) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.0 2 Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.1 (null) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=1 Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ /cib/status: Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/21, version=0.16.1) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/lrm: OK (rc=0, origin=el8-a01n02/crmd/22, version=0.16.1) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.1 2 Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.2 (null) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=2 Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ /cib/status: Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/23, version=0.16.2) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-25.raw Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/26, version=0.16.2) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.2 2 Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.3 (null) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=3 Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition, @join=member, @expected=member Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition, @join=member, @expected=member Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/27, version=0.16.3) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.3 2 Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.4 (null) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=4, @dc-uuid=2 Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n02/crmd/28, version=0.16.4) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_file_write_with_digest) info: Wrote version 0.16.0 of the CIB to disk (digest: 17149bee2ab28b054d280a7e9e1e7493) Jul 14 22:20:44 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.WdIR5j (digest: /var/lib/pacemaker/cib/cib.ZOqELV) Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.4 2 Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.5 (null) Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=5 Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_update_resource Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node1' not in cache Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=0, reply=1, notify=0 Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/31, version=0.16.5) Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node1' Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=0, reply=1, notify=1 Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=0, reply=1, notify=0 Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_lrm_rsc_op) info: Performing key=1:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=virsh_node1_monitor_0 Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.5 2 Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.6 (null) Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=6 Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/32, version=0.16.6) Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.6 2 Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.7 (null) Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=7 Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;5:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=5, @rc-code=7, @op-status=0, @exec-time=10, @queue-time=1 Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/33, version=0.16.7) Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.7 2 Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.8 (null) Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=8 Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;6:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=9, @rc-code=7, @op-status=0 Jul 14 22:20:45 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/34, version=0.16.8) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.8 2 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.9 (null) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=9 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/35, version=0.16.9) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.9 2 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.10 (null) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=10 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/36, version=0.16.10) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.10 2 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.11 (null) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=11 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;7:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=14, @rc-code=7, @op-status=0, @exec-time=63, @queue-time=1 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/37, version=0.16.11) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.11 2 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.12 (null) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=12 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;8:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=19, @rc-code=7, @op-status=0, @exec-time=58 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/38, version=0.16.12) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-controld [14958] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 15809 exited with rc=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/8) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=5, reply=1, notify=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:5 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (stonith_api_signon) debug: Attempting fencer connection by pacemaker-execd with mainloop Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.12 2 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.13 (null) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=13 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/8, version=0.16.13) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (stonith_api_signon) debug: Connection to fencer by pacemaker-execd succeeded (registration token: b20e9c24-c0a7-4dbe-94b6-1f965223c156) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:5 exit-code:7 exec-time:7ms queue-time:0ms Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-controld [14958] (process_lrm_event) notice: Result of probe operation for virsh_node1 on el8-a01n01: 7 (not running) | call=5 key=virsh_node1_monitor_0 confirmed=true cib-update=9 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/9) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.13 2 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node2' not in cache Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=0, reply=1, notify=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.14 (null) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=14 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;1:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=5, @rc-code=7, @op-status=0, @last-rc-change=1594765246, @last-run=1594765246, @exec-time=7 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node2' Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=0, reply=1, notify=1 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/9, version=0.16.14) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=0, reply=1, notify=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_lrm_rsc_op) info: Performing key=2:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=virsh_node2_monitor_0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=9, reply=1, notify=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/10) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_execute) debug: executing - rsc:virsh_node2 action:monitor call_id:9 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_finished) debug: finished - rsc:virsh_node2 action:monitor call_id:9 exit-code:7 exec-time:0ms queue-time:0ms Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-controld [14958] (process_lrm_event) notice: Result of probe operation for virsh_node2 on el8-a01n01: 7 (not running) | call=9 key=virsh_node2_monitor_0 confirmed=true cib-update=11 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.14 2 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.15 (null) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=15 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_lrm_rsc_op) info: Performing key=9:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=virsh_node1_start_0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/10, version=0.16.15) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/11) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/12) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=10, reply=1, notify=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_execute) info: executing - rsc:virsh_node1 action:start call_id:10 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.15 2 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.16 (null) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=16 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;2:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=9, @rc-code=7, @op-status=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/11, version=0.16.16) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.16 2 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.17 (null) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=17 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_start_0, @operation=start, @transition-key=9:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;9:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/12, version=0.16.17) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.17 2 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.18 (null) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=18 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=11:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;11:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765246, @last-run=1594765246 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/39, version=0.16.18) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor' not in cache Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=0, reply=1, notify=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor:0' not in cache Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=0, reply=1, notify=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_rsc_register) info: Cached agent information for 'hypervisor' Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=0, reply=1, notify=1 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=0, reply=1, notify=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_lrm_rsc_op) info: Performing key=3:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=hypervisor_monitor_0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=15, reply=1, notify=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:15 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/13) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.18 2 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.19 (null) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=19 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor' Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/13, version=0.16.19) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:15 exit-code:7 exec-time:14ms queue-time:2ms Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-controld [14958] (process_lrm_event) notice: Result of probe operation for hypervisor on el8-a01n01: 7 (not running) | call=15 key=hypervisor_monitor_0 confirmed=true cib-update=14 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/14) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.19 2 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.20 (null) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=20 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;3:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=15, @rc-code=7, @op-status=0, @exec-time=14, @queue-time=2 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/14, version=0.16.20) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_get_rsc_info) info: Agent information for 'drbd' not in cache Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=0, reply=1, notify=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_get_rsc_info) info: Agent information for 'drbd:0' not in cache Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=0, reply=1, notify=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_rsc_register) info: Cached agent information for 'drbd' Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=0, reply=1, notify=1 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=0, reply=1, notify=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_lrm_rsc_op) info: Performing key=4:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=drbd_monitor_0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.20 2 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.21 (null) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=21 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=15:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;15:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765246, @last-run=1594765246, @exec-time=0, Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/40, version=0.16.21) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/15) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.21 2 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=20, reply=1, notify=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.22 (null) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=22 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/15, version=0.16.22) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_finished) debug: finished - rsc:drbd action:monitor call_id:20 exit-code:7 exec-time:19ms queue-time:0ms Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-controld [14958] (process_lrm_event) notice: Result of probe operation for drbd on el8-a01n01: 7 (not running) | call=20 key=drbd_monitor_0 confirmed=true cib-update=16 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/16) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.22 2 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.23 (null) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=23 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;4:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=20, @rc-code=7, @op-status=0, @exec-time=19 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/16, version=0.16.23) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_lrm_rsc_op) info: Performing key=13:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=hypervisor_start_0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=21, reply=1, notify=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_execute) info: executing - rsc:hypervisor action:start call_id:21 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (systemd_unit_exec) debug: Performing asynchronous start op on systemd unit libvirtd named 'hypervisor' Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/17) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (systemd_unit_exec_with_unit) debug: Calling StartUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.23 2 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.24 (null) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=24 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=13:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;13:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0, @queue-time=0 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/17, version=0.16.24) Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (systemd_exec_result) info: Call to start passed: /org/freedesktop/systemd1/job/1336 Jul 14 22:20:46 el8-a01n01.digimer.ca pacemaker-execd [14955] (action_complete) debug: hypervisor start may still be in progress: re-scheduling (elapsed=305ms, remaining=99695ms, start_delay=2000ms) Jul 14 22:20:47 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.24 2 Jul 14 22:20:47 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.25 (null) Jul 14 22:20:47 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=25 Jul 14 22:20:47 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;11:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=20, @rc-code=0, @op-status=0, @exec-time=791 Jul 14 22:20:47 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/41, version=0.16.25) Jul 14 22:20:47 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.25 2 Jul 14 22:20:47 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.26 (null) Jul 14 22:20:47 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=26 Jul 14 22:20:47 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']: Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/4, version=0.16.37) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.37 2 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.38 (null) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=38 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=7:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;7:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765270, @last-run=1594765270, @exec-time=0 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/48, version=0.16.38) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.38 2 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.39 (null) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=39 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;7:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=25, @rc-code=0, @op-status=0 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_lrm_rsc_op) info: Performing key=8:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=virsh_node2_start_0 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/49, version=0.16.39) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=24, reply=1, notify=0 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_execute) info: executing - rsc:virsh_node2 action:start call_id:24 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.39 2 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.40 (null) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=40 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=10:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;10:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765270, @last-run=1594765270, @exec-time=0, @q Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/50, version=0.16.40) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/24) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemakerd [14952] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk_shutdown_worker) notice: Shutting down Pacemaker Jul 14 22:21:10 el8-a01n01.digimer.ca pacemakerd [14952] (stop_child) notice: Stopping pacemaker-controld | sent signal 15 to process 14958 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_shutdown) notice: Shutting down cluster resource manager | limit=1200000ms Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_shutdown_req) info: Sending shutdown request to all peers (DC is el8-a01n02) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_update_peer_expected) info: handle_request: Node el8-a01n01[1] - expected state is now down (was (null)) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-attrd [14956] (attrd_peer_update) notice: Setting shutdown[el8-a01n01]: (unset) -> 1594765270 | from el8-a01n02 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.40 2 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.41 (null) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=41 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=8:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;8:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765270, @last-run=1594765270 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/24, version=0.16.41) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.41 2 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.42 (null) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=42 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']: Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: ++ Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/5, version=0.16.42) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_finished) info: finished - rsc:virsh_node2 action:start call_id:24 exit-code:0 exec-time:749ms queue-time:0ms Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-controld [14958] (process_lrm_event) notice: Result of start operation for virsh_node2 on el8-a01n01: 0 (ok) | call=24 key=virsh_node2_start_0 confirmed=true cib-update=25 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/25) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.42 2 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.43 (null) Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=43 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;8:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=24, @rc-code=0, @op-status=0, @exec-time=749 Jul 14 22:21:10 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/25, version=0.16.43) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.43 2 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.44 (null) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=44 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;10:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=27, @rc-code=0, @op-status=0, @exec-time=2237 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/51, version=0.16.44) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:22 exit-code:0 exec-time:0ms queue-time:0ms Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=0, reply=1, notify=0 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_lrm_rsc_op) info: Performing key=3:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=virsh_node1_stop_0 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/26) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=26, reply=1, notify=0 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_execute) info: executing - rsc:virsh_node1 action:stop call_id:26 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_finished) info: finished - rsc:virsh_node1 action:stop call_id:26 exit-code:0 exec-time:0ms queue-time:0ms Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.44 2 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.45 (null) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=45 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_stop_0, @operation=stop, @transition-key=3:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;3:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765272, @last-run=1594765272, @exec-time=0 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-controld [14958] (process_lrm_event) info: Result of monitor operation for virsh_node1 on el8-a01n01: Cancelled | call=22 key=virsh_node1_monitor_60000 confirmed=true Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/26, version=0.16.45) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-controld [14958] (process_lrm_event) notice: Result of stop operation for virsh_node1 on el8-a01n01: 0 (ok) | call=26 key=virsh_node1_stop_0 confirmed=true cib-update=27 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/27) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_lrm_rsc_op) info: Performing key=4:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=virsh_node2_stop_0 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=27, reply=1, notify=0 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_execute) info: executing - rsc:virsh_node2 action:stop call_id:27 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_finished) info: finished - rsc:virsh_node2 action:stop call_id:27 exit-code:0 exec-time:0ms queue-time:0ms Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.45 2 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.46 (null) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=46 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:0;3:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=26, @rc-code=0, @op-status=0 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-controld [14958] (process_lrm_event) notice: Result of stop operation for virsh_node2 on el8-a01n01: 0 (ok) | call=27 key=virsh_node2_stop_0 confirmed=true cib-update=29 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/27, version=0.16.46) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/28) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-execd [14955] (cancel_recurring_action) info: Cancelling systemd operation hypervisor_status_60000 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:23 exit-code:0 exec-time:0ms queue-time:0ms Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=0, reply=1, notify=0 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_lrm_rsc_op) info: Performing key=5:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=hypervisor_stop_0 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/29) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-execd [14955] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 5125103c-578c-42d5-9219-4148b4fc6cd0: rc=29, reply=1, notify=0 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_execute) info: executing - rsc:hypervisor action:stop call_id:29 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-execd [14955] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit libvirtd named 'hypervisor' Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-controld [14958] (process_lrm_event) info: Result of monitor operation for hypervisor on el8-a01n01: Cancelled | call=23 key=hypervisor_monitor_60000 confirmed=true Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.46 2 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.47 (null) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=47 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=4:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;4:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765272, @last-run=1594765272, @exec-time=0 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-execd [14955] (systemd_unit_exec_with_unit) debug: Calling StopUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/28, version=0.16.47) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.47 2 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.48 (null) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=48 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;4:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=27, @rc-code=0, @op-status=0 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/29, version=0.16.48) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/30) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.48 2 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.49 (null) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=49 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=5:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;5:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765272, @last-run=1594765272, @exec-time=0, @que Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/30, version=0.16.49) Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-execd [14955] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/1426 Jul 14 22:21:12 el8-a01n01.digimer.ca pacemaker-execd [14955] (action_complete) debug: hypervisor stop may still be in progress: re-scheduling (elapsed=198ms, remaining=99802ms, start_delay=2000ms) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:29 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor' Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (action_complete) debug: hypervisor stop is now complete (elapsed=2203ms, remaining=97797ms): not running (7) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:29 exit-code:0 exec-time:2203ms queue-time:0ms Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/31) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (process_lrm_event) notice: Result of stop operation for hypervisor on el8-a01n01: 0 (ok) | call=29 key=hypervisor_stop_0 confirmed=true cib-update=31 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.49 2 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.50 (null) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=50 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;5:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=29, @rc-code=0, @op-status=0, @exec-time=2203 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/31, version=0.16.50) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (handle_request) info: Shutting down Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_state_transition) notice: State transition S_NOT_DC -> S_STOPPING | input=I_STOP cause=C_HA_MESSAGE origin=route_message Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (tengine_stonith_connection_destroy) info: Fencing daemon disconnected Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_lrm_control) info: Disconnecting from the executor Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (lrmd_api_disconnect) info: Disconnecting IPC local executor connection Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (lrmd_ipc_connection_destroy) info: IPC connection destroyed Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (lrm_connection_destroy) info: Disconnected from executor Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (lrmd_api_disconnect) info: Disconnecting IPC local executor connection Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_lrm_control) notice: Disconnected from the executor Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_cluster_disconnect) info: Disconnecting from cluster infrastructure: corosync Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_ipcs_dispatch_connection_request) debug: HUP conn (/dev/shm/qb-14955-14958-9-DajKTi/qb) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_ipcs_disconnect) debug: qb_ipcs_disconnect(/dev/shm/qb-14955-14958-9-DajKTi/qb) state:2 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (crm_client_destroy) debug: Destroying 0 events Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_rb_close_helper) debug: Free'ing ringbuffer: /dev/shm/qb-14955-14958-9-DajKTi/qb-response-lrmd-header Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_rb_close_helper) debug: Free'ing ringbuffer: /dev/shm/qb-14955-14958-9-DajKTi/qb-event-lrmd-header Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_rb_close_helper) debug: Free'ing ringbuffer: /dev/shm/qb-14955-14958-9-DajKTi/qb-request-lrmd-header Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (terminate_cs_connection) notice: Disconnected from Corosync Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_ha_control) info: Disconnected from the cluster Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_cib_control) info: Disconnecting from the CIB manager Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (crmd_cib_connection_destroy) info: Connection to the CIB manager terminated Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_cib_control) notice: Disconnected from the CIB manager Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_exit) info: Performing A_EXIT_0 - gracefully exiting the controller Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (do_exit) info: [pacemaker-controld] stopped (0) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (crmd_exit) info: Dropping I_TERMINATE: [ state=S_STOPPING cause=C_FSA_INTERNAL origin=do_stop ] Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (crmd_cs_destroy) info: Corosync connection closed Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (crmd_cs_destroy) info: Corosync connection closed Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (crmd_init) info: pacemaker-controld[14958] exiting with status 0 (OK) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-controld [14958] (crm_xml_cleanup) info: Cleaning up memory from libxml2 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk_child_exit) info: pacemaker-controld[14958] exited with status 0 (OK) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemakerd [14952] (stop_child) notice: Stopping pacemaker-schedulerd | sent signal 15 to process 14957 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-schedulerd[14957] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-schedulerd[14957] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-schedulerd[14957] (crm_xml_cleanup) info: Cleaning up memory from libxml2 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk_child_exit) info: pacemaker-schedulerd[14957] exited with status 0 (OK) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemakerd [14952] (stop_child) notice: Stopping pacemaker-attrd | sent signal 15 to process 14956 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-attrd [14956] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-attrd [14956] (main) info: Shutting down attribute manager Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-attrd [14956] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-attrd [14956] (attrd_cib_destroy_cb) info: Connection disconnection complete Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-attrd [14956] (crm_xml_cleanup) info: Cleaning up memory from libxml2 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: --- 0.16.50 2 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: Diff: +++ 0.16.51 (null) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib: @num_updates=51 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_dc_release, @expected=down Jul 14 22:21:14 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk_child_exit) info: pacemaker-attrd[14956] exited with status 0 (OK) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemakerd [14952] (stop_child) notice: Stopping pacemaker-execd | sent signal 15 to process 14955 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (lrmd_exit) info: Terminating with 0 clients Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (stonith_api_del_notification) debug: Removing callback for st_notify_disconnect events Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (stonith_api_signoff) debug: Disconnecting from the fencer Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect() Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-14954-14955-12-M4eiAk/qb-request-stonith-ng-header Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-14954-14955-12-M4eiAk/qb-response-stonith-ng-header Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-14954-14955-12-M4eiAk/qb-event-stonith-ng-header Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (qb_ipcs_unref) debug: qb_ipcs_unref() - destroying Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-execd [14955] (crm_xml_cleanup) info: Cleaning up memory from libxml2 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk_child_exit) info: pacemaker-execd[14955] exited with status 0 (OK) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemakerd [14952] (stop_child) notice: Stopping pacemaker-fenced | sent signal 15 to process 14954 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-fenced [14954] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-fenced [14954] (stonith_shutdown) info: Terminating with 0 clients Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-fenced [14954] (cib_connection_destroy) info: Connection to the CIB manager closed Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-fenced [14954] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (qb_ipcs_event_sendv) warning: new_event_notification (/dev/shm/qb-14953-14956-11-3i7Ugc/qb): Broken pipe (32) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_notify_send_one) warning: Notification of client attrd/0fae3a85-ac60-4a43-af94-30ea8585d317 failed Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (qb_ipcs_event_sendv) warning: new_event_notification (/dev/shm/qb-14953-14954-13-AS0Bwn/qb): Broken pipe (32) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_notify_send_one) warning: Notification of client stonithd/1510c981-5b24-4295-9677-0ba36a5b674d failed Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/53, version=0.16.51) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-fenced [14954] (crm_xml_cleanup) info: Cleaning up memory from libxml2 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_process_shutdown_req) info: Peer el8-a01n02 is requesting to shut down Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (pcmk_cpg_membership) info: Group cib event 2: el8-a01n02 (node 2 pid 2197) left via cpg_leave Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now offline Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now lost | nodeid=2 previous=member source=crm_update_peer_proc Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_reap_dead_member) info: Removing node with name el8-a01n02 and id 2 from membership cache Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (reap_crm_member) notice: Purged 1 peer with id=2 and/or uname=el8-a01n02 from the membership cache Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (pcmk_cpg_membership) info: Group cib event 2: el8-a01n01 (node 1 pid 14953) is member Jul 14 22:21:14 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk_child_exit) info: pacemaker-fenced[14954] exited with status 0 (OK) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemakerd [14952] (stop_child) notice: Stopping pacemaker-based | sent signal 15 to process 14953 Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_shutdown) info: Disconnected 0 clients Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (cib_shutdown) info: All clients disconnected (0) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (terminate_cib) info: initiate_exit: Exiting from mainloop... Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_cluster_disconnect) info: Disconnecting from cluster infrastructure: corosync Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (terminate_cs_connection) notice: Disconnected from Corosync Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (crm_cluster_disconnect) info: Disconnecting from cluster infrastructure: corosync Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (cluster_disconnect_cpg) info: No CPG connection Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (terminate_cs_connection) notice: Disconnected from Corosync Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:21:14 el8-a01n01.digimer.ca pacemaker-based [14953] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:21:14 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk_child_exit) info: pacemaker-based[14953] exited with status 0 (OK) Jul 14 22:21:14 el8-a01n01.digimer.ca pacemakerd [14952] (pcmk_shutdown_worker) notice: Shutdown complete Jul 14 22:21:14 el8-a01n01.digimer.ca pacemakerd [14952] (qb_ipcs_us_withdraw) info: withdrawing server sockets Jul 14 22:21:14 el8-a01n01.digimer.ca pacemakerd [14952] (crm_xml_cleanup) info: Cleaning up memory from libxml2