[root@el8-a01n02 ~]# cat /var/log/pacemaker/pacemaker.log Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_log_init) info: Cannot change active directory to /var/lib/pacemaker/cores: No such file or directory (2) Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (get_cluster_type) info: Detected an active 'corosync' cluster Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (mcp_read_config) info: Reading configure for stack: corosync Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (main) notice: Starting Pacemaker 2.0.3-5.el8_2.1 | build=4b1f869f0f features: generated-manpages agent-manpages ncurses libqb-logging libqb-ipc systemd nagios corosync-native atomic-attrd acls Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (main) info: Maximum core file size is: 18446744073709551615 Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (qb_ipcs_us_publish) info: server name: pacemakerd Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to lrmd IPC: Connection refused Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to cib_ro IPC: Connection refused Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to crmd IPC: Connection refused Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to attrd IPC: Connection refused Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to pengine IPC: Connection refused Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to stonith-ng IPC: Connection refused Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_get_peer) info: Created entry 9bbf4396-4299-4d59-82ee-27fa8b85610b/0x55d304462f90 for node el8-a01n02/2 (1 total) Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_get_peer) info: Node 2 is now known as el8-a01n02 Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_get_peer) info: Node 2 has uuid 2 Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (cluster_connect_quorum) warning: Quorum lost Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_get_peer) info: Created entry 146915c5-e9a4-400e-a4d5-0c8bb9fade2a/0x55d3046648b0 for node el8-a01n01/1 (2 total) Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_get_peer) info: Node 1 is now known as el8-a01n01 Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_get_peer) info: Node 1 has uuid 1 Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Using uid=189 and group=189 for process pacemaker-based Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Forked child 4565 for process pacemaker-based Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Forked child 4566 for process pacemaker-fenced Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Forked child 4567 for process pacemaker-execd Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Using uid=189 and group=189 for process pacemaker-attrd Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Forked child 4568 for process pacemaker-attrd Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Using uid=189 and group=189 for process pacemaker-schedulerd Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Forked child 4569 for process pacemaker-schedulerd Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Using uid=189 and group=189 for process pacemaker-controld Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Forked child 4570 for process pacemaker-controld Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-attrd [4568] (main) notice: Starting Pacemaker node attribute manager Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (main) notice: Pacemaker daemon successfully started and accepting connections Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk_quorum_notification) info: Quorum still lost | membership=140 members=1 Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_reap_unseen_nodes) info: State of node el8-a01n01[1] is still unknown Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk_cpg_membership) info: Group pacemakerd event 0: node 2 pid 4564 joined via cpg_join Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk_cpg_membership) info: Group pacemakerd event 0: el8-a01n02 (node 2 pid 4564) is member Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (main) notice: Starting Pacemaker local executor Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (qb_ipcs_us_publish) info: server name: lrmd Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (main) notice: Pacemaker local executor successfully started and accepting connections Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (main) notice: Starting Pacemaker CIB manager Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (get_cluster_type) info: Verifying cluster type: 'corosync' Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (get_cluster_type) info: Assuming an active 'corosync' cluster Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (pcmk__daemon_can_write) notice: /var/lib/pacemaker/cib/cib.xml not found: No such file or directory Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (pcmk__daemon_can_write) notice: /var/lib/pacemaker/cib/cib.xml.sig not found: No such file or directory Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (retrieveCib) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.xml (digest: /var/lib/pacemaker/cib/cib.xml.sig) Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_read_and_verify) warning: Could not verify cluster configuration file /var/lib/pacemaker/cib/cib.xml: No such file or directory (2) Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (readCibXmlFile) warning: Primary configuration corrupt or unusable, trying backups in /var/lib/pacemaker/cib Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (readCibXmlFile) warning: Continuing with an empty configuration. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-schedulerd[4569] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-schedulerd[4569] (main) notice: Starting Pacemaker scheduler Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-schedulerd[4569] (qb_ipcs_us_publish) info: server name: pengine Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-schedulerd[4569] (main) notice: Pacemaker scheduler successfully started and accepting connections Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (main) notice: Starting Pacemaker fencer Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (get_cluster_type) info: Verifying cluster type: 'corosync' Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (get_cluster_type) info: Assuming an active 'corosync' cluster Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_get_peer) info: Created entry 54e0653e-2879-48de-b644-14da24f28319/0x55a6ed64dd60 for node el8-a01n02/2 (1 total) Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_get_peer) info: Node 2 is now known as el8-a01n02 Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-controld [4570] (main) notice: Starting Pacemaker controller Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_log) info: Input I_STARTUP received in state S_STARTING from crmd_init Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_get_peer) info: Node 2 has uuid 2 Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (init_cs_connection_once) info: Connection to 'corosync': established Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_get_peer) info: Created entry 51324133-b4ae-420b-a11b-c4739181e652/0x557965bd37c0 for node el8-a01n02/2 (1 total) Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_get_peer) info: Node 2 is now known as el8-a01n02 Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_get_peer) info: Node 2 has uuid 2 Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (init_cs_connection_once) info: Connection to 'corosync': established Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (qb_ipcs_us_publish) info: server name: cib_ro Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (qb_ipcs_us_publish) info: server name: cib_rw Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (qb_ipcs_us_publish) info: server name: cib_shm Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (main) notice: Pacemaker CIB manager successfully started and accepting connections Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (pcmk_cpg_membership) info: Group cib event 0: node 2 pid 4565 joined via cpg_join Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (pcmk_cpg_membership) info: Group cib event 0: el8-a01n02 (node 2 pid 4565) is member Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_read_and_verify) warning: Could not verify cluster configuration file /var/lib/pacemaker/cib/cib.xml: No such file or directory (2) Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-0.raw Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.0.0 of the CIB to disk (digest: 2469c254d4b2872e864ff6b9c730ac0c) Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.zYQYpv (digest: /var/lib/pacemaker/cib/cib.VT7O7Z) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (main) info: CIB connection active Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (get_cluster_type) info: Verifying cluster type: 'corosync' Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (get_cluster_type) info: Assuming an active 'corosync' cluster Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_get_peer) info: Created entry 0ad36a2d-dd0d-4180-824d-83e6b89fee3e/0x55bd38224a20 for node el8-a01n02/2 (1 total) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_get_peer) info: Node 2 is now known as el8-a01n02 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_get_peer) info: Node 2 has uuid 2 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (init_cs_connection_once) info: Connection to 'corosync': established Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (main) info: Cluster connection active Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_erase_attrs) info: Clearing transient attributes from CIB | xpath=//node_state[@uname='el8-a01n02']/transient_attributes Jul 14 21:28:47 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk_quorum_notification) notice: Quorum acquired | membership=144 members=2 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemakerd [4564] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification Jul 14 21:28:47 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk_cpg_membership) info: Group pacemakerd event 1: node 1 pid 375401 joined via cluster join Jul 14 21:28:47 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n01 (node 1 pid 375401) is member Jul 14 21:28:47 el8-a01n02.digimer.ca pacemakerd [4564] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online Jul 14 21:28:47 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n02 (node 2 pid 4564) is member Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (pcmk_cpg_membership) info: Group cib event 1: node 1 pid 375409 joined via cluster join Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (qb_ipcs_us_publish) info: server name: attrd Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (main) notice: Pacemaker node attribute manager successfully started and accepting connections Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (pcmk_cpg_membership) info: Group attrd event 0: node 2 pid 4568 joined via cpg_join Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (pcmk_cpg_membership) info: Group attrd event 0: el8-a01n02 (node 2 pid 4568) is member Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n02]: (unset) -> 2 | from el8-a01n02 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_start_election_if_needed) info: Starting an election to determine the writer Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (election_check) info: election-attrd won by local node Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_declare_winner) notice: Recorded local node as attribute writer (was unset) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Processed 1 private change for #attrd-protocol, id=n/a, set=n/a Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_get_peer) info: Created entry 84fb1d58-897c-4c44-8d51-5e4a37114af8/0x557965bdac50 for node el8-a01n01/1 (2 total) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_get_peer) info: Node 1 is now known as el8-a01n01 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_get_peer) info: Node 1 has uuid 1 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n01 (node 1 pid 375409) is member Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n02 (node 2 pid 4565) is member Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n02']/transient_attributes to all (origin=local/attrd/2) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/transient_attributes: OK (rc=0, origin=el8-a01n02/attrd/2, version=0.0.0) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (get_cluster_type) info: Verifying cluster type: 'corosync' Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (get_cluster_type) info: Assuming an active 'corosync' cluster Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_get_peer) info: Created entry 8b024f81-724f-4922-992e-509d1eede034/0x5615c858fe20 for node el8-a01n02/2 (1 total) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_get_peer) info: Node 2 is now known as el8-a01n02 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (peer_update_callback) info: Cluster node el8-a01n02 is now in unknown state Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_get_peer) info: Node 2 has uuid 2 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (peer_update_callback) info: Node el8-a01n02 is now a peer | DC= old=0x0000000 new=0x4000000 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (init_cs_connection_once) info: Connection to 'corosync': established Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (cluster_connect_quorum) notice: Quorum acquired Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (setup_cib) info: Watching for stonith topology changes Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (qb_ipcs_us_publish) info: server name: stonith-ng Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (main) notice: Pacemaker fencer successfully started and accepting connections Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (init_cib_cache_cb) info: Updating device list from the cib: init Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (cib_devices_update) info: Updating devices to version 0.0.0 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (unpack_nodes) info: Creating a fake local node Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (pcmk_cpg_membership) info: Group stonith-ng event 0: node 2 pid 4566 joined via cpg_join Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (pcmk_cpg_membership) info: Group stonith-ng event 0: el8-a01n02 (node 2 pid 4566) is member Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (pcmk_cpg_membership) info: Group stonith-ng event 1: node 1 pid 375410 joined via cluster join Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_get_peer) info: Created entry 7a289904-7d3d-4548-8266-a2d86be2b071/0x5615c8592700 for node el8-a01n01/1 (2 total) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_get_peer) info: Node 1 is now known as el8-a01n01 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (peer_update_callback) info: Cluster node el8-a01n01 is now in unknown state Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_get_peer) info: Node 1 has uuid 1 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (pcmk_cpg_membership) info: Group attrd event 1: node 1 pid 375412 joined via cpg_join Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_get_peer) info: Created entry 3c30217b-b590-46e5-8bfb-97b297c96171/0x55a6ed7596e0 for node el8-a01n01/1 (2 total) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_get_peer) info: Node 1 is now known as el8-a01n01 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_get_peer) info: Node 1 has uuid 1 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n01 (node 1 pid 375410) is member Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n02 (node 2 pid 4566) is member Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_get_peer) info: Created entry 5eb72a08-a13c-41f2-8ded-452f6a55d402/0x55bd382276d0 for node el8-a01n01/1 (2 total) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_get_peer) info: Node 1 is now known as el8-a01n01 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_get_peer) info: Node 1 has uuid 1 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (pcmk_cpg_membership) info: Group attrd event 1: el8-a01n01 (node 1 pid 375412) is member Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (pcmk_cpg_membership) info: Group attrd event 1: el8-a01n02 (node 2 pid 4568) is member Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_ha_control) info: Connected to the cluster Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (lrmd_ipc_connect) info: Connecting to executor Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-execd [4567] (crm_client_new) debug: Connecting 0x5567f4193fb0 for uid=189 gid=189 pid=4570 id=2eb86a7f-4b61-4d54-8801-a8f9e7673b7c Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-execd [4567] (handle_new_connection) debug: IPC credentials authenticated (/dev/shm/qb-4567-4570-9-3C903B/qb) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-execd [4567] (qb_ipcs_shm_connect) debug: connecting to client [4570] Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-execd [4567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-execd [4567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-execd [4567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/4) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed register operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.0.0 2 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.1.0 (null) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=1 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/nodes: Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/nodes: Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/4, version=0.1.0) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_control) info: Connection to the executor established Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_started) info: Delaying start, no membership data (0000000000100000) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_started) info: Delaying start, no membership data (0000000000100000) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (pcmk_quorum_notification) info: Quorum retained | membership=144 members=2 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (peer_update_callback) info: Cluster node el8-a01n01 is now member (was in unknown state) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (peer_update_callback) info: Cluster node el8-a01n02 is now member (was in unknown state) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n01]: (unset) -> 2 | from el8-a01n01 Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Processed 2 private changes for #attrd-protocol, id=n/a, set=n/a Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_started) info: Delaying start, Config not read (0000000000000040) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (qb_ipcs_us_publish) info: server name: crmd Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_started) notice: Pacemaker controller successfully started and accepting connections Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_log) info: Input I_PENDING received in state S_STARTING from do_started Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_state_transition) notice: State transition S_STARTING -> S_PENDING | input=I_PENDING cause=C_FSA_INTERNAL origin=do_started Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-1.raw Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.1.0 of the CIB to disk (digest: 7aefab66c16403ca8289ab3d2d263d82) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/transient_attributes: OK (rc=0, origin=el8-a01n01/attrd/2, version=0.1.0) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.miPs12 (digest: /var/lib/pacemaker/cib/cib.PYE6LA) Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/4, version=0.1.0) Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (pcmk_cpg_membership) info: Group crmd event 0: node 2 pid 4570 joined via cpg_join Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (pcmk_cpg_membership) info: Group crmd event 0: el8-a01n02 (node 2 pid 4570) is member Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (pcmk_cpg_membership) info: Group crmd event 1: node 1 pid 375414 joined via cpg_join Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (pcmk_cpg_membership) info: Group crmd event 1: el8-a01n01 (node 1 pid 375414) is member Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (peer_update_callback) info: Node el8-a01n01 is now a peer | DC= old=0x0000000 new=0x4000000 Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (pcmk_cpg_membership) info: Group crmd event 1: el8-a01n02 (node 2 pid 4570) is member Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 30 seconds Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (te_connect_stonith) notice: Fencer successfully connected Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.1.0 2 Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.2.0 (null) Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=2 Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/resources: Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.2.0) Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-fenced [4566] (update_cib_stonith_devices_v2) info: Updating device list from the cib: create resources Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-fenced [4566] (cib_devices_update) info: Updating devices to version 0.2.0 Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-2.raw Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.2.0 of the CIB to disk (digest: 0d075a040dee76aba3d9bb362ddfe793) Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.F1NKGV (digest: /var/lib/pacemaker/cib/cib.7jVe6l) Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-fenced [4566] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 4580 exited with rc=0 Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-fenced [4566] (stonith_device_register) notice: Added 'virsh_node1' to the device list (1 active devices) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_timer_popped) info: Election Trigger just popped | input=I_DC_TIMEOUT time=20000ms Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_log) warning: Input I_DC_TIMEOUT received in state S_PENDING from crm_timer_popped Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_state_transition) info: State transition S_PENDING -> S_ELECTION | input=I_DC_TIMEOUT cause=C_TIMER_POPPED origin=crm_timer_popped Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (election_count_vote) info: election-DC round 1 (owner node ID 1) lost: vote from el8-a01n01 (Uptime) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_state_transition) notice: State transition S_ELECTION -> S_PENDING | input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_dc_release) info: DC role released Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_te_control) info: Transitioner is now inactive Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_log) info: Input I_RELEASE_SUCCESS received in state S_PENDING from do_dc_release Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n01/crmd/8, version=0.2.0) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.2.0 2 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.3.0 (null) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=3 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/crm_config: Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n01/crmd/10, version=0.3.0) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.3.0 2 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.4.0 (null) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=4 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']: Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n01/crmd/12, version=0.4.0) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-3.raw Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.4.0 2 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.5.0 (null) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=5 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']: Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n01/crmd/14, version=0.5.0) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.3.0 of the CIB to disk (digest: be8b332ecdd2a858160b9164e32749f2) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.Ee13h3 (digest: /var/lib/pacemaker/cib/cib.S8UXiB) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-4.raw Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (update_dc) info: Set DC to el8-a01n01 (3.3.0) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_update_peer_expected) info: update_dc: Node el8-a01n01[1] - expected state is now member (was (null)) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.5.0 2 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.0 (null) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=6 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']: Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n01/crmd/16, version=0.6.0) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_replace) info: Digest matched on replace from el8-a01n01: dc9656c0a868cfbd5503dbb3659ae0b0 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_replace) info: Replaced 0.6.0 with 0.6.0 from el8-a01n01 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=el8-a01n01/crmd/21, version=0.6.0) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_log) info: Input I_NOT_DC received in state S_PENDING from do_cl_join_finalize_respond Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_state_transition) notice: State transition S_PENDING -> S_NOT_DC | input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.5.0 of the CIB to disk (digest: 2efada062c6ed14e51d7c9f2521b85a4) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.8qILj3 (digest: /var/lib/pacemaker/cib/cib.iTmXnB) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/22, version=0.6.0) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/23, version=0.6.0) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/lrm: OK (rc=0, origin=el8-a01n01/crmd/24, version=0.6.0) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.0 2 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.1 (null) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=1 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status: Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/25, version=0.6.1) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/lrm: OK (rc=0, origin=el8-a01n01/crmd/26, version=0.6.1) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-5.raw Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.6.0 of the CIB to disk (digest: 26a4a88a0d55ef6a920ee0c95c0ed3d1) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.1 2 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.2 (null) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=2 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status: Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.TqtDn3 (digest: /var/lib/pacemaker/cib/cib.bgZ0tB) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/27, version=0.6.2) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/30, version=0.6.2) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.2 2 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.3 (null) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=3 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition, @join=member, @expected=member Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition, @join=member, @expected=member Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/31, version=0.6.3) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.3 2 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.4 (null) Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=4, @have-quorum=1, @dc-uuid=1 Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n01/crmd/32, version=0.6.4) Jul 14 21:29:09 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node1' not in cache Jul 14 21:29:09 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0 Jul 14 21:29:09 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node1' Jul 14 21:29:09 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=1 Jul 14 21:29:09 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0 Jul 14 21:29:09 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=2:0:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=virsh_node1_monitor_0 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-controld [4570] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 4585 exited with rc=0 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/12) Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=5, reply=1, notify=0 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:5 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (stonith_api_signon) debug: Attempting fencer connection by pacemaker-execd with mainloop Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.4 2 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.5 (null) Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=5 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_update_resource Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/12, version=0.6.5) Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (stonith_api_signon) debug: Connection to fencer by pacemaker-execd succeeded (registration token: a13c92b4-20bd-4071-8612-2cc435d107f4) Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:5 exit-code:7 exec-time:12ms queue-time:0ms Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of probe operation for virsh_node1 on el8-a01n02: 7 (not running) | call=5 key=virsh_node1_monitor_0 confirmed=true cib-update=13 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/13) Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.5 2 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.6 (null) Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=6 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;2:0:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=5, @rc-code=7, @op-status=0, @last-rc-change=1594762150, @last-run=1594762150, @exec-time=12 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/13, version=0.6.6) Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.6 2 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.7 (null) Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=7 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/35, version=0.6.7) Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.7 2 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.8 (null) Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=8 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;1:0:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=5, @rc-code=7, @op-status=0, @last-rc-change=1594762150, @last-run=1594762150, @exec-time=6 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/36, version=0.6.8) Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.8 2 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.9 (null) Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=9 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_start_0, @operation=start, @transition-key=3:0:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;3:0:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0 Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/37, version=0.6.9) Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.9 2 Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.0 (null) Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=7, @num_updates=0 Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/resources: Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.7.0) Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-fenced [4566] (update_cib_stonith_devices_v2) info: Updating device list from the cib: create resources Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-fenced [4566] (cib_devices_update) info: Updating devices to version 0.7.0 Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-fenced [4566] (stonith_device_register) notice: Added 'virsh_node2' to the device list (2 active devices) Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-6.raw Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.7.0 of the CIB to disk (digest: ed4816be9cba1c0fb4b22dfa9b7cb0b2) Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.Wlegkg (digest: /var/lib/pacemaker/cib/cib.XVdhn1) Jul 14 21:29:17 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_procfs_pid_of) info: Found pacemaker-based active as process 4565 Jul 14 21:29:17 el8-a01n02.digimer.ca pacemaker-controld [4570] (throttle_send_command) info: New throttle mode: negligible load (was undetermined) Jul 14 21:29:18 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_ping) info: Reporting our current digest to el8-a01n01: 38bdea493108431f663dd4ee0010b3ce for 0.7.0 (0x557965c88030 0) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.0 2 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.1 (null) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=1 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:1;3:0:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=6, @rc-code=1, @op-status=0, @exec-time=11453, @queue-time=1 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']: INFINITY | from el8-a01n01 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 4 with 1 change for fail-count-virsh_node1#start_0 (id n/a, set n/a) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting last-failure-virsh_node1#start_0[el8-a01n01]: (unset) -> 1594762161 | from el8-a01n01 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 5 with 1 change for last-failure-virsh_node1#start_0 (id n/a, set n/a) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/4) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/5) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.1 2 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.2 (null) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=2 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']: Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/4, version=0.7.2) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 4 result for fail-count-virsh_node1#start_0: OK | rc=0 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * fail-count-virsh_node1#start_0[el8-a01n01]=INFINITY Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.2 2 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.3 (null) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=3 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']: Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/5, version=0.7.3) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 5 result for last-failure-virsh_node1#start_0: OK | rc=0 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * last-failure-virsh_node1#start_0[el8-a01n01]=1594762161 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.3 2 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.4 (null) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=4 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_stop_0, @operation=stop, @transition-key=1:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;1:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762161, @last-run=1594762161, @exec-time=0, @q Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/42, version=0.7.4) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node2' not in cache Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node2' Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=1 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=3:2:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=virsh_node2_monitor_0 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=9, reply=1, notify=0 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:virsh_node2 action:monitor call_id:9 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:virsh_node2 action:monitor call_id:9 exit-code:7 exec-time:0ms queue-time:0ms Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/14) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of probe operation for virsh_node2 on el8-a01n02: 7 (not running) | call=9 key=virsh_node2_monitor_0 confirmed=true cib-update=15 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.4 2 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.5 (null) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=5 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/14, version=0.7.5) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/15) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.5 2 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.6 (null) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=6 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/43, version=0.7.6) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.6 2 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.7 (null) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=7 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;3:2:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=9, @rc-code=7, @op-status=0 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/15, version=0.7.7) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.7 2 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.8 (null) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=8 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:0;1:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=7, @rc-code=0, @op-status=0, @exec-time=1 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/44, version=0.7.8) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.8 2 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.9 (null) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=9 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;2:2:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=11, @rc-code=7, @op-status=0 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/45, version=0.7.9) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=4:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=virsh_node1_start_0 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/16) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=10, reply=1, notify=0 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) info: executing - rsc:virsh_node1 action:start call_id:10 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.9 2 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.10 (null) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=10 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_start_0, @operation=start, @transition-key=4:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;4:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762161, @last-run=1594762161, @exec-time=0 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/16, version=0.7.10) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.10 2 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.11 (null) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=11 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=6:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;6:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/46, version=0.7.11) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.11 2 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.8.0 (null) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=8, @num_updates=0 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/resources: Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.8.0) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-fenced [4566] (update_cib_stonith_devices_v2) info: Updating device list from the cib: create resources Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-fenced [4566] (cib_devices_update) info: Updating devices to version 0.8.0 Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-7.raw Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.8.0 of the CIB to disk (digest: 2c9ff6874f28590448a2f242c01625bf) Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.tjiQRF (digest: /var/lib/pacemaker/cib/cib.9WdCsQ) Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.8.0 2 Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.9.0 (null) Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: -- /cib/configuration/resources/primitive[@id='hypervisor'] Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=9 Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/resources: Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-fenced [4566] (stonith_device_remove) info: Device 'hypervisor' not found (2 active devices) Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-fenced [4566] (update_cib_stonith_devices_v2) info: Updating device list from the cib: create resources Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-fenced [4566] (cib_devices_update) info: Updating devices to version 0.9.0 Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_replace operation for section configuration: OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.9.0) Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-8.raw Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.9.0 of the CIB to disk (digest: db64a16c52c066583a5b1a74240d556a) Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.1zzlfH (digest: /var/lib/pacemaker/cib/cib.0kqxdT) Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.9.0 2 Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.10.0 (null) Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=10 Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/resources: Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.10.0) Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-fenced [4566] (update_cib_stonith_devices_v2) info: Updating device list from the cib: create resources Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-fenced [4566] (cib_devices_update) info: Updating devices to version 0.10.0 Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-9.raw Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.10.0 of the CIB to disk (digest: 84477666f918034297fa4b47c59c0a6a) Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.pHbaKI (digest: /var/lib/pacemaker/cib/cib.Ysl4cW) Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.10.0 2 Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.0 (null) Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: -- /cib/configuration/resources/primitive[@id='drbd'] Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=11 Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/resources: Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_replace operation for section configuration: OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.11.0) Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-fenced [4566] (stonith_device_remove) info: Device 'drbd' not found (2 active devices) Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-fenced [4566] (update_cib_stonith_devices_v2) info: Updating device list from the cib: create resources Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-fenced [4566] (cib_devices_update) info: Updating devices to version 0.11.0 Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-10.raw Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.11.0 of the CIB to disk (digest: 2cddade26531e2c2dfcb3bac3401f84c) Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.AKYe3J (digest: /var/lib/pacemaker/cib/cib.UTocPY) Jul 14 21:29:26 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_1:4588:stderr [ 2020-07-14 21:29:26,710 ERROR: Unable to connect/login to fencing device ] Jul 14 21:29:26 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_1:4588:stderr [ ] Jul 14 21:29:26 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_1:4588:stderr [ ] Jul 14 21:29:26 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4588] stderr: [ 2020-07-14 21:29:26,710 ERROR: Unable to connect/login to fencing device ] Jul 14 21:29:26 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4588] stderr: [ ] Jul 14 21:29:26 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4588] stderr: [ ] Jul 14 21:29:26 el8-a01n02.digimer.ca pacemaker-fenced [4566] (internal_stonith_action_execute) info: Attempt 2 to execute fence_virsh (monitor). remaining timeout is 15 Jul 14 21:29:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_ping) info: Reporting our current digest to el8-a01n01: b7814e3f7afd5c57f438342d72812427 for 0.11.0 (0x557965cbb5b0 0) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_2:4608:stderr [ 2020-07-14 21:29:32,910 ERROR: Unable to connect/login to fencing device ] Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_2:4608:stderr [ ] Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_2:4608:stderr [ ] Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4608] stderr: [ 2020-07-14 21:29:32,910 ERROR: Unable to connect/login to fencing device ] Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4608] stderr: [ ] Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4608] stderr: [ ] Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-fenced [4566] (update_remaining_timeout) info: Attempted to execute agent fence_virsh (monitor) the maximum number of times (2) allowed Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_operation) notice: Operation 'monitor' [4608] for device 'virsh_node1' returned: -201 (Generic Pacemaker error) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) info: finished - rsc:virsh_node1 action:start call_id:10 exit-code:1 exec-time:11419ms queue-time:0ms Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of start operation for virsh_node1 on el8-a01n02: 1 (error) | call=10 key=virsh_node1_start_0 confirmed=true cib-update=17 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/17) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.0 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.1 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=1 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:1;4:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=10, @rc-code=1, @op-status=0, @exec-time=11419 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']: INFINITY | from el8-a01n01 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/6) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 6 with 2 changes for fail-count-virsh_node1#start_0 (id n/a, set n/a) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting last-failure-virsh_node1#start_0[el8-a01n02]: (unset) -> 1594762173 | from el8-a01n01 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.1 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.2 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']: Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 7 with 2 changes for last-failure-virsh_node1#start_0 (id n/a, set n/a) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/6, version=0.11.2) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/7) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 6 result for fail-count-virsh_node1#start_0: OK | rc=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * fail-count-virsh_node1#start_0[el8-a01n01]=INFINITY Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * fail-count-virsh_node1#start_0[el8-a01n02]=INFINITY Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.2 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.3 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=3 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/7, version=0.11.3) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 7 result for last-failure-virsh_node1#start_0: OK | rc=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * last-failure-virsh_node1#start_0[el8-a01n01]=1594762161 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * last-failure-virsh_node1#start_0[el8-a01n02]=1594762173 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.3 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.4 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=4 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:1;6:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=12, @rc-code=1, @op-status=0, @exec-time=11463 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']: INFINITY | from el8-a01n01 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 8 with 1 change for fail-count-virsh_node2#start_0 (id n/a, set n/a) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/8) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting last-failure-virsh_node2#start_0[el8-a01n01]: (unset) -> 1594762173 | from el8-a01n01 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 9 with 1 change for last-failure-virsh_node2#start_0 (id n/a, set n/a) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/9) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.4 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.5 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=5 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']: Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/8, version=0.11.5) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 8 result for fail-count-virsh_node2#start_0: OK | rc=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * fail-count-virsh_node2#start_0[el8-a01n01]=INFINITY Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.5 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.6 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=6 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']: Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/9, version=0.11.6) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 9 result for last-failure-virsh_node2#start_0: OK | rc=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * last-failure-virsh_node2#start_0[el8-a01n01]=1594762173 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=2:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=virsh_node1_stop_0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/18) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=11, reply=1, notify=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) info: executing - rsc:virsh_node1 action:stop call_id:11 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) info: finished - rsc:virsh_node1 action:stop call_id:11 exit-code:0 exec-time:1ms queue-time:0ms Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.6 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.7 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=7 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=1:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;1:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762173, @last-run=1594762173, @exec-time=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/51, version=0.11.7) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.7 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.8 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=8 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_stop_0, @operation=stop, @transition-key=2:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;2:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762173, @last-run=1594762173, @exec-time=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/18, version=0.11.8) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of stop operation for virsh_node1 on el8-a01n02: 0 (ok) | call=11 key=virsh_node1_stop_0 confirmed=true cib-update=19 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/19) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.8 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.9 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=9 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:0;2:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=11, @rc-code=0, @op-status=0, @exec-time=1 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/19, version=0.11.9) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.9 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.10 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=10 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor' not in cache Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/52, version=0.11.10) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor:1' not in cache Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_rsc_register) info: Cached agent information for 'hypervisor' Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=1 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=5:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=hypervisor_monitor_0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/20) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=16, reply=1, notify=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:16 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.10 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.11 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=11 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/20, version=0.11.11) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor' Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:16 exit-code:7 exec-time:11ms queue-time:0ms Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.11 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.12 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=12 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/53, version=0.11.12) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/21) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of probe operation for hypervisor on el8-a01n02: 7 (not running) | call=16 key=hypervisor_monitor_0 confirmed=true cib-update=21 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.12 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.13 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=13 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;5:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=16, @rc-code=7, @op-status=0, @exec-time=11 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/21, version=0.11.13) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_get_rsc_info) info: Agent information for 'drbd' not in cache Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_get_rsc_info) info: Agent information for 'drbd:1' not in cache Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_rsc_register) info: Cached agent information for 'drbd' Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=1 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=6:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=drbd_monitor_0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/22) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.13 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.14 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=14 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;1:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=13, @rc-code=0, @op-status=0, @exec-time=1, @queue-time=1 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=21, reply=1, notify=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:21 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/54, version=0.11.14) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.14 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.15 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=15 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/22, version=0.11.15) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.15 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.16 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=16 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;3:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=18, @rc-code=7, @op-status=0, @exec-time=8 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/55, version=0.11.16) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:drbd action:monitor call_id:21 exit-code:7 exec-time:14ms queue-time:1ms Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of probe operation for drbd on el8-a01n02: 7 (not running) | call=21 key=drbd_monitor_0 confirmed=true cib-update=23 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.16 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.17 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=17 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;4:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=23, @rc-code=7, @op-status=0, @exec-time=11, @queue-time=1 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/56, version=0.11.17) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/23) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.17 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.18 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=18 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;6:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=21, @rc-code=7, @op-status=0, @exec-time=14, @queue-time=1 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/23, version=0.11.18) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=7:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=virsh_node2_start_0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/24) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=22, reply=1, notify=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) info: executing - rsc:virsh_node2 action:start call_id:22 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.18 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.19 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=19 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=7:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;7:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762173, @last-run=1594762173 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/24, version=0.11.19) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=11:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=hypervisor_start_0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/25) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=23, reply=1, notify=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) info: executing - rsc:hypervisor action:start call_id:23 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.19 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.20 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=20 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=11:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;11:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous start op on systemd unit libvirtd named 'hypervisor' Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/25, version=0.11.20) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec_with_unit) debug: Calling StartUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=19:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=drbd_start_0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/26) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=24, reply=1, notify=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) info: executing - rsc:drbd action:start call_id:24 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous start op on systemd unit drbd named 'drbd' Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.20 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.21 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=21 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_start_0, @operation=start, @transition-key=19:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;19:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0, @queue-time=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/26, version=0.11.21) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.21 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.22 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=22 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=9:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;9:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/57, version=0.11.22) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.22 2 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.23 (null) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=23 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_start_0, @operation=start, @transition-key=17:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;17:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0, @queue-time=0 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/58, version=0.11.23) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_exec_result) info: Call to start passed: /org/freedesktop/systemd1/job/4074 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: hypervisor start may still be in progress: re-scheduling (elapsed=217ms, remaining=99783ms, start_delay=2000ms) Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec_with_unit) debug: Calling StartUnit for drbd: /org/freedesktop/systemd1/unit/drbd_2eservice Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_exec_result) info: Call to start passed: /org/freedesktop/systemd1/job/4158 Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd start may still be in progress: re-scheduling (elapsed=404ms, remaining=99596ms, start_delay=2000ms) Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:23 Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor' Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: hypervisor start is now complete (elapsed=2222ms, remaining=97778ms): ok (0) Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:23 exit-code:0 exec-time:2222ms queue-time:0ms Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/27) Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of start operation for hypervisor on el8-a01n02: 0 (ok) | call=23 key=hypervisor_start_0 confirmed=true cib-update=27 Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.23 2 Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.24 (null) Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=24 Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;11:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=23, @rc-code=0, @op-status=0, @exec-time=2222 Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/27, version=0.11.24) Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=12:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=hypervisor_monitor_60000 Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=25, reply=1, notify=0 Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:25 Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/28) Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor' Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.24 2 Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.25 (null) Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=25 Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']: INFINITY | from el8-a01n01 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/10) Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 10 with 2 changes for fail-count-virsh_node2#start_0 (id n/a, set n/a) Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting last-failure-virsh_node2#start_0[el8-a01n02]: (unset) -> 1594762184 | from el8-a01n01 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 11 with 2 changes for last-failure-virsh_node2#start_0 (id n/a, set n/a) Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.36 2 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.37 (null) Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=37 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/10, version=0.11.37) Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 10 result for fail-count-virsh_node2#start_0: OK | rc=0 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * fail-count-virsh_node2#start_0[el8-a01n01]=INFINITY Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * fail-count-virsh_node2#start_0[el8-a01n02]=INFINITY Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/11) Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.37 2 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.38 (null) Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=38 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/11, version=0.11.38) Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 11 result for last-failure-virsh_node2#start_0: OK | rc=0 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * last-failure-virsh_node2#start_0[el8-a01n01]=1594762173 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * last-failure-virsh_node2#start_0[el8-a01n02]=1594762184 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=3:6:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=virsh_node2_stop_0 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=27, reply=1, notify=0 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) info: executing - rsc:virsh_node2 action:stop call_id:27 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/34) Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) info: finished - rsc:virsh_node2 action:stop call_id:27 exit-code:0 exec-time:1ms queue-time:0ms Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of stop operation for virsh_node2 on el8-a01n02: 0 (ok) | call=27 key=virsh_node2_stop_0 confirmed=true cib-update=35 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.38 2 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.39 (null) Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=39 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=3:6:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;3:6:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762184, @last-run=1594762184, @exec-time=0 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/34, version=0.11.39) Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/35) Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.39 2 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.40 (null) Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=40 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;3:6:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=27, @rc-code=0, @op-status=0, @exec-time=1 Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/35, version=0.11.40) Jul 14 21:29:49 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_ping) info: Reporting our current digest to el8-a01n01: 8251df6e54b59aa0e832ca3aef06d2cb for 0.11.40 (0x557965cbb5b0 0) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemakerd [4564] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk_shutdown_worker) notice: Shutting down Pacemaker Jul 14 21:30:28 el8-a01n02.digimer.ca pacemakerd [4564] (stop_child) notice: Stopping pacemaker-controld | sent signal 15 to process 4570 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_shutdown) notice: Shutting down cluster resource manager | limit=1200000ms Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_shutdown_req) info: Sending shutdown request to all peers (DC is el8-a01n01) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_update_peer_expected) info: handle_request: Node el8-a01n02[2] - expected state is now down (was (null)) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting shutdown[el8-a01n02]: (unset) -> 1594762228 | from el8-a01n01 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 12 with 1 change for shutdown (id n/a, set n/a) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/12) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.40 2 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.41 (null) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=41 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/12, version=0.11.41) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 12 result for shutdown: OK | rc=0 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * shutdown[el8-a01n02]=1594762228 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (cancel_recurring_action) info: Cancelling systemd operation hypervisor_status_60000 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:25 exit-code:0 exec-time:0ms queue-time:0ms Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=7:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=hypervisor_stop_0 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/36) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=29, reply=1, notify=0 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) info: executing - rsc:hypervisor action:stop call_id:29 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit libvirtd named 'hypervisor' Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) info: Result of monitor operation for hypervisor on el8-a01n02: Cancelled | call=25 key=hypervisor_monitor_60000 confirmed=true Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.41 2 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.42 (null) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=42 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=7:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;7:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762228, @last-run=1594762228, @exec-time=0 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (cancel_recurring_action) info: Cancelling systemd operation drbd_status_60000 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:drbd action:monitor call_id:26 exit-code:0 exec-time:0ms queue-time:0ms Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=14:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=drbd_stop_0 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/36, version=0.11.42) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/37) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=31, reply=1, notify=0 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) info: executing - rsc:drbd action:stop call_id:31 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit drbd named 'drbd' Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) info: Result of monitor operation for drbd on el8-a01n02: Cancelled | call=26 key=drbd_monitor_60000 confirmed=true Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.42 2 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.43 (null) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=43 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_stop_0, @operation=stop, @transition-key=14:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;14:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762228, @last-run=1594762228, @exec-time=0, @queue-time=0 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/37, version=0.11.43) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec_with_unit) debug: Calling StopUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec_with_unit) debug: Calling StopUnit for drbd: /org/freedesktop/systemd1/unit/drbd_2eservice Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_update_peer_expected) info: handle_request: Node el8-a01n01[1] - expected state is now down (was member) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting shutdown[el8-a01n01]: (unset) -> 1594762228 | from el8-a01n01 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 13 with 2 changes for shutdown (id n/a, set n/a) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/13) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.43 2 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.44 (null) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=44 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']: Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/13, version=0.11.44) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 13 result for shutdown: OK | rc=0 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * shutdown[el8-a01n01]=1594762228 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * shutdown[el8-a01n02]=1594762228 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/4253 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: hypervisor stop may still be in progress: re-scheduling (elapsed=220ms, remaining=99780ms, start_delay=2000ms) Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/4254 Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=380ms, remaining=99620ms, start_delay=2000ms) Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:29 Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor' Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: hypervisor stop is now complete (elapsed=2224ms, remaining=97776ms): not running (7) Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:29 exit-code:0 exec-time:2224ms queue-time:0ms Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of stop operation for hypervisor on el8-a01n02: 0 (ok) | call=29 key=hypervisor_stop_0 confirmed=true cib-update=38 Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/38) Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.44 2 Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.45 (null) Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=45 Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;7:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=29, @rc-code=0, @op-status=0, @exec-time=2224 Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/38, version=0.11.45) Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=2385ms, remaining=97615ms, start_delay=2000ms) Jul 14 21:30:32 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:30:32 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:30:32 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:30:32 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=4387ms, remaining=95613ms, start_delay=2000ms) Jul 14 21:30:34 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:30:34 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:30:34 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:30:34 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=6391ms, remaining=93609ms, start_delay=2000ms) Jul 14 21:30:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_ping) info: Reporting our current digest to el8-a01n01: afb079b332593e47dff6eff041966396 for 0.11.45 (0x557965cbb5b0 0) Jul 14 21:30:36 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:30:36 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:30:36 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:30:36 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=8394ms, remaining=91606ms, start_delay=2000ms) Jul 14 21:30:38 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:30:38 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:30:38 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:30:38 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=10397ms, remaining=89603ms, start_delay=2000ms) Jul 14 21:30:40 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:30:40 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:30:40 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:30:40 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=12401ms, remaining=87599ms, start_delay=2000ms) Jul 14 21:30:42 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:30:42 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:30:42 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:30:42 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=14405ms, remaining=85595ms, start_delay=2000ms) Jul 14 21:30:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:30:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:30:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:30:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=16412ms, remaining=83588ms, start_delay=2000ms) Jul 14 21:30:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:30:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:30:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:30:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=18417ms, remaining=81583ms, start_delay=2000ms) Jul 14 21:30:48 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:30:48 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:30:48 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:30:48 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=20420ms, remaining=79580ms, start_delay=2000ms) Jul 14 21:30:50 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:30:50 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:30:50 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:30:50 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=22423ms, remaining=77577ms, start_delay=2000ms) Jul 14 21:30:52 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:30:52 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:30:52 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:30:52 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=24430ms, remaining=75570ms, start_delay=2000ms) Jul 14 21:30:54 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:30:54 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:30:54 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:30:54 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=26435ms, remaining=73565ms, start_delay=2000ms) Jul 14 21:30:56 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:30:56 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:30:56 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:30:56 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=28442ms, remaining=71558ms, start_delay=2000ms) Jul 14 21:30:58 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:30:58 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:30:58 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:30:58 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=30445ms, remaining=69555ms, start_delay=2000ms) Jul 14 21:31:00 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:00 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:00 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:00 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=32449ms, remaining=67551ms, start_delay=2000ms) Jul 14 21:31:02 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:02 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:02 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:02 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=34452ms, remaining=65548ms, start_delay=2000ms) Jul 14 21:31:04 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:04 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:04 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:04 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=36455ms, remaining=63545ms, start_delay=2000ms) Jul 14 21:31:06 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:06 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:06 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:06 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=38459ms, remaining=61541ms, start_delay=2000ms) Jul 14 21:31:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=40462ms, remaining=59538ms, start_delay=2000ms) Jul 14 21:31:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=42469ms, remaining=57531ms, start_delay=2000ms) Jul 14 21:31:12 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:12 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:12 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:12 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=44472ms, remaining=55528ms, start_delay=2000ms) Jul 14 21:31:14 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:14 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:14 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:14 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=46475ms, remaining=53525ms, start_delay=2000ms) Jul 14 21:31:16 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:16 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:16 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:16 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=48481ms, remaining=51519ms, start_delay=2000ms) Jul 14 21:31:18 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:18 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:18 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:18 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=50485ms, remaining=49515ms, start_delay=2000ms) Jul 14 21:31:20 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:20 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:20 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:20 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=52488ms, remaining=47512ms, start_delay=2000ms) Jul 14 21:31:22 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:22 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:22 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:22 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=54492ms, remaining=45508ms, start_delay=2000ms) Jul 14 21:31:24 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:24 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:24 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:24 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=56495ms, remaining=43505ms, start_delay=2000ms) Jul 14 21:31:26 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:26 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:26 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:26 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=58502ms, remaining=41498ms, start_delay=2000ms) Jul 14 21:31:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=60507ms, remaining=39493ms, start_delay=2000ms) Jul 14 21:31:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=62511ms, remaining=37489ms, start_delay=2000ms) Jul 14 21:31:32 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:32 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:32 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:32 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=64516ms, remaining=35484ms, start_delay=2000ms) Jul 14 21:31:34 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:34 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:34 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:34 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=66519ms, remaining=33481ms, start_delay=2000ms) Jul 14 21:31:36 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:36 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:36 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:36 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=68522ms, remaining=31478ms, start_delay=2000ms) Jul 14 21:31:38 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:38 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:38 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:38 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=70526ms, remaining=29474ms, start_delay=2000ms) Jul 14 21:31:40 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:40 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:40 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:40 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=72529ms, remaining=27471ms, start_delay=2000ms) Jul 14 21:31:42 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:42 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:42 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:42 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=74532ms, remaining=25468ms, start_delay=2000ms) Jul 14 21:31:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=76536ms, remaining=23464ms, start_delay=2000ms) Jul 14 21:31:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=78541ms, remaining=21459ms, start_delay=2000ms) Jul 14 21:31:48 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:48 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:48 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:48 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=80545ms, remaining=19455ms, start_delay=2000ms) Jul 14 21:31:50 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:50 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:50 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:50 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=82549ms, remaining=17451ms, start_delay=2000ms) Jul 14 21:31:52 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:52 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:52 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:52 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=84553ms, remaining=15447ms, start_delay=2000ms) Jul 14 21:31:54 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:54 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:54 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:54 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=86561ms, remaining=13439ms, start_delay=2000ms) Jul 14 21:31:56 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:56 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:56 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:56 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=88565ms, remaining=11435ms, start_delay=2000ms) Jul 14 21:31:58 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:31:58 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:31:58 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:31:58 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=90569ms, remaining=9431ms, start_delay=2000ms) Jul 14 21:32:00 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:32:00 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:32:00 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:32:00 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=92571ms, remaining=7429ms, start_delay=2000ms) Jul 14 21:32:02 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:32:02 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:32:02 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:32:02 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=94574ms, remaining=5426ms, start_delay=2000ms) Jul 14 21:32:04 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:32:04 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:32:04 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:32:04 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=96578ms, remaining=3422ms, start_delay=1711ms) Jul 14 21:32:06 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:32:06 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:32:06 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:32:06 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=98293ms, remaining=1707ms, start_delay=853ms) Jul 14 21:32:07 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:32:07 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:32:07 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:32:07 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=99149ms, remaining=851ms, start_delay=425ms) Jul 14 21:32:07 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:32:07 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:32:07 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:32:07 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=99580ms, remaining=420ms, start_delay=210ms) Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=99793ms, remaining=207ms, start_delay=103ms) Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=99901ms, remaining=99ms, start_delay=49ms) Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=99953ms, remaining=47ms, start_delay=23ms) Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=99979ms, remaining=21ms, start_delay=10ms) Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd' Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) notice: Giving up on drbd stop (rc=0): timeout (elapsed=99991ms, remaining=9ms) Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:drbd action:monitor call_id:31 exit-code:198 exec-time:99991ms queue-time:0ms Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) error: Result of stop operation for drbd on el8-a01n02: Timed Out | call=31 key=drbd_stop_0 timeout=100000ms Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/39) Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.45 2 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.46 (null) Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=46 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=2:198;14:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=31, @rc-code=198, @op-status=2, @exec-time=99991 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']: Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/14, version=0.11.47) Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 14 result for fail-count-drbd#stop_0: OK | rc=0 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * fail-count-drbd#stop_0[el8-a01n02]=INFINITY Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.47 2 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.48 (null) Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=48 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/15, version=0.11.48) Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 15 result for last-failure-drbd#stop_0: OK | rc=0 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * last-failure-drbd#stop_0[el8-a01n02]=1594762328 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.48 2 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.49 (null) Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=49 Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=5:9:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;5:9:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762328, @last-run=1594762328, @exec-time=0, @queu Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/72, version=0.11.49) Jul 14 21:32:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.49 2 Jul 14 21:32:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.50 (null) Jul 14 21:32:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=50 Jul 14 21:32:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;5:9:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=29, @rc-code=0, @op-status=0, @exec-time=2271 Jul 14 21:32:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/73, version=0.11.50) Jul 14 21:32:15 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_ping) info: Reporting our current digest to el8-a01n01: 55204ae50dead0b498c548c5d466607c for 0.11.50 (0x557965cbb5b0 0) Jul 14 21:32:19 el8-a01n02.digimer.ca pacemaker-fenced [4566] (remote_op_done) notice: Operation 'reboot' targeting el8-a01n02 on for pacemaker-controld.375414@el8-a01n01.2bbcc655: No route to host Jul 14 21:32:19 el8-a01n02.digimer.ca pacemaker-controld [4570] (tengine_stonith_notify) notice: Peer el8-a01n02 was not terminated (reboot) by on behalf of pacemaker-controld.375414: No route to host | initiator=el8-a01n01 ref=2bbcc655-c155-4908-a792-c918b427a38e Jul 14 21:32:31 el8-a01n02.digimer.ca pacemaker-fenced [4566] (remote_op_done) notice: Operation 'reboot' targeting el8-a01n02 on for pacemaker-controld.375414@el8-a01n01.75793d25: No route to host Jul 14 21:32:31 el8-a01n02.digimer.ca pacemaker-controld [4570] (tengine_stonith_notify) notice: Peer el8-a01n02 was not terminated (reboot) by on behalf of pacemaker-controld.375414: No route to host | initiator=el8-a01n01 ref=75793d25-0284-4916-823f-ebadcf09771b Jul 14 21:32:42 el8-a01n02.digimer.ca pacemaker-fenced [4566] (remote_op_done) notice: Operation 'reboot' targeting el8-a01n02 on for pacemaker-controld.375414@el8-a01n01.5be1f6ba: No route to host Jul 14 21:32:42 el8-a01n02.digimer.ca pacemaker-controld [4570] (tengine_stonith_notify) notice: Peer el8-a01n02 was not terminated (reboot) by on behalf of pacemaker-controld.375414: No route to host | initiator=el8-a01n01 ref=5be1f6ba-66c0-4d35-920f-a083bf59809c Jul 14 21:32:54 el8-a01n02.digimer.ca pacemaker-fenced [4566] (remote_op_done) notice: Operation 'reboot' targeting el8-a01n02 on for pacemaker-controld.375414@el8-a01n01.9fc8382b: No route to host Jul 14 21:32:54 el8-a01n02.digimer.ca pacemaker-controld [4570] (tengine_stonith_notify) notice: Peer el8-a01n02 was not terminated (reboot) by on behalf of pacemaker-controld.375414: No route to host | initiator=el8-a01n01 ref=9fc8382b-26df-4e45-8f59-affe54245144 Jul 14 21:33:05 el8-a01n02.digimer.ca pacemaker-fenced [4566] (remote_op_done) notice: Operation 'reboot' targeting el8-a01n02 on for pacemaker-controld.375414@el8-a01n01.d3fc9798: No route to host Jul 14 21:33:05 el8-a01n02.digimer.ca pacemaker-controld [4570] (tengine_stonith_notify) notice: Peer el8-a01n02 was not terminated (reboot) by on behalf of pacemaker-controld.375414: No route to host | initiator=el8-a01n01 ref=d3fc9798-6df8-485b-a7d3-e904d314cf81 Jul 14 21:33:17 el8-a01n02.digimer.ca pacemaker-fenced [4566] (remote_op_done) notice: Operation 'reboot' targeting el8-a01n02 on for pacemaker-controld.375414@el8-a01n01.3e12455e: No route to host Jul 14 21:33:17 el8-a01n02.digimer.ca pacemaker-controld [4570] (tengine_stonith_notify) notice: Peer el8-a01n02 was not terminated (reboot) by on behalf of pacemaker-controld.375414: No route to host | initiator=el8-a01n01 ref=3e12455e-82d1-4460-90db-9dcb37d6de0e