API tools faq
paste
digimer

Untitled

digimer
Jul 14th, 2020
153
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 216.30 KB | None | 0 0
raw download clone embed print report
  1. [root@el8-a01n02 ~]# cat /var/log/pacemaker/pacemaker.log
  2. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_log_init) info: Cannot change active directory to /var/lib/pacemaker/cores: No such file or directory (2)
  3. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (get_cluster_type) info: Detected an active 'corosync' cluster
  4. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (mcp_read_config) info: Reading configure for stack: corosync
  5. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (main) notice: Starting Pacemaker 2.0.3-5.el8_2.1 | build=4b1f869f0f features: generated-manpages agent-manpages ncurses libqb-logging libqb-ipc systemd nagios corosync-native atomic-attrd acls
  6. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (main) info: Maximum core file size is: 18446744073709551615
  7. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (qb_ipcs_us_publish) info: server name: pacemakerd
  8. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to lrmd IPC: Connection refused
  9. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to cib_ro IPC: Connection refused
  10. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to crmd IPC: Connection refused
  11. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to attrd IPC: Connection refused
  12. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to pengine IPC: Connection refused
  13. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to stonith-ng IPC: Connection refused
  14. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_get_peer) info: Created entry 9bbf4396-4299-4d59-82ee-27fa8b85610b/0x55d304462f90 for node el8-a01n02/2 (1 total)
  15. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  16. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_get_peer) info: Node 2 has uuid 2
  17. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online
  18. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (cluster_connect_quorum) warning: Quorum lost
  19. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_get_peer) info: Created entry 146915c5-e9a4-400e-a4d5-0c8bb9fade2a/0x55d3046648b0 for node el8-a01n01/1 (2 total)
  20. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  21. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_get_peer) info: Node 1 has uuid 1
  22. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Using uid=189 and group=189 for process pacemaker-based
  23. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Forked child 4565 for process pacemaker-based
  24. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Forked child 4566 for process pacemaker-fenced
  25. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Forked child 4567 for process pacemaker-execd
  26. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Using uid=189 and group=189 for process pacemaker-attrd
  27. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Forked child 4568 for process pacemaker-attrd
  28. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Using uid=189 and group=189 for process pacemaker-schedulerd
  29. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Forked child 4569 for process pacemaker-schedulerd
  30. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Using uid=189 and group=189 for process pacemaker-controld
  31. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (start_child) info: Forked child 4570 for process pacemaker-controld
  32. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  33. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-attrd [4568] (main) notice: Starting Pacemaker node attribute manager
  34. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (main) notice: Pacemaker daemon successfully started and accepting connections
  35. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk_quorum_notification) info: Quorum still lost | membership=140 members=1
  36. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification
  37. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (crm_reap_unseen_nodes) info: State of node el8-a01n01[1] is still unknown
  38. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk_cpg_membership) info: Group pacemakerd event 0: node 2 pid 4564 joined via cpg_join
  39. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk_cpg_membership) info: Group pacemakerd event 0: el8-a01n02 (node 2 pid 4564) is member
  40. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  41. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (main) notice: Starting Pacemaker local executor
  42. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (qb_ipcs_us_publish) info: server name: lrmd
  43. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (main) notice: Pacemaker local executor successfully started and accepting connections
  44. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  45. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (main) notice: Starting Pacemaker CIB manager
  46. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (get_cluster_type) info: Verifying cluster type: 'corosync'
  47. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (get_cluster_type) info: Assuming an active 'corosync' cluster
  48. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (pcmk__daemon_can_write) notice: /var/lib/pacemaker/cib/cib.xml not found: No such file or directory
  49. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (pcmk__daemon_can_write) notice: /var/lib/pacemaker/cib/cib.xml.sig not found: No such file or directory
  50. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (retrieveCib) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.xml (digest: /var/lib/pacemaker/cib/cib.xml.sig)
  51. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_read_and_verify) warning: Could not verify cluster configuration file /var/lib/pacemaker/cib/cib.xml: No such file or directory (2)
  52. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (readCibXmlFile) warning: Primary configuration corrupt or unusable, trying backups in /var/lib/pacemaker/cib
  53. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (readCibXmlFile) warning: Continuing with an empty configuration.
  54. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-schedulerd[4569] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  55. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-schedulerd[4569] (main) notice: Starting Pacemaker scheduler
  56. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-schedulerd[4569] (qb_ipcs_us_publish) info: server name: pengine
  57. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-schedulerd[4569] (main) notice: Pacemaker scheduler successfully started and accepting connections
  58. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  59. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (main) notice: Starting Pacemaker fencer
  60. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (get_cluster_type) info: Verifying cluster type: 'corosync'
  61. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (get_cluster_type) info: Assuming an active 'corosync' cluster
  62. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  63. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  64. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_get_peer) info: Created entry 54e0653e-2879-48de-b644-14da24f28319/0x55a6ed64dd60 for node el8-a01n02/2 (1 total)
  65. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  66. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  67. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-controld [4570] (main) notice: Starting Pacemaker controller
  68. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_log) info: Input I_STARTUP received in state S_STARTING from crmd_init
  69. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_get_peer) info: Node 2 has uuid 2
  70. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online
  71. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  72. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-fenced [4566] (init_cs_connection_once) info: Connection to 'corosync': established
  73. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_get_peer) info: Created entry 51324133-b4ae-420b-a11b-c4739181e652/0x557965bd37c0 for node el8-a01n02/2 (1 total)
  74. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  75. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_get_peer) info: Node 2 has uuid 2
  76. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online
  77. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  78. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (init_cs_connection_once) info: Connection to 'corosync': established
  79. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (qb_ipcs_us_publish) info: server name: cib_ro
  80. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (qb_ipcs_us_publish) info: server name: cib_rw
  81. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (qb_ipcs_us_publish) info: server name: cib_shm
  82. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (main) notice: Pacemaker CIB manager successfully started and accepting connections
  83. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (pcmk_cpg_membership) info: Group cib event 0: node 2 pid 4565 joined via cpg_join
  84. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (pcmk_cpg_membership) info: Group cib event 0: el8-a01n02 (node 2 pid 4565) is member
  85. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_read_and_verify) warning: Could not verify cluster configuration file /var/lib/pacemaker/cib/cib.xml: No such file or directory (2)
  86. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-0.raw
  87. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.0.0 of the CIB to disk (digest: 2469c254d4b2872e864ff6b9c730ac0c)
  88. Jul 14 21:28:46 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.zYQYpv (digest: /var/lib/pacemaker/cib/cib.VT7O7Z)
  89. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (main) info: CIB connection active
  90. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (get_cluster_type) info: Verifying cluster type: 'corosync'
  91. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (get_cluster_type) info: Assuming an active 'corosync' cluster
  92. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  93. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_get_peer) info: Created entry 0ad36a2d-dd0d-4180-824d-83e6b89fee3e/0x55bd38224a20 for node el8-a01n02/2 (1 total)
  94. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  95. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_get_peer) info: Node 2 has uuid 2
  96. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online
  97. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  98. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (init_cs_connection_once) info: Connection to 'corosync': established
  99. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (main) info: Cluster connection active
  100. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_erase_attrs) info: Clearing transient attributes from CIB | xpath=//node_state[@uname='el8-a01n02']/transient_attributes
  101. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk_quorum_notification) notice: Quorum acquired | membership=144 members=2
  102. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemakerd [4564] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification
  103. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk_cpg_membership) info: Group pacemakerd event 1: node 1 pid 375401 joined via cluster join
  104. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n01 (node 1 pid 375401) is member
  105. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemakerd [4564] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online
  106. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n02 (node 2 pid 4564) is member
  107. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (pcmk_cpg_membership) info: Group cib event 1: node 1 pid 375409 joined via cluster join
  108. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (qb_ipcs_us_publish) info: server name: attrd
  109. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (main) notice: Pacemaker node attribute manager successfully started and accepting connections
  110. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (pcmk_cpg_membership) info: Group attrd event 0: node 2 pid 4568 joined via cpg_join
  111. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (pcmk_cpg_membership) info: Group attrd event 0: el8-a01n02 (node 2 pid 4568) is member
  112. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n02]: (unset) -> 2 | from el8-a01n02
  113. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_start_election_if_needed) info: Starting an election to determine the writer
  114. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (election_check) info: election-attrd won by local node
  115. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_declare_winner) notice: Recorded local node as attribute writer (was unset)
  116. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Processed 1 private change for #attrd-protocol, id=n/a, set=n/a
  117. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_get_peer) info: Created entry 84fb1d58-897c-4c44-8d51-5e4a37114af8/0x557965bdac50 for node el8-a01n01/1 (2 total)
  118. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  119. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_get_peer) info: Node 1 has uuid 1
  120. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n01 (node 1 pid 375409) is member
  121. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online
  122. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  123. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n02 (node 2 pid 4565) is member
  124. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n02']/transient_attributes to all (origin=local/attrd/2)
  125. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/transient_attributes: OK (rc=0, origin=el8-a01n02/attrd/2, version=0.0.0)
  126. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (get_cluster_type) info: Verifying cluster type: 'corosync'
  127. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (get_cluster_type) info: Assuming an active 'corosync' cluster
  128. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  129. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_get_peer) info: Created entry 8b024f81-724f-4922-992e-509d1eede034/0x5615c858fe20 for node el8-a01n02/2 (1 total)
  130. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  131. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (peer_update_callback) info: Cluster node el8-a01n02 is now in unknown state
  132. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_get_peer) info: Node 2 has uuid 2
  133. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online
  134. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (peer_update_callback) info: Node el8-a01n02 is now a peer | DC=<none> old=0x0000000 new=0x4000000
  135. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds
  136. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (init_cs_connection_once) info: Connection to 'corosync': established
  137. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (cluster_connect_quorum) notice: Quorum acquired
  138. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (setup_cib) info: Watching for stonith topology changes
  139. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (qb_ipcs_us_publish) info: server name: stonith-ng
  140. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (main) notice: Pacemaker fencer successfully started and accepting connections
  141. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (init_cib_cache_cb) info: Updating device list from the cib: init
  142. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (cib_devices_update) info: Updating devices to version 0.0.0
  143. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (unpack_nodes) info: Creating a fake local node
  144. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (pcmk_cpg_membership) info: Group stonith-ng event 0: node 2 pid 4566 joined via cpg_join
  145. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (pcmk_cpg_membership) info: Group stonith-ng event 0: el8-a01n02 (node 2 pid 4566) is member
  146. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (pcmk_cpg_membership) info: Group stonith-ng event 1: node 1 pid 375410 joined via cluster join
  147. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_get_peer) info: Created entry 7a289904-7d3d-4548-8266-a2d86be2b071/0x5615c8592700 for node el8-a01n01/1 (2 total)
  148. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  149. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (peer_update_callback) info: Cluster node el8-a01n01 is now in unknown state
  150. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_get_peer) info: Node 1 has uuid 1
  151. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (pcmk_cpg_membership) info: Group attrd event 1: node 1 pid 375412 joined via cpg_join
  152. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_get_peer) info: Created entry 3c30217b-b590-46e5-8bfb-97b297c96171/0x55a6ed7596e0 for node el8-a01n01/1 (2 total)
  153. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  154. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_get_peer) info: Node 1 has uuid 1
  155. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n01 (node 1 pid 375410) is member
  156. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online
  157. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  158. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-fenced [4566] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n02 (node 2 pid 4566) is member
  159. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_get_peer) info: Created entry 5eb72a08-a13c-41f2-8ded-452f6a55d402/0x55bd382276d0 for node el8-a01n01/1 (2 total)
  160. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  161. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_get_peer) info: Node 1 has uuid 1
  162. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (pcmk_cpg_membership) info: Group attrd event 1: el8-a01n01 (node 1 pid 375412) is member
  163. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online
  164. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  165. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (pcmk_cpg_membership) info: Group attrd event 1: el8-a01n02 (node 2 pid 4568) is member
  166. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_ha_control) info: Connected to the cluster
  167. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (lrmd_ipc_connect) info: Connecting to executor
  168. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-execd [4567] (crm_client_new) debug: Connecting 0x5567f4193fb0 for uid=189 gid=189 pid=4570 id=2eb86a7f-4b61-4d54-8801-a8f9e7673b7c
  169. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-execd [4567] (handle_new_connection) debug: IPC credentials authenticated (/dev/shm/qb-4567-4570-9-3C903B/qb)
  170. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-execd [4567] (qb_ipcs_shm_connect) debug: connecting to client [4570]
  171. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-execd [4567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  172. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-execd [4567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  173. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-execd [4567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  174. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/4)
  175. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed register operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0
  176. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.0.0 2
  177. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.1.0 (null)
  178. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=1
  179. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/nodes: <node id="1" uname="el8-a01n01"/>
  180. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/nodes: <node id="2" uname="el8-a01n02"/>
  181. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/4, version=0.1.0)
  182. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_control) info: Connection to the executor established
  183. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_started) info: Delaying start, no membership data (0000000000100000)
  184. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_started) info: Delaying start, no membership data (0000000000100000)
  185. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (pcmk_quorum_notification) info: Quorum retained | membership=144 members=2
  186. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification
  187. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (peer_update_callback) info: Cluster node el8-a01n01 is now member (was in unknown state)
  188. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification
  189. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (peer_update_callback) info: Cluster node el8-a01n02 is now member (was in unknown state)
  190. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n01]: (unset) -> 2 | from el8-a01n01
  191. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Processed 2 private changes for #attrd-protocol, id=n/a, set=n/a
  192. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_started) info: Delaying start, Config not read (0000000000000040)
  193. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (qb_ipcs_us_publish) info: server name: crmd
  194. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_started) notice: Pacemaker controller successfully started and accepting connections
  195. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_log) info: Input I_PENDING received in state S_STARTING from do_started
  196. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_state_transition) notice: State transition S_STARTING -> S_PENDING | input=I_PENDING cause=C_FSA_INTERNAL origin=do_started
  197. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-1.raw
  198. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.1.0 of the CIB to disk (digest: 7aefab66c16403ca8289ab3d2d263d82)
  199. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/transient_attributes: OK (rc=0, origin=el8-a01n01/attrd/2, version=0.1.0)
  200. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.miPs12 (digest: /var/lib/pacemaker/cib/cib.PYE6LA)
  201. Jul 14 21:28:47 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/4, version=0.1.0)
  202. Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (pcmk_cpg_membership) info: Group crmd event 0: node 2 pid 4570 joined via cpg_join
  203. Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (pcmk_cpg_membership) info: Group crmd event 0: el8-a01n02 (node 2 pid 4570) is member
  204. Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (pcmk_cpg_membership) info: Group crmd event 1: node 1 pid 375414 joined via cpg_join
  205. Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (pcmk_cpg_membership) info: Group crmd event 1: el8-a01n01 (node 1 pid 375414) is member
  206. Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online
  207. Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (peer_update_callback) info: Node el8-a01n01 is now a peer | DC=<none> old=0x0000000 new=0x4000000
  208. Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds
  209. Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (pcmk_cpg_membership) info: Group crmd event 1: el8-a01n02 (node 2 pid 4570) is member
  210. Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 30 seconds
  211. Jul 14 21:28:48 el8-a01n02.digimer.ca pacemaker-controld [4570] (te_connect_stonith) notice: Fencer successfully connected
  212. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.1.0 2
  213. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.2.0 (null)
  214. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=2
  215. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/resources: <primitive class="stonith" id="virsh_node1" type="fence_virsh"/>
  216. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <instance_attributes id="virsh_node1-instance_attributes">
  217. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="virsh_node1-instance_attributes-delay" name="delay" value="15"/>
  218. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="virsh_node1-instance_attributes-ipaddr" name="ipaddr" value="10.20.255.250"/>
  219. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="virsh_node1-instance_attributes-login" name="login" value="root"/>
  220. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="virsh_node1-instance_attributes-password_script" name="password_script" value="/root/stonith.pw"/>
  221. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="virsh_node1-instance_attributes-pcmk_host_list" name="pcmk_host_list" value="el8-a01n01"/>
  222. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="virsh_node1-instance_attributes-port" name="port" value="el8-a01n01"/>
  223. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </instance_attributes>
  224. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <operations>
  225. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <op id="virsh_node1-monitor-interval-60" interval="60" name="monitor"/>
  226. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </operations>
  227. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </primitive>
  228. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.2.0)
  229. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-fenced [4566] (update_cib_stonith_devices_v2) info: Updating device list from the cib: create resources
  230. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-fenced [4566] (cib_devices_update) info: Updating devices to version 0.2.0
  231. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-2.raw
  232. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.2.0 of the CIB to disk (digest: 0d075a040dee76aba3d9bb362ddfe793)
  233. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.F1NKGV (digest: /var/lib/pacemaker/cib/cib.7jVe6l)
  234. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-fenced [4566] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 4580 exited with rc=0
  235. Jul 14 21:29:06 el8-a01n02.digimer.ca pacemaker-fenced [4566] (stonith_device_register) notice: Added 'virsh_node1' to the device list (1 active devices)
  236. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_timer_popped) info: Election Trigger just popped | input=I_DC_TIMEOUT time=20000ms
  237. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_log) warning: Input I_DC_TIMEOUT received in state S_PENDING from crm_timer_popped
  238. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_state_transition) info: State transition S_PENDING -> S_ELECTION | input=I_DC_TIMEOUT cause=C_TIMER_POPPED origin=crm_timer_popped
  239. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (election_count_vote) info: election-DC round 1 (owner node ID 1) lost: vote from el8-a01n01 (Uptime)
  240. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_state_transition) notice: State transition S_ELECTION -> S_PENDING | input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote
  241. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_dc_release) info: DC role released
  242. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_te_control) info: Transitioner is now inactive
  243. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_log) info: Input I_RELEASE_SUCCESS received in state S_PENDING from do_dc_release
  244. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n01/crmd/8, version=0.2.0)
  245. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.2.0 2
  246. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.3.0 (null)
  247. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=3
  248. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/crm_config: <cluster_property_set id="cib-bootstrap-options"/>
  249. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="cib-bootstrap-options-have-watchdog" name="have-watchdog" value="false"/>
  250. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </cluster_property_set>
  251. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n01/crmd/10, version=0.3.0)
  252. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.3.0 2
  253. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.4.0 (null)
  254. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=4
  255. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']: <nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="2.0.3-5.el8_2.1-4b1f869f0f"/>
  256. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n01/crmd/12, version=0.4.0)
  257. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-3.raw
  258. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.4.0 2
  259. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.5.0 (null)
  260. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=5
  261. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']: <nvpair id="cib-bootstrap-options-cluster-infrastructure" name="cluster-infrastructure" value="corosync"/>
  262. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n01/crmd/14, version=0.5.0)
  263. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.3.0 of the CIB to disk (digest: be8b332ecdd2a858160b9164e32749f2)
  264. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.Ee13h3 (digest: /var/lib/pacemaker/cib/cib.S8UXiB)
  265. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-4.raw
  266. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (update_dc) info: Set DC to el8-a01n01 (3.3.0)
  267. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_update_peer_expected) info: update_dc: Node el8-a01n01[1] - expected state is now member (was (null))
  268. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.5.0 2
  269. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.0 (null)
  270. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=6
  271. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']: <nvpair id="cib-bootstrap-options-cluster-name" name="cluster-name" value="el8-anvil-01"/>
  272. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n01/crmd/16, version=0.6.0)
  273. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_replace) info: Digest matched on replace from el8-a01n01: dc9656c0a868cfbd5503dbb3659ae0b0
  274. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_replace) info: Replaced 0.6.0 with 0.6.0 from el8-a01n01
  275. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=el8-a01n01/crmd/21, version=0.6.0)
  276. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_log) info: Input I_NOT_DC received in state S_PENDING from do_cl_join_finalize_respond
  277. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_state_transition) notice: State transition S_PENDING -> S_NOT_DC | input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond
  278. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.5.0 of the CIB to disk (digest: 2efada062c6ed14e51d7c9f2521b85a4)
  279. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.8qILj3 (digest: /var/lib/pacemaker/cib/cib.iTmXnB)
  280. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/22, version=0.6.0)
  281. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/23, version=0.6.0)
  282. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/lrm: OK (rc=0, origin=el8-a01n01/crmd/24, version=0.6.0)
  283. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.0 2
  284. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.1 (null)
  285. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=1
  286. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status: <node_state id="1" uname="el8-a01n01" in_ccm="true" crmd="online" crm-debug-origin="do_lrm_query_internal"/>
  287. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <lrm id="1">
  288. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <lrm_resources/>
  289. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </lrm>
  290. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </node_state>
  291. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/25, version=0.6.1)
  292. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/lrm: OK (rc=0, origin=el8-a01n01/crmd/26, version=0.6.1)
  293. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-5.raw
  294. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.6.0 of the CIB to disk (digest: 26a4a88a0d55ef6a920ee0c95c0ed3d1)
  295. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.1 2
  296. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.2 (null)
  297. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=2
  298. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status: <node_state id="2" uname="el8-a01n02" in_ccm="true" crmd="online" crm-debug-origin="do_lrm_query_internal"/>
  299. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <lrm id="2">
  300. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <lrm_resources/>
  301. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </lrm>
  302. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </node_state>
  303. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.TqtDn3 (digest: /var/lib/pacemaker/cib/cib.bgZ0tB)
  304. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/27, version=0.6.2)
  305. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/30, version=0.6.2)
  306. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.2 2
  307. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.3 (null)
  308. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=3
  309. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition, @join=member, @expected=member
  310. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition, @join=member, @expected=member
  311. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/31, version=0.6.3)
  312. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.3 2
  313. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.4 (null)
  314. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=4, @have-quorum=1, @dc-uuid=1
  315. Jul 14 21:29:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n01/crmd/32, version=0.6.4)
  316. Jul 14 21:29:09 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node1' not in cache
  317. Jul 14 21:29:09 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0
  318. Jul 14 21:29:09 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node1'
  319. Jul 14 21:29:09 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=1
  320. Jul 14 21:29:09 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0
  321. Jul 14 21:29:09 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=2:0:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=virsh_node1_monitor_0
  322. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-controld [4570] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 4585 exited with rc=0
  323. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/12)
  324. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=5, reply=1, notify=0
  325. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:5
  326. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (stonith_api_signon) debug: Attempting fencer connection by pacemaker-execd with mainloop
  327. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.4 2
  328. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.5 (null)
  329. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=5
  330. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_update_resource
  331. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="virsh_node1" type="fence_virsh" class="stonith"/>
  332. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node1_last_0" operation_key="virsh_node1_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="2:0:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" transition-magic="-1:193;2:0:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-statu
  333. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </lrm_resource>
  334. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  335. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  336. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  337. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/12, version=0.6.5)
  338. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (stonith_api_signon) debug: Connection to fencer by pacemaker-execd succeeded (registration token: a13c92b4-20bd-4071-8612-2cc435d107f4)
  339. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:5 exit-code:7 exec-time:12ms queue-time:0ms
  340. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of probe operation for virsh_node1 on el8-a01n02: 7 (not running) | call=5 key=virsh_node1_monitor_0 confirmed=true cib-update=13
  341. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/13)
  342. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.5 2
  343. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.6 (null)
  344. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=6
  345. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;2:0:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=5, @rc-code=7, @op-status=0, @last-rc-change=1594762150, @last-run=1594762150, @exec-time=12
  346. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/13, version=0.6.6)
  347. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.6 2
  348. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.7 (null)
  349. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=7
  350. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource
  351. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="virsh_node1" type="fence_virsh" class="stonith"/>
  352. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node1_last_0" operation_key="virsh_node1_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="1:0:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" transition-magic="-1:193;1:0:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-statu
  353. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </lrm_resource>
  354. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/35, version=0.6.7)
  355. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.7 2
  356. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.8 (null)
  357. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=8
  358. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;1:0:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=5, @rc-code=7, @op-status=0, @last-rc-change=1594762150, @last-run=1594762150, @exec-time=6
  359. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/36, version=0.6.8)
  360. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.8 2
  361. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.6.9 (null)
  362. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=9
  363. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_start_0, @operation=start, @transition-key=3:0:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;3:0:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0
  364. Jul 14 21:29:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/37, version=0.6.9)
  365. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.6.9 2
  366. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.0 (null)
  367. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=7, @num_updates=0
  368. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/resources: <primitive class="stonith" id="virsh_node2" type="fence_virsh"/>
  369. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <instance_attributes id="virsh_node2-instance_attributes">
  370. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="virsh_node2-instance_attributes-ipaddr" name="ipaddr" value="10.20.255.250"/>
  371. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="virsh_node2-instance_attributes-login" name="login" value="root"/>
  372. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="virsh_node2-instance_attributes-password_script" name="password_script" value="/root/stonith.pw"/>
  373. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="virsh_node2-instance_attributes-pcmk_host_list" name="pcmk_host_list" value="el8-a01n02"/>
  374. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="virsh_node2-instance_attributes-port" name="port" value="el8-a01n02"/>
  375. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </instance_attributes>
  376. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <operations>
  377. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <op id="virsh_node2-monitor-interval-60" interval="60" name="monitor"/>
  378. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </operations>
  379. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </primitive>
  380. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.7.0)
  381. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-fenced [4566] (update_cib_stonith_devices_v2) info: Updating device list from the cib: create resources
  382. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-fenced [4566] (cib_devices_update) info: Updating devices to version 0.7.0
  383. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-fenced [4566] (stonith_device_register) notice: Added 'virsh_node2' to the device list (2 active devices)
  384. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-6.raw
  385. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.7.0 of the CIB to disk (digest: ed4816be9cba1c0fb4b22dfa9b7cb0b2)
  386. Jul 14 21:29:13 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.Wlegkg (digest: /var/lib/pacemaker/cib/cib.XVdhn1)
  387. Jul 14 21:29:17 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_procfs_pid_of) info: Found pacemaker-based active as process 4565
  388. Jul 14 21:29:17 el8-a01n02.digimer.ca pacemaker-controld [4570] (throttle_send_command) info: New throttle mode: negligible load (was undetermined)
  389. Jul 14 21:29:18 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_ping) info: Reporting our current digest to el8-a01n01: 38bdea493108431f663dd4ee0010b3ce for 0.7.0 (0x557965c88030 0)
  390. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.0 2
  391. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.1 (null)
  392. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=1
  393. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:1;3:0:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=6, @rc-code=1, @op-status=0, @exec-time=11453, @queue-time=1
  394. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']: <lrm_rsc_op id="virsh_node1_last_failure_0" operation_key="virsh_node1_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="3:0:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" transition-magic="0:1;3:0:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" exit-reason="" on_node="el8-a01n01" call
  395. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/38, version=0.7.1)
  396. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting fail-count-virsh_node1#start_0[el8-a01n01]: (unset) -> INFINITY | from el8-a01n01
  397. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 4 with 1 change for fail-count-virsh_node1#start_0 (id n/a, set n/a)
  398. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting last-failure-virsh_node1#start_0[el8-a01n01]: (unset) -> 1594762161 | from el8-a01n01
  399. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 5 with 1 change for last-failure-virsh_node1#start_0 (id n/a, set n/a)
  400. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/4)
  401. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/5)
  402. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.1 2
  403. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.2 (null)
  404. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=2
  405. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']: <transient_attributes id="1"/>
  406. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <instance_attributes id="status-1">
  407. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="status-1-fail-count-virsh_node1.start_0" name="fail-count-virsh_node1#start_0" value="INFINITY"/>
  408. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </instance_attributes>
  409. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </transient_attributes>
  410. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/4, version=0.7.2)
  411. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 4 result for fail-count-virsh_node1#start_0: OK | rc=0
  412. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * fail-count-virsh_node1#start_0[el8-a01n01]=INFINITY
  413. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.2 2
  414. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.3 (null)
  415. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=3
  416. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']: <nvpair id="status-1-last-failure-virsh_node1.start_0" name="last-failure-virsh_node1#start_0" value="1594762161"/>
  417. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/5, version=0.7.3)
  418. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 5 result for last-failure-virsh_node1#start_0: OK | rc=0
  419. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * last-failure-virsh_node1#start_0[el8-a01n01]=1594762161
  420. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.3 2
  421. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.4 (null)
  422. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=4
  423. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_stop_0, @operation=stop, @transition-key=1:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;1:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762161, @last-run=1594762161, @exec-time=0, @q
  424. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/42, version=0.7.4)
  425. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node2' not in cache
  426. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0
  427. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node2'
  428. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=1
  429. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0
  430. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=3:2:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=virsh_node2_monitor_0
  431. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=9, reply=1, notify=0
  432. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:virsh_node2 action:monitor call_id:9
  433. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:virsh_node2 action:monitor call_id:9 exit-code:7 exec-time:0ms queue-time:0ms
  434. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/14)
  435. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of probe operation for virsh_node2 on el8-a01n02: 7 (not running) | call=9 key=virsh_node2_monitor_0 confirmed=true cib-update=15
  436. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.4 2
  437. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.5 (null)
  438. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=5
  439. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="virsh_node2" type="fence_virsh" class="stonith"/>
  440. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node2_last_0" operation_key="virsh_node2_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="3:2:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" transition-magic="-1:193;3:2:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-statu
  441. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </lrm_resource>
  442. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/14, version=0.7.5)
  443. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/15)
  444. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.5 2
  445. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.6 (null)
  446. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=6
  447. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="virsh_node2" type="fence_virsh" class="stonith"/>
  448. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node2_last_0" operation_key="virsh_node2_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="2:2:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" transition-magic="-1:193;2:2:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-statu
  449. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </lrm_resource>
  450. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/43, version=0.7.6)
  451. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.6 2
  452. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.7 (null)
  453. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=7
  454. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;3:2:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=9, @rc-code=7, @op-status=0
  455. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/15, version=0.7.7)
  456. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.7 2
  457. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.8 (null)
  458. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=8
  459. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:0;1:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=7, @rc-code=0, @op-status=0, @exec-time=1
  460. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/44, version=0.7.8)
  461. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.8 2
  462. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.9 (null)
  463. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=9
  464. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;2:2:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=11, @rc-code=7, @op-status=0
  465. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/45, version=0.7.9)
  466. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=4:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=virsh_node1_start_0
  467. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/16)
  468. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=10, reply=1, notify=0
  469. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) info: executing - rsc:virsh_node1 action:start call_id:10
  470. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.9 2
  471. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.10 (null)
  472. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=10
  473. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_start_0, @operation=start, @transition-key=4:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;4:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762161, @last-run=1594762161, @exec-time=0
  474. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/16, version=0.7.10)
  475. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.10 2
  476. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.7.11 (null)
  477. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=11
  478. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=6:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;6:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1
  479. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/46, version=0.7.11)
  480. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.7.11 2
  481. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.8.0 (null)
  482. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=8, @num_updates=0
  483. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/resources: <primitive class="systemd" id="hypervisor" type="libvirtd"/>
  484. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <operations>
  485. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <op id="hypervisor-monitor-interval-60" interval="60" name="monitor"/>
  486. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <op id="hypervisor-start-interval-0s" interval="0s" name="start" timeout="100"/>
  487. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <op id="hypervisor-stop-interval-0s" interval="0s" name="stop" timeout="100"/>
  488. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </operations>
  489. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </primitive>
  490. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.8.0)
  491. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-fenced [4566] (update_cib_stonith_devices_v2) info: Updating device list from the cib: create resources
  492. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-fenced [4566] (cib_devices_update) info: Updating devices to version 0.8.0
  493. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-7.raw
  494. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.8.0 of the CIB to disk (digest: 2c9ff6874f28590448a2f242c01625bf)
  495. Jul 14 21:29:21 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.tjiQRF (digest: /var/lib/pacemaker/cib/cib.9WdCsQ)
  496. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.8.0 2
  497. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.9.0 (null)
  498. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: -- /cib/configuration/resources/primitive[@id='hypervisor']
  499. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=9
  500. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/resources: <clone id="hypervisor-clone"/>
  501. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <primitive class="systemd" id="hypervisor" type="libvirtd">
  502. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <operations>
  503. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <op id="hypervisor-monitor-interval-60" interval="60" name="monitor"/>
  504. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <op id="hypervisor-start-interval-0s" interval="0s" name="start" timeout="100"/>
  505. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <op id="hypervisor-stop-interval-0s" interval="0s" name="stop" timeout="100"/>
  506. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </operations>
  507. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </primitive>
  508. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <meta_attributes id="hypervisor-clone-meta_attributes">
  509. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="hypervisor-clone-meta_attributes-clone-max" name="clone-max" value="2"/>
  510. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="hypervisor-clone-meta_attributes-notify" name="notify" value="false"/>
  511. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </meta_attributes>
  512. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </clone>
  513. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-fenced [4566] (stonith_device_remove) info: Device 'hypervisor' not found (2 active devices)
  514. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-fenced [4566] (update_cib_stonith_devices_v2) info: Updating device list from the cib: create resources
  515. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-fenced [4566] (cib_devices_update) info: Updating devices to version 0.9.0
  516. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_replace operation for section configuration: OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.9.0)
  517. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-8.raw
  518. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.9.0 of the CIB to disk (digest: db64a16c52c066583a5b1a74240d556a)
  519. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.1zzlfH (digest: /var/lib/pacemaker/cib/cib.0kqxdT)
  520. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.9.0 2
  521. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.10.0 (null)
  522. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=10
  523. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/resources: <primitive class="systemd" id="drbd" type="drbd"/>
  524. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <operations>
  525. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <op id="drbd-monitor-interval-60" interval="60" name="monitor"/>
  526. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <op id="drbd-start-interval-0s" interval="0s" name="start" timeout="100"/>
  527. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <op id="drbd-stop-interval-0s" interval="0s" name="stop" timeout="100"/>
  528. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </operations>
  529. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </primitive>
  530. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.10.0)
  531. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-fenced [4566] (update_cib_stonith_devices_v2) info: Updating device list from the cib: create resources
  532. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-fenced [4566] (cib_devices_update) info: Updating devices to version 0.10.0
  533. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-9.raw
  534. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.10.0 of the CIB to disk (digest: 84477666f918034297fa4b47c59c0a6a)
  535. Jul 14 21:29:22 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.pHbaKI (digest: /var/lib/pacemaker/cib/cib.Ysl4cW)
  536. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.10.0 2
  537. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.0 (null)
  538. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: -- /cib/configuration/resources/primitive[@id='drbd']
  539. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @epoch=11
  540. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/configuration/resources: <clone id="drbd-clone"/>
  541. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <primitive class="systemd" id="drbd" type="drbd">
  542. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <operations>
  543. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <op id="drbd-monitor-interval-60" interval="60" name="monitor"/>
  544. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <op id="drbd-start-interval-0s" interval="0s" name="start" timeout="100"/>
  545. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <op id="drbd-stop-interval-0s" interval="0s" name="stop" timeout="100"/>
  546. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </operations>
  547. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </primitive>
  548. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <meta_attributes id="drbd-clone-meta_attributes">
  549. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="drbd-clone-meta_attributes-clone-max" name="clone-max" value="2"/>
  550. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="drbd-clone-meta_attributes-notify" name="notify" value="false"/>
  551. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </meta_attributes>
  552. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </clone>
  553. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_replace operation for section configuration: OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.11.0)
  554. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-fenced [4566] (stonith_device_remove) info: Device 'drbd' not found (2 active devices)
  555. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-fenced [4566] (update_cib_stonith_devices_v2) info: Updating device list from the cib: create resources
  556. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-fenced [4566] (cib_devices_update) info: Updating devices to version 0.11.0
  557. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-10.raw
  558. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Wrote version 0.11.0 of the CIB to disk (digest: 2cddade26531e2c2dfcb3bac3401f84c)
  559. Jul 14 21:29:23 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.AKYe3J (digest: /var/lib/pacemaker/cib/cib.UTocPY)
  560. Jul 14 21:29:26 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_1:4588:stderr [ 2020-07-14 21:29:26,710 ERROR: Unable to connect/login to fencing device ]
  561. Jul 14 21:29:26 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_1:4588:stderr [ ]
  562. Jul 14 21:29:26 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_1:4588:stderr [ ]
  563. Jul 14 21:29:26 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4588] stderr: [ 2020-07-14 21:29:26,710 ERROR: Unable to connect/login to fencing device ]
  564. Jul 14 21:29:26 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4588] stderr: [ ]
  565. Jul 14 21:29:26 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4588] stderr: [ ]
  566. Jul 14 21:29:26 el8-a01n02.digimer.ca pacemaker-fenced [4566] (internal_stonith_action_execute) info: Attempt 2 to execute fence_virsh (monitor). remaining timeout is 15
  567. Jul 14 21:29:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_ping) info: Reporting our current digest to el8-a01n01: b7814e3f7afd5c57f438342d72812427 for 0.11.0 (0x557965cbb5b0 0)
  568. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_2:4608:stderr [ 2020-07-14 21:29:32,910 ERROR: Unable to connect/login to fencing device ]
  569. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_2:4608:stderr [ ]
  570. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_2:4608:stderr [ ]
  571. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4608] stderr: [ 2020-07-14 21:29:32,910 ERROR: Unable to connect/login to fencing device ]
  572. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4608] stderr: [ ]
  573. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4608] stderr: [ ]
  574. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-fenced [4566] (update_remaining_timeout) info: Attempted to execute agent fence_virsh (monitor) the maximum number of times (2) allowed
  575. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_operation) notice: Operation 'monitor' [4608] for device 'virsh_node1' returned: -201 (Generic Pacemaker error)
  576. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) info: finished - rsc:virsh_node1 action:start call_id:10 exit-code:1 exec-time:11419ms queue-time:0ms
  577. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of start operation for virsh_node1 on el8-a01n02: 1 (error) | call=10 key=virsh_node1_start_0 confirmed=true cib-update=17
  578. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/17)
  579. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.0 2
  580. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.1 (null)
  581. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=1
  582. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:1;4:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=10, @rc-code=1, @op-status=0, @exec-time=11419
  583. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']: <lrm_rsc_op id="virsh_node1_last_failure_0" operation_key="virsh_node1_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="4:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" transition-magic="0:1;4:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" exit-reason="" on_node="el8-a01n02" call
  584. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/17, version=0.11.1)
  585. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting fail-count-virsh_node1#start_0[el8-a01n02]: (unset) -> INFINITY | from el8-a01n01
  586. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/6)
  587. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 6 with 2 changes for fail-count-virsh_node1#start_0 (id n/a, set n/a)
  588. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting last-failure-virsh_node1#start_0[el8-a01n02]: (unset) -> 1594762173 | from el8-a01n01
  589. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.1 2
  590. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.2 (null)
  591. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=2
  592. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']: <transient_attributes id="2"/>
  593. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <instance_attributes id="status-2">
  594. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <nvpair id="status-2-fail-count-virsh_node1.start_0" name="fail-count-virsh_node1#start_0" value="INFINITY"/>
  595. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </instance_attributes>
  596. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </transient_attributes>
  597. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 7 with 2 changes for last-failure-virsh_node1#start_0 (id n/a, set n/a)
  598. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/6, version=0.11.2)
  599. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/7)
  600. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 6 result for fail-count-virsh_node1#start_0: OK | rc=0
  601. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * fail-count-virsh_node1#start_0[el8-a01n01]=INFINITY
  602. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * fail-count-virsh_node1#start_0[el8-a01n02]=INFINITY
  603. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.2 2
  604. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.3 (null)
  605. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=3
  606. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: <nvpair id="status-2-last-failure-virsh_node1.start_0" name="last-failure-virsh_node1#start_0" value="1594762173"/>
  607. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/7, version=0.11.3)
  608. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 7 result for last-failure-virsh_node1#start_0: OK | rc=0
  609. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * last-failure-virsh_node1#start_0[el8-a01n01]=1594762161
  610. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * last-failure-virsh_node1#start_0[el8-a01n02]=1594762173
  611. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.3 2
  612. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.4 (null)
  613. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=4
  614. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:1;6:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=12, @rc-code=1, @op-status=0, @exec-time=11463
  615. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']: <lrm_rsc_op id="virsh_node2_last_failure_0" operation_key="virsh_node2_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="6:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" transition-magic="0:1;6:2:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" exit-reason="" on_node="el8-a01n01" call
  616. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/47, version=0.11.4)
  617. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting fail-count-virsh_node2#start_0[el8-a01n01]: (unset) -> INFINITY | from el8-a01n01
  618. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 8 with 1 change for fail-count-virsh_node2#start_0 (id n/a, set n/a)
  619. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/8)
  620. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting last-failure-virsh_node2#start_0[el8-a01n01]: (unset) -> 1594762173 | from el8-a01n01
  621. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 9 with 1 change for last-failure-virsh_node2#start_0 (id n/a, set n/a)
  622. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/9)
  623. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.4 2
  624. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.5 (null)
  625. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=5
  626. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']: <nvpair id="status-1-fail-count-virsh_node2.start_0" name="fail-count-virsh_node2#start_0" value="INFINITY"/>
  627. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/8, version=0.11.5)
  628. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 8 result for fail-count-virsh_node2#start_0: OK | rc=0
  629. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * fail-count-virsh_node2#start_0[el8-a01n01]=INFINITY
  630. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.5 2
  631. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.6 (null)
  632. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=6
  633. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']: <nvpair id="status-1-last-failure-virsh_node2.start_0" name="last-failure-virsh_node2#start_0" value="1594762173"/>
  634. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/9, version=0.11.6)
  635. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 9 result for last-failure-virsh_node2#start_0: OK | rc=0
  636. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * last-failure-virsh_node2#start_0[el8-a01n01]=1594762173
  637. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=2:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=virsh_node1_stop_0
  638. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/18)
  639. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=11, reply=1, notify=0
  640. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) info: executing - rsc:virsh_node1 action:stop call_id:11
  641. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) info: finished - rsc:virsh_node1 action:stop call_id:11 exit-code:0 exec-time:1ms queue-time:0ms
  642. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.6 2
  643. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.7 (null)
  644. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=7
  645. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=1:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;1:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762173, @last-run=1594762173, @exec-time=0
  646. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/51, version=0.11.7)
  647. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.7 2
  648. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.8 (null)
  649. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=8
  650. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_stop_0, @operation=stop, @transition-key=2:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;2:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762173, @last-run=1594762173, @exec-time=0
  651. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/18, version=0.11.8)
  652. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of stop operation for virsh_node1 on el8-a01n02: 0 (ok) | call=11 key=virsh_node1_stop_0 confirmed=true cib-update=19
  653. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/19)
  654. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.8 2
  655. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.9 (null)
  656. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=9
  657. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:0;2:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=11, @rc-code=0, @op-status=0, @exec-time=1
  658. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/19, version=0.11.9)
  659. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.9 2
  660. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.10 (null)
  661. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=10
  662. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="hypervisor" type="libvirtd" class="systemd"/>
  663. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor' not in cache
  664. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <lrm_rsc_op id="hypervisor_last_0" operation_key="hypervisor_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="3:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" transition-magic="-1:193;3:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-status=
  665. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </lrm_resource>
  666. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0
  667. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/52, version=0.11.10)
  668. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor:1' not in cache
  669. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0
  670. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_rsc_register) info: Cached agent information for 'hypervisor'
  671. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=1
  672. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0
  673. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=5:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=hypervisor_monitor_0
  674. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/20)
  675. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=16, reply=1, notify=0
  676. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:16
  677. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.10 2
  678. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.11 (null)
  679. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=11
  680. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="hypervisor" type="libvirtd" class="systemd"/>
  681. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <lrm_rsc_op id="hypervisor_last_0" operation_key="hypervisor_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="5:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" transition-magic="-1:193;5:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-status=
  682. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </lrm_resource>
  683. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/20, version=0.11.11)
  684. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  685. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  686. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:16 exit-code:7 exec-time:11ms queue-time:0ms
  687. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.11 2
  688. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.12 (null)
  689. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=12
  690. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="drbd" type="drbd" class="systemd"/>
  691. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <lrm_rsc_op id="drbd_last_0" operation_key="drbd_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="4:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" transition-magic="-1:193;4:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-status="-1" interva
  692. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </lrm_resource>
  693. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/53, version=0.11.12)
  694. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/21)
  695. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of probe operation for hypervisor on el8-a01n02: 7 (not running) | call=16 key=hypervisor_monitor_0 confirmed=true cib-update=21
  696. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.12 2
  697. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.13 (null)
  698. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=13
  699. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;5:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=16, @rc-code=7, @op-status=0, @exec-time=11
  700. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/21, version=0.11.13)
  701. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_get_rsc_info) info: Agent information for 'drbd' not in cache
  702. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0
  703. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_get_rsc_info) info: Agent information for 'drbd:1' not in cache
  704. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0
  705. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_rsc_register) info: Cached agent information for 'drbd'
  706. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=1
  707. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0
  708. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=6:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=drbd_monitor_0
  709. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/22)
  710. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.13 2
  711. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.14 (null)
  712. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=14
  713. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;1:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=13, @rc-code=0, @op-status=0, @exec-time=1, @queue-time=1
  714. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=21, reply=1, notify=0
  715. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:21
  716. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/54, version=0.11.14)
  717. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  718. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  719. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.14 2
  720. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.15 (null)
  721. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=15
  722. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="drbd" type="drbd" class="systemd"/>
  723. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ <lrm_rsc_op id="drbd_last_0" operation_key="drbd_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="6:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" transition-magic="-1:193;6:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-status="-1" interva
  724. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ </lrm_resource>
  725. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/22, version=0.11.15)
  726. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.15 2
  727. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.16 (null)
  728. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=16
  729. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;3:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=18, @rc-code=7, @op-status=0, @exec-time=8
  730. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/55, version=0.11.16)
  731. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:drbd action:monitor call_id:21 exit-code:7 exec-time:14ms queue-time:1ms
  732. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of probe operation for drbd on el8-a01n02: 7 (not running) | call=21 key=drbd_monitor_0 confirmed=true cib-update=23
  733. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.16 2
  734. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.17 (null)
  735. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=17
  736. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;4:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=23, @rc-code=7, @op-status=0, @exec-time=11, @queue-time=1
  737. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/56, version=0.11.17)
  738. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/23)
  739. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.17 2
  740. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.18 (null)
  741. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=18
  742. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;6:4:7:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=21, @rc-code=7, @op-status=0, @exec-time=14, @queue-time=1
  743. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/23, version=0.11.18)
  744. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=7:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=virsh_node2_start_0
  745. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/24)
  746. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=22, reply=1, notify=0
  747. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) info: executing - rsc:virsh_node2 action:start call_id:22
  748. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.18 2
  749. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.19 (null)
  750. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=19
  751. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=7:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;7:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762173, @last-run=1594762173
  752. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/24, version=0.11.19)
  753. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=11:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=hypervisor_start_0
  754. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/25)
  755. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=23, reply=1, notify=0
  756. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) info: executing - rsc:hypervisor action:start call_id:23
  757. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.19 2
  758. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.20 (null)
  759. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=20
  760. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=11:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;11:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0
  761. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous start op on systemd unit libvirtd named 'hypervisor'
  762. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/25, version=0.11.20)
  763. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec_with_unit) debug: Calling StartUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice
  764. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=19:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=drbd_start_0
  765. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/26)
  766. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=24, reply=1, notify=0
  767. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) info: executing - rsc:drbd action:start call_id:24
  768. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous start op on systemd unit drbd named 'drbd'
  769. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.20 2
  770. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.21 (null)
  771. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=21
  772. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_start_0, @operation=start, @transition-key=19:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;19:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0, @queue-time=0
  773. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/26, version=0.11.21)
  774. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.21 2
  775. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.22 (null)
  776. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=22
  777. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=9:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;9:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0
  778. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/57, version=0.11.22)
  779. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.22 2
  780. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.23 (null)
  781. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=23
  782. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_start_0, @operation=start, @transition-key=17:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;17:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0, @queue-time=0
  783. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/58, version=0.11.23)
  784. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_exec_result) info: Call to start passed: /org/freedesktop/systemd1/job/4074
  785. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: hypervisor start may still be in progress: re-scheduling (elapsed=217ms, remaining=99783ms, start_delay=2000ms)
  786. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec_with_unit) debug: Calling StartUnit for drbd: /org/freedesktop/systemd1/unit/drbd_2eservice
  787. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_exec_result) info: Call to start passed: /org/freedesktop/systemd1/job/4158
  788. Jul 14 21:29:33 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd start may still be in progress: re-scheduling (elapsed=404ms, remaining=99596ms, start_delay=2000ms)
  789. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:23
  790. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  791. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  792. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: hypervisor start is now complete (elapsed=2222ms, remaining=97778ms): ok (0)
  793. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:23 exit-code:0 exec-time:2222ms queue-time:0ms
  794. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/27)
  795. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of start operation for hypervisor on el8-a01n02: 0 (ok) | call=23 key=hypervisor_start_0 confirmed=true cib-update=27
  796. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.23 2
  797. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.24 (null)
  798. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=24
  799. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;11:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=23, @rc-code=0, @op-status=0, @exec-time=2222
  800. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/27, version=0.11.24)
  801. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=12:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=hypervisor_monitor_60000
  802. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=25, reply=1, notify=0
  803. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:25
  804. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/28)
  805. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  806. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.24 2
  807. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.25 (null)
  808. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=25
  809. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']: <lrm_rsc_op id="hypervisor_monitor_60000" operation_key="hypervisor_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="12:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" transition-magic="-1:193;12:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" exit-reason="" on_node="el8-a01
  810. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/28, version=0.11.25)
  811. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  812. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:25 exit-code:0 exec-time:3ms queue-time:0ms
  813. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of monitor operation for hypervisor on el8-a01n02: 0 (ok) | call=25 key=hypervisor_monitor_60000 confirmed=false cib-update=29
  814. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/29)
  815. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.25 2
  816. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.26 (null)
  817. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=26
  818. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_monitor_60000']: @transition-magic=0:0;12:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=25, @rc-code=0, @op-status=0, @exec-time=3
  819. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/29, version=0.11.26)
  820. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.26 2
  821. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.27 (null)
  822. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=27
  823. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;9:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=24, @rc-code=0, @op-status=0, @exec-time=2255, @queue-time=1
  824. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/59, version=0.11.27)
  825. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.27 2
  826. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.28 (null)
  827. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=28
  828. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']: <lrm_rsc_op id="hypervisor_monitor_60000" operation_key="hypervisor_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="10:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" transition-magic="-1:193;10:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" exit-reason="" on_node="el8-a01
  829. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/60, version=0.11.28)
  830. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.28 2
  831. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.29 (null)
  832. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=29
  833. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_monitor_60000']: @transition-magic=0:0;10:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=26, @rc-code=0, @op-status=0, @exec-time=4
  834. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/61, version=0.11.29)
  835. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:24
  836. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  837. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  838. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd start is now complete (elapsed=2409ms, remaining=97591ms): ok (0)
  839. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:drbd action:monitor call_id:24 exit-code:0 exec-time:2409ms queue-time:1ms
  840. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of start operation for drbd on el8-a01n02: 0 (ok) | call=24 key=drbd_start_0 confirmed=true cib-update=30
  841. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/30)
  842. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.29 2
  843. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.30 (null)
  844. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=30
  845. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;19:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=24, @rc-code=0, @op-status=0, @exec-time=2409, @queue-time=1
  846. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/30, version=0.11.30)
  847. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=20:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=drbd_monitor_60000
  848. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/31)
  849. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=26, reply=1, notify=0
  850. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:26
  851. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  852. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  853. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:drbd action:monitor call_id:26 exit-code:0 exec-time:3ms queue-time:0ms
  854. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.30 2
  855. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.31 (null)
  856. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=31
  857. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']: <lrm_rsc_op id="drbd_monitor_60000" operation_key="drbd_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="20:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" transition-magic="-1:193;20:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" exit-reason="" on_node="el8-a01n02" call-id="-1"
  858. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of monitor operation for drbd on el8-a01n02: 0 (ok) | call=26 key=drbd_monitor_60000 confirmed=false cib-update=32
  859. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/31, version=0.11.31)
  860. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/32)
  861. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.31 2
  862. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.32 (null)
  863. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=32
  864. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_monitor_60000']: @transition-magic=0:0;20:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=26, @rc-code=0, @op-status=0, @exec-time=3
  865. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/32, version=0.11.32)
  866. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.32 2
  867. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.33 (null)
  868. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=33
  869. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;17:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=25, @rc-code=0, @op-status=0, @exec-time=2511, @queue-time=1
  870. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/62, version=0.11.33)
  871. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.33 2
  872. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.34 (null)
  873. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=34
  874. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']: <lrm_rsc_op id="drbd_monitor_60000" operation_key="drbd_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="18:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" transition-magic="-1:193;18:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" exit-reason="" on_node="el8-a01n01" call-id="-1"
  875. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/63, version=0.11.34)
  876. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.34 2
  877. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.35 (null)
  878. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=35
  879. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_monitor_60000']: @transition-magic=0:0;18:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=27, @rc-code=0, @op-status=0, @exec-time=3
  880. Jul 14 21:29:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/64, version=0.11.35)
  881. Jul 14 21:29:38 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_1:4612:stderr [ 2020-07-14 21:29:38,334 ERROR: Unable to connect/login to fencing device ]
  882. Jul 14 21:29:38 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_1:4612:stderr [ ]
  883. Jul 14 21:29:38 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_1:4612:stderr [ ]
  884. Jul 14 21:29:38 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4612] stderr: [ 2020-07-14 21:29:38,334 ERROR: Unable to connect/login to fencing device ]
  885. Jul 14 21:29:38 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4612] stderr: [ ]
  886. Jul 14 21:29:38 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4612] stderr: [ ]
  887. Jul 14 21:29:38 el8-a01n02.digimer.ca pacemaker-fenced [4566] (internal_stonith_action_execute) info: Attempt 2 to execute fence_virsh (monitor). remaining timeout is 15
  888. Jul 14 21:29:40 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_ping) info: Reporting our current digest to el8-a01n01: 72c0a0507a5f4ea0970661199a542caa for 0.11.35 (0x557965cbb5b0 0)
  889. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_2:4701:stderr [ 2020-07-14 21:29:44,551 ERROR: Unable to connect/login to fencing device ]
  890. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_2:4701:stderr [ ]
  891. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-fenced [4566] (operation_finished) notice: fence_virsh_monitor_2:4701:stderr [ ]
  892. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4701] stderr: [ 2020-07-14 21:29:44,551 ERROR: Unable to connect/login to fencing device ]
  893. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4701] stderr: [ ]
  894. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_action) warning: fence_virsh[4701] stderr: [ ]
  895. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-fenced [4566] (update_remaining_timeout) info: Attempted to execute agent fence_virsh (monitor) the maximum number of times (2) allowed
  896. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-fenced [4566] (log_operation) notice: Operation 'monitor' [4701] for device 'virsh_node2' returned: -201 (Generic Pacemaker error)
  897. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) info: finished - rsc:virsh_node2 action:start call_id:22 exit-code:1 exec-time:11492ms queue-time:0ms
  898. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of start operation for virsh_node2 on el8-a01n02: 1 (error) | call=22 key=virsh_node2_start_0 confirmed=true cib-update=33
  899. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/33)
  900. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.35 2
  901. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.36 (null)
  902. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=36
  903. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:1;7:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=22, @rc-code=1, @op-status=0, @exec-time=11492
  904. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']: <lrm_rsc_op id="virsh_node2_last_failure_0" operation_key="virsh_node2_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="7:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" transition-magic="0:1;7:4:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" exit-reason="" on_node="el8-a01n02" call
  905. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/33, version=0.11.36)
  906. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting fail-count-virsh_node2#start_0[el8-a01n02]: (unset) -> INFINITY | from el8-a01n01
  907. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/10)
  908. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 10 with 2 changes for fail-count-virsh_node2#start_0 (id n/a, set n/a)
  909. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting last-failure-virsh_node2#start_0[el8-a01n02]: (unset) -> 1594762184 | from el8-a01n01
  910. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 11 with 2 changes for last-failure-virsh_node2#start_0 (id n/a, set n/a)
  911. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.36 2
  912. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.37 (null)
  913. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=37
  914. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: <nvpair id="status-2-fail-count-virsh_node2.start_0" name="fail-count-virsh_node2#start_0" value="INFINITY"/>
  915. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/10, version=0.11.37)
  916. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 10 result for fail-count-virsh_node2#start_0: OK | rc=0
  917. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * fail-count-virsh_node2#start_0[el8-a01n01]=INFINITY
  918. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * fail-count-virsh_node2#start_0[el8-a01n02]=INFINITY
  919. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/11)
  920. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.37 2
  921. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.38 (null)
  922. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=38
  923. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: <nvpair id="status-2-last-failure-virsh_node2.start_0" name="last-failure-virsh_node2#start_0" value="1594762184"/>
  924. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/11, version=0.11.38)
  925. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 11 result for last-failure-virsh_node2#start_0: OK | rc=0
  926. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * last-failure-virsh_node2#start_0[el8-a01n01]=1594762173
  927. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * last-failure-virsh_node2#start_0[el8-a01n02]=1594762184
  928. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=3:6:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=virsh_node2_stop_0
  929. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=27, reply=1, notify=0
  930. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) info: executing - rsc:virsh_node2 action:stop call_id:27
  931. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/34)
  932. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) info: finished - rsc:virsh_node2 action:stop call_id:27 exit-code:0 exec-time:1ms queue-time:0ms
  933. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of stop operation for virsh_node2 on el8-a01n02: 0 (ok) | call=27 key=virsh_node2_stop_0 confirmed=true cib-update=35
  934. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.38 2
  935. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.39 (null)
  936. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=39
  937. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=3:6:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;3:6:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762184, @last-run=1594762184, @exec-time=0
  938. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/34, version=0.11.39)
  939. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/35)
  940. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.39 2
  941. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.40 (null)
  942. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=40
  943. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;3:6:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=27, @rc-code=0, @op-status=0, @exec-time=1
  944. Jul 14 21:29:44 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/35, version=0.11.40)
  945. Jul 14 21:29:49 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_ping) info: Reporting our current digest to el8-a01n01: 8251df6e54b59aa0e832ca3aef06d2cb for 0.11.40 (0x557965cbb5b0 0)
  946. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemakerd [4564] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  947. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemakerd [4564] (pcmk_shutdown_worker) notice: Shutting down Pacemaker
  948. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemakerd [4564] (stop_child) notice: Stopping pacemaker-controld | sent signal 15 to process 4570
  949. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  950. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_shutdown) notice: Shutting down cluster resource manager | limit=1200000ms
  951. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_shutdown_req) info: Sending shutdown request to all peers (DC is el8-a01n01)
  952. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_update_peer_expected) info: handle_request: Node el8-a01n02[2] - expected state is now down (was (null))
  953. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting shutdown[el8-a01n02]: (unset) -> 1594762228 | from el8-a01n01
  954. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 12 with 1 change for shutdown (id n/a, set n/a)
  955. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/12)
  956. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.40 2
  957. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.41 (null)
  958. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=41
  959. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: <nvpair id="status-2-shutdown" name="shutdown" value="1594762228"/>
  960. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/12, version=0.11.41)
  961. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 12 result for shutdown: OK | rc=0
  962. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * shutdown[el8-a01n02]=1594762228
  963. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (cancel_recurring_action) info: Cancelling systemd operation hypervisor_status_60000
  964. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:25 exit-code:0 exec-time:0ms queue-time:0ms
  965. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0
  966. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=7:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=hypervisor_stop_0
  967. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/36)
  968. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=29, reply=1, notify=0
  969. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) info: executing - rsc:hypervisor action:stop call_id:29
  970. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit libvirtd named 'hypervisor'
  971. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) info: Result of monitor operation for hypervisor on el8-a01n02: Cancelled | call=25 key=hypervisor_monitor_60000 confirmed=true
  972. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.41 2
  973. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.42 (null)
  974. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=42
  975. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=7:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;7:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762228, @last-run=1594762228, @exec-time=0
  976. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (cancel_recurring_action) info: Cancelling systemd operation drbd_status_60000
  977. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:drbd action:monitor call_id:26 exit-code:0 exec-time:0ms queue-time:0ms
  978. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=0, reply=1, notify=0
  979. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (do_lrm_rsc_op) info: Performing key=14:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11 op=drbd_stop_0
  980. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/36, version=0.11.42)
  981. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/37)
  982. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2eb86a7f-4b61-4d54-8801-a8f9e7673b7c: rc=31, reply=1, notify=0
  983. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) info: executing - rsc:drbd action:stop call_id:31
  984. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit drbd named 'drbd'
  985. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) info: Result of monitor operation for drbd on el8-a01n02: Cancelled | call=26 key=drbd_monitor_60000 confirmed=true
  986. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.42 2
  987. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.43 (null)
  988. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=43
  989. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_stop_0, @operation=stop, @transition-key=14:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;14:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762228, @last-run=1594762228, @exec-time=0, @queue-time=0
  990. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/37, version=0.11.43)
  991. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec_with_unit) debug: Calling StopUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice
  992. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec_with_unit) debug: Calling StopUnit for drbd: /org/freedesktop/systemd1/unit/drbd_2eservice
  993. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-controld [4570] (crm_update_peer_expected) info: handle_request: Node el8-a01n01[1] - expected state is now down (was member)
  994. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting shutdown[el8-a01n01]: (unset) -> 1594762228 | from el8-a01n01
  995. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 13 with 2 changes for shutdown (id n/a, set n/a)
  996. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/13)
  997. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.43 2
  998. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.44 (null)
  999. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=44
  1000. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']: <nvpair id="status-1-shutdown" name="shutdown" value="1594762228"/>
  1001. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/13, version=0.11.44)
  1002. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 13 result for shutdown: OK | rc=0
  1003. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * shutdown[el8-a01n01]=1594762228
  1004. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * shutdown[el8-a01n02]=1594762228
  1005. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/4253
  1006. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: hypervisor stop may still be in progress: re-scheduling (elapsed=220ms, remaining=99780ms, start_delay=2000ms)
  1007. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/4254
  1008. Jul 14 21:30:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=380ms, remaining=99620ms, start_delay=2000ms)
  1009. Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:29
  1010. Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  1011. Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1012. Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: hypervisor stop is now complete (elapsed=2224ms, remaining=97776ms): not running (7)
  1013. Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:29 exit-code:0 exec-time:2224ms queue-time:0ms
  1014. Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) notice: Result of stop operation for hypervisor on el8-a01n02: 0 (ok) | call=29 key=hypervisor_stop_0 confirmed=true cib-update=38
  1015. Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/38)
  1016. Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.44 2
  1017. Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.45 (null)
  1018. Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=45
  1019. Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;7:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=29, @rc-code=0, @op-status=0, @exec-time=2224
  1020. Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/38, version=0.11.45)
  1021. Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1022. Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1023. Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1024. Jul 14 21:30:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=2385ms, remaining=97615ms, start_delay=2000ms)
  1025. Jul 14 21:30:32 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1026. Jul 14 21:30:32 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1027. Jul 14 21:30:32 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1028. Jul 14 21:30:32 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=4387ms, remaining=95613ms, start_delay=2000ms)
  1029. Jul 14 21:30:34 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1030. Jul 14 21:30:34 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1031. Jul 14 21:30:34 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1032. Jul 14 21:30:34 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=6391ms, remaining=93609ms, start_delay=2000ms)
  1033. Jul 14 21:30:35 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_ping) info: Reporting our current digest to el8-a01n01: afb079b332593e47dff6eff041966396 for 0.11.45 (0x557965cbb5b0 0)
  1034. Jul 14 21:30:36 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1035. Jul 14 21:30:36 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1036. Jul 14 21:30:36 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1037. Jul 14 21:30:36 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=8394ms, remaining=91606ms, start_delay=2000ms)
  1038. Jul 14 21:30:38 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1039. Jul 14 21:30:38 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1040. Jul 14 21:30:38 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1041. Jul 14 21:30:38 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=10397ms, remaining=89603ms, start_delay=2000ms)
  1042. Jul 14 21:30:40 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1043. Jul 14 21:30:40 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1044. Jul 14 21:30:40 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1045. Jul 14 21:30:40 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=12401ms, remaining=87599ms, start_delay=2000ms)
  1046. Jul 14 21:30:42 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1047. Jul 14 21:30:42 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1048. Jul 14 21:30:42 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1049. Jul 14 21:30:42 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=14405ms, remaining=85595ms, start_delay=2000ms)
  1050. Jul 14 21:30:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1051. Jul 14 21:30:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1052. Jul 14 21:30:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1053. Jul 14 21:30:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=16412ms, remaining=83588ms, start_delay=2000ms)
  1054. Jul 14 21:30:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1055. Jul 14 21:30:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1056. Jul 14 21:30:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1057. Jul 14 21:30:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=18417ms, remaining=81583ms, start_delay=2000ms)
  1058. Jul 14 21:30:48 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1059. Jul 14 21:30:48 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1060. Jul 14 21:30:48 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1061. Jul 14 21:30:48 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=20420ms, remaining=79580ms, start_delay=2000ms)
  1062. Jul 14 21:30:50 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1063. Jul 14 21:30:50 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1064. Jul 14 21:30:50 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1065. Jul 14 21:30:50 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=22423ms, remaining=77577ms, start_delay=2000ms)
  1066. Jul 14 21:30:52 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1067. Jul 14 21:30:52 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1068. Jul 14 21:30:52 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1069. Jul 14 21:30:52 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=24430ms, remaining=75570ms, start_delay=2000ms)
  1070. Jul 14 21:30:54 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1071. Jul 14 21:30:54 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1072. Jul 14 21:30:54 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1073. Jul 14 21:30:54 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=26435ms, remaining=73565ms, start_delay=2000ms)
  1074. Jul 14 21:30:56 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1075. Jul 14 21:30:56 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1076. Jul 14 21:30:56 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1077. Jul 14 21:30:56 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=28442ms, remaining=71558ms, start_delay=2000ms)
  1078. Jul 14 21:30:58 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1079. Jul 14 21:30:58 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1080. Jul 14 21:30:58 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1081. Jul 14 21:30:58 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=30445ms, remaining=69555ms, start_delay=2000ms)
  1082. Jul 14 21:31:00 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1083. Jul 14 21:31:00 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1084. Jul 14 21:31:00 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1085. Jul 14 21:31:00 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=32449ms, remaining=67551ms, start_delay=2000ms)
  1086. Jul 14 21:31:02 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1087. Jul 14 21:31:02 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1088. Jul 14 21:31:02 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1089. Jul 14 21:31:02 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=34452ms, remaining=65548ms, start_delay=2000ms)
  1090. Jul 14 21:31:04 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1091. Jul 14 21:31:04 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1092. Jul 14 21:31:04 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1093. Jul 14 21:31:04 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=36455ms, remaining=63545ms, start_delay=2000ms)
  1094. Jul 14 21:31:06 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1095. Jul 14 21:31:06 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1096. Jul 14 21:31:06 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1097. Jul 14 21:31:06 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=38459ms, remaining=61541ms, start_delay=2000ms)
  1098. Jul 14 21:31:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1099. Jul 14 21:31:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1100. Jul 14 21:31:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1101. Jul 14 21:31:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=40462ms, remaining=59538ms, start_delay=2000ms)
  1102. Jul 14 21:31:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1103. Jul 14 21:31:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1104. Jul 14 21:31:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1105. Jul 14 21:31:10 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=42469ms, remaining=57531ms, start_delay=2000ms)
  1106. Jul 14 21:31:12 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1107. Jul 14 21:31:12 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1108. Jul 14 21:31:12 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1109. Jul 14 21:31:12 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=44472ms, remaining=55528ms, start_delay=2000ms)
  1110. Jul 14 21:31:14 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1111. Jul 14 21:31:14 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1112. Jul 14 21:31:14 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1113. Jul 14 21:31:14 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=46475ms, remaining=53525ms, start_delay=2000ms)
  1114. Jul 14 21:31:16 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1115. Jul 14 21:31:16 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1116. Jul 14 21:31:16 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1117. Jul 14 21:31:16 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=48481ms, remaining=51519ms, start_delay=2000ms)
  1118. Jul 14 21:31:18 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1119. Jul 14 21:31:18 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1120. Jul 14 21:31:18 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1121. Jul 14 21:31:18 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=50485ms, remaining=49515ms, start_delay=2000ms)
  1122. Jul 14 21:31:20 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1123. Jul 14 21:31:20 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1124. Jul 14 21:31:20 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1125. Jul 14 21:31:20 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=52488ms, remaining=47512ms, start_delay=2000ms)
  1126. Jul 14 21:31:22 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1127. Jul 14 21:31:22 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1128. Jul 14 21:31:22 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1129. Jul 14 21:31:22 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=54492ms, remaining=45508ms, start_delay=2000ms)
  1130. Jul 14 21:31:24 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1131. Jul 14 21:31:24 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1132. Jul 14 21:31:24 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1133. Jul 14 21:31:24 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=56495ms, remaining=43505ms, start_delay=2000ms)
  1134. Jul 14 21:31:26 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1135. Jul 14 21:31:26 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1136. Jul 14 21:31:26 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1137. Jul 14 21:31:26 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=58502ms, remaining=41498ms, start_delay=2000ms)
  1138. Jul 14 21:31:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1139. Jul 14 21:31:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1140. Jul 14 21:31:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1141. Jul 14 21:31:28 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=60507ms, remaining=39493ms, start_delay=2000ms)
  1142. Jul 14 21:31:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1143. Jul 14 21:31:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1144. Jul 14 21:31:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1145. Jul 14 21:31:30 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=62511ms, remaining=37489ms, start_delay=2000ms)
  1146. Jul 14 21:31:32 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1147. Jul 14 21:31:32 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1148. Jul 14 21:31:32 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1149. Jul 14 21:31:32 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=64516ms, remaining=35484ms, start_delay=2000ms)
  1150. Jul 14 21:31:34 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1151. Jul 14 21:31:34 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1152. Jul 14 21:31:34 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1153. Jul 14 21:31:34 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=66519ms, remaining=33481ms, start_delay=2000ms)
  1154. Jul 14 21:31:36 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1155. Jul 14 21:31:36 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1156. Jul 14 21:31:36 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1157. Jul 14 21:31:36 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=68522ms, remaining=31478ms, start_delay=2000ms)
  1158. Jul 14 21:31:38 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1159. Jul 14 21:31:38 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1160. Jul 14 21:31:38 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1161. Jul 14 21:31:38 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=70526ms, remaining=29474ms, start_delay=2000ms)
  1162. Jul 14 21:31:40 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1163. Jul 14 21:31:40 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1164. Jul 14 21:31:40 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1165. Jul 14 21:31:40 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=72529ms, remaining=27471ms, start_delay=2000ms)
  1166. Jul 14 21:31:42 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1167. Jul 14 21:31:42 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1168. Jul 14 21:31:42 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1169. Jul 14 21:31:42 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=74532ms, remaining=25468ms, start_delay=2000ms)
  1170. Jul 14 21:31:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1171. Jul 14 21:31:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1172. Jul 14 21:31:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1173. Jul 14 21:31:44 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=76536ms, remaining=23464ms, start_delay=2000ms)
  1174. Jul 14 21:31:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1175. Jul 14 21:31:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1176. Jul 14 21:31:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1177. Jul 14 21:31:46 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=78541ms, remaining=21459ms, start_delay=2000ms)
  1178. Jul 14 21:31:48 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1179. Jul 14 21:31:48 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1180. Jul 14 21:31:48 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1181. Jul 14 21:31:48 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=80545ms, remaining=19455ms, start_delay=2000ms)
  1182. Jul 14 21:31:50 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1183. Jul 14 21:31:50 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1184. Jul 14 21:31:50 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1185. Jul 14 21:31:50 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=82549ms, remaining=17451ms, start_delay=2000ms)
  1186. Jul 14 21:31:52 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1187. Jul 14 21:31:52 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1188. Jul 14 21:31:52 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1189. Jul 14 21:31:52 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=84553ms, remaining=15447ms, start_delay=2000ms)
  1190. Jul 14 21:31:54 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1191. Jul 14 21:31:54 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1192. Jul 14 21:31:54 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1193. Jul 14 21:31:54 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=86561ms, remaining=13439ms, start_delay=2000ms)
  1194. Jul 14 21:31:56 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1195. Jul 14 21:31:56 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1196. Jul 14 21:31:56 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1197. Jul 14 21:31:56 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=88565ms, remaining=11435ms, start_delay=2000ms)
  1198. Jul 14 21:31:58 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1199. Jul 14 21:31:58 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1200. Jul 14 21:31:58 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1201. Jul 14 21:31:58 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=90569ms, remaining=9431ms, start_delay=2000ms)
  1202. Jul 14 21:32:00 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1203. Jul 14 21:32:00 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1204. Jul 14 21:32:00 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1205. Jul 14 21:32:00 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=92571ms, remaining=7429ms, start_delay=2000ms)
  1206. Jul 14 21:32:02 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1207. Jul 14 21:32:02 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1208. Jul 14 21:32:02 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1209. Jul 14 21:32:02 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=94574ms, remaining=5426ms, start_delay=2000ms)
  1210. Jul 14 21:32:04 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1211. Jul 14 21:32:04 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1212. Jul 14 21:32:04 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1213. Jul 14 21:32:04 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=96578ms, remaining=3422ms, start_delay=1711ms)
  1214. Jul 14 21:32:06 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1215. Jul 14 21:32:06 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1216. Jul 14 21:32:06 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1217. Jul 14 21:32:06 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=98293ms, remaining=1707ms, start_delay=853ms)
  1218. Jul 14 21:32:07 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1219. Jul 14 21:32:07 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1220. Jul 14 21:32:07 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1221. Jul 14 21:32:07 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=99149ms, remaining=851ms, start_delay=425ms)
  1222. Jul 14 21:32:07 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1223. Jul 14 21:32:07 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1224. Jul 14 21:32:07 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1225. Jul 14 21:32:07 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=99580ms, remaining=420ms, start_delay=210ms)
  1226. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1227. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1228. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1229. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=99793ms, remaining=207ms, start_delay=103ms)
  1230. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1231. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1232. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1233. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=99901ms, remaining=99ms, start_delay=49ms)
  1234. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1235. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1236. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1237. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=99953ms, remaining=47ms, start_delay=23ms)
  1238. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1239. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1240. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1241. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=99979ms, remaining=21ms, start_delay=10ms)
  1242. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:31
  1243. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1244. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1245. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (action_complete) notice: Giving up on drbd stop (rc=0): timeout (elapsed=99991ms, remaining=9ms)
  1246. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-execd [4567] (log_finished) debug: finished - rsc:drbd action:monitor call_id:31 exit-code:198 exec-time:99991ms queue-time:0ms
  1247. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-controld [4570] (process_lrm_event) error: Result of stop operation for drbd on el8-a01n02: Timed Out | call=31 key=drbd_stop_0 timeout=100000ms
  1248. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/39)
  1249. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.45 2
  1250. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.46 (null)
  1251. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=46
  1252. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=2:198;14:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=31, @rc-code=198, @op-status=2, @exec-time=99991
  1253. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']: <lrm_rsc_op id="drbd_last_failure_0" operation_key="drbd_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="14:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" transition-magic="2:198;14:7:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11" exit-reason="" on_node="el8-a01n02" call-id="31" rc-code="1
  1254. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/39, version=0.11.46)
  1255. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting fail-count-drbd#stop_0[el8-a01n02]: (unset) -> INFINITY | from el8-a01n01
  1256. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 14 with 1 change for fail-count-drbd#stop_0 (id n/a, set n/a)
  1257. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_peer_update) notice: Setting last-failure-drbd#stop_0[el8-a01n02]: (unset) -> 1594762328 | from el8-a01n01
  1258. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-attrd [4568] (write_attribute) info: Sent CIB request 15 with 1 change for last-failure-drbd#stop_0 (id n/a, set n/a)
  1259. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/14)
  1260. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/15)
  1261. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.46 2
  1262. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.47 (null)
  1263. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=47
  1264. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: <nvpair id="status-2-fail-count-drbd.stop_0" name="fail-count-drbd#stop_0" value="INFINITY"/>
  1265. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/14, version=0.11.47)
  1266. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 14 result for fail-count-drbd#stop_0: OK | rc=0
  1267. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * fail-count-drbd#stop_0[el8-a01n02]=INFINITY
  1268. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.47 2
  1269. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.48 (null)
  1270. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=48
  1271. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: <nvpair id="status-2-last-failure-drbd.stop_0" name="last-failure-drbd#stop_0" value="1594762328"/>
  1272. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/15, version=0.11.48)
  1273. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: CIB update 15 result for last-failure-drbd#stop_0: OK | rc=0
  1274. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-attrd [4568] (attrd_cib_callback) info: * last-failure-drbd#stop_0[el8-a01n02]=1594762328
  1275. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.48 2
  1276. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.49 (null)
  1277. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=49
  1278. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=5:9:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @transition-magic=-1:193;5:9:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762328, @last-run=1594762328, @exec-time=0, @queu
  1279. Jul 14 21:32:08 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/72, version=0.11.49)
  1280. Jul 14 21:32:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: --- 0.11.49 2
  1281. Jul 14 21:32:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: Diff: +++ 0.11.50 (null)
  1282. Jul 14 21:32:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib: @num_updates=50
  1283. Jul 14 21:32:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;5:9:0:0df6240b-f9a0-4c7b-bf60-fd2dab145e11, @call-id=29, @rc-code=0, @op-status=0, @exec-time=2271
  1284. Jul 14 21:32:10 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/73, version=0.11.50)
  1285. Jul 14 21:32:15 el8-a01n02.digimer.ca pacemaker-based [4565] (cib_process_ping) info: Reporting our current digest to el8-a01n01: 55204ae50dead0b498c548c5d466607c for 0.11.50 (0x557965cbb5b0 0)
  1286. Jul 14 21:32:19 el8-a01n02.digimer.ca pacemaker-fenced [4566] (remote_op_done) notice: Operation 'reboot' targeting el8-a01n02 on <no-one> for pacemaker-controld.375414@el8-a01n01.2bbcc655: No route to host
  1287. Jul 14 21:32:19 el8-a01n02.digimer.ca pacemaker-controld [4570] (tengine_stonith_notify) notice: Peer el8-a01n02 was not terminated (reboot) by <anyone> on behalf of pacemaker-controld.375414: No route to host | initiator=el8-a01n01 ref=2bbcc655-c155-4908-a792-c918b427a38e
  1288. Jul 14 21:32:31 el8-a01n02.digimer.ca pacemaker-fenced [4566] (remote_op_done) notice: Operation 'reboot' targeting el8-a01n02 on <no-one> for pacemaker-controld.375414@el8-a01n01.75793d25: No route to host
  1289. Jul 14 21:32:31 el8-a01n02.digimer.ca pacemaker-controld [4570] (tengine_stonith_notify) notice: Peer el8-a01n02 was not terminated (reboot) by <anyone> on behalf of pacemaker-controld.375414: No route to host | initiator=el8-a01n01 ref=75793d25-0284-4916-823f-ebadcf09771b
  1290. Jul 14 21:32:42 el8-a01n02.digimer.ca pacemaker-fenced [4566] (remote_op_done) notice: Operation 'reboot' targeting el8-a01n02 on <no-one> for pacemaker-controld.375414@el8-a01n01.5be1f6ba: No route to host
  1291. Jul 14 21:32:42 el8-a01n02.digimer.ca pacemaker-controld [4570] (tengine_stonith_notify) notice: Peer el8-a01n02 was not terminated (reboot) by <anyone> on behalf of pacemaker-controld.375414: No route to host | initiator=el8-a01n01 ref=5be1f6ba-66c0-4d35-920f-a083bf59809c
  1292. Jul 14 21:32:54 el8-a01n02.digimer.ca pacemaker-fenced [4566] (remote_op_done) notice: Operation 'reboot' targeting el8-a01n02 on <no-one> for pacemaker-controld.375414@el8-a01n01.9fc8382b: No route to host
  1293. Jul 14 21:32:54 el8-a01n02.digimer.ca pacemaker-controld [4570] (tengine_stonith_notify) notice: Peer el8-a01n02 was not terminated (reboot) by <anyone> on behalf of pacemaker-controld.375414: No route to host | initiator=el8-a01n01 ref=9fc8382b-26df-4e45-8f59-affe54245144
  1294. Jul 14 21:33:05 el8-a01n02.digimer.ca pacemaker-fenced [4566] (remote_op_done) notice: Operation 'reboot' targeting el8-a01n02 on <no-one> for pacemaker-controld.375414@el8-a01n01.d3fc9798: No route to host
  1295. Jul 14 21:33:05 el8-a01n02.digimer.ca pacemaker-controld [4570] (tengine_stonith_notify) notice: Peer el8-a01n02 was not terminated (reboot) by <anyone> on behalf of pacemaker-controld.375414: No route to host | initiator=el8-a01n01 ref=d3fc9798-6df8-485b-a7d3-e904d314cf81
  1296. Jul 14 21:33:17 el8-a01n02.digimer.ca pacemaker-fenced [4566] (remote_op_done) notice: Operation 'reboot' targeting el8-a01n02 on <no-one> for pacemaker-controld.375414@el8-a01n01.3e12455e: No route to host
  1297. Jul 14 21:33:17 el8-a01n02.digimer.ca pacemaker-controld [4570] (tengine_stonith_notify) notice: Peer el8-a01n02 was not terminated (reboot) by <anyone> on behalf of pacemaker-controld.375414: No route to host | initiator=el8-a01n01 ref=3e12455e-82d1-4460-90db-9dcb37d6de0e
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!