API tools faq
paste
digimer

Untitled

digimer
Jul 14th, 2020
151
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 329.85 KB | None | 0 0
raw download clone embed print report
  1. [root@el8-a01n01 ~]# cat /var/log/pacemaker/el8-a01n02/pacemaker.log.drbd-down-cluster-down-ok-after
  2. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  3. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (get_cluster_type) info: Detected an active 'corosync' cluster
  4. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (mcp_read_config) info: Reading configure for stack: corosync
  5. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (main) notice: Starting Pacemaker 2.0.3-5.el8_2.1 | build=4b1f869f0f features: generated-manpages agent-manpages ncurses libqb-logging libqb-ipc systemd nagios corosync-native atomic-attrd acls
  6. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (main) info: Maximum core file size is: 18446744073709551615
  7. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (qb_ipcs_us_publish) info: server name: pacemakerd
  8. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to lrmd IPC: Connection refused
  9. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to cib_ro IPC: Connection refused
  10. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to crmd IPC: Connection refused
  11. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to attrd IPC: Connection refused
  12. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to pengine IPC: Connection refused
  13. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk__ipc_is_authentic_process_active) info: Could not connect to stonith-ng IPC: Connection refused
  14. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (crm_get_peer) info: Created entry 22c2f6bc-5d5f-4cd7-a4d6-2a20d5967896/0x5609e2d8b8c0 for node el8-a01n02/2 (1 total)
  15. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  16. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (crm_get_peer) info: Node 2 has uuid 2
  17. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online
  18. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (cluster_connect_quorum) warning: Quorum lost
  19. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (crm_get_peer) info: Created entry d415b99b-afd1-4753-b59d-a154ca8562a4/0x5609e2d8d860 for node el8-a01n01/1 (2 total)
  20. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  21. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (crm_get_peer) info: Node 1 has uuid 1
  22. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (start_child) info: Using uid=189 and group=189 for process pacemaker-based
  23. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (start_child) info: Forked child 1565 for process pacemaker-based
  24. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (start_child) info: Forked child 1566 for process pacemaker-fenced
  25. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (start_child) info: Forked child 1567 for process pacemaker-execd
  26. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (start_child) info: Using uid=189 and group=189 for process pacemaker-attrd
  27. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (start_child) info: Forked child 1568 for process pacemaker-attrd
  28. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (start_child) info: Using uid=189 and group=189 for process pacemaker-schedulerd
  29. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (start_child) info: Forked child 1569 for process pacemaker-schedulerd
  30. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (start_child) info: Using uid=189 and group=189 for process pacemaker-controld
  31. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (start_child) info: Forked child 1570 for process pacemaker-controld
  32. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (main) notice: Pacemaker daemon successfully started and accepting connections
  33. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk_quorum_notification) info: Quorum still lost | membership=180 members=1
  34. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification
  35. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (crm_reap_unseen_nodes) info: State of node el8-a01n01[1] is still unknown
  36. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk_cpg_membership) info: Group pacemakerd event 0: node 2 pid 1564 joined via cpg_join
  37. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk_cpg_membership) info: Group pacemakerd event 0: el8-a01n02 (node 2 pid 1564) is member
  38. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-fenced [1566] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  39. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-fenced [1566] (main) notice: Starting Pacemaker fencer
  40. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-fenced [1566] (get_cluster_type) info: Verifying cluster type: 'corosync'
  41. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-fenced [1566] (get_cluster_type) info: Assuming an active 'corosync' cluster
  42. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-fenced [1566] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  43. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  44. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-based [1565] (main) notice: Starting Pacemaker CIB manager
  45. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-based [1565] (get_cluster_type) info: Verifying cluster type: 'corosync'
  46. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-based [1565] (get_cluster_type) info: Assuming an active 'corosync' cluster
  47. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-based [1565] (retrieveCib) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.xml (digest: /var/lib/pacemaker/cib/cib.xml.sig)
  48. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-fenced [1566] (crm_get_peer) info: Created entry 315e2037-f298-4c9d-af2b-2b51a10a1360/0x55e92fd65d60 for node el8-a01n02/2 (1 total)
  49. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-fenced [1566] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  50. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  51. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (main) notice: Starting Pacemaker scheduler
  52. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (qb_ipcs_us_publish) info: server name: pengine
  53. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (main) notice: Pacemaker scheduler successfully started and accepting connections
  54. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-execd [1567] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  55. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-execd [1567] (main) notice: Starting Pacemaker local executor
  56. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_ipcs_us_publish) info: server name: lrmd
  57. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-execd [1567] (main) notice: Pacemaker local executor successfully started and accepting connections
  58. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  59. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-controld [1570] (main) notice: Starting Pacemaker controller
  60. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_log) info: Input I_STARTUP received in state S_STARTING from crmd_init
  61. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-fenced [1566] (crm_get_peer) info: Node 2 has uuid 2
  62. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-fenced [1566] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online
  63. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-fenced [1566] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  64. Jul 14 22:18:33 el8-a01n02.digimer.ca pacemaker-fenced [1566] (init_cs_connection_once) info: Connection to 'corosync': established
  65. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-attrd [1568] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  66. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-attrd [1568] (main) notice: Starting Pacemaker node attribute manager
  67. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  68. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_get_peer) info: Created entry 0ed6ddb1-4bbc-446e-8f17-340f3b44e169/0x55b6e7f3eef0 for node el8-a01n02/2 (1 total)
  69. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  70. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_get_peer) info: Node 2 has uuid 2
  71. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online
  72. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  73. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-based [1565] (init_cs_connection_once) info: Connection to 'corosync': established
  74. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-based [1565] (qb_ipcs_us_publish) info: server name: cib_ro
  75. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-based [1565] (qb_ipcs_us_publish) info: server name: cib_rw
  76. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-based [1565] (qb_ipcs_us_publish) info: server name: cib_shm
  77. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-based [1565] (main) notice: Pacemaker CIB manager successfully started and accepting connections
  78. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-based [1565] (pcmk_cpg_membership) info: Group cib event 0: node 2 pid 1565 joined via cpg_join
  79. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-based [1565] (pcmk_cpg_membership) info: Group cib event 0: el8-a01n02 (node 2 pid 1565) is member
  80. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-21.raw
  81. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_file_write_with_digest) info: Wrote version 0.15.0 of the CIB to disk (digest: ebc4019e128491fe42d0179b85b82cf6)
  82. Jul 14 22:18:34 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.o8mHNY (digest: /var/lib/pacemaker/cib/cib.rvvc6K)
  83. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-controld [1570] (get_cluster_type) info: Verifying cluster type: 'corosync'
  84. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-controld [1570] (get_cluster_type) info: Assuming an active 'corosync' cluster
  85. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  86. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk_cpg_membership) info: Group pacemakerd event 1: node 1 pid 7329 joined via cluster join
  87. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n01 (node 1 pid 7329) is member
  88. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemakerd [1564] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online
  89. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n02 (node 2 pid 1564) is member
  90. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemakerd [1564] (crm_cs_flush) info: Sent 0 CPG messages (1 remaining, last=7): Try again (6)
  91. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk_quorum_notification) notice: Quorum acquired | membership=188 members=2
  92. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-based [1565] (pcmk_cpg_membership) info: Group cib event 1: node 1 pid 7330 joined via cluster join
  93. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemakerd [1564] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification
  94. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_get_peer) info: Created entry 450b0f8c-fe2a-45e4-bda8-8649f97bb82a/0x55b6e7f45ba0 for node el8-a01n01/1 (2 total)
  95. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  96. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_get_peer) info: Node 1 has uuid 1
  97. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-based [1565] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n01 (node 1 pid 7330) is member
  98. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online
  99. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  100. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-based [1565] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n02 (node 2 pid 1565) is member
  101. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (setup_cib) info: Watching for stonith topology changes
  102. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (qb_ipcs_us_publish) info: server name: stonith-ng
  103. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (main) notice: Pacemaker fencer successfully started and accepting connections
  104. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (pcmk_cpg_membership) info: Group stonith-ng event 0: node 2 pid 1566 joined via cpg_join
  105. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (pcmk_cpg_membership) info: Group stonith-ng event 0: el8-a01n02 (node 2 pid 1566) is member
  106. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (pcmk_cpg_membership) info: Group stonith-ng event 1: node 1 pid 7331 joined via cluster join
  107. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (main) info: CIB connection active
  108. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (get_cluster_type) info: Verifying cluster type: 'corosync'
  109. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (get_cluster_type) info: Assuming an active 'corosync' cluster
  110. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  111. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (crm_get_peer) info: Created entry b41ef672-fa8b-4b00-8e0b-219a51c42514/0x55e92fe6d990 for node el8-a01n01/1 (2 total)
  112. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  113. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (crm_get_peer) info: Node 1 has uuid 1
  114. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n01 (node 1 pid 7331) is member
  115. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online
  116. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  117. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n02 (node 2 pid 1566) is member
  118. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (init_cib_cache_cb) info: Updating device list from the cib: init
  119. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (cib_devices_update) info: Updating devices to version 0.15.0
  120. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (crm_get_peer) info: Created entry d8f661bc-891d-4844-9ffb-d1eb8bd7a5fd/0x562bfbb0fa20 for node el8-a01n02/2 (1 total)
  121. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  122. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (crm_get_peer) info: Node 2 has uuid 2
  123. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online
  124. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  125. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (init_cs_connection_once) info: Connection to 'corosync': established
  126. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (main) info: Cluster connection active
  127. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (attrd_erase_attrs) info: Clearing transient attributes from CIB | xpath=//node_state[@uname='el8-a01n02']/transient_attributes
  128. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n02']/transient_attributes to all (origin=local/attrd/2)
  129. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (qb_ipcs_us_publish) info: server name: attrd
  130. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (main) notice: Pacemaker node attribute manager successfully started and accepting connections
  131. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (pcmk_cpg_membership) info: Group attrd event 0: node 2 pid 1568 joined via cpg_join
  132. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (crm_get_peer) info: Created entry 100e2574-8d63-4677-a809-bca2f920c12e/0x562bfbb126a0 for node el8-a01n01/1 (2 total)
  133. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  134. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (crm_get_peer) info: Node 1 has uuid 1
  135. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (pcmk_cpg_membership) info: Group attrd event 0: el8-a01n01 (node 1 pid 7333) is member
  136. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online
  137. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  138. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (pcmk_cpg_membership) info: Group attrd event 0: el8-a01n02 (node 2 pid 1568) is member
  139. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n02]: (unset) -> 2 | from el8-a01n02
  140. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (attrd_start_election_if_needed) info: Starting an election to determine the writer
  141. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/transient_attributes: OK (rc=0, origin=el8-a01n02/attrd/2, version=0.15.0)
  142. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 1572 exited with rc=0
  143. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (stonith_device_register) notice: Added 'virsh_node1' to the device list (1 active devices)
  144. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-fenced [1566] (stonith_device_register) notice: Added 'virsh_node2' to the device list (2 active devices)
  145. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n01]: (unset) -> 2 | from el8-a01n01
  146. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (election_count_vote) info: election-attrd round 1 (owner node ID 1) pass: vote from el8-a01n01 (Uptime)
  147. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (election_check) info: election-attrd won by local node
  148. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (attrd_declare_winner) notice: Recorded local node as attribute writer (was unset)
  149. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-attrd [1568] (write_attribute) info: Processed 2 private changes for #attrd-protocol, id=n/a, set=n/a
  150. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/transient_attributes: OK (rc=0, origin=el8-a01n01/attrd/2, version=0.15.0)
  151. Jul 14 22:18:35 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/4, version=0.15.0)
  152. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_get_peer) info: Created entry 27319cf9-d821-44c4-93f4-6e9f2f5541cb/0x55cce43f4e50 for node el8-a01n02/2 (1 total)
  153. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  154. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (peer_update_callback) info: Cluster node el8-a01n02 is now in unknown state
  155. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_get_peer) info: Node 2 has uuid 2
  156. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online
  157. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (peer_update_callback) info: Node el8-a01n02 is now a peer | DC=<none> old=0x0000000 new=0x4000000
  158. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds
  159. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (init_cs_connection_once) info: Connection to 'corosync': established
  160. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (cluster_connect_quorum) notice: Quorum acquired
  161. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_get_peer) info: Created entry f51bfaf2-441c-4ece-9c5a-a3b9941c5280/0x55cce43f7750 for node el8-a01n01/1 (2 total)
  162. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  163. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (peer_update_callback) info: Cluster node el8-a01n01 is now in unknown state
  164. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_get_peer) info: Node 1 has uuid 1
  165. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_ha_control) info: Connected to the cluster
  166. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (lrmd_ipc_connect) info: Connecting to executor
  167. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/4)
  168. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-execd [1567] (crm_client_new) debug: Connecting 0x55f87aac1fb0 for uid=189 gid=189 pid=1570 id=801acf2c-48e6-470c-97f7-eae1b7d56a84
  169. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-execd [1567] (handle_new_connection) debug: IPC credentials authenticated (/dev/shm/qb-1567-1570-9-qIYT85/qb)
  170. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_ipcs_shm_connect) debug: connecting to client [1570]
  171. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  172. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  173. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  174. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/4, version=0.15.0)
  175. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed register operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=0
  176. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_lrm_control) info: Connection to the executor established
  177. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_started) info: Delaying start, no membership data (0000000000100000)
  178. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_started) info: Delaying start, no membership data (0000000000100000)
  179. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (pcmk_quorum_notification) info: Quorum retained | membership=188 members=2
  180. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification
  181. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (peer_update_callback) info: Cluster node el8-a01n01 is now member (was in unknown state)
  182. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification
  183. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (peer_update_callback) info: Cluster node el8-a01n02 is now member (was in unknown state)
  184. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_started) info: Delaying start, Config not read (0000000000000040)
  185. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (qb_ipcs_us_publish) info: server name: crmd
  186. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_started) notice: Pacemaker controller successfully started and accepting connections
  187. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_log) info: Input I_PENDING received in state S_STARTING from do_started
  188. Jul 14 22:18:36 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_state_transition) notice: State transition S_STARTING -> S_PENDING | input=I_PENDING cause=C_FSA_INTERNAL origin=do_started
  189. Jul 14 22:18:37 el8-a01n02.digimer.ca pacemaker-controld [1570] (pcmk_cpg_membership) info: Group crmd event 0: node 2 pid 1570 joined via cpg_join
  190. Jul 14 22:18:37 el8-a01n02.digimer.ca pacemaker-controld [1570] (pcmk_cpg_membership) info: Group crmd event 0: el8-a01n01 (node 1 pid 7335) is member
  191. Jul 14 22:18:37 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online
  192. Jul 14 22:18:37 el8-a01n02.digimer.ca pacemaker-controld [1570] (peer_update_callback) info: Node el8-a01n01 is now a peer | DC=<none> old=0x0000000 new=0x4000000
  193. Jul 14 22:18:37 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds
  194. Jul 14 22:18:37 el8-a01n02.digimer.ca pacemaker-controld [1570] (pcmk_cpg_membership) info: Group crmd event 0: el8-a01n02 (node 2 pid 1570) is member
  195. Jul 14 22:18:37 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 30 seconds
  196. Jul 14 22:18:37 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_connect_stonith) notice: Fencer successfully connected
  197. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (election_count_vote) info: election-DC round 1 (owner node ID 1) pass: vote from el8-a01n01 (Uptime)
  198. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_state_transition) info: State transition S_PENDING -> S_ELECTION | input=I_ELECTION cause=C_FSA_INTERNAL origin=do_election_count_vote
  199. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (election_check) info: election-DC won by local node
  200. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_log) info: Input I_ELECTION_DC received in state S_ELECTION from election_win_cb
  201. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_state_transition) notice: State transition S_ELECTION -> S_INTEGRATION | input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=election_win_cb
  202. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_te_control) info: Registering TE UUID: 22298a5d-b80a-4cd7-adde-aba652405a4e
  203. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (set_graph_functions) info: Setting custom graph functions
  204. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_dc_takeover) info: Taking over DC status for this partition
  205. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_readwrite) info: We are now in R/W mode
  206. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_master operation for section 'all': OK (rc=0, origin=local/crmd/7, version=0.15.0)
  207. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/8)
  208. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/10)
  209. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/12)
  210. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (join_make_offer) info: Making join-1 offers based on membership event 188
  211. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (join_make_offer) info: Sending join-1 offer to el8-a01n01
  212. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (join_make_offer) info: Sending join-1 offer to el8-a01n02
  213. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_dc_join_offer_all) info: Waiting on join-1 requests from 2 outstanding nodes
  214. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/14)
  215. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n02/crmd/8, version=0.15.0)
  216. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n02/crmd/10, version=0.15.0)
  217. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n02/crmd/12, version=0.15.0)
  218. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n02/crmd/14, version=0.15.0)
  219. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (update_dc) info: Set DC to el8-a01n02 (3.3.0)
  220. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_update_peer_expected) info: update_dc: Node el8-a01n02[2] - expected state is now member (was (null))
  221. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_update_peer_expected) info: do_dc_join_filter_offer: Node el8-a01n01[1] - expected state is now member (was (null))
  222. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_state_transition) info: State transition S_INTEGRATION -> S_FINALIZE_JOIN | input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state
  223. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_replace) info: Digest matched on replace from el8-a01n02: 2efe3adb921c1da251a57d1a819d3e7a
  224. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_replace) info: Replaced 0.15.0 with 0.15.0 from el8-a01n02
  225. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=el8-a01n02/crmd/17, version=0.15.0)
  226. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/18)
  227. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/19)
  228. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (controld_delete_node_state) info: Deleting resource history for node el8-a01n01 (via CIB call 20) | xpath=//node_state[@uname='el8-a01n01']/lrm
  229. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (controld_delete_node_state) info: Deleting resource history for node el8-a01n02 (via CIB call 22) | xpath=//node_state[@uname='el8-a01n02']/lrm
  230. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/18, version=0.15.0)
  231. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/19, version=0.15.0)
  232. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n01']/lrm to all (origin=local/crmd/20)
  233. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/21)
  234. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n02']/lrm to all (origin=local/crmd/22)
  235. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/23)
  236. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/lrm: OK (rc=0, origin=el8-a01n02/crmd/20, version=0.15.0)
  237. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.0 2
  238. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.1 (null)
  239. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=1
  240. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status: <node_state id="1" uname="el8-a01n01" in_ccm="true" crmd="online" crm-debug-origin="do_lrm_query_internal"/>
  241. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ <lrm id="1">
  242. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ <lrm_resources/>
  243. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ </lrm>
  244. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ </node_state>
  245. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/21, version=0.15.1)
  246. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_state_transition) info: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE | input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state
  247. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (abort_transition_graph) info: Transition -1 aborted: Peer Cancelled | source=do_te_invoke:129 complete=true
  248. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/lrm: OK (rc=0, origin=el8-a01n02/crmd/22, version=0.15.1)
  249. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.1 2
  250. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.2 (null)
  251. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=2
  252. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status: <node_state id="2" uname="el8-a01n02" in_ccm="true" crmd="online" crm-debug-origin="do_lrm_query_internal"/>
  253. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ <lrm id="2">
  254. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ <lrm_resources/>
  255. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ </lrm>
  256. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ </node_state>
  257. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/23, version=0.15.2)
  258. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/26)
  259. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/27)
  260. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/28)
  261. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/26, version=0.15.2)
  262. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.2 2
  263. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.3 (null)
  264. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=3
  265. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition, @join=member, @expected=member
  266. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition, @join=member, @expected=member
  267. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/27, version=0.15.3)
  268. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.3 2
  269. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.4 (null)
  270. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=4, @dc-uuid=2
  271. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-22.raw
  272. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n02/crmd/28, version=0.15.4)
  273. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_file_write_with_digest) info: Wrote version 0.15.0 of the CIB to disk (digest: 82d7477d0ff671fbf6bcda5e74ef2f1d)
  274. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.o0xCzY (digest: /var/lib/pacemaker/cib/cib.LnuEmO)
  275. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (determine_online_status_fencing) info: Node el8-a01n01 is active
  276. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (determine_online_status) info: Node el8-a01n01 is online
  277. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (determine_online_status_fencing) info: Node el8-a01n02 is active
  278. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (determine_online_status) info: Node el8-a01n02 is online
  279. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (unpack_node_loop) info: Node 1 is already processed
  280. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (unpack_node_loop) info: Node 2 is already processed
  281. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (unpack_node_loop) info: Node 1 is already processed
  282. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (unpack_node_loop) info: Node 2 is already processed
  283. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (common_print) info: virsh_node1 (stonith:fence_virsh): Stopped
  284. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (common_print) info: virsh_node2 (stonith:fence_virsh): Stopped
  285. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  286. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (short_print) info: Stopped: [ el8-a01n01 el8-a01n02 ]
  287. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (clone_print) info: Clone Set: drbd-clone [drbd]
  288. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (short_print) info: Stopped: [ el8-a01n01 el8-a01n02 ]
  289. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (RecurringOp) info: Start recurring monitor (60s) for virsh_node1 on el8-a01n01
  290. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (RecurringOp) info: Start recurring monitor (60s) for virsh_node2 on el8-a01n02
  291. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (RecurringOp) info: Start recurring monitor (60s) for hypervisor:0 on el8-a01n01
  292. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (RecurringOp) info: Start recurring monitor (60s) for hypervisor:1 on el8-a01n02
  293. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (RecurringOp) info: Start recurring monitor (60s) for drbd:0 on el8-a01n01
  294. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (RecurringOp) info: Start recurring monitor (60s) for drbd:1 on el8-a01n02
  295. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogAction) notice: * Start virsh_node1 ( el8-a01n01 )
  296. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogAction) notice: * Start virsh_node2 ( el8-a01n02 )
  297. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogAction) notice: * Start hypervisor:0 ( el8-a01n01 )
  298. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogAction) notice: * Start hypervisor:1 ( el8-a01n02 )
  299. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogAction) notice: * Start drbd:0 ( el8-a01n01 )
  300. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogAction) notice: * Start drbd:1 ( el8-a01n02 )
  301. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (pcmk__log_transition_summary) notice: Calculated transition 0, saving inputs in /var/lib/pacemaker/pengine/pe-input-7.bz2
  302. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response
  303. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_te_invoke) info: Processing graph 0 (ref=pe_calc-dc-1594765136-9) derived from /var/lib/pacemaker/pengine/pe-input-7.bz2
  304. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating monitor operation virsh_node1_monitor_0 locally on el8-a01n02 | action 5
  305. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node1' not in cache
  306. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=0
  307. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node1'
  308. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=1
  309. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=0
  310. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_lrm_rsc_op) info: Performing key=5:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e op=virsh_node1_monitor_0
  311. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 1658 exited with rc=0
  312. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/30)
  313. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=5, reply=1, notify=0
  314. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:5
  315. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (stonith_api_signon) debug: Attempting fencer connection by pacemaker-execd with mainloop
  316. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.4 2
  317. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.5 (null)
  318. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=5
  319. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_update_resource
  320. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="virsh_node1" type="fence_virsh" class="stonith"/>
  321. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node1_last_0" operation_key="virsh_node1_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="5:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e" transition-magic="-1:193;5:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-statu
  322. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ </lrm_resource>
  323. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/30, version=0.15.5)
  324. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  325. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  326. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  327. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (stonith_api_signon) debug: Connection to fencer by pacemaker-execd succeeded (registration token: f5d7a2eb-edf7-47c3-9690-9e31641359e8)
  328. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:5 exit-code:7 exec-time:5ms queue-time:0ms
  329. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating monitor operation virsh_node1_monitor_0 on el8-a01n01 | action 1
  330. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating monitor operation virsh_node2_monitor_0 locally on el8-a01n02 | action 6
  331. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node2' not in cache
  332. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=0
  333. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node2'
  334. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=1
  335. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=0
  336. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_lrm_rsc_op) info: Performing key=6:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e op=virsh_node2_monitor_0
  337. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=9, reply=1, notify=0
  338. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) debug: executing - rsc:virsh_node2 action:monitor call_id:9
  339. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) debug: finished - rsc:virsh_node2 action:monitor call_id:9 exit-code:7 exec-time:0ms queue-time:0ms
  340. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/31)
  341. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.5 2
  342. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.6 (null)
  343. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=6
  344. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="virsh_node2" type="fence_virsh" class="stonith"/>
  345. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node2_last_0" operation_key="virsh_node2_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="6:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e" transition-magic="-1:193;6:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-statu
  346. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ </lrm_resource>
  347. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/31, version=0.15.6)
  348. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating monitor operation virsh_node2_monitor_0 on el8-a01n01 | action 2
  349. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_lrm_event) notice: Result of probe operation for virsh_node1 on el8-a01n02: 7 (not running) | call=5 key=virsh_node1_monitor_0 confirmed=true cib-update=32
  350. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/32)
  351. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_lrm_event) notice: Result of probe operation for virsh_node2 on el8-a01n02: 7 (not running) | call=9 key=virsh_node2_monitor_0 confirmed=true cib-update=33
  352. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.6 2
  353. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.7 (null)
  354. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=7
  355. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;5:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=5, @rc-code=7, @op-status=0, @exec-time=5
  356. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 5 (virsh_node1_monitor_0 on el8-a01n02) confirmed: not running | rc=7 call-id=5
  357. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating monitor operation hypervisor:1_monitor_0 locally on el8-a01n02 | action 7
  358. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor' not in cache
  359. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=0
  360. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor:1' not in cache
  361. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=0
  362. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_rsc_register) info: Cached agent information for 'hypervisor'
  363. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=1
  364. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/32, version=0.15.7)
  365. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/33)
  366. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.7 2
  367. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.8 (null)
  368. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=8
  369. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;6:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=9, @rc-code=7, @op-status=0
  370. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/33, version=0.15.8)
  371. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=0
  372. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_lrm_rsc_op) info: Performing key=7:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e op=hypervisor_monitor_0
  373. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=14, reply=1, notify=0
  374. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:14
  375. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  376. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/34)
  377. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.8 2
  378. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.9 (null)
  379. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=9
  380. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="hypervisor" type="libvirtd" class="systemd"/>
  381. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ <lrm_rsc_op id="hypervisor_last_0" operation_key="hypervisor_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="7:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e" transition-magic="-1:193;7:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-status=
  382. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ </lrm_resource>
  383. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/34, version=0.15.9)
  384. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 6 (virsh_node2_monitor_0 on el8-a01n02) confirmed: not running | rc=7 call-id=9
  385. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating monitor operation drbd:1_monitor_0 locally on el8-a01n02 | action 8
  386. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_get_rsc_info) info: Agent information for 'drbd' not in cache
  387. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=0
  388. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_get_rsc_info) info: Agent information for 'drbd:1' not in cache
  389. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=0
  390. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_rsc_register) info: Cached agent information for 'drbd'
  391. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=1
  392. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=0
  393. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_lrm_rsc_op) info: Performing key=8:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e op=drbd_monitor_0
  394. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=19, reply=1, notify=0
  395. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:19
  396. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  397. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/35)
  398. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.9 2
  399. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.10 (null)
  400. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=10
  401. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="drbd" type="drbd" class="systemd"/>
  402. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ <lrm_rsc_op id="drbd_last_0" operation_key="drbd_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="8:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e" transition-magic="-1:193;8:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-status="-1" interva
  403. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ </lrm_resource>
  404. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/35, version=0.15.10)
  405. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  406. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  407. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:14 exit-code:7 exec-time:45ms queue-time:0ms
  408. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_lrm_event) notice: Result of probe operation for hypervisor on el8-a01n02: 7 (not running) | call=14 key=hypervisor_monitor_0 confirmed=true cib-update=36
  409. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/36)
  410. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.10 2
  411. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.11 (null)
  412. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=11
  413. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;7:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=14, @rc-code=7, @op-status=0, @exec-time=45
  414. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 7 (hypervisor_monitor_0 on el8-a01n02) confirmed: not running | rc=7 call-id=14
  415. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/36, version=0.15.11)
  416. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) debug: finished - rsc:drbd action:monitor call_id:19 exit-code:7 exec-time:37ms queue-time:0ms
  417. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_lrm_event) notice: Result of probe operation for drbd on el8-a01n02: 7 (not running) | call=19 key=drbd_monitor_0 confirmed=true cib-update=37
  418. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/37)
  419. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.11 2
  420. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.12 (null)
  421. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=12
  422. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;8:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=19, @rc-code=7, @op-status=0, @exec-time=37
  423. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/37, version=0.15.12)
  424. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 8 (drbd_monitor_0 on el8-a01n02) confirmed: not running | rc=7 call-id=19
  425. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.12 2
  426. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.13 (null)
  427. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=13
  428. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource
  429. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="virsh_node1" type="fence_virsh" class="stonith"/>
  430. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node1_last_0" operation_key="virsh_node1_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="1:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e" transition-magic="-1:193;1:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-statu
  431. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ </lrm_resource>
  432. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/9, version=0.15.13)
  433. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.13 2
  434. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.14 (null)
  435. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=14
  436. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="virsh_node2" type="fence_virsh" class="stonith"/>
  437. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node2_last_0" operation_key="virsh_node2_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="2:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e" transition-magic="-1:193;2:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-statu
  438. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ </lrm_resource>
  439. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/10, version=0.15.14)
  440. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.14 2
  441. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.15 (null)
  442. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=15
  443. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;1:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=5, @rc-code=7, @op-status=0, @exec-time=5
  444. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/11, version=0.15.15)
  445. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 1 (virsh_node1_monitor_0 on el8-a01n01) confirmed: not running | rc=7 call-id=5
  446. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating start operation virsh_node1_start_0 on el8-a01n01 | action 9
  447. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.15 2
  448. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.16 (null)
  449. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=16
  450. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;2:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=9, @rc-code=7, @op-status=0
  451. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/12, version=0.15.16)
  452. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 2 (virsh_node2_monitor_0 on el8-a01n01) confirmed: not running | rc=7 call-id=9
  453. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating start operation virsh_node2_start_0 locally on el8-a01n02 | action 11
  454. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_lrm_rsc_op) info: Performing key=11:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e op=virsh_node2_start_0
  455. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=20, reply=1, notify=0
  456. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) info: executing - rsc:virsh_node2 action:start call_id:20
  457. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating monitor operation hypervisor:0_monitor_0 on el8-a01n01 | action 3
  458. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.16 2
  459. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.17 (null)
  460. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=17
  461. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_start_0, @operation=start, @transition-key=9:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;9:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0
  462. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/13, version=0.15.17)
  463. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/38)
  464. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.17 2
  465. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.18 (null)
  466. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=18
  467. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=11:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;11:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1
  468. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/38, version=0.15.18)
  469. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.18 2
  470. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.19 (null)
  471. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=19
  472. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="hypervisor" type="libvirtd" class="systemd"/>
  473. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ <lrm_rsc_op id="hypervisor_last_0" operation_key="hypervisor_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="3:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e" transition-magic="-1:193;3:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-status=
  474. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ </lrm_resource>
  475. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/14, version=0.15.19)
  476. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.19 2
  477. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.20 (null)
  478. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=20
  479. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;3:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=15, @rc-code=7, @op-status=0, @exec-time=78
  480. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/15, version=0.15.20)
  481. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 3 (hypervisor_monitor_0 on el8-a01n01) confirmed: not running | rc=7 call-id=15
  482. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating monitor operation drbd:0_monitor_0 on el8-a01n01 | action 4
  483. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating start operation hypervisor:1_start_0 locally on el8-a01n02 | action 15
  484. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_lrm_rsc_op) info: Performing key=15:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e op=hypervisor_start_0
  485. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=21, reply=1, notify=0
  486. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) info: executing - rsc:hypervisor action:start call_id:21
  487. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_unit_exec) debug: Performing asynchronous start op on systemd unit libvirtd named 'hypervisor'
  488. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/39)
  489. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.20 2
  490. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.21 (null)
  491. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=21
  492. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="drbd" type="drbd" class="systemd"/>
  493. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ <lrm_rsc_op id="drbd_last_0" operation_key="drbd_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="4:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e" transition-magic="-1:193;4:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-status="-1" interva
  494. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ </lrm_resource>
  495. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/16, version=0.15.21)
  496. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.21 2
  497. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.22 (null)
  498. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=22
  499. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=15:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;15:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0
  500. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/39, version=0.15.22)
  501. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_unit_exec_with_unit) debug: Calling StartUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice
  502. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.22 2
  503. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.23 (null)
  504. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=23
  505. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;4:0:7:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=20, @rc-code=7, @op-status=0, @exec-time=16
  506. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/17, version=0.15.23)
  507. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 4 (drbd_monitor_0 on el8-a01n01) confirmed: not running | rc=7 call-id=20
  508. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating start operation hypervisor:0_start_0 on el8-a01n01 | action 13
  509. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.23 2
  510. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.24 (null)
  511. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=24
  512. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=13:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;13:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0
  513. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/18, version=0.15.24)
  514. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_exec_result) info: Call to start passed: /org/freedesktop/systemd1/job/826
  515. Jul 14 22:18:56 el8-a01n02.digimer.ca pacemaker-execd [1567] (action_complete) debug: hypervisor start may still be in progress: re-scheduling (elapsed=270ms, remaining=99730ms, start_delay=2000ms)
  516. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) info: finished - rsc:virsh_node2 action:start call_id:20 exit-code:0 exec-time:810ms queue-time:0ms
  517. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_lrm_event) notice: Result of start operation for virsh_node2 on el8-a01n02: 0 (ok) | call=20 key=virsh_node2_start_0 confirmed=true cib-update=40
  518. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/40)
  519. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.24 2
  520. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.25 (null)
  521. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=25
  522. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;11:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=20, @rc-code=0, @op-status=0, @exec-time=810
  523. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/40, version=0.15.25)
  524. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 11 (virsh_node2_start_0 on el8-a01n02) confirmed: ok | rc=0 call-id=20
  525. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating monitor operation virsh_node2_monitor_60000 locally on el8-a01n02 | action 12
  526. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_lrm_rsc_op) info: Performing key=12:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e op=virsh_node2_monitor_60000
  527. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=22, reply=1, notify=0
  528. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) debug: executing - rsc:virsh_node2 action:monitor call_id:22
  529. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/41)
  530. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.25 2
  531. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.26 (null)
  532. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=26
  533. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']: <lrm_rsc_op id="virsh_node2_monitor_60000" operation_key="virsh_node2_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="12:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e" transition-magic="-1:193;12:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e" exit-reason="" on_node="el8-
  534. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/41, version=0.15.26)
  535. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.26 2
  536. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.27 (null)
  537. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=27
  538. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:0;9:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=10, @rc-code=0, @op-status=0, @exec-time=962, @queue-time=1
  539. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/19, version=0.15.27)
  540. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 9 (virsh_node1_start_0 on el8-a01n01) confirmed: ok | rc=0 call-id=10
  541. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating monitor operation virsh_node1_monitor_60000 on el8-a01n01 | action 10
  542. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.27 2
  543. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.28 (null)
  544. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=28
  545. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']: <lrm_rsc_op id="virsh_node1_monitor_60000" operation_key="virsh_node1_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="10:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e" transition-magic="-1:193;10:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e" exit-reason="" on_node="el8-
  546. Jul 14 22:18:57 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/20, version=0.15.28)
  547. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) debug: finished - rsc:virsh_node2 action:monitor call_id:22 exit-code:0 exec-time:765ms queue-time:0ms
  548. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/42)
  549. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_lrm_event) notice: Result of monitor operation for virsh_node2 on el8-a01n02: 0 (ok) | call=22 key=virsh_node2_monitor_60000 confirmed=false cib-update=42
  550. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.28 2
  551. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.29 (null)
  552. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=29
  553. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_monitor_60000']: @transition-magic=0:0;12:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=22, @rc-code=0, @op-status=0, @exec-time=765
  554. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/42, version=0.15.29)
  555. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 12 (virsh_node2_monitor_60000 on el8-a01n02) confirmed: ok | rc=0 call-id=22
  556. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating start operation drbd:1_start_0 locally on el8-a01n02 | action 23
  557. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_lrm_rsc_op) info: Performing key=23:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e op=drbd_start_0
  558. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/43)
  559. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.29 2
  560. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.30 (null)
  561. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=30
  562. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_start_0, @operation=start, @transition-key=23:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;23:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765138, @last-run=1594765138, @exec-time=0
  563. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=23, reply=1, notify=0
  564. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) info: executing - rsc:drbd action:start call_id:23
  565. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_unit_exec) debug: Performing asynchronous start op on systemd unit drbd named 'drbd'
  566. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/43, version=0.15.30)
  567. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_unit_exec_with_unit) debug: Calling StartUnit for drbd: /org/freedesktop/systemd1/unit/drbd_2eservice
  568. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.30 2
  569. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.31 (null)
  570. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=31
  571. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_monitor_60000']: @transition-magic=0:0;10:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=22, @rc-code=0, @op-status=0, @exec-time=870, @queue-time=1
  572. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/21, version=0.15.31)
  573. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 10 (virsh_node1_monitor_60000 on el8-a01n01) confirmed: ok | rc=0 call-id=22
  574. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating start operation drbd:0_start_0 on el8-a01n01 | action 21
  575. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_exec_result) info: Call to start passed: /org/freedesktop/systemd1/job/910
  576. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (action_complete) debug: drbd start may still be in progress: re-scheduling (elapsed=264ms, remaining=99736ms, start_delay=2000ms)
  577. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.31 2
  578. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.32 (null)
  579. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=32
  580. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_start_0, @operation=start, @transition-key=21:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;21:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765138, @last-run=1594765138, @exec-time=0
  581. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/22, version=0.15.32)
  582. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:21
  583. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  584. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  585. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (action_complete) debug: hypervisor start is now complete (elapsed=2275ms, remaining=97725ms): ok (0)
  586. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:21 exit-code:0 exec-time:2275ms queue-time:0ms
  587. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_lrm_event) notice: Result of start operation for hypervisor on el8-a01n02: 0 (ok) | call=21 key=hypervisor_start_0 confirmed=true cib-update=44
  588. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/44)
  589. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.32 2
  590. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.33 (null)
  591. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=33
  592. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;15:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=21, @rc-code=0, @op-status=0, @exec-time=2275
  593. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 15 (hypervisor_start_0 on el8-a01n02) confirmed: ok | rc=0 call-id=21
  594. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating monitor operation hypervisor:1_monitor_60000 locally on el8-a01n02 | action 16
  595. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/44, version=0.15.33)
  596. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_lrm_rsc_op) info: Performing key=16:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e op=hypervisor_monitor_60000
  597. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=24, reply=1, notify=0
  598. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:24
  599. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  600. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/45)
  601. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.33 2
  602. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.34 (null)
  603. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=34
  604. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']: <lrm_rsc_op id="hypervisor_monitor_60000" operation_key="hypervisor_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="16:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e" transition-magic="-1:193;16:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e" exit-reason="" on_node="el8-a01
  605. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/45, version=0.15.34)
  606. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  607. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:5ms queue-time:0ms
  608. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_lrm_event) notice: Result of monitor operation for hypervisor on el8-a01n02: 0 (ok) | call=24 key=hypervisor_monitor_60000 confirmed=false cib-update=46
  609. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/46)
  610. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.34 2
  611. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.35 (null)
  612. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=35
  613. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_monitor_60000']: @transition-magic=0:0;16:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=24, @rc-code=0, @op-status=0, @exec-time=5
  614. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/46, version=0.15.35)
  615. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 16 (hypervisor_monitor_60000 on el8-a01n02) confirmed: ok | rc=0 call-id=24
  616. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.35 2
  617. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.36 (null)
  618. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=36
  619. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;13:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=21, @rc-code=0, @op-status=0, @exec-time=2346
  620. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 13 (hypervisor_start_0 on el8-a01n01) confirmed: ok | rc=0 call-id=21
  621. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating monitor operation hypervisor:0_monitor_60000 on el8-a01n01 | action 14
  622. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/23, version=0.15.36)
  623. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.36 2
  624. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.37 (null)
  625. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=37
  626. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']: <lrm_rsc_op id="hypervisor_monitor_60000" operation_key="hypervisor_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="14:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e" transition-magic="-1:193;14:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e" exit-reason="" on_node="el8-a01
  627. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/24, version=0.15.37)
  628. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.37 2
  629. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.38 (null)
  630. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=38
  631. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_monitor_60000']: @transition-magic=0:0;14:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=24, @rc-code=0, @op-status=0, @exec-time=10
  632. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/25, version=0.15.38)
  633. Jul 14 22:18:58 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 14 (hypervisor_monitor_60000 on el8-a01n01) confirmed: ok | rc=0 call-id=24
  634. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:23
  635. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  636. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-execd [1567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  637. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-execd [1567] (action_complete) debug: drbd start is now complete (elapsed=2267ms, remaining=97733ms): ok (0)
  638. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) debug: finished - rsc:drbd action:monitor call_id:23 exit-code:0 exec-time:2267ms queue-time:0ms
  639. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_lrm_event) notice: Result of start operation for drbd on el8-a01n02: 0 (ok) | call=23 key=drbd_start_0 confirmed=true cib-update=47
  640. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/47)
  641. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.38 2
  642. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.39 (null)
  643. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=39
  644. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;23:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=23, @rc-code=0, @op-status=0, @exec-time=2267
  645. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/47, version=0.15.39)
  646. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 23 (drbd_start_0 on el8-a01n02) confirmed: ok | rc=0 call-id=23
  647. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating monitor operation drbd:1_monitor_60000 locally on el8-a01n02 | action 24
  648. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_lrm_rsc_op) info: Performing key=24:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e op=drbd_monitor_60000
  649. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=25, reply=1, notify=0
  650. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:25
  651. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/48)
  652. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  653. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.39 2
  654. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.40 (null)
  655. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=40
  656. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']: <lrm_rsc_op id="drbd_monitor_60000" operation_key="drbd_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="24:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e" transition-magic="-1:193;24:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e" exit-reason="" on_node="el8-a01n02" call-id="-1"
  657. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/48, version=0.15.40)
  658. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-execd [1567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  659. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) debug: finished - rsc:drbd action:monitor call_id:25 exit-code:0 exec-time:4ms queue-time:0ms
  660. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_lrm_event) notice: Result of monitor operation for drbd on el8-a01n02: 0 (ok) | call=25 key=drbd_monitor_60000 confirmed=false cib-update=49
  661. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/49)
  662. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.40 2
  663. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.41 (null)
  664. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=41
  665. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_monitor_60000']: @transition-magic=0:0;24:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=25, @rc-code=0, @op-status=0, @exec-time=4
  666. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/49, version=0.15.41)
  667. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 24 (drbd_monitor_60000 on el8-a01n02) confirmed: ok | rc=0 call-id=25
  668. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.41 2
  669. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.42 (null)
  670. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=42
  671. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;21:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=23, @rc-code=0, @op-status=0, @exec-time=2218, @queue-time=1
  672. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/26, version=0.15.42)
  673. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 21 (drbd_start_0 on el8-a01n01) confirmed: ok | rc=0 call-id=23
  674. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating monitor operation drbd:0_monitor_60000 on el8-a01n01 | action 22
  675. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.42 2
  676. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.43 (null)
  677. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=43
  678. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']: <lrm_rsc_op id="drbd_monitor_60000" operation_key="drbd_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="22:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e" transition-magic="-1:193;22:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e" exit-reason="" on_node="el8-a01n01" call-id="-1"
  679. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/27, version=0.15.43)
  680. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.43 2
  681. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.15.44 (null)
  682. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=44
  683. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_monitor_60000']: @transition-magic=0:0;22:0:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=25, @rc-code=0, @op-status=0, @exec-time=6
  684. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/28, version=0.15.44)
  685. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 0 action 22 (drbd_monitor_60000 on el8-a01n01) confirmed: ok | rc=0 call-id=25
  686. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-controld [1570] (run_graph) notice: Transition 0 (Complete=24, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-7.bz2): Complete
  687. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_log) info: Input I_TE_SUCCESS received in state S_TRANSITION_ENGINE from notify_crmd
  688. Jul 14 22:19:00 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_state_transition) notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd
  689. Jul 14 22:19:05 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_ping) info: Reporting our current digest to el8-a01n02: 6d5e9e37570d7b60632f978059877856 for 0.15.44 (0x55b6e7f46990 0)
  690. Jul 14 22:19:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_procfs_pid_of) info: Found pacemaker-based active as process 1565
  691. Jul 14 22:19:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (throttle_send_command) info: New throttle mode: negligible load (was undetermined)
  692. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.15.44 2
  693. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.0 3d93222d8ce2098e91b58274347b8c85
  694. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @epoch=16, @num_updates=0
  695. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/configuration/resources/clone[@id='drbd-clone']/primitive[@id='drbd']/meta_attributes[@id='drbd-meta_attributes']: <nvpair id="drbd-meta_attributes-target-role" name="target-role" value="Stopped"/>
  696. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.16.0)
  697. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-controld [1570] (abort_transition_graph) info: Transition 0 aborted by drbd-meta_attributes-target-role doing create target-role=Stopped: Configuration change | cib=0.16.0 source=te_update_diff_v2:490 path=/cib/configuration/resources/clone[@id='drbd-clone']/primitive[@id='drbd']/meta_attributes[@id='drbd-meta_attributes'] complete=true
  698. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-fenced [1566] (update_cib_stonith_devices_v2) info: Updating device list from the cib: create meta_attributes[@id='drbd-meta_attributes']
  699. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-fenced [1566] (cib_devices_update) info: Updating devices to version 0.16.0
  700. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_state_transition) notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph
  701. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-23.raw
  702. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (determine_online_status_fencing) info: Node el8-a01n01 is active
  703. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (determine_online_status) info: Node el8-a01n01 is online
  704. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (determine_online_status_fencing) info: Node el8-a01n02 is active
  705. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (determine_online_status) info: Node el8-a01n02 is online
  706. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (unpack_node_loop) info: Node 1 is already processed
  707. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (unpack_node_loop) info: Node 2 is already processed
  708. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (unpack_node_loop) info: Node 1 is already processed
  709. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (unpack_node_loop) info: Node 2 is already processed
  710. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (common_print) info: virsh_node1 (stonith:fence_virsh): Started el8-a01n01
  711. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (common_print) info: virsh_node2 (stonith:fence_virsh): Started el8-a01n02
  712. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  713. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (short_print) info: Started: [ el8-a01n01 el8-a01n02 ]
  714. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (clone_print) info: Clone Set: drbd-clone [drbd]
  715. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (short_print) info: Started: [ el8-a01n01 el8-a01n02 ]
  716. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (native_color) info: Resource drbd:0 cannot run anywhere
  717. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (native_color) info: Resource drbd:1 cannot run anywhere
  718. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogActions) info: Leave virsh_node1 (Started el8-a01n01)
  719. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogActions) info: Leave virsh_node2 (Started el8-a01n02)
  720. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogActions) info: Leave hypervisor:0 (Started el8-a01n01)
  721. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogActions) info: Leave hypervisor:1 (Started el8-a01n02)
  722. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogAction) notice: * Stop drbd:0 ( el8-a01n01 ) due to node availability
  723. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogAction) notice: * Stop drbd:1 ( el8-a01n02 ) due to node availability
  724. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (pcmk__log_transition_summary) notice: Calculated transition 1, saving inputs in /var/lib/pacemaker/pengine/pe-input-8.bz2
  725. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response
  726. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_te_invoke) info: Processing graph 1 (ref=pe_calc-dc-1594765184-31) derived from /var/lib/pacemaker/pengine/pe-input-8.bz2
  727. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating stop operation drbd_stop_0 on el8-a01n01 | action 19
  728. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating stop operation drbd_stop_0 locally on el8-a01n02 | action 20
  729. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-execd [1567] (cancel_recurring_action) info: Cancelling systemd operation drbd_status_60000
  730. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) debug: finished - rsc:drbd action:monitor call_id:25 exit-code:0 exec-time:0ms queue-time:0ms
  731. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=0
  732. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_lrm_rsc_op) info: Performing key=20:1:0:22298a5d-b80a-4cd7-adde-aba652405a4e op=drbd_stop_0
  733. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/51)
  734. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_file_write_with_digest) info: Wrote version 0.16.0 of the CIB to disk (digest: 9ac5ee247228cbebb26d752a5874dee5)
  735. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.0 2
  736. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.1 (null)
  737. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=1
  738. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_stop_0, @operation=stop, @transition-key=20:1:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;20:1:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765184, @last-run=1594765184, @exec-time=0
  739. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.RQ0HBh (digest: /var/lib/pacemaker/cib/cib.rif2Tq)
  740. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/51, version=0.16.1)
  741. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.1 2
  742. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.2 (null)
  743. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=2
  744. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_stop_0, @operation=stop, @transition-key=19:1:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;19:1:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765184, @last-run=1594765184, @exec-time=0, @queue-time=0
  745. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/29, version=0.16.2)
  746. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=27, reply=1, notify=0
  747. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) info: executing - rsc:drbd action:stop call_id:27
  748. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit drbd named 'drbd'
  749. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_lrm_event) info: Result of monitor operation for drbd on el8-a01n02: Cancelled | call=25 key=drbd_monitor_60000 confirmed=true
  750. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_unit_exec_with_unit) debug: Calling StopUnit for drbd: /org/freedesktop/systemd1/unit/drbd_2eservice
  751. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/1004
  752. Jul 14 22:19:44 el8-a01n02.digimer.ca pacemaker-execd [1567] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=197ms, remaining=99803ms, start_delay=2000ms)
  753. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) debug: executing - rsc:drbd action:monitor call_id:27
  754. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  755. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-execd [1567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  756. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-execd [1567] (action_complete) debug: drbd stop is now complete (elapsed=2205ms, remaining=97795ms): not running (7)
  757. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) debug: finished - rsc:drbd action:monitor call_id:27 exit-code:0 exec-time:2205ms queue-time:0ms
  758. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_lrm_event) notice: Result of stop operation for drbd on el8-a01n02: 0 (ok) | call=27 key=drbd_stop_0 confirmed=true cib-update=52
  759. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/52)
  760. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.2 2
  761. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.3 (null)
  762. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=3
  763. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;20:1:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=27, @rc-code=0, @op-status=0, @exec-time=2205
  764. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/52, version=0.16.3)
  765. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 1 action 20 (drbd_stop_0 on el8-a01n02) confirmed: ok | rc=0 call-id=27
  766. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.3 2
  767. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.4 (null)
  768. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=4
  769. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;19:1:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=27, @rc-code=0, @op-status=0, @exec-time=2244
  770. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/30, version=0.16.4)
  771. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 1 action 19 (drbd_stop_0 on el8-a01n01) confirmed: ok | rc=0 call-id=27
  772. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-controld [1570] (run_graph) notice: Transition 1 (Complete=4, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-8.bz2): Complete
  773. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_log) info: Input I_TE_SUCCESS received in state S_TRANSITION_ENGINE from notify_crmd
  774. Jul 14 22:19:46 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_state_transition) notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd
  775. Jul 14 22:19:51 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_ping) info: Reporting our current digest to el8-a01n02: c9fc5e41d0d91479ef3a8c683462f6ab for 0.16.4 (0x55b6e80d0050 0)
  776. Jul 14 22:19:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) debug: executing - rsc:virsh_node2 action:monitor call_id:22
  777. Jul 14 22:19:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) debug: finished - rsc:virsh_node2 action:monitor call_id:22 exit-code:0 exec-time:645ms queue-time:0ms
  778. Jul 14 22:19:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (recurring_action_timer) debug: Scheduling another invocation of hypervisor_status_60000
  779. Jul 14 22:19:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  780. Jul 14 22:19:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  781. Jul 14 22:19:58 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  782. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemakerd [1564] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  783. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk_shutdown_worker) notice: Shutting down Pacemaker
  784. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemakerd [1564] (stop_child) notice: Stopping pacemaker-controld | sent signal 15 to process 1570
  785. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  786. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_shutdown) notice: Shutting down cluster resource manager | limit=1200000ms
  787. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_log) info: Input I_SHUTDOWN received in state S_IDLE from crm_shutdown
  788. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_state_transition) notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_SHUTDOWN cause=C_SHUTDOWN origin=crm_shutdown
  789. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_shutdown_req) info: Sending shutdown request to all peers (DC is el8-a01n02)
  790. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_update_peer_expected) info: handle_request: Node el8-a01n02[2] - expected state is now down (was member)
  791. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (handle_shutdown_request) info: Creating shutdown request for el8-a01n02 (state=S_POLICY_ENGINE)
  792. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-attrd [1568] (attrd_peer_update) notice: Setting shutdown[el8-a01n02]: (unset) -> 1594765206 | from el8-a01n02
  793. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-attrd [1568] (write_attribute) info: Sent CIB request 4 with 1 change for shutdown (id n/a, set n/a)
  794. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/4)
  795. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.4 2
  796. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.5 (null)
  797. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=5
  798. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']: <transient_attributes id="2"/>
  799. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ <instance_attributes id="status-2">
  800. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ <nvpair id="status-2-shutdown" name="shutdown" value="1594765206"/>
  801. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ </instance_attributes>
  802. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ </transient_attributes>
  803. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/4, version=0.16.5)
  804. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (abort_transition_graph) info: Transition 1 aborted by transient_attributes.2 'create': Transient attribute change | cib=0.16.5 source=abort_unless_down:329 path=/cib/status/node_state[@id='2'] complete=true
  805. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-attrd [1568] (attrd_cib_callback) info: CIB update 4 result for shutdown: OK | rc=0
  806. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-attrd [1568] (attrd_cib_callback) info: * shutdown[el8-a01n02]=1594765206
  807. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (determine_online_status_fencing) info: Node el8-a01n01 is active
  808. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (determine_online_status) info: Node el8-a01n01 is online
  809. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (determine_online_status) info: Node el8-a01n02 is shutting down
  810. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (unpack_node_loop) info: Node 1 is already processed
  811. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (unpack_node_loop) info: Node 2 is already processed
  812. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (unpack_node_loop) info: Node 1 is already processed
  813. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (unpack_node_loop) info: Node 2 is already processed
  814. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (common_print) info: virsh_node1 (stonith:fence_virsh): Started el8-a01n01
  815. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (common_print) info: virsh_node2 (stonith:fence_virsh): Started el8-a01n02
  816. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  817. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (short_print) info: Started: [ el8-a01n01 el8-a01n02 ]
  818. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (clone_print) info: Clone Set: drbd-clone [drbd]
  819. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (short_print) info: Stopped (disabled): [ el8-a01n01 el8-a01n02 ]
  820. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (native_color) info: Resource hypervisor:1 cannot run anywhere
  821. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (native_color) info: Resource drbd:0 cannot run anywhere
  822. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (native_color) info: Resource drbd:1 cannot run anywhere
  823. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (RecurringOp) info: Start recurring monitor (60s) for virsh_node2 on el8-a01n01
  824. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (sched_shutdown_op) notice: Scheduling shutdown of node el8-a01n02
  825. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogNodeActions) notice: * Shutdown el8-a01n02
  826. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogActions) info: Leave virsh_node1 (Started el8-a01n01)
  827. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogAction) notice: * Move virsh_node2 ( el8-a01n02 -> el8-a01n01 )
  828. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogActions) info: Leave hypervisor:0 (Started el8-a01n01)
  829. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogAction) notice: * Stop hypervisor:1 ( el8-a01n02 ) due to node availability
  830. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogActions) info: Leave drbd:0 (Stopped)
  831. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogActions) info: Leave drbd:1 (Stopped)
  832. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (pcmk__log_transition_summary) notice: Calculated transition 2, saving inputs in /var/lib/pacemaker/pengine/pe-input-9.bz2
  833. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response
  834. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_te_invoke) info: Processing graph 2 (ref=pe_calc-dc-1594765206-35) derived from /var/lib/pacemaker/pengine/pe-input-9.bz2
  835. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating stop operation virsh_node2_stop_0 locally on el8-a01n02 | action 7
  836. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) debug: finished - rsc:virsh_node2 action:monitor call_id:22 exit-code:0 exec-time:0ms queue-time:0ms
  837. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=0
  838. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_lrm_rsc_op) info: Performing key=7:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e op=virsh_node2_stop_0
  839. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/54)
  840. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=29, reply=1, notify=0
  841. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) info: executing - rsc:virsh_node2 action:stop call_id:29
  842. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) info: finished - rsc:virsh_node2 action:stop call_id:29 exit-code:0 exec-time:0ms queue-time:0ms
  843. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.5 2
  844. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.6 (null)
  845. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=6
  846. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=7:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;7:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765206, @last-run=1594765206, @exec-time=0
  847. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/54, version=0.16.6)
  848. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_lrm_event) info: Result of monitor operation for virsh_node2 on el8-a01n02: Cancelled | call=22 key=virsh_node2_monitor_60000 confirmed=true
  849. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_lrm_event) notice: Result of stop operation for virsh_node2 on el8-a01n02: 0 (ok) | call=29 key=virsh_node2_stop_0 confirmed=true cib-update=55
  850. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/55)
  851. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.6 2
  852. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.7 (null)
  853. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=7
  854. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;7:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=29, @rc-code=0, @op-status=0
  855. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/55, version=0.16.7)
  856. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 2 action 7 (virsh_node2_stop_0 on el8-a01n02) confirmed: ok | rc=0 call-id=29
  857. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating start operation virsh_node2_start_0 on el8-a01n01 | action 8
  858. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating stop operation hypervisor_stop_0 locally on el8-a01n02 | action 12
  859. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-execd [1567] (cancel_recurring_action) info: Cancelling systemd operation hypervisor_status_60000
  860. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  861. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=0, reply=1, notify=0
  862. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_lrm_rsc_op) info: Performing key=12:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e op=hypervisor_stop_0
  863. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-execd [1567] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 801acf2c-48e6-470c-97f7-eae1b7d56a84: rc=31, reply=1, notify=0
  864. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) info: executing - rsc:hypervisor action:stop call_id:31
  865. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit libvirtd named 'hypervisor'
  866. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.7 2
  867. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.8 (null)
  868. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=8
  869. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=8:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;8:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765206, @last-run=1594765206
  870. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_unit_exec_with_unit) debug: Calling StopUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice
  871. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/31, version=0.16.8)
  872. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/56)
  873. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.8 2
  874. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.9 (null)
  875. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=9
  876. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=12:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;12:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765206, @last-run=1594765206, @exec-time=0
  877. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_lrm_event) info: Result of monitor operation for hypervisor on el8-a01n02: Cancelled | call=24 key=hypervisor_monitor_60000 confirmed=true
  878. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/56, version=0.16.9)
  879. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_update_peer_expected) info: handle_request: Node el8-a01n01[1] - expected state is now down (was member)
  880. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (handle_shutdown_request) info: Creating shutdown request for el8-a01n01 (state=S_TRANSITION_ENGINE)
  881. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-attrd [1568] (attrd_peer_update) notice: Setting shutdown[el8-a01n01]: (unset) -> 1594765206 | from el8-a01n02
  882. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-attrd [1568] (write_attribute) info: Sent CIB request 5 with 2 changes for shutdown (id n/a, set n/a)
  883. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/5)
  884. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.9 2
  885. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.10 (null)
  886. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=10
  887. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']: <transient_attributes id="1"/>
  888. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ <instance_attributes id="status-1">
  889. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ <nvpair id="status-1-shutdown" name="shutdown" value="1594765206"/>
  890. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ </instance_attributes>
  891. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: ++ </transient_attributes>
  892. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/5, version=0.16.10)
  893. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-controld [1570] (abort_transition_graph) notice: Transition 2 aborted by transient_attributes.1 'create': Transient attribute change | cib=0.16.10 source=abort_unless_down:329 path=/cib/status/node_state[@id='1'] complete=false
  894. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-attrd [1568] (attrd_cib_callback) info: CIB update 5 result for shutdown: OK | rc=0
  895. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-attrd [1568] (attrd_cib_callback) info: * shutdown[el8-a01n01]=1594765206
  896. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-attrd [1568] (attrd_cib_callback) info: * shutdown[el8-a01n02]=1594765206
  897. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/1011
  898. Jul 14 22:20:06 el8-a01n02.digimer.ca pacemaker-execd [1567] (action_complete) debug: hypervisor stop may still be in progress: re-scheduling (elapsed=192ms, remaining=99808ms, start_delay=2000ms)
  899. Jul 14 22:20:07 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.10 2
  900. Jul 14 22:20:07 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.11 (null)
  901. Jul 14 22:20:07 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=11
  902. Jul 14 22:20:07 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;8:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=28, @rc-code=0, @op-status=0, @exec-time=697
  903. Jul 14 22:20:07 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/32, version=0.16.11)
  904. Jul 14 22:20:07 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 2 action 8 (virsh_node2_start_0 on el8-a01n01) confirmed: ok | rc=0 call-id=28
  905. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:31
  906. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-execd [1567] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  907. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-execd [1567] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  908. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-execd [1567] (action_complete) debug: hypervisor stop is now complete (elapsed=2198ms, remaining=97802ms): not running (7)
  909. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-execd [1567] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:31 exit-code:0 exec-time:2198ms queue-time:0ms
  910. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_lrm_event) notice: Result of stop operation for hypervisor on el8-a01n02: 0 (ok) | call=31 key=hypervisor_stop_0 confirmed=true cib-update=57
  911. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/57)
  912. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.11 2
  913. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.12 (null)
  914. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=12
  915. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;12:2:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=31, @rc-code=0, @op-status=0, @exec-time=2198
  916. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 2 action 12 (hypervisor_stop_0 on el8-a01n02) confirmed: ok | rc=0 call-id=31
  917. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/57, version=0.16.12)
  918. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-controld [1570] (run_graph) notice: Transition 2 (Complete=5, Pending=0, Fired=0, Skipped=2, Incomplete=2, Source=/var/lib/pacemaker/pengine/pe-input-9.bz2): Stopped
  919. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_state_transition) info: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd
  920. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (determine_online_status) info: Node el8-a01n01 is shutting down
  921. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (determine_online_status) info: Node el8-a01n02 is shutting down
  922. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (unpack_node_loop) info: Node 1 is already processed
  923. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (unpack_node_loop) info: Node 2 is already processed
  924. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (unpack_node_loop) info: Node 1 is already processed
  925. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (unpack_node_loop) info: Node 2 is already processed
  926. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (common_print) info: virsh_node1 (stonith:fence_virsh): Started el8-a01n01
  927. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (common_print) info: virsh_node2 (stonith:fence_virsh): Started el8-a01n01
  928. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  929. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (short_print) info: Started: [ el8-a01n01 ]
  930. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (short_print) info: Stopped: [ el8-a01n02 ]
  931. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (clone_print) info: Clone Set: drbd-clone [drbd]
  932. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (short_print) info: Stopped (disabled): [ el8-a01n01 el8-a01n02 ]
  933. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (native_color) info: Resource virsh_node1 cannot run anywhere
  934. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (native_color) info: Resource virsh_node2 cannot run anywhere
  935. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (native_color) info: Resource hypervisor:0 cannot run anywhere
  936. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (native_color) info: Resource hypervisor:1 cannot run anywhere
  937. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (native_color) info: Resource drbd:0 cannot run anywhere
  938. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (native_color) info: Resource drbd:1 cannot run anywhere
  939. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (sched_shutdown_op) notice: Scheduling shutdown of node el8-a01n01
  940. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (sched_shutdown_op) notice: Scheduling shutdown of node el8-a01n02
  941. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogNodeActions) notice: * Shutdown el8-a01n02
  942. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogNodeActions) notice: * Shutdown el8-a01n01
  943. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogAction) notice: * Stop virsh_node1 ( el8-a01n01 ) due to node availability
  944. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogAction) notice: * Stop virsh_node2 ( el8-a01n01 ) due to node availability
  945. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogAction) notice: * Stop hypervisor:0 ( el8-a01n01 ) due to node availability
  946. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogActions) info: Leave hypervisor:1 (Stopped)
  947. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogActions) info: Leave drbd:0 (Stopped)
  948. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (LogActions) info: Leave drbd:1 (Stopped)
  949. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (pcmk__log_transition_summary) notice: Calculated transition 3, saving inputs in /var/lib/pacemaker/pengine/pe-input-10.bz2
  950. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response
  951. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_te_invoke) info: Processing graph 3 (ref=pe_calc-dc-1594765208-39) derived from /var/lib/pacemaker/pengine/pe-input-10.bz2
  952. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating stop operation virsh_node1_stop_0 on el8-a01n01 | action 3
  953. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating stop operation virsh_node2_stop_0 on el8-a01n01 | action 4
  954. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_rsc_command) notice: Initiating stop operation hypervisor_stop_0 on el8-a01n01 | action 5
  955. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.12 2
  956. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.13 (null)
  957. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=13
  958. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_stop_0, @operation=stop, @transition-key=3:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;3:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765208, @last-run=1594765208, @exec-time=0, @q
  959. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/33, version=0.16.13)
  960. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.13 2
  961. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.14 (null)
  962. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=14
  963. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=4:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;4:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765208, @last-run=1594765208, @exec-time=0
  964. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/34, version=0.16.14)
  965. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.14 2
  966. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.15 (null)
  967. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=15
  968. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:0;3:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=30, @rc-code=0, @op-status=0, @queue-time=1
  969. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/35, version=0.16.15)
  970. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 3 action 3 (virsh_node1_stop_0 on el8-a01n01) confirmed: ok | rc=0 call-id=30
  971. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.15 2
  972. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.16 (null)
  973. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=16
  974. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=5:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @transition-magic=-1:193;5:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765208, @last-run=1594765208, @exec-time=0
  975. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/36, version=0.16.16)
  976. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.16 2
  977. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.17 (null)
  978. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=17
  979. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;4:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=31, @rc-code=0, @op-status=0, @exec-time=1
  980. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/37, version=0.16.17)
  981. Jul 14 22:20:08 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 3 action 4 (virsh_node2_stop_0 on el8-a01n01) confirmed: ok | rc=0 call-id=31
  982. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.17 2
  983. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.18 (null)
  984. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=18
  985. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;5:3:0:22298a5d-b80a-4cd7-adde-aba652405a4e, @call-id=33, @rc-code=0, @op-status=0, @exec-time=2260
  986. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/38, version=0.16.18)
  987. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (process_graph_event) info: Transition 3 action 5 (hypervisor_stop_0 on el8-a01n01) confirmed: ok | rc=0 call-id=33
  988. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_crm_command) info: Executing crm-event (14) without waiting: do_shutdown on el8-a01n01
  989. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_crm_command) info: Executing crm-event (15) locally without waiting: do_shutdown on el8-a01n02
  990. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (te_crm_command) info: crm-event (15) is a local shutdown
  991. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (run_graph) notice: Transition 3 (Complete=7, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-10.bz2): Complete
  992. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_log) info: Input I_STOP received in state S_TRANSITION_ENGINE from notify_crmd
  993. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_state_transition) info: State transition S_TRANSITION_ENGINE -> S_STOPPING | input=I_STOP cause=C_FSA_INTERNAL origin=notify_crmd
  994. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_dc_release) info: DC role released
  995. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (pe_ipc_destroy) info: Connection to the scheduler released
  996. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_te_control) info: Transitioner is now inactive
  997. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/59)
  998. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (tengine_stonith_connection_destroy) info: Fencing daemon disconnected
  999. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_lrm_control) info: Disconnecting from the executor
  1000. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (lrmd_api_disconnect) info: Disconnecting IPC local executor connection
  1001. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_ipcs_dispatch_connection_request) debug: HUP conn (/dev/shm/qb-1567-1570-9-qIYT85/qb)
  1002. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_ipcs_disconnect) debug: qb_ipcs_disconnect(/dev/shm/qb-1567-1570-9-qIYT85/qb) state:2
  1003. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-execd [1567] (crm_client_destroy) debug: Destroying 0 events
  1004. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_rb_close_helper) debug: Free'ing ringbuffer: /dev/shm/qb-1567-1570-9-qIYT85/qb-response-lrmd-header
  1005. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_rb_close_helper) debug: Free'ing ringbuffer: /dev/shm/qb-1567-1570-9-qIYT85/qb-event-lrmd-header
  1006. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_rb_close_helper) debug: Free'ing ringbuffer: /dev/shm/qb-1567-1570-9-qIYT85/qb-request-lrmd-header
  1007. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: --- 0.16.18 2
  1008. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: Diff: +++ 0.16.19 (null)
  1009. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib: @num_updates=19
  1010. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_dc_release, @expected=down
  1011. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (lrmd_ipc_connection_destroy) info: IPC connection destroyed
  1012. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (lrm_connection_destroy) info: Disconnected from executor
  1013. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (lrmd_api_disconnect) info: Disconnecting IPC local executor connection
  1014. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/59, version=0.16.19)
  1015. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_lrm_control) notice: Disconnected from the executor
  1016. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_cluster_disconnect) info: Disconnecting from cluster infrastructure: corosync
  1017. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-attrd [1568] (pcmk_cpg_membership) info: Group attrd event 1: el8-a01n01 (node 1 pid 7333) left for unknown reason
  1018. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-attrd [1568] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now offline
  1019. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-attrd [1568] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now lost | nodeid=1 previous=member source=crm_update_peer_proc
  1020. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-attrd [1568] (attrd_peer_remove) notice: Removing all el8-a01n01 attributes for peer loss
  1021. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-attrd [1568] (crm_reap_dead_member) info: Removing node with name el8-a01n01 and id 1 from membership cache
  1022. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-attrd [1568] (reap_crm_member) notice: Purged 1 peer with id=1 and/or uname=el8-a01n01 from the membership cache
  1023. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-attrd [1568] (pcmk_cpg_membership) info: Group attrd event 1: el8-a01n02 (node 2 pid 1568) is member
  1024. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (terminate_cs_connection) notice: Disconnected from Corosync
  1025. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_ha_control) info: Disconnected from the cluster
  1026. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_cib_control) info: Disconnecting from the CIB manager
  1027. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (crmd_cib_connection_destroy) info: Connection to the CIB manager terminated
  1028. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_cib_control) notice: Disconnected from the CIB manager
  1029. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  1030. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_exit) info: Performing A_EXIT_0 - gracefully exiting the controller
  1031. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (do_exit) info: [pacemaker-controld] stopped (0)
  1032. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (crmd_exit) info: Dropping I_RELEASE_SUCCESS: [ state=S_STOPPING cause=C_FSA_INTERNAL origin=do_dc_release ]
  1033. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (crmd_exit) info: Dropping I_TERMINATE: [ state=S_STOPPING cause=C_FSA_INTERNAL origin=do_stop ]
  1034. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (crmd_cs_destroy) info: Corosync connection closed
  1035. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (crmd_cs_destroy) info: Corosync connection closed
  1036. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (crmd_init) info: pacemaker-controld[1570] exiting with status 0 (OK)
  1037. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-controld [1570] (crm_xml_cleanup) info: Cleaning up memory from libxml2
  1038. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk_child_exit) info: pacemaker-controld[1570] exited with status 0 (OK)
  1039. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemakerd [1564] (stop_child) notice: Stopping pacemaker-schedulerd | sent signal 15 to process 1569
  1040. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-fenced [1566] (pcmk_cpg_membership) info: Group stonith-ng event 2: el8-a01n01 (node 1 pid 7331) left for unknown reason
  1041. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-fenced [1566] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now offline
  1042. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-fenced [1566] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now lost | nodeid=1 previous=member source=crm_update_peer_proc
  1043. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_process_shutdown_req) info: Peer el8-a01n01 is requesting to shut down
  1044. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-fenced [1566] (crm_reap_dead_member) info: Removing node with name el8-a01n01 and id 1 from membership cache
  1045. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-fenced [1566] (reap_crm_member) notice: Purged 1 peer with id=1 and/or uname=el8-a01n01 from the membership cache
  1046. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-fenced [1566] (pcmk_cpg_membership) info: Group stonith-ng event 2: el8-a01n02 (node 2 pid 1566) is member
  1047. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (pcmk_cpg_membership) info: Group cib event 2: el8-a01n01 (node 1 pid 7330) left via cpg_leave
  1048. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now offline
  1049. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now lost | nodeid=1 previous=member source=crm_update_peer_proc
  1050. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  1051. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  1052. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-schedulerd[1569] (crm_xml_cleanup) info: Cleaning up memory from libxml2
  1053. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_reap_dead_member) info: Removing node with name el8-a01n01 and id 1 from membership cache
  1054. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (reap_crm_member) notice: Purged 1 peer with id=1 and/or uname=el8-a01n01 from the membership cache
  1055. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (pcmk_cpg_membership) info: Group cib event 2: el8-a01n02 (node 2 pid 1565) is member
  1056. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk_child_exit) info: pacemaker-schedulerd[1569] exited with status 0 (OK)
  1057. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemakerd [1564] (stop_child) notice: Stopping pacemaker-attrd | sent signal 15 to process 1568
  1058. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-attrd [1568] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  1059. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-attrd [1568] (main) info: Shutting down attribute manager
  1060. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-attrd [1568] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  1061. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-attrd [1568] (attrd_cib_destroy_cb) info: Connection disconnection complete
  1062. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-attrd [1568] (crm_xml_cleanup) info: Cleaning up memory from libxml2
  1063. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk_child_exit) info: pacemaker-attrd[1568] exited with status 0 (OK)
  1064. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemakerd [1564] (stop_child) notice: Stopping pacemaker-execd | sent signal 15 to process 1567
  1065. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-execd [1567] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  1066. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-execd [1567] (lrmd_exit) info: Terminating with 0 clients
  1067. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-execd [1567] (stonith_api_del_notification) debug: Removing callback for st_notify_disconnect events
  1068. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-execd [1567] (stonith_api_signoff) debug: Disconnecting from the fencer
  1069. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
  1070. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-1566-1567-12-4eRucb/qb-request-stonith-ng-header
  1071. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-1566-1567-12-4eRucb/qb-response-stonith-ng-header
  1072. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-1566-1567-12-4eRucb/qb-event-stonith-ng-header
  1073. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  1074. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-execd [1567] (qb_ipcs_unref) debug: qb_ipcs_unref() - destroying
  1075. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-execd [1567] (crm_xml_cleanup) info: Cleaning up memory from libxml2
  1076. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk_cpg_membership) info: Group pacemakerd event 2: el8-a01n01 (node 1 pid 7329) left via cpg_leave
  1077. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemakerd [1564] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now offline
  1078. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk_cpg_membership) info: Group pacemakerd event 2: el8-a01n02 (node 2 pid 1564) is member
  1079. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk_child_exit) info: pacemaker-execd[1567] exited with status 0 (OK)
  1080. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemakerd [1564] (stop_child) notice: Stopping pacemaker-fenced | sent signal 15 to process 1566
  1081. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-fenced [1566] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  1082. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-fenced [1566] (stonith_shutdown) info: Terminating with 0 clients
  1083. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-fenced [1566] (cib_connection_destroy) info: Connection to the CIB manager closed
  1084. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-fenced [1566] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  1085. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-fenced [1566] (crm_xml_cleanup) info: Cleaning up memory from libxml2
  1086. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk_child_exit) info: pacemaker-fenced[1566] exited with status 0 (OK)
  1087. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemakerd [1564] (stop_child) notice: Stopping pacemaker-based | sent signal 15 to process 1565
  1088. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  1089. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_shutdown) info: Disconnected 0 clients
  1090. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (cib_shutdown) info: All clients disconnected (0)
  1091. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (terminate_cib) info: initiate_exit: Exiting from mainloop...
  1092. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_cluster_disconnect) info: Disconnecting from cluster infrastructure: corosync
  1093. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (terminate_cs_connection) notice: Disconnected from Corosync
  1094. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (crm_cluster_disconnect) info: Disconnecting from cluster infrastructure: corosync
  1095. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (cluster_disconnect_cpg) info: No CPG connection
  1096. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (terminate_cs_connection) notice: Disconnected from Corosync
  1097. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  1098. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  1099. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemaker-based [1565] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  1100. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk_child_exit) info: pacemaker-based[1565] exited with status 0 (OK)
  1101. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemakerd [1564] (pcmk_shutdown_worker) notice: Shutdown complete
  1102. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemakerd [1564] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  1103. Jul 14 22:20:11 el8-a01n02.digimer.ca pacemakerd [1564] (crm_xml_cleanup) info: Cleaning up memory from libxml2
  1104. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  1105. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (get_cluster_type) info: Detected an active 'corosync' cluster
  1106. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (mcp_read_config) info: Reading configure for stack: corosync
  1107. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (main) notice: Starting Pacemaker 2.0.3-5.el8_2.1 | build=4b1f869f0f features: generated-manpages agent-manpages ncurses libqb-logging libqb-ipc systemd nagios corosync-native atomic-attrd acls
  1108. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (main) info: Maximum core file size is: 18446744073709551615
  1109. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (qb_ipcs_us_publish) info: server name: pacemakerd
  1110. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk__ipc_is_authentic_process_active) info: Could not connect to lrmd IPC: Connection refused
  1111. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk__ipc_is_authentic_process_active) info: Could not connect to cib_ro IPC: Connection refused
  1112. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk__ipc_is_authentic_process_active) info: Could not connect to crmd IPC: Connection refused
  1113. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk__ipc_is_authentic_process_active) info: Could not connect to attrd IPC: Connection refused
  1114. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk__ipc_is_authentic_process_active) info: Could not connect to pengine IPC: Connection refused
  1115. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk__ipc_is_authentic_process_active) info: Could not connect to stonith-ng IPC: Connection refused
  1116. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (crm_get_peer) info: Created entry c08dd45a-d8a6-4164-b247-bb01cf124bb7/0x557a6b70a860 for node el8-a01n02/2 (1 total)
  1117. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  1118. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (crm_get_peer) info: Node 2 has uuid 2
  1119. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online
  1120. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (cluster_connect_quorum) warning: Quorum lost
  1121. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (crm_get_peer) info: Created entry b7084b25-d019-459a-935b-84803bd35fa2/0x557a6b70c800 for node el8-a01n01/1 (2 total)
  1122. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  1123. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (crm_get_peer) info: Node 1 has uuid 1
  1124. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (start_child) info: Using uid=189 and group=189 for process pacemaker-based
  1125. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (start_child) info: Forked child 2197 for process pacemaker-based
  1126. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (start_child) info: Forked child 2198 for process pacemaker-fenced
  1127. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (start_child) info: Forked child 2199 for process pacemaker-execd
  1128. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (start_child) info: Using uid=189 and group=189 for process pacemaker-attrd
  1129. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (start_child) info: Forked child 2200 for process pacemaker-attrd
  1130. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (start_child) info: Using uid=189 and group=189 for process pacemaker-schedulerd
  1131. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (start_child) info: Forked child 2201 for process pacemaker-schedulerd
  1132. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (start_child) info: Using uid=189 and group=189 for process pacemaker-controld
  1133. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (start_child) info: Forked child 2202 for process pacemaker-controld
  1134. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (main) notice: Pacemaker daemon successfully started and accepting connections
  1135. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk_quorum_notification) info: Quorum still lost | membership=193 members=1
  1136. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification
  1137. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (crm_reap_unseen_nodes) info: State of node el8-a01n01[1] is still unknown
  1138. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk_cpg_membership) info: Group pacemakerd event 0: node 2 pid 2196 joined via cpg_join
  1139. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk_cpg_membership) info: Group pacemakerd event 0: el8-a01n02 (node 2 pid 2196) is member
  1140. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  1141. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (main) notice: Starting Pacemaker CIB manager
  1142. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (get_cluster_type) info: Verifying cluster type: 'corosync'
  1143. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (get_cluster_type) info: Assuming an active 'corosync' cluster
  1144. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (retrieveCib) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.xml (digest: /var/lib/pacemaker/cib/cib.xml.sig)
  1145. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-fenced [2198] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  1146. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-fenced [2198] (main) notice: Starting Pacemaker fencer
  1147. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-fenced [2198] (get_cluster_type) info: Verifying cluster type: 'corosync'
  1148. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-fenced [2198] (get_cluster_type) info: Assuming an active 'corosync' cluster
  1149. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-fenced [2198] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  1150. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-fenced [2198] (crm_get_peer) info: Created entry b957efac-537d-4b52-b0e1-1df16a9a9aa1/0x555e50d78d60 for node el8-a01n02/2 (1 total)
  1151. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-fenced [2198] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  1152. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-attrd [2200] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  1153. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-attrd [2200] (main) notice: Starting Pacemaker node attribute manager
  1154. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-fenced [2198] (crm_get_peer) info: Node 2 has uuid 2
  1155. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-fenced [2198] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online
  1156. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-fenced [2198] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  1157. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-fenced [2198] (init_cs_connection_once) info: Connection to 'corosync': established
  1158. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-execd [2199] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  1159. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-execd [2199] (main) notice: Starting Pacemaker local executor
  1160. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_ipcs_us_publish) info: server name: lrmd
  1161. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-execd [2199] (main) notice: Pacemaker local executor successfully started and accepting connections
  1162. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  1163. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  1164. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (main) notice: Starting Pacemaker scheduler
  1165. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (qb_ipcs_us_publish) info: server name: pengine
  1166. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (main) notice: Pacemaker scheduler successfully started and accepting connections
  1167. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  1168. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-controld [2202] (main) notice: Starting Pacemaker controller
  1169. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_log) info: Input I_STARTUP received in state S_STARTING from crmd_init
  1170. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (crm_get_peer) info: Created entry aec45592-ab77-4b0c-948d-dc89bb4377ad/0x55efd7b56320 for node el8-a01n02/2 (1 total)
  1171. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  1172. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (crm_get_peer) info: Node 2 has uuid 2
  1173. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online
  1174. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  1175. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (init_cs_connection_once) info: Connection to 'corosync': established
  1176. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (qb_ipcs_us_publish) info: server name: cib_ro
  1177. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (qb_ipcs_us_publish) info: server name: cib_rw
  1178. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (qb_ipcs_us_publish) info: server name: cib_shm
  1179. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (main) notice: Pacemaker CIB manager successfully started and accepting connections
  1180. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (pcmk_cpg_membership) info: Group cib event 0: node 2 pid 2197 joined via cpg_join
  1181. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (pcmk_cpg_membership) info: Group cib event 0: el8-a01n02 (node 2 pid 2197) is member
  1182. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-24.raw
  1183. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_file_write_with_digest) info: Wrote version 0.16.0 of the CIB to disk (digest: 71670f38eb62dc5320749c7be8e07fc8)
  1184. Jul 14 22:20:22 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.Z1jOmz (digest: /var/lib/pacemaker/cib/cib.r1Lvrw)
  1185. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk_quorum_notification) notice: Quorum acquired | membership=197 members=2
  1186. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemakerd [2196] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification
  1187. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk_cpg_membership) info: Group pacemakerd event 1: node 1 pid 14952 joined via cluster join
  1188. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-based [2197] (pcmk_cpg_membership) info: Group cib event 1: node 1 pid 14953 joined via cluster join
  1189. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n01 (node 1 pid 14952) is member
  1190. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemakerd [2196] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online
  1191. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n02 (node 2 pid 2196) is member
  1192. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-based [2197] (crm_get_peer) info: Created entry f900687c-f4f5-4693-a9cd-1c00b457cce3/0x55efd7be2a40 for node el8-a01n01/1 (2 total)
  1193. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-based [2197] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  1194. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-based [2197] (crm_get_peer) info: Node 1 has uuid 1
  1195. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-based [2197] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n01 (node 1 pid 14953) is member
  1196. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-based [2197] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online
  1197. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-based [2197] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  1198. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-based [2197] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n02 (node 2 pid 2197) is member
  1199. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (main) info: CIB connection active
  1200. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (get_cluster_type) info: Verifying cluster type: 'corosync'
  1201. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (get_cluster_type) info: Assuming an active 'corosync' cluster
  1202. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  1203. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (crm_get_peer) info: Created entry ec71a618-b081-4cf2-aa59-4d7a98647b95/0x55958b358a20 for node el8-a01n02/2 (1 total)
  1204. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  1205. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (crm_get_peer) info: Node 2 has uuid 2
  1206. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online
  1207. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  1208. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (init_cs_connection_once) info: Connection to 'corosync': established
  1209. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (setup_cib) info: Watching for stonith topology changes
  1210. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (qb_ipcs_us_publish) info: server name: stonith-ng
  1211. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (main) notice: Pacemaker fencer successfully started and accepting connections
  1212. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (pcmk_cpg_membership) info: Group stonith-ng event 0: node 2 pid 2198 joined via cpg_join
  1213. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (pcmk_cpg_membership) info: Group stonith-ng event 0: el8-a01n02 (node 2 pid 2198) is member
  1214. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (pcmk_cpg_membership) info: Group stonith-ng event 1: node 1 pid 14954 joined via cluster join
  1215. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (crm_get_peer) info: Created entry 1a57dca3-8dda-4e43-b870-43d449ef5d3e/0x555e50e80990 for node el8-a01n01/1 (2 total)
  1216. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  1217. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (crm_get_peer) info: Node 1 has uuid 1
  1218. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n01 (node 1 pid 14954) is member
  1219. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online
  1220. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  1221. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n02 (node 2 pid 2198) is member
  1222. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (init_cib_cache_cb) info: Updating device list from the cib: init
  1223. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (cib_devices_update) info: Updating devices to version 0.16.0
  1224. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (main) info: Cluster connection active
  1225. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (attrd_erase_attrs) info: Clearing transient attributes from CIB | xpath=//node_state[@uname='el8-a01n02']/transient_attributes
  1226. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n02']/transient_attributes to all (origin=local/attrd/2)
  1227. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (qb_ipcs_us_publish) info: server name: attrd
  1228. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (main) notice: Pacemaker node attribute manager successfully started and accepting connections
  1229. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (pcmk_cpg_membership) info: Group attrd event 0: node 2 pid 2200 joined via cpg_join
  1230. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (pcmk_cpg_membership) info: Group attrd event 0: el8-a01n02 (node 2 pid 2200) is member
  1231. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (pcmk_cpg_membership) info: Group attrd event 1: node 1 pid 14956 joined via cpg_join
  1232. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (crm_get_peer) info: Created entry 7650bc57-0770-4c55-8845-1beb2395b45c/0x55958b35b6d0 for node el8-a01n01/1 (2 total)
  1233. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  1234. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (crm_get_peer) info: Node 1 has uuid 1
  1235. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (pcmk_cpg_membership) info: Group attrd event 1: el8-a01n01 (node 1 pid 14956) is member
  1236. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online
  1237. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  1238. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (pcmk_cpg_membership) info: Group attrd event 1: el8-a01n02 (node 2 pid 2200) is member
  1239. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n02]: (unset) -> 2 | from el8-a01n02
  1240. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (attrd_start_election_if_needed) info: Starting an election to determine the writer
  1241. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/transient_attributes: OK (rc=0, origin=el8-a01n02/attrd/2, version=0.16.0)
  1242. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (get_cluster_type) info: Verifying cluster type: 'corosync'
  1243. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (get_cluster_type) info: Assuming an active 'corosync' cluster
  1244. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  1245. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_get_peer) info: Created entry 1e227e11-f834-414a-b9d1-47581bbff31d/0x564c02e57e20 for node el8-a01n02/2 (1 total)
  1246. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  1247. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (peer_update_callback) info: Cluster node el8-a01n02 is now in unknown state
  1248. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_get_peer) info: Node 2 has uuid 2
  1249. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n02[2] - corosync-cpg is now online
  1250. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (peer_update_callback) info: Node el8-a01n02 is now a peer | DC=<none> old=0x0000000 new=0x4000000
  1251. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds
  1252. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (init_cs_connection_once) info: Connection to 'corosync': established
  1253. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (cluster_connect_quorum) notice: Quorum acquired
  1254. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_get_peer) info: Created entry fc531f01-5566-4adc-abf2-052d50d7779a/0x564c02e5a700 for node el8-a01n01/1 (2 total)
  1255. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  1256. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (peer_update_callback) info: Cluster node el8-a01n01 is now in unknown state
  1257. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_get_peer) info: Node 1 has uuid 1
  1258. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/transient_attributes: OK (rc=0, origin=el8-a01n01/attrd/2, version=0.16.0)
  1259. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_ha_control) info: Connected to the cluster
  1260. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (lrmd_ipc_connect) info: Connecting to executor
  1261. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/4)
  1262. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-execd [2199] (crm_client_new) debug: Connecting 0x55ebe3a73fb0 for uid=189 gid=189 pid=2202 id=c96da60c-8915-453c-8c89-13f67117ca10
  1263. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-execd [2199] (handle_new_connection) debug: IPC credentials authenticated (/dev/shm/qb-2199-2202-9-AHVNzI/qb)
  1264. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_ipcs_shm_connect) debug: connecting to client [2202]
  1265. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  1266. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  1267. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  1268. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n01]: (unset) -> 2 | from el8-a01n01
  1269. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/4, version=0.16.0)
  1270. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed register operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=0, reply=1, notify=0
  1271. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_lrm_control) info: Connection to the executor established
  1272. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_started) info: Delaying start, no membership data (0000000000100000)
  1273. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_started) info: Delaying start, no membership data (0000000000100000)
  1274. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (pcmk_quorum_notification) info: Quorum retained | membership=197 members=2
  1275. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification
  1276. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (peer_update_callback) info: Cluster node el8-a01n01 is now member (was in unknown state)
  1277. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification
  1278. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (peer_update_callback) info: Cluster node el8-a01n02 is now member (was in unknown state)
  1279. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_started) info: Delaying start, Config not read (0000000000000040)
  1280. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (qb_ipcs_us_publish) info: server name: crmd
  1281. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_started) notice: Pacemaker controller successfully started and accepting connections
  1282. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_log) info: Input I_PENDING received in state S_STARTING from do_started
  1283. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_state_transition) notice: State transition S_STARTING -> S_PENDING | input=I_PENDING cause=C_FSA_INTERNAL origin=do_started
  1284. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (election_count_vote) info: election-attrd round 1 (owner node ID 1) pass: vote from el8-a01n01 (Uptime)
  1285. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (election_check) info: election-attrd won by local node
  1286. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (attrd_declare_winner) notice: Recorded local node as attribute writer (was unset)
  1287. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-attrd [2200] (write_attribute) info: Processed 2 private changes for #attrd-protocol, id=n/a, set=n/a
  1288. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/4, version=0.16.0)
  1289. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 2214 exited with rc=0
  1290. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (stonith_device_register) notice: Added 'virsh_node1' to the device list (1 active devices)
  1291. Jul 14 22:20:23 el8-a01n02.digimer.ca pacemaker-fenced [2198] (stonith_device_register) notice: Added 'virsh_node2' to the device list (2 active devices)
  1292. Jul 14 22:20:24 el8-a01n02.digimer.ca pacemaker-controld [2202] (pcmk_cpg_membership) info: Group crmd event 0: node 2 pid 2202 joined via cpg_join
  1293. Jul 14 22:20:24 el8-a01n02.digimer.ca pacemaker-controld [2202] (pcmk_cpg_membership) info: Group crmd event 0: el8-a01n01 (node 1 pid 14958) is member
  1294. Jul 14 22:20:24 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n01[1] - corosync-cpg is now online
  1295. Jul 14 22:20:24 el8-a01n02.digimer.ca pacemaker-controld [2202] (peer_update_callback) info: Node el8-a01n01 is now a peer | DC=<none> old=0x0000000 new=0x4000000
  1296. Jul 14 22:20:24 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds
  1297. Jul 14 22:20:24 el8-a01n02.digimer.ca pacemaker-controld [2202] (pcmk_cpg_membership) info: Group crmd event 0: el8-a01n02 (node 2 pid 2202) is member
  1298. Jul 14 22:20:24 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 30 seconds
  1299. Jul 14 22:20:24 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_connect_stonith) notice: Fencer successfully connected
  1300. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_timer_popped) info: Election Trigger just popped | input=I_DC_TIMEOUT time=20000ms
  1301. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_log) warning: Input I_DC_TIMEOUT received in state S_PENDING from crm_timer_popped
  1302. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_state_transition) info: State transition S_PENDING -> S_ELECTION | input=I_DC_TIMEOUT cause=C_TIMER_POPPED origin=crm_timer_popped
  1303. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (election_check) info: election-DC won by local node
  1304. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_log) info: Input I_ELECTION_DC received in state S_ELECTION from election_win_cb
  1305. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_state_transition) notice: State transition S_ELECTION -> S_INTEGRATION | input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=election_win_cb
  1306. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_te_control) info: Registering TE UUID: adab90af-4d5d-488e-8027-cf0e0ac80cd9
  1307. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (set_graph_functions) info: Setting custom graph functions
  1308. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_dc_takeover) info: Taking over DC status for this partition
  1309. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_readwrite) info: We are now in R/W mode
  1310. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_master operation for section 'all': OK (rc=0, origin=local/crmd/7, version=0.16.0)
  1311. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/8)
  1312. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/10)
  1313. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/12)
  1314. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (join_make_offer) info: Making join-1 offers based on membership event 197
  1315. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (join_make_offer) info: Sending join-1 offer to el8-a01n01
  1316. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/14)
  1317. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n02/crmd/8, version=0.16.0)
  1318. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n02/crmd/10, version=0.16.0)
  1319. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n02/crmd/12, version=0.16.0)
  1320. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n02/crmd/14, version=0.16.0)
  1321. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (join_make_offer) info: Sending join-1 offer to el8-a01n02
  1322. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_dc_join_offer_all) info: Waiting on join-1 requests from 2 outstanding nodes
  1323. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (update_dc) info: Set DC to el8-a01n02 (3.3.0)
  1324. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_update_peer_expected) info: update_dc: Node el8-a01n02[2] - expected state is now member (was (null))
  1325. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_update_peer_expected) info: do_dc_join_filter_offer: Node el8-a01n01[1] - expected state is now member (was (null))
  1326. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_state_transition) info: State transition S_INTEGRATION -> S_FINALIZE_JOIN | input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state
  1327. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_replace) info: Digest matched on replace from el8-a01n02: be52010831fd154fc90da9ca9708e392
  1328. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_replace) info: Replaced 0.16.0 with 0.16.0 from el8-a01n02
  1329. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=el8-a01n02/crmd/17, version=0.16.0)
  1330. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/18)
  1331. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/19)
  1332. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (controld_delete_node_state) info: Deleting resource history for node el8-a01n02 (via CIB call 20) | xpath=//node_state[@uname='el8-a01n02']/lrm
  1333. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/18, version=0.16.0)
  1334. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/19, version=0.16.0)
  1335. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n02']/lrm to all (origin=local/crmd/20)
  1336. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (controld_delete_node_state) info: Deleting resource history for node el8-a01n01 (via CIB call 22) | xpath=//node_state[@uname='el8-a01n01']/lrm
  1337. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/21)
  1338. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/lrm: OK (rc=0, origin=el8-a01n02/crmd/20, version=0.16.0)
  1339. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.0 2
  1340. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.1 (null)
  1341. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=1
  1342. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ /cib/status: <node_state id="2" uname="el8-a01n02" in_ccm="true" crmd="online" crm-debug-origin="do_lrm_query_internal"/>
  1343. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ <lrm id="2">
  1344. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ <lrm_resources/>
  1345. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ </lrm>
  1346. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ </node_state>
  1347. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/21, version=0.16.1)
  1348. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n01']/lrm to all (origin=local/crmd/22)
  1349. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_state_transition) info: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE | input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state
  1350. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-controld [2202] (abort_transition_graph) info: Transition -1 aborted: Peer Cancelled | source=do_te_invoke:129 complete=true
  1351. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/23)
  1352. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-25.raw
  1353. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/lrm: OK (rc=0, origin=el8-a01n02/crmd/22, version=0.16.1)
  1354. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.1 2
  1355. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.2 (null)
  1356. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=2
  1357. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ /cib/status: <node_state id="1" uname="el8-a01n01" in_ccm="true" crmd="online" crm-debug-origin="do_lrm_query_internal"/>
  1358. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ <lrm id="1">
  1359. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ <lrm_resources/>
  1360. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ </lrm>
  1361. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ </node_state>
  1362. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_file_write_with_digest) info: Wrote version 0.16.0 of the CIB to disk (digest: 17149bee2ab28b054d280a7e9e1e7493)
  1363. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/23, version=0.16.2)
  1364. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/26)
  1365. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/27)
  1366. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.OZvF35 (digest: /var/lib/pacemaker/cib/cib.ajyCY6)
  1367. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/28)
  1368. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/26, version=0.16.2)
  1369. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.2 2
  1370. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.3 (null)
  1371. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=3
  1372. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition, @join=member, @expected=member
  1373. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition, @join=member, @expected=member
  1374. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/27, version=0.16.3)
  1375. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.3 2
  1376. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.4 (null)
  1377. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=4, @dc-uuid=2
  1378. Jul 14 22:20:44 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n02/crmd/28, version=0.16.4)
  1379. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (determine_online_status_fencing) info: Node el8-a01n02 is active
  1380. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (determine_online_status) info: Node el8-a01n02 is online
  1381. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (determine_online_status_fencing) info: Node el8-a01n01 is active
  1382. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (determine_online_status) info: Node el8-a01n01 is online
  1383. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (unpack_node_loop) info: Node 2 is already processed
  1384. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (unpack_node_loop) info: Node 1 is already processed
  1385. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (unpack_node_loop) info: Node 2 is already processed
  1386. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (unpack_node_loop) info: Node 1 is already processed
  1387. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (common_print) info: virsh_node1 (stonith:fence_virsh): Stopped
  1388. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (common_print) info: virsh_node2 (stonith:fence_virsh): Stopped
  1389. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  1390. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (short_print) info: Stopped: [ el8-a01n01 el8-a01n02 ]
  1391. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (clone_print) info: Clone Set: drbd-clone [drbd]
  1392. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (short_print) info: Stopped (disabled): [ el8-a01n01 el8-a01n02 ]
  1393. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (native_color) info: Resource drbd:0 cannot run anywhere
  1394. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (native_color) info: Resource drbd:1 cannot run anywhere
  1395. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (RecurringOp) info: Start recurring monitor (60s) for virsh_node1 on el8-a01n01
  1396. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (RecurringOp) info: Start recurring monitor (60s) for virsh_node2 on el8-a01n02
  1397. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (RecurringOp) info: Start recurring monitor (60s) for hypervisor:0 on el8-a01n01
  1398. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (RecurringOp) info: Start recurring monitor (60s) for hypervisor:1 on el8-a01n02
  1399. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogAction) notice: * Start virsh_node1 ( el8-a01n01 )
  1400. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogAction) notice: * Start virsh_node2 ( el8-a01n02 )
  1401. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogAction) notice: * Start hypervisor:0 ( el8-a01n01 )
  1402. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogAction) notice: * Start hypervisor:1 ( el8-a01n02 )
  1403. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogActions) info: Leave drbd:0 (Stopped)
  1404. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogActions) info: Leave drbd:1 (Stopped)
  1405. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (pcmk__log_transition_summary) notice: Calculated transition 0, saving inputs in /var/lib/pacemaker/pengine/pe-input-11.bz2
  1406. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response
  1407. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_te_invoke) info: Processing graph 0 (ref=pe_calc-dc-1594765245-9) derived from /var/lib/pacemaker/pengine/pe-input-11.bz2
  1408. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating monitor operation virsh_node1_monitor_0 locally on el8-a01n02 | action 5
  1409. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node1' not in cache
  1410. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=0, reply=1, notify=0
  1411. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node1'
  1412. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=0, reply=1, notify=1
  1413. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=0, reply=1, notify=0
  1414. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_lrm_rsc_op) info: Performing key=5:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=virsh_node1_monitor_0
  1415. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-controld [2202] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 2295 exited with rc=0
  1416. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/31)
  1417. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=5, reply=1, notify=0
  1418. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:5
  1419. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (stonith_api_signon) debug: Attempting fencer connection by pacemaker-execd with mainloop
  1420. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating monitor operation virsh_node1_monitor_0 on el8-a01n01 | action 1
  1421. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating monitor operation virsh_node2_monitor_0 locally on el8-a01n02 | action 6
  1422. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.4 2
  1423. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.5 (null)
  1424. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=5
  1425. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_update_resource
  1426. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="virsh_node1" type="fence_virsh" class="stonith"/>
  1427. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node1_last_0" operation_key="virsh_node1_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="5:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9" transition-magic="-1:193;5:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-statu
  1428. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ </lrm_resource>
  1429. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/31, version=0.16.5)
  1430. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  1431. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  1432. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  1433. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (stonith_api_signon) debug: Connection to fencer by pacemaker-execd succeeded (registration token: 0db933c7-c42d-4876-a437-d2dc005d3acd)
  1434. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:5 exit-code:7 exec-time:10ms queue-time:1ms
  1435. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node2' not in cache
  1436. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=0, reply=1, notify=0
  1437. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node2'
  1438. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=0, reply=1, notify=1
  1439. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=0, reply=1, notify=0
  1440. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_lrm_rsc_op) info: Performing key=6:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=virsh_node2_monitor_0
  1441. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=9, reply=1, notify=0
  1442. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_execute) debug: executing - rsc:virsh_node2 action:monitor call_id:9
  1443. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_finished) debug: finished - rsc:virsh_node2 action:monitor call_id:9 exit-code:7 exec-time:0ms queue-time:0ms
  1444. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/32)
  1445. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating monitor operation virsh_node2_monitor_0 on el8-a01n01 | action 2
  1446. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.5 2
  1447. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.6 (null)
  1448. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=6
  1449. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="virsh_node2" type="fence_virsh" class="stonith"/>
  1450. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node2_last_0" operation_key="virsh_node2_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="6:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9" transition-magic="-1:193;6:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-statu
  1451. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ </lrm_resource>
  1452. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/32, version=0.16.6)
  1453. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_lrm_event) notice: Result of probe operation for virsh_node1 on el8-a01n02: 7 (not running) | call=5 key=virsh_node1_monitor_0 confirmed=true cib-update=33
  1454. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_lrm_event) notice: Result of probe operation for virsh_node2 on el8-a01n02: 7 (not running) | call=9 key=virsh_node2_monitor_0 confirmed=true cib-update=34
  1455. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/33)
  1456. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/34)
  1457. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.6 2
  1458. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.7 (null)
  1459. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=7
  1460. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;5:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=5, @rc-code=7, @op-status=0, @exec-time=10, @queue-time=1
  1461. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/33, version=0.16.7)
  1462. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.7 2
  1463. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.8 (null)
  1464. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=8
  1465. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;6:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=9, @rc-code=7, @op-status=0
  1466. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/34, version=0.16.8)
  1467. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 0 action 5 (virsh_node1_monitor_0 on el8-a01n02) confirmed: not running | rc=7 call-id=5
  1468. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 0 action 6 (virsh_node2_monitor_0 on el8-a01n02) confirmed: not running | rc=7 call-id=9
  1469. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating monitor operation hypervisor:1_monitor_0 locally on el8-a01n02 | action 7
  1470. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor' not in cache
  1471. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=0, reply=1, notify=0
  1472. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor:1' not in cache
  1473. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=0, reply=1, notify=0
  1474. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_rsc_register) info: Cached agent information for 'hypervisor'
  1475. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=0, reply=1, notify=1
  1476. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=0, reply=1, notify=0
  1477. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_lrm_rsc_op) info: Performing key=7:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=hypervisor_monitor_0
  1478. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/35)
  1479. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=14, reply=1, notify=0
  1480. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:14
  1481. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating monitor operation drbd:0_monitor_0 locally on el8-a01n02 | action 8
  1482. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.8 2
  1483. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.9 (null)
  1484. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=9
  1485. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="hypervisor" type="libvirtd" class="systemd"/>
  1486. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ <lrm_rsc_op id="hypervisor_last_0" operation_key="hypervisor_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="7:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9" transition-magic="-1:193;7:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-status=
  1487. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ </lrm_resource>
  1488. Jul 14 22:20:45 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/35, version=0.16.9)
  1489. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  1490. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_get_rsc_info) info: Agent information for 'drbd' not in cache
  1491. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=0, reply=1, notify=0
  1492. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_get_rsc_info) info: Agent information for 'drbd:0' not in cache
  1493. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=0, reply=1, notify=0
  1494. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_rsc_register) info: Cached agent information for 'drbd'
  1495. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=0, reply=1, notify=1
  1496. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=0, reply=1, notify=0
  1497. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_lrm_rsc_op) info: Performing key=8:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=drbd_monitor_0
  1498. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/36)
  1499. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=19, reply=1, notify=0
  1500. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_execute) debug: executing - rsc:drbd action:monitor call_id:19
  1501. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1502. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.9 2
  1503. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.10 (null)
  1504. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=10
  1505. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="drbd" type="drbd" class="systemd"/>
  1506. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ <lrm_rsc_op id="drbd_last_0" operation_key="drbd_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="8:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9" transition-magic="-1:193;8:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-status="-1" interva
  1507. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ </lrm_resource>
  1508. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/36, version=0.16.10)
  1509. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1510. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1511. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:14 exit-code:7 exec-time:63ms queue-time:1ms
  1512. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_lrm_event) notice: Result of probe operation for hypervisor on el8-a01n02: 7 (not running) | call=14 key=hypervisor_monitor_0 confirmed=true cib-update=37
  1513. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/37)
  1514. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.10 2
  1515. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.11 (null)
  1516. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=11
  1517. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;7:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=14, @rc-code=7, @op-status=0, @exec-time=63, @queue-time=1
  1518. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 0 action 7 (hypervisor_monitor_0 on el8-a01n02) confirmed: not running | rc=7 call-id=14
  1519. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/37, version=0.16.11)
  1520. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_finished) debug: finished - rsc:drbd action:monitor call_id:19 exit-code:7 exec-time:58ms queue-time:0ms
  1521. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_lrm_event) notice: Result of probe operation for drbd on el8-a01n02: 7 (not running) | call=19 key=drbd_monitor_0 confirmed=true cib-update=38
  1522. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/38)
  1523. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.11 2
  1524. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.12 (null)
  1525. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=12
  1526. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;8:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=19, @rc-code=7, @op-status=0, @exec-time=58
  1527. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/38, version=0.16.12)
  1528. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 0 action 8 (drbd_monitor_0 on el8-a01n02) confirmed: not running | rc=7 call-id=19
  1529. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.12 2
  1530. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.13 (null)
  1531. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=13
  1532. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource
  1533. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="virsh_node1" type="fence_virsh" class="stonith"/>
  1534. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node1_last_0" operation_key="virsh_node1_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="1:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9" transition-magic="-1:193;1:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-statu
  1535. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ </lrm_resource>
  1536. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/8, version=0.16.13)
  1537. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.13 2
  1538. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.14 (null)
  1539. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=14
  1540. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;1:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=5, @rc-code=7, @op-status=0, @last-rc-change=1594765246, @last-run=1594765246, @exec-time=7
  1541. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 0 action 1 (virsh_node1_monitor_0 on el8-a01n01) confirmed: not running | rc=7 call-id=5
  1542. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating start operation virsh_node1_start_0 on el8-a01n01 | action 9
  1543. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/9, version=0.16.14)
  1544. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.14 2
  1545. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.15 (null)
  1546. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=15
  1547. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="virsh_node2" type="fence_virsh" class="stonith"/>
  1548. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node2_last_0" operation_key="virsh_node2_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="2:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9" transition-magic="-1:193;2:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-statu
  1549. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ </lrm_resource>
  1550. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/10, version=0.16.15)
  1551. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.15 2
  1552. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.16 (null)
  1553. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=16
  1554. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;2:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=9, @rc-code=7, @op-status=0
  1555. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/11, version=0.16.16)
  1556. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.16 2
  1557. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 0 action 2 (virsh_node2_monitor_0 on el8-a01n01) confirmed: not running | rc=7 call-id=9
  1558. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.17 (null)
  1559. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=17
  1560. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating start operation virsh_node2_start_0 locally on el8-a01n02 | action 11
  1561. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_start_0, @operation=start, @transition-key=9:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;9:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0
  1562. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_lrm_rsc_op) info: Performing key=11:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=virsh_node2_start_0
  1563. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/12, version=0.16.17)
  1564. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/39)
  1565. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=20, reply=1, notify=0
  1566. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_execute) info: executing - rsc:virsh_node2 action:start call_id:20
  1567. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating monitor operation hypervisor:0_monitor_0 on el8-a01n01 | action 3
  1568. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.17 2
  1569. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.18 (null)
  1570. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=18
  1571. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=11:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;11:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765246, @last-run=1594765246
  1572. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/39, version=0.16.18)
  1573. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.18 2
  1574. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.19 (null)
  1575. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=19
  1576. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="hypervisor" type="libvirtd" class="systemd"/>
  1577. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ <lrm_rsc_op id="hypervisor_last_0" operation_key="hypervisor_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="3:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9" transition-magic="-1:193;3:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-status=
  1578. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ </lrm_resource>
  1579. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/13, version=0.16.19)
  1580. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.19 2
  1581. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.20 (null)
  1582. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=20
  1583. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;3:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=15, @rc-code=7, @op-status=0, @exec-time=14, @queue-time=2
  1584. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/14, version=0.16.20)
  1585. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 0 action 3 (hypervisor_monitor_0 on el8-a01n01) confirmed: not running | rc=7 call-id=15
  1586. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating monitor operation drbd:0_monitor_0 on el8-a01n01 | action 4
  1587. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating start operation hypervisor:1_start_0 locally on el8-a01n02 | action 15
  1588. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_lrm_rsc_op) info: Performing key=15:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=hypervisor_start_0
  1589. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/40)
  1590. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=21, reply=1, notify=0
  1591. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_execute) info: executing - rsc:hypervisor action:start call_id:21
  1592. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (systemd_unit_exec) debug: Performing asynchronous start op on systemd unit libvirtd named 'hypervisor'
  1593. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.20 2
  1594. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.21 (null)
  1595. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=21
  1596. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=15:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;15:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765246, @last-run=1594765246, @exec-time=0, @
  1597. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/40, version=0.16.21)
  1598. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.21 2
  1599. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.22 (null)
  1600. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=22
  1601. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="drbd" type="drbd" class="systemd"/>
  1602. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ <lrm_rsc_op id="drbd_last_0" operation_key="drbd_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="4:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9" transition-magic="-1:193;4:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-status="-1" interva
  1603. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ </lrm_resource>
  1604. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (systemd_unit_exec_with_unit) debug: Calling StartUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice
  1605. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/15, version=0.16.22)
  1606. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.22 2
  1607. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.23 (null)
  1608. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=23
  1609. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;4:0:7:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=20, @rc-code=7, @op-status=0, @exec-time=19
  1610. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/16, version=0.16.23)
  1611. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 0 action 4 (drbd_monitor_0 on el8-a01n01) confirmed: not running | rc=7 call-id=20
  1612. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating start operation hypervisor:0_start_0 on el8-a01n01 | action 13
  1613. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.23 2
  1614. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.24 (null)
  1615. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=24
  1616. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=13:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;13:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0, @queue-time=0
  1617. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/17, version=0.16.24)
  1618. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (systemd_exec_result) info: Call to start passed: /org/freedesktop/systemd1/job/1182
  1619. Jul 14 22:20:46 el8-a01n02.digimer.ca pacemaker-execd [2199] (action_complete) debug: hypervisor start may still be in progress: re-scheduling (elapsed=224ms, remaining=99776ms, start_delay=2000ms)
  1620. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_finished) info: finished - rsc:virsh_node2 action:start call_id:20 exit-code:0 exec-time:791ms queue-time:0ms
  1621. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/41)
  1622. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_lrm_event) notice: Result of start operation for virsh_node2 on el8-a01n02: 0 (ok) | call=20 key=virsh_node2_start_0 confirmed=true cib-update=41
  1623. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.24 2
  1624. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.25 (null)
  1625. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=25
  1626. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;11:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=20, @rc-code=0, @op-status=0, @exec-time=791
  1627. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/41, version=0.16.25)
  1628. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 0 action 11 (virsh_node2_start_0 on el8-a01n02) confirmed: ok | rc=0 call-id=20
  1629. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating monitor operation virsh_node2_monitor_60000 locally on el8-a01n02 | action 12
  1630. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_lrm_rsc_op) info: Performing key=12:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=virsh_node2_monitor_60000
  1631. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/42)
  1632. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=22, reply=1, notify=0
  1633. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_execute) debug: executing - rsc:virsh_node2 action:monitor call_id:22
  1634. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.25 2
  1635. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.26 (null)
  1636. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=26
  1637. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']: <lrm_rsc_op id="virsh_node2_monitor_60000" operation_key="virsh_node2_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="12:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9" transition-magic="-1:193;12:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9" exit-reason="" on_node="el8-
  1638. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/42, version=0.16.26)
  1639. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.26 2
  1640. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.27 (null)
  1641. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=27
  1642. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:0;9:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=10, @rc-code=0, @op-status=0, @exec-time=922
  1643. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/18, version=0.16.27)
  1644. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 0 action 9 (virsh_node1_start_0 on el8-a01n01) confirmed: ok | rc=0 call-id=10
  1645. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating monitor operation virsh_node1_monitor_60000 on el8-a01n01 | action 10
  1646. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.27 2
  1647. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.28 (null)
  1648. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=28
  1649. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']: <lrm_rsc_op id="virsh_node1_monitor_60000" operation_key="virsh_node1_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="10:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9" transition-magic="-1:193;10:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9" exit-reason="" on_node="el8-
  1650. Jul 14 22:20:47 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/19, version=0.16.28)
  1651. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_finished) debug: finished - rsc:virsh_node2 action:monitor call_id:22 exit-code:0 exec-time:743ms queue-time:0ms
  1652. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_lrm_event) notice: Result of monitor operation for virsh_node2 on el8-a01n02: 0 (ok) | call=22 key=virsh_node2_monitor_60000 confirmed=false cib-update=43
  1653. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/43)
  1654. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.28 2
  1655. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.29 (null)
  1656. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=29
  1657. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_monitor_60000']: @transition-magic=0:0;12:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=22, @rc-code=0, @op-status=0, @exec-time=743
  1658. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/43, version=0.16.29)
  1659. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 0 action 12 (virsh_node2_monitor_60000 on el8-a01n02) confirmed: ok | rc=0 call-id=22
  1660. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.29 2
  1661. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.30 (null)
  1662. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=30
  1663. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_monitor_60000']: @transition-magic=0:0;10:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=22, @rc-code=0, @op-status=0, @exec-time=903
  1664. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/20, version=0.16.30)
  1665. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 0 action 10 (virsh_node1_monitor_60000 on el8-a01n01) confirmed: ok | rc=0 call-id=22
  1666. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:21
  1667. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-execd [2199] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  1668. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-execd [2199] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1669. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-execd [2199] (action_complete) debug: hypervisor start is now complete (elapsed=2234ms, remaining=97766ms): ok (0)
  1670. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:21 exit-code:0 exec-time:2234ms queue-time:1ms
  1671. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_lrm_event) notice: Result of start operation for hypervisor on el8-a01n02: 0 (ok) | call=21 key=hypervisor_start_0 confirmed=true cib-update=44
  1672. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/44)
  1673. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.30 2
  1674. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.31 (null)
  1675. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=31
  1676. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;15:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=21, @rc-code=0, @op-status=0, @exec-time=2234, @queue-time=1
  1677. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/44, version=0.16.31)
  1678. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 0 action 15 (hypervisor_start_0 on el8-a01n02) confirmed: ok | rc=0 call-id=21
  1679. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating monitor operation hypervisor:1_monitor_60000 locally on el8-a01n02 | action 16
  1680. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_lrm_rsc_op) info: Performing key=16:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=hypervisor_monitor_60000
  1681. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/45)
  1682. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=23, reply=1, notify=0
  1683. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:23
  1684. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-execd [2199] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  1685. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.31 2
  1686. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.32 (null)
  1687. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=32
  1688. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']: <lrm_rsc_op id="hypervisor_monitor_60000" operation_key="hypervisor_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="16:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9" transition-magic="-1:193;16:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9" exit-reason="" on_node="el8-a01
  1689. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/45, version=0.16.32)
  1690. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-execd [2199] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1691. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:23 exit-code:0 exec-time:7ms queue-time:0ms
  1692. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_lrm_event) notice: Result of monitor operation for hypervisor on el8-a01n02: 0 (ok) | call=23 key=hypervisor_monitor_60000 confirmed=false cib-update=46
  1693. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/46)
  1694. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.32 2
  1695. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.33 (null)
  1696. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=33
  1697. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_monitor_60000']: @transition-magic=0:0;16:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=23, @rc-code=0, @op-status=0, @exec-time=7
  1698. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/46, version=0.16.33)
  1699. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 0 action 16 (hypervisor_monitor_60000 on el8-a01n02) confirmed: ok | rc=0 call-id=23
  1700. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.33 2
  1701. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.34 (null)
  1702. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=34
  1703. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;13:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=21, @rc-code=0, @op-status=0, @exec-time=2312, @queue-time=1
  1704. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/21, version=0.16.34)
  1705. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 0 action 13 (hypervisor_start_0 on el8-a01n01) confirmed: ok | rc=0 call-id=21
  1706. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating monitor operation hypervisor:0_monitor_60000 on el8-a01n01 | action 14
  1707. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.34 2
  1708. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.35 (null)
  1709. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=35
  1710. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']: <lrm_rsc_op id="hypervisor_monitor_60000" operation_key="hypervisor_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="14:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9" transition-magic="-1:193;14:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9" exit-reason="" on_node="el8-a01
  1711. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/22, version=0.16.35)
  1712. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.35 2
  1713. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.36 (null)
  1714. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=36
  1715. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_monitor_60000']: @transition-magic=0:0;14:0:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=23, @rc-code=0, @op-status=0, @exec-time=4
  1716. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/23, version=0.16.36)
  1717. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 0 action 14 (hypervisor_monitor_60000 on el8-a01n01) confirmed: ok | rc=0 call-id=23
  1718. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-controld [2202] (run_graph) notice: Transition 0 (Complete=18, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-11.bz2): Complete
  1719. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_log) info: Input I_TE_SUCCESS received in state S_TRANSITION_ENGINE from notify_crmd
  1720. Jul 14 22:20:48 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_state_transition) notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd
  1721. Jul 14 22:20:53 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_procfs_pid_of) info: Found pacemaker-based active as process 2197
  1722. Jul 14 22:20:53 el8-a01n02.digimer.ca pacemaker-controld [2202] (throttle_send_command) info: New throttle mode: negligible load (was undetermined)
  1723. Jul 14 22:20:53 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_ping) info: Reporting our current digest to el8-a01n02: f89b1d2175a83990aeb829badbe52c67 for 0.16.36 (0x55efd7cf2cb0 0)
  1724. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemakerd [2196] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  1725. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk_shutdown_worker) notice: Shutting down Pacemaker
  1726. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemakerd [2196] (stop_child) notice: Stopping pacemaker-controld | sent signal 15 to process 2202
  1727. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  1728. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_shutdown) notice: Shutting down cluster resource manager | limit=1200000ms
  1729. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_log) info: Input I_SHUTDOWN received in state S_IDLE from crm_shutdown
  1730. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_state_transition) notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_SHUTDOWN cause=C_SHUTDOWN origin=crm_shutdown
  1731. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_shutdown_req) info: Sending shutdown request to all peers (DC is el8-a01n02)
  1732. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_update_peer_expected) info: handle_request: Node el8-a01n02[2] - expected state is now down (was member)
  1733. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (handle_shutdown_request) info: Creating shutdown request for el8-a01n02 (state=S_POLICY_ENGINE)
  1734. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-attrd [2200] (attrd_peer_update) notice: Setting shutdown[el8-a01n02]: (unset) -> 1594765270 | from el8-a01n02
  1735. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-attrd [2200] (write_attribute) info: Sent CIB request 4 with 1 change for shutdown (id n/a, set n/a)
  1736. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/4)
  1737. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.36 2
  1738. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.37 (null)
  1739. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=37
  1740. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']: <transient_attributes id="2"/>
  1741. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ <instance_attributes id="status-2">
  1742. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ <nvpair id="status-2-shutdown" name="shutdown" value="1594765270"/>
  1743. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ </instance_attributes>
  1744. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ </transient_attributes>
  1745. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/4, version=0.16.37)
  1746. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (abort_transition_graph) info: Transition 0 aborted by transient_attributes.2 'create': Transient attribute change | cib=0.16.37 source=abort_unless_down:329 path=/cib/status/node_state[@id='2'] complete=true
  1747. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-attrd [2200] (attrd_cib_callback) info: CIB update 4 result for shutdown: OK | rc=0
  1748. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-attrd [2200] (attrd_cib_callback) info: * shutdown[el8-a01n02]=1594765270
  1749. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (determine_online_status) info: Node el8-a01n02 is shutting down
  1750. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (determine_online_status_fencing) info: Node el8-a01n01 is active
  1751. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (determine_online_status) info: Node el8-a01n01 is online
  1752. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (unpack_node_loop) info: Node 2 is already processed
  1753. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (unpack_node_loop) info: Node 1 is already processed
  1754. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (unpack_node_loop) info: Node 2 is already processed
  1755. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (unpack_node_loop) info: Node 1 is already processed
  1756. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (common_print) info: virsh_node1 (stonith:fence_virsh): Started el8-a01n01
  1757. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (common_print) info: virsh_node2 (stonith:fence_virsh): Started el8-a01n02
  1758. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  1759. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (short_print) info: Started: [ el8-a01n01 el8-a01n02 ]
  1760. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (clone_print) info: Clone Set: drbd-clone [drbd]
  1761. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (short_print) info: Stopped (disabled): [ el8-a01n01 el8-a01n02 ]
  1762. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (native_color) info: Resource hypervisor:0 cannot run anywhere
  1763. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (native_color) info: Resource drbd:0 cannot run anywhere
  1764. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (native_color) info: Resource drbd:1 cannot run anywhere
  1765. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (RecurringOp) info: Start recurring monitor (60s) for virsh_node2 on el8-a01n01
  1766. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (sched_shutdown_op) notice: Scheduling shutdown of node el8-a01n02
  1767. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogNodeActions) notice: * Shutdown el8-a01n02
  1768. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogActions) info: Leave virsh_node1 (Started el8-a01n01)
  1769. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogAction) notice: * Move virsh_node2 ( el8-a01n02 -> el8-a01n01 )
  1770. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogAction) notice: * Stop hypervisor:0 ( el8-a01n02 ) due to node availability
  1771. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogActions) info: Leave hypervisor:1 (Started el8-a01n01)
  1772. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogActions) info: Leave drbd:0 (Stopped)
  1773. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogActions) info: Leave drbd:1 (Stopped)
  1774. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (pcmk__log_transition_summary) notice: Calculated transition 1, saving inputs in /var/lib/pacemaker/pengine/pe-input-12.bz2
  1775. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response
  1776. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_te_invoke) info: Processing graph 1 (ref=pe_calc-dc-1594765270-28) derived from /var/lib/pacemaker/pengine/pe-input-12.bz2
  1777. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating stop operation virsh_node2_stop_0 locally on el8-a01n02 | action 7
  1778. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_finished) debug: finished - rsc:virsh_node2 action:monitor call_id:22 exit-code:0 exec-time:0ms queue-time:0ms
  1779. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=0, reply=1, notify=0
  1780. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_lrm_rsc_op) info: Performing key=7:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=virsh_node2_stop_0
  1781. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/48)
  1782. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=25, reply=1, notify=0
  1783. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_execute) info: executing - rsc:virsh_node2 action:stop call_id:25
  1784. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_finished) info: finished - rsc:virsh_node2 action:stop call_id:25 exit-code:0 exec-time:0ms queue-time:0ms
  1785. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_lrm_event) info: Result of monitor operation for virsh_node2 on el8-a01n02: Cancelled | call=22 key=virsh_node2_monitor_60000 confirmed=true
  1786. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_lrm_event) notice: Result of stop operation for virsh_node2 on el8-a01n02: 0 (ok) | call=25 key=virsh_node2_stop_0 confirmed=true cib-update=49
  1787. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.37 2
  1788. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.38 (null)
  1789. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=38
  1790. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=7:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;7:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765270, @last-run=1594765270, @exec-time=0
  1791. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/48, version=0.16.38)
  1792. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/49)
  1793. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating stop operation hypervisor_stop_0 locally on el8-a01n02 | action 10
  1794. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.38 2
  1795. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.39 (null)
  1796. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=39
  1797. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;7:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=25, @rc-code=0, @op-status=0
  1798. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-execd [2199] (cancel_recurring_action) info: Cancelling systemd operation hypervisor_status_60000
  1799. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:23 exit-code:0 exec-time:0ms queue-time:0ms
  1800. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=0, reply=1, notify=0
  1801. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_lrm_rsc_op) info: Performing key=10:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9 op=hypervisor_stop_0
  1802. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/49, version=0.16.39)
  1803. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-execd [2199] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from c96da60c-8915-453c-8c89-13f67117ca10: rc=27, reply=1, notify=0
  1804. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_execute) info: executing - rsc:hypervisor action:stop call_id:27
  1805. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-execd [2199] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit libvirtd named 'hypervisor'
  1806. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 1 action 7 (virsh_node2_stop_0 on el8-a01n02) confirmed: ok | rc=0 call-id=25
  1807. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_lrm_event) info: Result of monitor operation for hypervisor on el8-a01n02: Cancelled | call=23 key=hypervisor_monitor_60000 confirmed=true
  1808. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating start operation virsh_node2_start_0 on el8-a01n01 | action 8
  1809. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-execd [2199] (systemd_unit_exec_with_unit) debug: Calling StopUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice
  1810. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/50)
  1811. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.39 2
  1812. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.40 (null)
  1813. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=40
  1814. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=10:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;10:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765270, @last-run=1594765270, @exec-time=0, @qu
  1815. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/50, version=0.16.40)
  1816. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.40 2
  1817. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_update_peer_expected) info: handle_request: Node el8-a01n01[1] - expected state is now down (was member)
  1818. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (handle_shutdown_request) info: Creating shutdown request for el8-a01n01 (state=S_TRANSITION_ENGINE)
  1819. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.41 (null)
  1820. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-attrd [2200] (attrd_peer_update) notice: Setting shutdown[el8-a01n01]: (unset) -> 1594765270 | from el8-a01n02
  1821. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-attrd [2200] (write_attribute) info: Sent CIB request 5 with 2 changes for shutdown (id n/a, set n/a)
  1822. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=41
  1823. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=8:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;8:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765270, @last-run=1594765270
  1824. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/24, version=0.16.41)
  1825. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/5)
  1826. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.41 2
  1827. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.42 (null)
  1828. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=42
  1829. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']: <transient_attributes id="1"/>
  1830. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ <instance_attributes id="status-1">
  1831. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ <nvpair id="status-1-shutdown" name="shutdown" value="1594765270"/>
  1832. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ </instance_attributes>
  1833. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: ++ </transient_attributes>
  1834. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/5, version=0.16.42)
  1835. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-attrd [2200] (attrd_cib_callback) info: CIB update 5 result for shutdown: OK | rc=0
  1836. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-attrd [2200] (attrd_cib_callback) info: * shutdown[el8-a01n01]=1594765270
  1837. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-attrd [2200] (attrd_cib_callback) info: * shutdown[el8-a01n02]=1594765270
  1838. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (abort_transition_graph) notice: Transition 1 aborted by transient_attributes.1 'create': Transient attribute change | cib=0.16.42 source=abort_unless_down:329 path=/cib/status/node_state[@id='1'] complete=false
  1839. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-execd [2199] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/1272
  1840. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-execd [2199] (action_complete) debug: hypervisor stop may still be in progress: re-scheduling (elapsed=231ms, remaining=99769ms, start_delay=2000ms)
  1841. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.42 2
  1842. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.43 (null)
  1843. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=43
  1844. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;8:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=24, @rc-code=0, @op-status=0, @exec-time=749
  1845. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/25, version=0.16.43)
  1846. Jul 14 22:21:10 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 1 action 8 (virsh_node2_start_0 on el8-a01n01) confirmed: ok | rc=0 call-id=24
  1847. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:27
  1848. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-execd [2199] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  1849. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-execd [2199] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1850. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-execd [2199] (action_complete) debug: hypervisor stop is now complete (elapsed=2237ms, remaining=97763ms): not running (7)
  1851. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-execd [2199] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:27 exit-code:0 exec-time:2237ms queue-time:0ms
  1852. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/51)
  1853. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_lrm_event) notice: Result of stop operation for hypervisor on el8-a01n02: 0 (ok) | call=27 key=hypervisor_stop_0 confirmed=true cib-update=51
  1854. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.43 2
  1855. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.44 (null)
  1856. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=44
  1857. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;10:1:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=27, @rc-code=0, @op-status=0, @exec-time=2237
  1858. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/51, version=0.16.44)
  1859. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 1 action 10 (hypervisor_stop_0 on el8-a01n02) confirmed: ok | rc=0 call-id=27
  1860. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-controld [2202] (run_graph) notice: Transition 1 (Complete=5, Pending=0, Fired=0, Skipped=2, Incomplete=2, Source=/var/lib/pacemaker/pengine/pe-input-12.bz2): Stopped
  1861. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_state_transition) info: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd
  1862. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (determine_online_status) info: Node el8-a01n02 is shutting down
  1863. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (determine_online_status) info: Node el8-a01n01 is shutting down
  1864. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (unpack_node_loop) info: Node 2 is already processed
  1865. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (unpack_node_loop) info: Node 1 is already processed
  1866. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (unpack_node_loop) info: Node 2 is already processed
  1867. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (unpack_node_loop) info: Node 1 is already processed
  1868. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (common_print) info: virsh_node1 (stonith:fence_virsh): Started el8-a01n01
  1869. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (common_print) info: virsh_node2 (stonith:fence_virsh): Started el8-a01n01
  1870. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  1871. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (short_print) info: Started: [ el8-a01n01 ]
  1872. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (short_print) info: Stopped: [ el8-a01n02 ]
  1873. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (clone_print) info: Clone Set: drbd-clone [drbd]
  1874. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (short_print) info: Stopped (disabled): [ el8-a01n01 el8-a01n02 ]
  1875. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (native_color) info: Resource virsh_node1 cannot run anywhere
  1876. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (native_color) info: Resource virsh_node2 cannot run anywhere
  1877. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (native_color) info: Resource hypervisor:0 cannot run anywhere
  1878. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (native_color) info: Resource hypervisor:1 cannot run anywhere
  1879. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (native_color) info: Resource drbd:0 cannot run anywhere
  1880. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (native_color) info: Resource drbd:1 cannot run anywhere
  1881. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (sched_shutdown_op) notice: Scheduling shutdown of node el8-a01n01
  1882. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (sched_shutdown_op) notice: Scheduling shutdown of node el8-a01n02
  1883. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogNodeActions) notice: * Shutdown el8-a01n02
  1884. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogNodeActions) notice: * Shutdown el8-a01n01
  1885. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogAction) notice: * Stop virsh_node1 ( el8-a01n01 ) due to node availability
  1886. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogAction) notice: * Stop virsh_node2 ( el8-a01n01 ) due to node availability
  1887. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogAction) notice: * Stop hypervisor:0 ( el8-a01n01 ) due to node availability
  1888. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogActions) info: Leave hypervisor:1 (Stopped)
  1889. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogActions) info: Leave drbd:0 (Stopped)
  1890. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (LogActions) info: Leave drbd:1 (Stopped)
  1891. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (pcmk__log_transition_summary) notice: Calculated transition 2, saving inputs in /var/lib/pacemaker/pengine/pe-input-13.bz2
  1892. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response
  1893. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_te_invoke) info: Processing graph 2 (ref=pe_calc-dc-1594765272-32) derived from /var/lib/pacemaker/pengine/pe-input-13.bz2
  1894. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating stop operation virsh_node1_stop_0 on el8-a01n01 | action 3
  1895. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating stop operation virsh_node2_stop_0 on el8-a01n01 | action 4
  1896. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_rsc_command) notice: Initiating stop operation hypervisor_stop_0 on el8-a01n01 | action 5
  1897. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.44 2
  1898. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.45 (null)
  1899. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=45
  1900. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_stop_0, @operation=stop, @transition-key=3:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;3:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765272, @last-run=1594765272, @exec-time=0
  1901. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/26, version=0.16.45)
  1902. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.45 2
  1903. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.46 (null)
  1904. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=46
  1905. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:0;3:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=26, @rc-code=0, @op-status=0
  1906. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 2 action 3 (virsh_node1_stop_0 on el8-a01n01) confirmed: ok | rc=0 call-id=26
  1907. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/27, version=0.16.46)
  1908. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.46 2
  1909. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.47 (null)
  1910. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=47
  1911. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=4:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;4:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765272, @last-run=1594765272, @exec-time=0
  1912. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/28, version=0.16.47)
  1913. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.47 2
  1914. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.48 (null)
  1915. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=48
  1916. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;4:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=27, @rc-code=0, @op-status=0
  1917. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/29, version=0.16.48)
  1918. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 2 action 4 (virsh_node2_stop_0 on el8-a01n01) confirmed: ok | rc=0 call-id=27
  1919. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.48 2
  1920. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.49 (null)
  1921. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=49
  1922. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=5:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @transition-magic=-1:193;5:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594765272, @last-run=1594765272, @exec-time=0, @queu
  1923. Jul 14 22:21:12 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/30, version=0.16.49)
  1924. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.49 2
  1925. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.50 (null)
  1926. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=50
  1927. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;5:2:0:adab90af-4d5d-488e-8027-cf0e0ac80cd9, @call-id=29, @rc-code=0, @op-status=0, @exec-time=2203
  1928. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/31, version=0.16.50)
  1929. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (process_graph_event) info: Transition 2 action 5 (hypervisor_stop_0 on el8-a01n01) confirmed: ok | rc=0 call-id=29
  1930. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_crm_command) info: Executing crm-event (14) without waiting: do_shutdown on el8-a01n01
  1931. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_crm_command) info: Executing crm-event (15) locally without waiting: do_shutdown on el8-a01n02
  1932. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (te_crm_command) info: crm-event (15) is a local shutdown
  1933. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (run_graph) notice: Transition 2 (Complete=7, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-13.bz2): Complete
  1934. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_log) info: Input I_STOP received in state S_TRANSITION_ENGINE from notify_crmd
  1935. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_state_transition) info: State transition S_TRANSITION_ENGINE -> S_STOPPING | input=I_STOP cause=C_FSA_INTERNAL origin=notify_crmd
  1936. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_dc_release) info: DC role released
  1937. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (pe_ipc_destroy) info: Connection to the scheduler released
  1938. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_te_control) info: Transitioner is now inactive
  1939. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (tengine_stonith_connection_destroy) info: Fencing daemon disconnected
  1940. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_lrm_control) info: Disconnecting from the executor
  1941. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (lrmd_api_disconnect) info: Disconnecting IPC local executor connection
  1942. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (lrmd_ipc_connection_destroy) info: IPC connection destroyed
  1943. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (lrm_connection_destroy) info: Disconnected from executor
  1944. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (lrmd_api_disconnect) info: Disconnecting IPC local executor connection
  1945. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_lrm_control) notice: Disconnected from the executor
  1946. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_cluster_disconnect) info: Disconnecting from cluster infrastructure: corosync
  1947. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_ipcs_dispatch_connection_request) debug: HUP conn (/dev/shm/qb-2199-2202-9-AHVNzI/qb)
  1948. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_ipcs_disconnect) debug: qb_ipcs_disconnect(/dev/shm/qb-2199-2202-9-AHVNzI/qb) state:2
  1949. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-execd [2199] (crm_client_destroy) debug: Destroying 0 events
  1950. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_rb_close_helper) debug: Free'ing ringbuffer: /dev/shm/qb-2199-2202-9-AHVNzI/qb-response-lrmd-header
  1951. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_rb_close_helper) debug: Free'ing ringbuffer: /dev/shm/qb-2199-2202-9-AHVNzI/qb-event-lrmd-header
  1952. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_rb_close_helper) debug: Free'ing ringbuffer: /dev/shm/qb-2199-2202-9-AHVNzI/qb-request-lrmd-header
  1953. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/53)
  1954. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (terminate_cs_connection) notice: Disconnected from Corosync
  1955. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_ha_control) info: Disconnected from the cluster
  1956. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_cib_control) info: Disconnecting from the CIB manager
  1957. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (crmd_cib_connection_destroy) info: Connection to the CIB manager terminated
  1958. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_cib_control) notice: Disconnected from the CIB manager
  1959. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  1960. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_exit) info: Performing A_EXIT_0 - gracefully exiting the controller
  1961. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (do_exit) info: [pacemaker-controld] stopped (0)
  1962. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (crmd_exit) info: Dropping I_RELEASE_SUCCESS: [ state=S_STOPPING cause=C_FSA_INTERNAL origin=do_dc_release ]
  1963. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (crmd_exit) info: Dropping I_TERMINATE: [ state=S_STOPPING cause=C_FSA_INTERNAL origin=do_stop ]
  1964. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (crmd_cs_destroy) info: Corosync connection closed
  1965. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (crmd_cs_destroy) info: Corosync connection closed
  1966. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (crmd_init) info: pacemaker-controld[2202] exiting with status 0 (OK)
  1967. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-controld [2202] (crm_xml_cleanup) info: Cleaning up memory from libxml2
  1968. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk_child_exit) info: pacemaker-controld[2202] exited with status 0 (OK)
  1969. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemakerd [2196] (stop_child) notice: Stopping pacemaker-schedulerd | sent signal 15 to process 2201
  1970. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  1971. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  1972. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-schedulerd[2201] (crm_xml_cleanup) info: Cleaning up memory from libxml2
  1973. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: --- 0.16.50 2
  1974. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: Diff: +++ 0.16.51 (null)
  1975. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib: @num_updates=51
  1976. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_dc_release, @expected=down
  1977. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk_child_exit) info: pacemaker-schedulerd[2201] exited with status 0 (OK)
  1978. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/53, version=0.16.51)
  1979. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemakerd [2196] (stop_child) notice: Stopping pacemaker-attrd | sent signal 15 to process 2200
  1980. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-attrd [2200] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  1981. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-attrd [2200] (main) info: Shutting down attribute manager
  1982. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-attrd [2200] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  1983. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-attrd [2200] (attrd_cib_destroy_cb) info: Connection disconnection complete
  1984. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-attrd [2200] (crm_xml_cleanup) info: Cleaning up memory from libxml2
  1985. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk_child_exit) info: pacemaker-attrd[2200] exited with status 0 (OK)
  1986. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemakerd [2196] (stop_child) notice: Stopping pacemaker-execd | sent signal 15 to process 2199
  1987. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-execd [2199] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  1988. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-execd [2199] (lrmd_exit) info: Terminating with 0 clients
  1989. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-execd [2199] (stonith_api_del_notification) debug: Removing callback for st_notify_disconnect events
  1990. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-execd [2199] (stonith_api_signoff) debug: Disconnecting from the fencer
  1991. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
  1992. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-2198-2199-12-fwH66A/qb-request-stonith-ng-header
  1993. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-2198-2199-12-fwH66A/qb-response-stonith-ng-header
  1994. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-2198-2199-12-fwH66A/qb-event-stonith-ng-header
  1995. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  1996. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-execd [2199] (qb_ipcs_unref) debug: qb_ipcs_unref() - destroying
  1997. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-execd [2199] (crm_xml_cleanup) info: Cleaning up memory from libxml2
  1998. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk_child_exit) info: pacemaker-execd[2199] exited with status 0 (OK)
  1999. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemakerd [2196] (stop_child) notice: Stopping pacemaker-fenced | sent signal 15 to process 2198
  2000. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-fenced [2198] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  2001. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-fenced [2198] (stonith_shutdown) info: Terminating with 0 clients
  2002. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-fenced [2198] (cib_connection_destroy) info: Connection to the CIB manager closed
  2003. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-fenced [2198] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  2004. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-fenced [2198] (crm_xml_cleanup) info: Cleaning up memory from libxml2
  2005. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk_child_exit) info: pacemaker-fenced[2198] exited with status 0 (OK)
  2006. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemakerd [2196] (stop_child) notice: Stopping pacemaker-based | sent signal 15 to process 2197
  2007. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  2008. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_shutdown) info: Disconnected 0 clients
  2009. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_shutdown) info: All clients disconnected (0)
  2010. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (initiate_exit) info: Sending disconnect notification to 2 peers...
  2011. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_shutdown_req) info: Peer el8-a01n02 is requesting to shut down
  2012. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_process_shutdown_req) info: Peer el8-a01n02 has acknowledged our shutdown request
  2013. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (terminate_cib) info: cib_process_shutdown_req: Exiting from mainloop...
  2014. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (crm_cluster_disconnect) info: Disconnecting from cluster infrastructure: corosync
  2015. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (terminate_cs_connection) notice: Disconnected from Corosync
  2016. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (crm_get_peer) info: Created entry d2a899f7-e3db-4826-a2c4-52db1fca8504/0x55efd7d11290 for node el8-a01n02/0 (1 total)
  2017. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (cib_peer_update_callback) info: No more peers
  2018. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (terminate_cib) info: cib_peer_update_callback: Exiting from mainloop...
  2019. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (crm_cluster_disconnect) info: Disconnecting from cluster infrastructure: corosync
  2020. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (cluster_disconnect_cpg) info: No CPG connection
  2021. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (terminate_cs_connection) notice: Disconnected from Corosync
  2022. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  2023. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  2024. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  2025. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemaker-based [2197] (crm_xml_cleanup) info: Cleaning up memory from libxml2
  2026. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk_child_exit) info: pacemaker-based[2197] exited with status 0 (OK)
  2027. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemakerd [2196] (pcmk_shutdown_worker) notice: Shutdown complete
  2028. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemakerd [2196] (qb_ipcs_us_withdraw) info: withdrawing server sockets
  2029. Jul 14 22:21:14 el8-a01n02.digimer.ca pacemakerd [2196] (crm_xml_cleanup) info: Cleaning up memory from libxml2
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!