API tools faq
paste
digimer

Untitled

digimer
Jul 14th, 2020
159
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 236.34 KB | None | 0 0
raw download clone embed print report
  1. [root@el8-a01n01 ~]# cat /var/log/pacemaker/el8-a01n01/pacemaker.log.fence-fix-to-crash
  2. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  3. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (get_cluster_type) info: Detected an active 'corosync' cluster
  4. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (mcp_read_config) info: Reading configure for stack: corosync
  5. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (main) notice: Starting Pacemaker 2.0.3-5.el8_2.1 | build=4b1f869f0f features: generated-manpages agent-manpages ncurses libqb-logging libqb-ipc systemd nagios corosync-native atomic-attrd acls
  6. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (main) info: Maximum core file size is: 18446744073709551615
  7. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (qb_ipcs_us_publish) info: server name: pacemakerd
  8. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (pcmk__ipc_is_authentic_process_active) info: Could not connect to lrmd IPC: Connection refused
  9. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (pcmk__ipc_is_authentic_process_active) info: Could not connect to cib_ro IPC: Connection refused
  10. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (pcmk__ipc_is_authentic_process_active) info: Could not connect to crmd IPC: Connection refused
  11. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (pcmk__ipc_is_authentic_process_active) info: Could not connect to attrd IPC: Connection refused
  12. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (pcmk__ipc_is_authentic_process_active) info: Could not connect to pengine IPC: Connection refused
  13. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (pcmk__ipc_is_authentic_process_active) info: Could not connect to stonith-ng IPC: Connection refused
  14. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (crm_get_peer) info: Created entry e413f253-82b9-44eb-ba68-3784c56675d2/0x563f1a7d78c0 for node el8-a01n01/1 (1 total)
  15. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  16. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (crm_get_peer) info: Node 1 has uuid 1
  17. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online
  18. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (cluster_connect_quorum) warning: Quorum lost
  19. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (crm_get_peer) info: Created entry 2a802492-cc46-41d4-b146-62ef14800e30/0x563f1a7d9860 for node el8-a01n02/2 (2 total)
  20. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  21. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (crm_get_peer) info: Node 2 has uuid 2
  22. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (start_child) info: Using uid=189 and group=189 for process pacemaker-based
  23. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (start_child) info: Forked child 26604 for process pacemaker-based
  24. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (start_child) info: Forked child 26605 for process pacemaker-fenced
  25. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (start_child) info: Forked child 26606 for process pacemaker-execd
  26. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (start_child) info: Using uid=189 and group=189 for process pacemaker-attrd
  27. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (start_child) info: Forked child 26607 for process pacemaker-attrd
  28. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (start_child) info: Using uid=189 and group=189 for process pacemaker-schedulerd
  29. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (start_child) info: Forked child 26608 for process pacemaker-schedulerd
  30. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (start_child) info: Using uid=189 and group=189 for process pacemaker-controld
  31. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (start_child) info: Forked child 26609 for process pacemaker-controld
  32. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (main) notice: Pacemaker daemon successfully started and accepting connections
  33. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (pcmk_quorum_notification) info: Quorum still lost | membership=149 members=1
  34. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification
  35. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (crm_reap_unseen_nodes) info: State of node el8-a01n02[2] is still unknown
  36. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (pcmk_cpg_membership) info: Group pacemakerd event 0: node 1 pid 26603 joined via cpg_join
  37. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (pcmk_cpg_membership) info: Group pacemakerd event 0: el8-a01n01 (node 1 pid 26603) is member
  38. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-execd [26606] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  39. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-execd [26606] (main) notice: Starting Pacemaker local executor
  40. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-execd [26606] (qb_ipcs_us_publish) info: server name: lrmd
  41. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-execd [26606] (main) notice: Pacemaker local executor successfully started and accepting connections
  42. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  43. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (main) notice: Starting Pacemaker CIB manager
  44. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (get_cluster_type) info: Verifying cluster type: 'corosync'
  45. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (get_cluster_type) info: Assuming an active 'corosync' cluster
  46. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (retrieveCib) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.xml (digest: /var/lib/pacemaker/cib/cib.xml.sig)
  47. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  48. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-controld [26609] (main) notice: Starting Pacemaker controller
  49. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_log) info: Input I_STARTUP received in state S_STARTING from crmd_init
  50. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-attrd [26607] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  51. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-attrd [26607] (main) notice: Starting Pacemaker node attribute manager
  52. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  53. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (main) notice: Starting Pacemaker scheduler
  54. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (qb_ipcs_us_publish) info: server name: pengine
  55. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (main) notice: Pacemaker scheduler successfully started and accepting connections
  56. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-fenced [26605] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  57. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-fenced [26605] (main) notice: Starting Pacemaker fencer
  58. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-fenced [26605] (get_cluster_type) info: Verifying cluster type: 'corosync'
  59. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-fenced [26605] (get_cluster_type) info: Assuming an active 'corosync' cluster
  60. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-fenced [26605] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  61. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-fenced [26605] (crm_get_peer) info: Created entry daa3b197-e233-434b-bda5-cc3562ac2f29/0x55b0fb9acd60 for node el8-a01n01/1 (1 total)
  62. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-fenced [26605] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  63. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  64. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-fenced [26605] (crm_get_peer) info: Node 1 has uuid 1
  65. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-fenced [26605] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online
  66. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-fenced [26605] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  67. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-fenced [26605] (init_cs_connection_once) info: Connection to 'corosync': established
  68. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (crm_get_peer) info: Created entry 61627068-208a-4610-a861-48370001e6ab/0x55db1c153a70 for node el8-a01n01/1 (1 total)
  69. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  70. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (crm_get_peer) info: Node 1 has uuid 1
  71. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online
  72. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  73. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (init_cs_connection_once) info: Connection to 'corosync': established
  74. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (qb_ipcs_us_publish) info: server name: cib_ro
  75. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (qb_ipcs_us_publish) info: server name: cib_rw
  76. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (qb_ipcs_us_publish) info: server name: cib_shm
  77. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (main) notice: Pacemaker CIB manager successfully started and accepting connections
  78. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (pcmk_cpg_membership) info: Group cib event 0: node 1 pid 26604 joined via cpg_join
  79. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (pcmk_cpg_membership) info: Group cib event 0: el8-a01n01 (node 1 pid 26604) is member
  80. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-10.raw
  81. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_file_write_with_digest) info: Wrote version 0.11.0 of the CIB to disk (digest: f200e1165b560ade835f6f7bb132756a)
  82. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.AYg3wi (digest: /var/lib/pacemaker/cib/cib.D9qnG9)
  83. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (pcmk_cpg_membership) info: Group cib event 1: node 2 pid 1608 joined via cluster join
  84. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n01 (node 1 pid 26604) is member
  85. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (pcmk_cpg_membership) info: Group pacemakerd event 1: node 2 pid 1607 joined via cluster join
  86. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n01 (node 1 pid 26603) is member
  87. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n02 (node 2 pid 1607) is member
  88. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online
  89. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (crm_cs_flush) info: Sent 0 CPG messages (1 remaining, last=7): Try again (6)
  90. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (pcmk_quorum_notification) notice: Quorum acquired | membership=153 members=2
  91. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemakerd [26603] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification
  92. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (crm_get_peer) info: Created entry 0e34cda3-713c-48a5-a9a5-ea3d695cc5d1/0x55db1c159070 for node el8-a01n02/2 (2 total)
  93. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  94. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (crm_get_peer) info: Node 2 has uuid 2
  95. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n02 (node 2 pid 1608) is member
  96. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online
  97. Jul 14 21:41:23 el8-a01n01.digimer.ca pacemaker-based [26604] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  98. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (get_cluster_type) info: Verifying cluster type: 'corosync'
  99. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (get_cluster_type) info: Assuming an active 'corosync' cluster
  100. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  101. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (main) info: CIB connection active
  102. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (get_cluster_type) info: Verifying cluster type: 'corosync'
  103. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (get_cluster_type) info: Assuming an active 'corosync' cluster
  104. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  105. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/transient_attributes: OK (rc=0, origin=el8-a01n02/attrd/2, version=0.11.0)
  106. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_get_peer) info: Created entry 2eb586c3-8f6c-48b9-895c-61812a68101b/0x564e14aa0eb0 for node el8-a01n01/1 (1 total)
  107. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  108. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (peer_update_callback) info: Cluster node el8-a01n01 is now in unknown state
  109. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (crm_get_peer) info: Created entry 8c3d04f7-0ef4-4574-8d44-253a7ef6ad6f/0x55fed5fe8a20 for node el8-a01n01/1 (1 total)
  110. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  111. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (setup_cib) info: Watching for stonith topology changes
  112. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (qb_ipcs_us_publish) info: server name: stonith-ng
  113. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (main) notice: Pacemaker fencer successfully started and accepting connections
  114. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (pcmk_cpg_membership) info: Group stonith-ng event 0: node 1 pid 26605 joined via cpg_join
  115. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (pcmk_cpg_membership) info: Group stonith-ng event 0: el8-a01n01 (node 1 pid 26605) is member
  116. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (pcmk_cpg_membership) info: Group stonith-ng event 1: node 2 pid 1609 joined via cluster join
  117. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n01 (node 1 pid 26605) is member
  118. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_get_peer) info: Node 1 has uuid 1
  119. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online
  120. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (peer_update_callback) info: Node el8-a01n01 is now a peer | DC=<none> old=0x0000000 new=0x4000000
  121. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds
  122. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (init_cs_connection_once) info: Connection to 'corosync': established
  123. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/4, version=0.11.0)
  124. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (crm_get_peer) info: Node 1 has uuid 1
  125. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online
  126. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  127. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (init_cs_connection_once) info: Connection to 'corosync': established
  128. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (crm_get_peer) info: Created entry 88185fb4-7459-4f48-bbb4-edbf893acdda/0x55b0fbab4990 for node el8-a01n02/2 (2 total)
  129. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  130. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (crm_get_peer) info: Node 2 has uuid 2
  131. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n02 (node 2 pid 1609) is member
  132. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online
  133. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  134. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (init_cib_cache_cb) info: Updating device list from the cib: init
  135. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (cib_devices_update) info: Updating devices to version 0.11.0
  136. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (cluster_connect_quorum) notice: Quorum acquired
  137. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (main) info: Cluster connection active
  138. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_erase_attrs) info: Clearing transient attributes from CIB | xpath=//node_state[@uname='el8-a01n01']/transient_attributes
  139. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n01']/transient_attributes to all (origin=local/attrd/2)
  140. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_get_peer) info: Created entry 5a7f9f65-24be-4e08-a470-b3b842126040/0x564e14aa3790 for node el8-a01n02/2 (2 total)
  141. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  142. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (peer_update_callback) info: Cluster node el8-a01n02 is now in unknown state
  143. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_get_peer) info: Node 2 has uuid 2
  144. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/transient_attributes: OK (rc=0, origin=el8-a01n01/attrd/2, version=0.11.0)
  145. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (qb_ipcs_us_publish) info: server name: attrd
  146. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (main) notice: Pacemaker node attribute manager successfully started and accepting connections
  147. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (pcmk_cpg_membership) info: Group attrd event 0: node 1 pid 26607 joined via cpg_join
  148. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (pcmk_cpg_membership) info: Group attrd event 0: el8-a01n01 (node 1 pid 26607) is member
  149. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/4)
  150. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (crm_get_peer) info: Created entry 34fa34fc-949b-4888-9ab0-9cc01fcfd679/0x55fed5fef060 for node el8-a01n02/2 (2 total)
  151. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  152. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (crm_get_peer) info: Node 2 has uuid 2
  153. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (pcmk_cpg_membership) info: Group attrd event 0: el8-a01n02 (node 2 pid 1611) is member
  154. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online
  155. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  156. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_ha_control) info: Connected to the cluster
  157. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (lrmd_ipc_connect) info: Connecting to executor
  158. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-execd [26606] (crm_client_new) debug: Connecting 0x564b0ba25fb0 for uid=189 gid=189 pid=26609 id=2396817f-4776-4134-be53-572580788be0
  159. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-execd [26606] (handle_new_connection) debug: IPC credentials authenticated (/dev/shm/qb-26606-26609-9-RYddSn/qb)
  160. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-execd [26606] (qb_ipcs_shm_connect) debug: connecting to client [26609]
  161. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-execd [26606] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  162. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-execd [26606] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  163. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-execd [26606] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  164. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed register operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=0
  165. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_check_for_new_writer) notice: Recorded new attribute writer: el8-a01n02 (was unset)
  166. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_message) info: Processing sync-response from el8-a01n02
  167. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n02]: (unset) -> 2 | from el8-a01n02
  168. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/4, version=0.11.0)
  169. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n01]: (unset) -> 2 | from el8-a01n01
  170. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_control) info: Connection to the executor established
  171. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_started) info: Delaying start, no membership data (0000000000100000)
  172. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_started) info: Delaying start, no membership data (0000000000100000)
  173. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (pcmk_quorum_notification) info: Quorum retained | membership=153 members=2
  174. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification
  175. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (peer_update_callback) info: Cluster node el8-a01n01 is now member (was in unknown state)
  176. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification
  177. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (peer_update_callback) info: Cluster node el8-a01n02 is now member (was in unknown state)
  178. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_started) info: Delaying start, Config not read (0000000000000040)
  179. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (qb_ipcs_us_publish) info: server name: crmd
  180. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_started) notice: Pacemaker controller successfully started and accepting connections
  181. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_log) info: Input I_PENDING received in state S_STARTING from do_started
  182. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_state_transition) notice: State transition S_STARTING -> S_PENDING | input=I_PENDING cause=C_FSA_INTERNAL origin=do_started
  183. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 26656 exited with rc=0
  184. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (stonith_device_register) notice: Added 'virsh_node1' to the device list (1 active devices)
  185. Jul 14 21:41:24 el8-a01n01.digimer.ca pacemaker-fenced [26605] (stonith_device_register) notice: Added 'virsh_node2' to the device list (2 active devices)
  186. Jul 14 21:41:25 el8-a01n01.digimer.ca pacemaker-controld [26609] (pcmk_cpg_membership) info: Group crmd event 0: node 1 pid 26609 joined via cpg_join
  187. Jul 14 21:41:25 el8-a01n01.digimer.ca pacemaker-controld [26609] (pcmk_cpg_membership) info: Group crmd event 0: el8-a01n01 (node 1 pid 26609) is member
  188. Jul 14 21:41:25 el8-a01n01.digimer.ca pacemaker-controld [26609] (pcmk_cpg_membership) info: Group crmd event 0: el8-a01n02 (node 2 pid 1613) is member
  189. Jul 14 21:41:25 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online
  190. Jul 14 21:41:25 el8-a01n01.digimer.ca pacemaker-controld [26609] (peer_update_callback) info: Node el8-a01n02 is now a peer | DC=<none> old=0x0000000 new=0x4000000
  191. Jul 14 21:41:25 el8-a01n01.digimer.ca pacemaker-controld [26609] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds
  192. Jul 14 21:41:25 el8-a01n01.digimer.ca pacemaker-controld [26609] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 30 seconds
  193. Jul 14 21:41:25 el8-a01n01.digimer.ca pacemaker-controld [26609] (te_connect_stonith) notice: Fencer successfully connected
  194. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-controld [26609] (election_count_vote) info: election-DC round 1 (owner node ID 2) lost: vote from el8-a01n02 (Uptime)
  195. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_log) info: Input I_PENDING received in state S_PENDING from do_election_count_vote
  196. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-controld [26609] (update_dc) info: Set DC to el8-a01n02 (3.3.0)
  197. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_update_peer_expected) info: update_dc: Node el8-a01n02[2] - expected state is now member (was (null))
  198. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n02/crmd/8, version=0.11.0)
  199. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n02/crmd/10, version=0.11.0)
  200. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n02/crmd/12, version=0.11.0)
  201. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n02/crmd/14, version=0.11.0)
  202. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_log) info: Input I_NOT_DC received in state S_PENDING from do_cl_join_finalize_respond
  203. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_state_transition) notice: State transition S_PENDING -> S_NOT_DC | input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond
  204. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_replace) info: Digest matched on replace from el8-a01n02: 58002fb42d021891f86a2c77da32294a
  205. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_replace) info: Replaced 0.11.0 with 0.11.0 from el8-a01n02
  206. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=el8-a01n02/crmd/17, version=0.11.0)
  207. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/18, version=0.11.0)
  208. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/19, version=0.11.0)
  209. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/lrm: OK (rc=0, origin=el8-a01n02/crmd/20, version=0.11.0)
  210. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.0 2
  211. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.1 (null)
  212. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=1
  213. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status: <node_state id="1" uname="el8-a01n01" in_ccm="true" crmd="online" crm-debug-origin="do_lrm_query_internal"/>
  214. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm id="1">
  215. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_resources/>
  216. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </lrm>
  217. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </node_state>
  218. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/21, version=0.11.1)
  219. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/lrm: OK (rc=0, origin=el8-a01n02/crmd/22, version=0.11.1)
  220. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.1 2
  221. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.2 (null)
  222. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=2
  223. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status: <node_state id="2" uname="el8-a01n02" in_ccm="true" crmd="online" crm-debug-origin="do_lrm_query_internal"/>
  224. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm id="2">
  225. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_resources/>
  226. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </lrm>
  227. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </node_state>
  228. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/23, version=0.11.2)
  229. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/26, version=0.11.2)
  230. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.2 2
  231. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.3 (null)
  232. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=3
  233. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition, @join=member, @expected=member
  234. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition, @join=member, @expected=member
  235. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/27, version=0.11.3)
  236. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.3 2
  237. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.4 (null)
  238. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=4, @dc-uuid=2
  239. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n02/crmd/28, version=0.11.4)
  240. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-11.raw
  241. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_file_write_with_digest) info: Wrote version 0.11.0 of the CIB to disk (digest: 9a696706e038f86744f31e5794c481e4)
  242. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.PtSIAw (digest: /var/lib/pacemaker/cib/cib.otFU1q)
  243. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node1' not in cache
  244. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=0
  245. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node1'
  246. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=1
  247. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=0
  248. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.4 2
  249. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.5 (null)
  250. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=5
  251. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_update_resource
  252. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="virsh_node1" type="fence_virsh" class="stonith"/>
  253. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node1_last_0" operation_key="virsh_node1_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="5:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="-1:193;5:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-stat
  254. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </lrm_resource>
  255. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=1:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16 op=virsh_node1_monitor_0
  256. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/30, version=0.11.5)
  257. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.5 2
  258. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.6 (null)
  259. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=6
  260. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="virsh_node2" type="fence_virsh" class="stonith"/>
  261. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node2_last_0" operation_key="virsh_node2_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="6:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="-1:193;6:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-stat
  262. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </lrm_resource>
  263. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/31, version=0.11.6)
  264. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.6 2
  265. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.7 (null)
  266. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=7
  267. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;5:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=5, @rc-code=7, @op-status=0, @exec-time=3
  268. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/32, version=0.11.7)
  269. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.7 2
  270. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.8 (null)
  271. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=8
  272. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;6:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=9, @rc-code=7, @op-status=0
  273. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/33, version=0.11.8)
  274. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.8 2
  275. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.9 (null)
  276. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=9
  277. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="hypervisor" type="libvirtd" class="systemd"/>
  278. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_rsc_op id="hypervisor_last_0" operation_key="hypervisor_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="7:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="-1:193;7:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-status
  279. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </lrm_resource>
  280. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/34, version=0.11.9)
  281. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.9 2
  282. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.10 (null)
  283. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=10
  284. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="drbd" type="drbd" class="systemd"/>
  285. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_rsc_op id="drbd_last_0" operation_key="drbd_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="8:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="-1:193;8:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-status="-1" interv
  286. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </lrm_resource>
  287. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/35, version=0.11.10)
  288. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.10 2
  289. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.11 (null)
  290. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=11
  291. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;7:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=14, @rc-code=7, @op-status=0, @exec-time=58
  292. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/36, version=0.11.11)
  293. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.11 2
  294. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.12 (null)
  295. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=12
  296. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;8:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=19, @rc-code=7, @op-status=0, @exec-time=59
  297. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/37, version=0.11.12)
  298. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-controld [26609] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 28195 exited with rc=0
  299. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/8)
  300. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2396817f-4776-4134-be53-572580788be0: rc=5, reply=1, notify=0
  301. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:5
  302. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (stonith_api_signon) debug: Attempting fencer connection by pacemaker-execd with mainloop
  303. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.12 2
  304. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.13 (null)
  305. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=13
  306. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource
  307. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="virsh_node1" type="fence_virsh" class="stonith"/>
  308. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node1_last_0" operation_key="virsh_node1_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="1:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="-1:193;1:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-stat
  309. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </lrm_resource>
  310. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/8, version=0.11.13)
  311. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  312. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  313. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  314. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (stonith_api_signon) debug: Connection to fencer by pacemaker-execd succeeded (registration token: 2f10ea00-a756-4d36-9021-1df969795aa5)
  315. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:5 exit-code:7 exec-time:5ms queue-time:0ms
  316. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) notice: Result of probe operation for virsh_node1 on el8-a01n01: 7 (not running) | call=5 key=virsh_node1_monitor_0 confirmed=true cib-update=9
  317. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/9)
  318. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.13 2
  319. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.14 (null)
  320. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=14
  321. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;1:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=5, @rc-code=7, @op-status=0, @exec-time=5
  322. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/9, version=0.11.14)
  323. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node2' not in cache
  324. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=0
  325. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node2'
  326. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=1
  327. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=0
  328. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=2:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16 op=virsh_node2_monitor_0
  329. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/10)
  330. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.14 2
  331. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.15 (null)
  332. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=15
  333. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="virsh_node2" type="fence_virsh" class="stonith"/>
  334. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node2_last_0" operation_key="virsh_node2_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="2:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="-1:193;2:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-stat
  335. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </lrm_resource>
  336. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/10, version=0.11.15)
  337. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2396817f-4776-4134-be53-572580788be0: rc=9, reply=1, notify=0
  338. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:virsh_node2 action:monitor call_id:9
  339. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:virsh_node2 action:monitor call_id:9 exit-code:7 exec-time:0ms queue-time:0ms
  340. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) notice: Result of probe operation for virsh_node2 on el8-a01n01: 7 (not running) | call=9 key=virsh_node2_monitor_0 confirmed=true cib-update=11
  341. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=9:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16 op=virsh_node1_start_0
  342. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/11)
  343. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/12)
  344. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.15 2
  345. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.16 (null)
  346. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=16
  347. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;2:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=9, @rc-code=7, @op-status=0
  348. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2396817f-4776-4134-be53-572580788be0: rc=10, reply=1, notify=0
  349. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) info: executing - rsc:virsh_node1 action:start call_id:10
  350. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/11, version=0.11.16)
  351. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.16 2
  352. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.17 (null)
  353. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=17
  354. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_start_0, @operation=start, @transition-key=9:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @transition-magic=-1:193;9:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0
  355. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/12, version=0.11.17)
  356. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor' not in cache
  357. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=0
  358. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor:0' not in cache
  359. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=0
  360. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_rsc_register) info: Cached agent information for 'hypervisor'
  361. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=1
  362. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=0
  363. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=3:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16 op=hypervisor_monitor_0
  364. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2396817f-4776-4134-be53-572580788be0: rc=15, reply=1, notify=0
  365. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/13)
  366. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:15
  367. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.17 2
  368. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.18 (null)
  369. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=18
  370. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="hypervisor" type="libvirtd" class="systemd"/>
  371. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_rsc_op id="hypervisor_last_0" operation_key="hypervisor_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="3:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="-1:193;3:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-status
  372. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </lrm_resource>
  373. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/13, version=0.11.18)
  374. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.18 2
  375. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.19 (null)
  376. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=19
  377. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=11:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @transition-magic=-1:193;11:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=-1, @rc-code=193, @op-status=-1
  378. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/38, version=0.11.19)
  379. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  380. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  381. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:15 exit-code:7 exec-time:88ms queue-time:0ms
  382. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) notice: Result of probe operation for hypervisor on el8-a01n01: 7 (not running) | call=15 key=hypervisor_monitor_0 confirmed=true cib-update=14
  383. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/14)
  384. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.19 2
  385. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.20 (null)
  386. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=20
  387. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;3:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=15, @rc-code=7, @op-status=0, @exec-time=88
  388. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/14, version=0.11.20)
  389. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_get_rsc_info) info: Agent information for 'drbd' not in cache
  390. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=0
  391. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_get_rsc_info) info: Agent information for 'drbd:0' not in cache
  392. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=0
  393. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_rsc_register) info: Cached agent information for 'drbd'
  394. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=1
  395. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=0
  396. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=4:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16 op=drbd_monitor_0
  397. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/15)
  398. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.20 2
  399. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.21 (null)
  400. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=21
  401. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="drbd" type="drbd" class="systemd"/>
  402. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_rsc_op id="drbd_last_0" operation_key="drbd_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="4:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="-1:193;4:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-status="-1" interv
  403. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </lrm_resource>
  404. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2396817f-4776-4134-be53-572580788be0: rc=20, reply=1, notify=0
  405. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:20
  406. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  407. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/15, version=0.11.21)
  408. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  409. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.21 2
  410. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.22 (null)
  411. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=22
  412. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=15:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @transition-magic=-1:193;15:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0
  413. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/39, version=0.11.22)
  414. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:drbd action:monitor call_id:20 exit-code:7 exec-time:20ms queue-time:0ms
  415. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) notice: Result of probe operation for drbd on el8-a01n01: 7 (not running) | call=20 key=drbd_monitor_0 confirmed=true cib-update=16
  416. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/16)
  417. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.22 2
  418. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.23 (null)
  419. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=23
  420. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;4:0:7:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=20, @rc-code=7, @op-status=0, @exec-time=20
  421. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/16, version=0.11.23)
  422. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=13:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16 op=hypervisor_start_0
  423. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/17)
  424. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2396817f-4776-4134-be53-572580788be0: rc=21, reply=1, notify=0
  425. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) info: executing - rsc:hypervisor action:start call_id:21
  426. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous start op on systemd unit libvirtd named 'hypervisor'
  427. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.23 2
  428. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.24 (null)
  429. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=24
  430. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=13:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @transition-magic=-1:193;13:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0
  431. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/17, version=0.11.24)
  432. Jul 14 21:41:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec_with_unit) debug: Calling StartUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice
  433. Jul 14 21:41:46 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_exec_result) info: Call to start passed: /org/freedesktop/systemd1/job/1012
  434. Jul 14 21:41:46 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: hypervisor start may still be in progress: re-scheduling (elapsed=355ms, remaining=99645ms, start_delay=2000ms)
  435. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.24 2
  436. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.25 (null)
  437. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=25
  438. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;15:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=21, @rc-code=0, @op-status=0, @exec-time=2253
  439. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/40, version=0.11.25)
  440. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.25 2
  441. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.26 (null)
  442. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=26
  443. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']: <lrm_rsc_op id="hypervisor_monitor_60000" operation_key="hypervisor_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="16:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="-1:193;16:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a0
  444. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/41, version=0.11.26)
  445. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.26 2
  446. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.27 (null)
  447. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=27
  448. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_monitor_60000']: @transition-magic=0:0;16:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=22, @rc-code=0, @op-status=0, @exec-time=2, @queue-time=1
  449. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/42, version=0.11.27)
  450. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.27 2
  451. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.28 (null)
  452. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=28
  453. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_start_0, @operation=start, @transition-key=23:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @transition-magic=-1:193;23:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762907, @last-run=1594762907, @exec-time=0
  454. Jul 14 21:41:47 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/43, version=0.11.28)
  455. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:21
  456. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  457. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  458. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: hypervisor start is now complete (elapsed=2364ms, remaining=97636ms): ok (0)
  459. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:21 exit-code:0 exec-time:2364ms queue-time:1ms
  460. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/18)
  461. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.28 2
  462. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.29 (null)
  463. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=29
  464. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;13:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=21, @rc-code=0, @op-status=0, @exec-time=2364, @queue-time=1
  465. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) notice: Result of start operation for hypervisor on el8-a01n01: 0 (ok) | call=21 key=hypervisor_start_0 confirmed=true cib-update=18
  466. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/18, version=0.11.29)
  467. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=14:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16 op=hypervisor_monitor_60000
  468. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/19)
  469. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2396817f-4776-4134-be53-572580788be0: rc=22, reply=1, notify=0
  470. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:22
  471. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  472. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.29 2
  473. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.30 (null)
  474. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=30
  475. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']: <lrm_rsc_op id="hypervisor_monitor_60000" operation_key="hypervisor_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="14:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="-1:193;14:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a0
  476. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/19, version=0.11.30)
  477. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  478. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:22 exit-code:0 exec-time:5ms queue-time:0ms
  479. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) notice: Result of monitor operation for hypervisor on el8-a01n01: 0 (ok) | call=22 key=hypervisor_monitor_60000 confirmed=false cib-update=20
  480. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/20)
  481. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.30 2
  482. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.31 (null)
  483. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=31
  484. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_monitor_60000']: @transition-magic=0:0;14:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=22, @rc-code=0, @op-status=0, @exec-time=5
  485. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/20, version=0.11.31)
  486. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=21:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16 op=drbd_start_0
  487. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2396817f-4776-4134-be53-572580788be0: rc=23, reply=1, notify=0
  488. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/21)
  489. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) info: executing - rsc:drbd action:start call_id:23
  490. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous start op on systemd unit drbd named 'drbd'
  491. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec_with_unit) debug: Calling StartUnit for drbd: /org/freedesktop/systemd1/unit/drbd_2eservice
  492. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.31 2
  493. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.32 (null)
  494. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=32
  495. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_start_0, @operation=start, @transition-key=21:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @transition-magic=-1:193;21:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762908, @last-run=1594762908, @exec-time=0
  496. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/21, version=0.11.32)
  497. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_exec_result) info: Call to start passed: /org/freedesktop/systemd1/job/1096
  498. Jul 14 21:41:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd start may still be in progress: re-scheduling (elapsed=210ms, remaining=99790ms, start_delay=2000ms)
  499. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.32 2
  500. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.33 (null)
  501. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=33
  502. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;23:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=23, @rc-code=0, @op-status=0, @exec-time=2191
  503. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/44, version=0.11.33)
  504. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.33 2
  505. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.34 (null)
  506. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=34
  507. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']: <lrm_rsc_op id="drbd_monitor_60000" operation_key="drbd_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="24:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="-1:193;24:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n02" call-id="-1"
  508. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/45, version=0.11.34)
  509. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.34 2
  510. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.35 (null)
  511. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=35
  512. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_monitor_60000']: @transition-magic=0:0;24:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=24, @rc-code=0, @op-status=0, @exec-time=4
  513. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/46, version=0.11.35)
  514. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:23
  515. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  516. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  517. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd start is now complete (elapsed=2216ms, remaining=97784ms): ok (0)
  518. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:drbd action:monitor call_id:23 exit-code:0 exec-time:2216ms queue-time:0ms
  519. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) notice: Result of start operation for drbd on el8-a01n01: 0 (ok) | call=23 key=drbd_start_0 confirmed=true cib-update=22
  520. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/22)
  521. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.35 2
  522. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.36 (null)
  523. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=36
  524. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;21:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=23, @rc-code=0, @op-status=0, @exec-time=2216
  525. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/22, version=0.11.36)
  526. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=22:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16 op=drbd_monitor_60000
  527. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2396817f-4776-4134-be53-572580788be0: rc=24, reply=1, notify=0
  528. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/23)
  529. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:24
  530. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  531. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.36 2
  532. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.37 (null)
  533. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=37
  534. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']: <lrm_rsc_op id="drbd_monitor_60000" operation_key="drbd_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="22:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="-1:193;22:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n01" call-id="-1"
  535. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/23, version=0.11.37)
  536. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  537. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:drbd action:monitor call_id:24 exit-code:0 exec-time:5ms queue-time:0ms
  538. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) notice: Result of monitor operation for drbd on el8-a01n01: 0 (ok) | call=24 key=drbd_monitor_60000 confirmed=false cib-update=24
  539. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/24)
  540. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.37 2
  541. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.11.38 (null)
  542. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=38
  543. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_monitor_60000']: @transition-magic=0:0;22:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=24, @rc-code=0, @op-status=0, @exec-time=5
  544. Jul 14 21:41:50 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/24, version=0.11.38)
  545. Jul 14 21:41:51 el8-a01n01.digimer.ca pacemaker-fenced [26605] (operation_finished) notice: fence_virsh_monitor_1:28212:stderr [ 2020-07-14 21:41:50,926 ERROR: Unable to connect/login to fencing device ]
  546. Jul 14 21:41:51 el8-a01n01.digimer.ca pacemaker-fenced [26605] (operation_finished) notice: fence_virsh_monitor_1:28212:stderr [ ]
  547. Jul 14 21:41:51 el8-a01n01.digimer.ca pacemaker-fenced [26605] (operation_finished) notice: fence_virsh_monitor_1:28212:stderr [ ]
  548. Jul 14 21:41:51 el8-a01n01.digimer.ca pacemaker-fenced [26605] (log_action) warning: fence_virsh[28212] stderr: [ 2020-07-14 21:41:50,926 ERROR: Unable to connect/login to fencing device ]
  549. Jul 14 21:41:51 el8-a01n01.digimer.ca pacemaker-fenced [26605] (log_action) warning: fence_virsh[28212] stderr: [ ]
  550. Jul 14 21:41:51 el8-a01n01.digimer.ca pacemaker-fenced [26605] (log_action) warning: fence_virsh[28212] stderr: [ ]
  551. Jul 14 21:41:51 el8-a01n01.digimer.ca pacemaker-fenced [26605] (internal_stonith_action_execute) info: Attempt 2 to execute fence_virsh (monitor). remaining timeout is 14
  552. Jul 14 21:41:54 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_procfs_pid_of) info: Found pacemaker-based active as process 26604
  553. Jul 14 21:41:54 el8-a01n01.digimer.ca pacemaker-controld [26609] (throttle_send_command) info: New throttle mode: negligible load (was undetermined)
  554. Jul 14 21:41:54 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_replace operation for section configuration to all (origin=local/cibadmin/2)
  555. Jul 14 21:41:54 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.11.38 2
  556. Jul 14 21:41:54 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.0 (null)
  557. Jul 14 21:41:54 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @epoch=12, @num_updates=0
  558. Jul 14 21:41:54 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/configuration/resources/primitive[@id='virsh_node1']/instance_attributes[@id='virsh_node1-instance_attributes']/nvpair[@id='virsh_node1-instance_attributes-ipaddr']: @value=192.168.122.1
  559. Jul 14 21:41:54 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_replace operation for section configuration: OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.12.0)
  560. Jul 14 21:41:54 el8-a01n01.digimer.ca pacemaker-fenced [26605] (update_cib_stonith_devices_v2) info: Updating device list from the cib: modify nvpair[@id='virsh_node1-instance_attributes-ipaddr']
  561. Jul 14 21:41:54 el8-a01n01.digimer.ca pacemaker-fenced [26605] (cib_devices_update) info: Updating devices to version 0.12.0
  562. Jul 14 21:41:54 el8-a01n01.digimer.ca pacemaker-fenced [26605] (stonith_device_register) info: Overwriting an existing entry for virsh_node1 from the cib
  563. Jul 14 21:41:54 el8-a01n01.digimer.ca pacemaker-fenced [26605] (stonith_device_register) notice: Added 'virsh_node1' to the device list (2 active devices)
  564. Jul 14 21:41:54 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-12.raw
  565. Jul 14 21:41:54 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_file_write_with_digest) info: Wrote version 0.12.0 of the CIB to disk (digest: daf4c048d1270debdb32bf77a2ca60d5)
  566. Jul 14 21:41:54 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.ol3r8i (digest: /var/lib/pacemaker/cib/cib.bZujRF)
  567. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.0 2
  568. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.1 (null)
  569. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=1
  570. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:1;11:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=20, @rc-code=1, @op-status=0, @exec-time=11535
  571. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']: <lrm_rsc_op id="virsh_node2_last_failure_0" operation_key="virsh_node2_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="11:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="0:1;11:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n02" c
  572. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/47, version=0.12.1)
  573. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting fail-count-virsh_node2#start_0[el8-a01n02]: (unset) -> INFINITY | from el8-a01n02
  574. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting last-failure-virsh_node2#start_0[el8-a01n02]: (unset) -> 1594762917 | from el8-a01n02
  575. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.1 2
  576. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.2 (null)
  577. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=2
  578. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']: <transient_attributes id="2"/>
  579. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <instance_attributes id="status-2">
  580. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <nvpair id="status-2-fail-count-virsh_node2.start_0" name="fail-count-virsh_node2#start_0" value="INFINITY"/>
  581. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </instance_attributes>
  582. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </transient_attributes>
  583. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/4, version=0.12.2)
  584. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.2 2
  585. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.3 (null)
  586. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=3
  587. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: <nvpair id="status-2-last-failure-virsh_node2.start_0" name="last-failure-virsh_node2#start_0" value="1594762917"/>
  588. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/5, version=0.12.3)
  589. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-fenced [26605] (operation_finished) notice: fence_virsh_monitor_2:28816:stderr [ 2020-07-14 21:41:57,161 ERROR: Unable to connect/login to fencing device ]
  590. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-fenced [26605] (operation_finished) notice: fence_virsh_monitor_2:28816:stderr [ ]
  591. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-fenced [26605] (operation_finished) notice: fence_virsh_monitor_2:28816:stderr [ ]
  592. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-fenced [26605] (log_action) warning: fence_virsh[28816] stderr: [ 2020-07-14 21:41:57,161 ERROR: Unable to connect/login to fencing device ]
  593. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-fenced [26605] (log_action) warning: fence_virsh[28816] stderr: [ ]
  594. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-fenced [26605] (log_action) warning: fence_virsh[28816] stderr: [ ]
  595. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-fenced [26605] (update_remaining_timeout) info: Attempted to execute agent fence_virsh (monitor) the maximum number of times (2) allowed
  596. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-fenced [26605] (log_operation) notice: Operation 'monitor' [28816] for device 'virsh_node1' returned: -201 (Generic Pacemaker error)
  597. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) info: finished - rsc:virsh_node1 action:start call_id:10 exit-code:1 exec-time:11666ms queue-time:0ms
  598. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) notice: Result of start operation for virsh_node1 on el8-a01n01: 1 (error) | call=10 key=virsh_node1_start_0 confirmed=true cib-update=25
  599. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/25)
  600. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.3 2
  601. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.4 (null)
  602. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=4
  603. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:1;9:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=10, @rc-code=1, @op-status=0, @exec-time=11666
  604. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']: <lrm_rsc_op id="virsh_node1_last_failure_0" operation_key="virsh_node1_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="9:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="0:1;9:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n01" cal
  605. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/25, version=0.12.4)
  606. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting fail-count-virsh_node1#start_0[el8-a01n01]: (unset) -> INFINITY | from el8-a01n02
  607. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting last-failure-virsh_node1#start_0[el8-a01n01]: (unset) -> 1594762917 | from el8-a01n02
  608. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.4 2
  609. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.5 (null)
  610. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=5
  611. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']: <transient_attributes id="1"/>
  612. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <instance_attributes id="status-1">
  613. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <nvpair id="status-1-fail-count-virsh_node1.start_0" name="fail-count-virsh_node1#start_0" value="INFINITY"/>
  614. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </instance_attributes>
  615. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </transient_attributes>
  616. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/6, version=0.12.5)
  617. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.5 2
  618. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.6 (null)
  619. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=6
  620. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']: <nvpair id="status-1-last-failure-virsh_node1.start_0" name="last-failure-virsh_node1#start_0" value="1594762917"/>
  621. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/7, version=0.12.6)
  622. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=2:2:0:11323fbf-740e-4c7f-8ee5-4d98173fee16 op=virsh_node1_stop_0
  623. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/26)
  624. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2396817f-4776-4134-be53-572580788be0: rc=25, reply=1, notify=0
  625. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) info: executing - rsc:virsh_node1 action:stop call_id:25
  626. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) info: finished - rsc:virsh_node1 action:stop call_id:25 exit-code:0 exec-time:1ms queue-time:0ms
  627. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.6 2
  628. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.7 (null)
  629. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=7
  630. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_stop_0, @operation=stop, @transition-key=2:2:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @transition-magic=-1:193;2:2:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762917, @last-run=1594762917, @exec-time=0
  631. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/26, version=0.12.7)
  632. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/27)
  633. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) notice: Result of stop operation for virsh_node1 on el8-a01n01: 0 (ok) | call=25 key=virsh_node1_stop_0 confirmed=true cib-update=27
  634. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-controld [26609] (update_attrd_clear_failures) info: Asking pacemaker-attrd to clear failure of all operations for virsh_node1 on cluster node el8-a01n01
  635. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.7 2
  636. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.8 (null)
  637. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=8
  638. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=5:2:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @transition-magic=-1:193;5:2:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762917, @last-run=1594762917, @exec-time=0
  639. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/51, version=0.12.8)
  640. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.8 2
  641. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.9 (null)
  642. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=9
  643. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:0;2:2:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=25, @rc-code=0, @op-status=0, @exec-time=1
  644. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/27, version=0.12.9)
  645. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_delete operation for section /cib/status/node_state[@uname='el8-a01n01']/lrm/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_failure_0'] to all (origin=local/crmd/28)
  646. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting last-failure-virsh_node1#start_0[el8-a01n01]: 1594762917 -> (unset) | from el8-a01n01
  647. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting fail-count-virsh_node1#start_0[el8-a01n01]: INFINITY -> (unset) | from el8-a01n01
  648. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.9 2
  649. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.10 (null)
  650. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_failure_0']
  651. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=10
  652. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_delete operation for section /cib/status/node_state[@uname='el8-a01n01']/lrm/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_failure_0']: OK (rc=0, origin=el8-a01n01/crmd/28, version=0.12.10)
  653. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.10 2
  654. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.11 (null)
  655. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-last-failure-virsh_node1.start_0']
  656. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=11
  657. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/8, version=0.12.11)
  658. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.11 2
  659. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.12 (null)
  660. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-fail-count-virsh_node1.start_0']
  661. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=12
  662. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/9, version=0.12.12)
  663. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.12 2
  664. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.13 (null)
  665. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=13
  666. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;5:2:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=25, @rc-code=0, @op-status=0, @exec-time=1
  667. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/52, version=0.12.13)
  668. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=5:3:0:11323fbf-740e-4c7f-8ee5-4d98173fee16 op=virsh_node1_start_0
  669. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/29)
  670. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2396817f-4776-4134-be53-572580788be0: rc=26, reply=1, notify=0
  671. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) info: executing - rsc:virsh_node1 action:start call_id:26
  672. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=7:3:0:11323fbf-740e-4c7f-8ee5-4d98173fee16 op=virsh_node2_start_0
  673. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2396817f-4776-4134-be53-572580788be0: rc=27, reply=1, notify=0
  674. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) info: executing - rsc:virsh_node2 action:start call_id:27
  675. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.13 2
  676. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.14 (null)
  677. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=14
  678. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_start_0, @operation=start, @transition-key=5:3:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @transition-magic=-1:193;5:3:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0, @op-digest=5954d092e67eaaaf96b50bb5a144e5ff, @op-
  679. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/29, version=0.12.14)
  680. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/30)
  681. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.14 2
  682. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.15 (null)
  683. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=15
  684. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=7:3:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @transition-magic=-1:193;7:3:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762917, @last-run=1594762917
  685. Jul 14 21:41:57 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/30, version=0.12.15)
  686. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) info: finished - rsc:virsh_node1 action:start call_id:26 exit-code:0 exec-time:668ms queue-time:0ms
  687. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) notice: Result of start operation for virsh_node1 on el8-a01n01: 0 (ok) | call=26 key=virsh_node1_start_0 confirmed=true cib-update=31
  688. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/31)
  689. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.15 2
  690. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.16 (null)
  691. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=16
  692. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:0;5:3:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=26, @rc-code=0, @op-status=0, @exec-time=668
  693. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/31, version=0.12.16)
  694. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=6:3:0:11323fbf-740e-4c7f-8ee5-4d98173fee16 op=virsh_node1_monitor_60000
  695. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/32)
  696. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2396817f-4776-4134-be53-572580788be0: rc=28, reply=1, notify=0
  697. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:28
  698. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.16 2
  699. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.17 (null)
  700. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=17
  701. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']: <lrm_rsc_op id="virsh_node1_monitor_60000" operation_key="virsh_node1_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="6:3:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="-1:193;6:3:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a
  702. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/32, version=0.12.17)
  703. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:28 exit-code:0 exec-time:675ms queue-time:0ms
  704. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) notice: Result of monitor operation for virsh_node1 on el8-a01n01: 0 (ok) | call=28 key=virsh_node1_monitor_60000 confirmed=false cib-update=33
  705. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/33)
  706. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.17 2
  707. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.12.18 (null)
  708. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=18
  709. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_monitor_60000']: @transition-magic=0:0;6:3:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=28, @rc-code=0, @op-status=0, @exec-time=675
  710. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/33, version=0.12.18)
  711. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_replace operation for section configuration to all (origin=local/cibadmin/2)
  712. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.12.18 2
  713. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.0 (null)
  714. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @epoch=13, @num_updates=0
  715. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/configuration/resources/primitive[@id='virsh_node2']/instance_attributes[@id='virsh_node2-instance_attributes']/nvpair[@id='virsh_node2-instance_attributes-ipaddr']: @value=192.168.122.1
  716. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-fenced [26605] (update_cib_stonith_devices_v2) info: Updating device list from the cib: modify nvpair[@id='virsh_node2-instance_attributes-ipaddr']
  717. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-fenced [26605] (cib_devices_update) info: Updating devices to version 0.13.0
  718. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-fenced [26605] (stonith_device_register) info: Overwriting an existing entry for virsh_node2 from the cib
  719. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_replace operation for section configuration: OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.13.0)
  720. Jul 14 21:41:58 el8-a01n01.digimer.ca pacemaker-fenced [26605] (stonith_device_register) notice: Added 'virsh_node2' to the device list (2 active devices)
  721. Jul 14 21:41:59 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-13.raw
  722. Jul 14 21:41:59 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_file_write_with_digest) info: Wrote version 0.13.0 of the CIB to disk (digest: 6876636a76b471e9db232dbf63bd5909)
  723. Jul 14 21:41:59 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.6SmAJ5 (digest: /var/lib/pacemaker/cib/cib.BjB2JE)
  724. Jul 14 21:42:02 el8-a01n01.digimer.ca pacemaker-fenced [26605] (operation_finished) notice: fence_virsh_monitor_1:29152:stderr [ 2020-07-14 21:42:02,447 ERROR: Unable to connect/login to fencing device ]
  725. Jul 14 21:42:02 el8-a01n01.digimer.ca pacemaker-fenced [26605] (operation_finished) notice: fence_virsh_monitor_1:29152:stderr [ ]
  726. Jul 14 21:42:02 el8-a01n01.digimer.ca pacemaker-fenced [26605] (operation_finished) notice: fence_virsh_monitor_1:29152:stderr [ ]
  727. Jul 14 21:42:02 el8-a01n01.digimer.ca pacemaker-fenced [26605] (log_action) warning: fence_virsh[29152] stderr: [ 2020-07-14 21:42:02,447 ERROR: Unable to connect/login to fencing device ]
  728. Jul 14 21:42:02 el8-a01n01.digimer.ca pacemaker-fenced [26605] (log_action) warning: fence_virsh[29152] stderr: [ ]
  729. Jul 14 21:42:02 el8-a01n01.digimer.ca pacemaker-fenced [26605] (log_action) warning: fence_virsh[29152] stderr: [ ]
  730. Jul 14 21:42:02 el8-a01n01.digimer.ca pacemaker-fenced [26605] (internal_stonith_action_execute) info: Attempt 2 to execute fence_virsh (monitor). remaining timeout is 15
  731. Jul 14 21:42:03 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_ping) info: Reporting our current digest to el8-a01n02: a781c069a7af021eb6ff95d7fc802f5d for 0.13.0 (0x55db1c261f20 0)
  732. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-fenced [26605] (operation_finished) notice: fence_virsh_monitor_2:29562:stderr [ 2020-07-14 21:42:08,652 ERROR: Unable to connect/login to fencing device ]
  733. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-fenced [26605] (operation_finished) notice: fence_virsh_monitor_2:29562:stderr [ ]
  734. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-fenced [26605] (operation_finished) notice: fence_virsh_monitor_2:29562:stderr [ ]
  735. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-fenced [26605] (log_action) warning: fence_virsh[29562] stderr: [ 2020-07-14 21:42:08,652 ERROR: Unable to connect/login to fencing device ]
  736. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-fenced [26605] (log_action) warning: fence_virsh[29562] stderr: [ ]
  737. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-fenced [26605] (log_action) warning: fence_virsh[29562] stderr: [ ]
  738. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-fenced [26605] (update_remaining_timeout) info: Attempted to execute agent fence_virsh (monitor) the maximum number of times (2) allowed
  739. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-fenced [26605] (log_operation) notice: Operation 'monitor' [29562] for device 'virsh_node2' returned: -201 (Generic Pacemaker error)
  740. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) info: finished - rsc:virsh_node2 action:start call_id:27 exit-code:1 exec-time:11422ms queue-time:0ms
  741. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) notice: Result of start operation for virsh_node2 on el8-a01n01: 1 (error) | call=27 key=virsh_node2_start_0 confirmed=true cib-update=34
  742. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/34)
  743. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.0 2
  744. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.1 (null)
  745. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=1
  746. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:1;7:3:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=27, @rc-code=1, @op-status=0, @exec-time=11422
  747. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']: <lrm_rsc_op id="virsh_node2_last_failure_0" operation_key="virsh_node2_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="7:3:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="0:1;7:3:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n01" cal
  748. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting fail-count-virsh_node2#start_0[el8-a01n01]: (unset) -> INFINITY | from el8-a01n02
  749. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting last-failure-virsh_node2#start_0[el8-a01n01]: (unset) -> 1594762928 | from el8-a01n02
  750. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/34, version=0.13.1)
  751. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.1 2
  752. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.2 (null)
  753. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=2
  754. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']: <nvpair id="status-1-fail-count-virsh_node2.start_0" name="fail-count-virsh_node2#start_0" value="INFINITY"/>
  755. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/10, version=0.13.2)
  756. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.2 2
  757. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.3 (null)
  758. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=3
  759. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']: <nvpair id="status-1-last-failure-virsh_node2.start_0" name="last-failure-virsh_node2#start_0" value="1594762928"/>
  760. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/11, version=0.13.3)
  761. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=3:4:0:11323fbf-740e-4c7f-8ee5-4d98173fee16 op=virsh_node2_stop_0
  762. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/35)
  763. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2396817f-4776-4134-be53-572580788be0: rc=29, reply=1, notify=0
  764. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) info: executing - rsc:virsh_node2 action:stop call_id:29
  765. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting fail-count-virsh_node2#start_0[el8-a01n02]: INFINITY -> (unset) | from el8-a01n02
  766. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-controld [26609] (update_attrd_clear_failures) info: Asking pacemaker-attrd to clear failure of all operations for virsh_node2 on cluster node el8-a01n01
  767. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting last-failure-virsh_node2#start_0[el8-a01n02]: 1594762917 -> (unset) | from el8-a01n02
  768. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) info: finished - rsc:virsh_node2 action:stop call_id:29 exit-code:0 exec-time:1ms queue-time:0ms
  769. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting fail-count-virsh_node2#start_0[el8-a01n01]: INFINITY -> (unset) | from el8-a01n01
  770. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting last-failure-virsh_node2#start_0[el8-a01n01]: 1594762928 -> (unset) | from el8-a01n01
  771. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) notice: Result of stop operation for virsh_node2 on el8-a01n01: 0 (ok) | call=29 key=virsh_node2_stop_0 confirmed=true cib-update=37
  772. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.3 2
  773. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.4 (null)
  774. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=4
  775. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=3:4:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @transition-magic=-1:193;3:4:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762928, @last-run=1594762928, @exec-time=0
  776. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/35, version=0.13.4)
  777. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.4 2
  778. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.5 (null)
  779. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_failure_0']
  780. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=5
  781. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_delete operation for section /cib/status/node_state[@uname='el8-a01n02']/lrm/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_failure_0']: OK (rc=0, origin=el8-a01n02/crmd/57, version=0.13.5)
  782. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_delete operation for section /cib/status/node_state[@uname='el8-a01n01']/lrm/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_failure_0'] to all (origin=local/crmd/36)
  783. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/37)
  784. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.5 2
  785. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.6 (null)
  786. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-fail-count-virsh_node2.start_0']
  787. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=6
  788. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/12, version=0.13.6)
  789. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.6 2
  790. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.7 (null)
  791. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-last-failure-virsh_node2.start_0']
  792. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=7
  793. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/13, version=0.13.7)
  794. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.7 2
  795. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.8 (null)
  796. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_failure_0']
  797. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=8
  798. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_delete operation for section /cib/status/node_state[@uname='el8-a01n01']/lrm/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_failure_0']: OK (rc=0, origin=el8-a01n01/crmd/36, version=0.13.8)
  799. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.8 2
  800. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.9 (null)
  801. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=9
  802. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;3:4:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=29, @rc-code=0, @op-status=0, @exec-time=1
  803. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/37, version=0.13.9)
  804. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.9 2
  805. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.10 (null)
  806. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-fail-count-virsh_node2.start_0']
  807. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=10
  808. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/14, version=0.13.10)
  809. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.10 2
  810. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.11 (null)
  811. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-last-failure-virsh_node2.start_0']
  812. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=11
  813. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/15, version=0.13.11)
  814. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.11 2
  815. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.12 (null)
  816. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=12
  817. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=8:5:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @transition-magic=-1:193;8:5:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594762928, @last-run=1594762928, @exec-time=0,
  818. Jul 14 21:42:08 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/60, version=0.13.12)
  819. Jul 14 21:42:09 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.12 2
  820. Jul 14 21:42:09 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.13 (null)
  821. Jul 14 21:42:09 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=13
  822. Jul 14 21:42:09 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;8:5:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=26, @rc-code=0, @op-status=0, @exec-time=696
  823. Jul 14 21:42:09 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/61, version=0.13.13)
  824. Jul 14 21:42:09 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.13 2
  825. Jul 14 21:42:09 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.14 (null)
  826. Jul 14 21:42:09 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=14
  827. Jul 14 21:42:09 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']: <lrm_rsc_op id="virsh_node2_monitor_60000" operation_key="virsh_node2_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="9:5:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="-1:193;9:5:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a
  828. Jul 14 21:42:09 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/62, version=0.13.14)
  829. Jul 14 21:42:10 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.14 2
  830. Jul 14 21:42:10 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.15 (null)
  831. Jul 14 21:42:10 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=15
  832. Jul 14 21:42:10 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_monitor_60000']: @transition-magic=0:0;9:5:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=27, @rc-code=0, @op-status=0, @exec-time=623
  833. Jul 14 21:42:10 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/63, version=0.13.15)
  834. Jul 14 21:42:15 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_ping) info: Reporting our current digest to el8-a01n02: 2bcc92dbb408e1ac1b2596aea324d92c for 0.13.15 (0x55db1c268910 0)
  835. Jul 14 21:42:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (recurring_action_timer) debug: Scheduling another invocation of hypervisor_status_60000
  836. Jul 14 21:42:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  837. Jul 14 21:42:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  838. Jul 14 21:42:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:22 exit-code:0 exec-time:0ms queue-time:0ms
  839. Jul 14 21:42:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (recurring_action_timer) debug: Scheduling another invocation of drbd_status_60000
  840. Jul 14 21:42:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  841. Jul 14 21:42:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  842. Jul 14 21:42:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:drbd action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  843. Jul 14 21:42:58 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:28
  844. Jul 14 21:43:00 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:28 exit-code:0 exec-time:1696ms queue-time:0ms
  845. Jul 14 21:43:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (recurring_action_timer) debug: Scheduling another invocation of hypervisor_status_60000
  846. Jul 14 21:43:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  847. Jul 14 21:43:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  848. Jul 14 21:43:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:22 exit-code:0 exec-time:0ms queue-time:0ms
  849. Jul 14 21:43:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (recurring_action_timer) debug: Scheduling another invocation of drbd_status_60000
  850. Jul 14 21:43:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  851. Jul 14 21:43:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  852. Jul 14 21:43:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:drbd action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  853. Jul 14 21:44:00 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:28
  854. Jul 14 21:44:01 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:28 exit-code:0 exec-time:726ms queue-time:0ms
  855. Jul 14 21:44:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (recurring_action_timer) debug: Scheduling another invocation of hypervisor_status_60000
  856. Jul 14 21:44:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  857. Jul 14 21:44:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  858. Jul 14 21:44:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:22 exit-code:0 exec-time:0ms queue-time:0ms
  859. Jul 14 21:44:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (recurring_action_timer) debug: Scheduling another invocation of drbd_status_60000
  860. Jul 14 21:44:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  861. Jul 14 21:44:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  862. Jul 14 21:44:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:drbd action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  863. Jul 14 21:45:01 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:28
  864. Jul 14 21:45:01 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:28 exit-code:0 exec-time:639ms queue-time:0ms
  865. Jul 14 21:45:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (recurring_action_timer) debug: Scheduling another invocation of hypervisor_status_60000
  866. Jul 14 21:45:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  867. Jul 14 21:45:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  868. Jul 14 21:45:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:22 exit-code:0 exec-time:0ms queue-time:0ms
  869. Jul 14 21:45:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (recurring_action_timer) debug: Scheduling another invocation of drbd_status_60000
  870. Jul 14 21:45:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  871. Jul 14 21:45:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  872. Jul 14 21:45:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:drbd action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  873. Jul 14 21:46:01 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:28
  874. Jul 14 21:46:02 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:28 exit-code:0 exec-time:654ms queue-time:0ms
  875. Jul 14 21:46:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (recurring_action_timer) debug: Scheduling another invocation of hypervisor_status_60000
  876. Jul 14 21:46:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  877. Jul 14 21:46:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  878. Jul 14 21:46:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:22 exit-code:0 exec-time:0ms queue-time:0ms
  879. Jul 14 21:46:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (recurring_action_timer) debug: Scheduling another invocation of drbd_status_60000
  880. Jul 14 21:46:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  881. Jul 14 21:46:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  882. Jul 14 21:46:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:drbd action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  883. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_update_peer_expected) info: handle_request: Node el8-a01n02[2] - expected state is now down (was member)
  884. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting shutdown[el8-a01n02]: (unset) -> 1594763213 | from el8-a01n02
  885. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.15 2
  886. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.16 (null)
  887. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=16
  888. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: <nvpair id="status-2-shutdown" name="shutdown" value="1594763213"/>
  889. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/16, version=0.13.16)
  890. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.16 2
  891. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.17 (null)
  892. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=17
  893. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=9:6:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @transition-magic=-1:193;9:6:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594763213, @last-run=1594763213, @exec-time=0
  894. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/65, version=0.13.17)
  895. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.17 2
  896. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.18 (null)
  897. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=18
  898. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=14:6:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @transition-magic=-1:193;14:6:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594763213, @last-run=1594763213, @exec-time=0
  899. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/66, version=0.13.18)
  900. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemakerd [26603] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  901. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemakerd [26603] (pcmk_shutdown_worker) notice: Shutting down Pacemaker
  902. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemakerd [26603] (stop_child) notice: Stopping pacemaker-controld | sent signal 15 to process 26609
  903. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  904. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_shutdown) notice: Shutting down cluster resource manager | limit=1200000ms
  905. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_shutdown_req) info: Sending shutdown request to all peers (DC is el8-a01n02)
  906. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_update_peer_expected) info: handle_request: Node el8-a01n01[1] - expected state is now down (was (null))
  907. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.18 2
  908. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.19 (null)
  909. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=19
  910. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_stop_0, @operation=stop, @transition-key=21:6:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @transition-magic=-1:193;21:6:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594763213, @last-run=1594763213, @exec-time=0
  911. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/67, version=0.13.19)
  912. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting shutdown[el8-a01n01]: (unset) -> 1594763213 | from el8-a01n02
  913. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.19 2
  914. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.20 (null)
  915. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=20
  916. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;9:6:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=29, @rc-code=0, @op-status=0, @exec-time=1
  917. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=10:6:0:11323fbf-740e-4c7f-8ee5-4d98173fee16 op=virsh_node2_start_0
  918. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) notice: Discarding attempt to perform action start on virsh_node2 in state S_NOT_DC (shutdown=true)
  919. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_shutdown_req) info: Sending shutdown request to all peers (DC is el8-a01n02)
  920. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/68, version=0.13.20)
  921. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.20 2
  922. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.21 (null)
  923. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=21
  924. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']: <nvpair id="status-1-shutdown" name="shutdown" value="1594763213"/>
  925. Jul 14 21:46:53 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/17, version=0.13.21)
  926. Jul 14 21:46:55 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.21 2
  927. Jul 14 21:46:55 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.22 (null)
  928. Jul 14 21:46:55 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=22
  929. Jul 14 21:46:55 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;14:6:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=31, @rc-code=0, @op-status=0, @exec-time=2208
  930. Jul 14 21:46:55 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/69, version=0.13.22)
  931. Jul 14 21:47:00 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_ping) info: Reporting our current digest to el8-a01n02: 416cfb2580c7286c9b7cd6f5bff3da81 for 0.13.22 (0x55db1c268910 0)
  932. Jul 14 21:47:02 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:28
  933. Jul 14 21:47:03 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:28 exit-code:0 exec-time:656ms queue-time:0ms
  934. Jul 14 21:47:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (recurring_action_timer) debug: Scheduling another invocation of hypervisor_status_60000
  935. Jul 14 21:47:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  936. Jul 14 21:47:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  937. Jul 14 21:47:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:22 exit-code:0 exec-time:0ms queue-time:0ms
  938. Jul 14 21:47:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (recurring_action_timer) debug: Scheduling another invocation of drbd_status_60000
  939. Jul 14 21:47:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  940. Jul 14 21:47:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  941. Jul 14 21:47:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:drbd action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  942. Jul 14 21:48:03 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:28
  943. Jul 14 21:48:03 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:28 exit-code:0 exec-time:613ms queue-time:0ms
  944. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.22 2
  945. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.23 (null)
  946. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=23
  947. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=2:198;21:6:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=33, @rc-code=198, @op-status=2, @exec-time=99991
  948. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']: <lrm_rsc_op id="drbd_last_failure_0" operation_key="drbd_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="21:6:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="2:198;21:6:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n02" call-id="33" rc-code="
  949. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/70, version=0.13.23)
  950. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting fail-count-drbd#stop_0[el8-a01n02]: (unset) -> INFINITY | from el8-a01n02
  951. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_update) notice: Setting last-failure-drbd#stop_0[el8-a01n02]: (unset) -> 1594763313 | from el8-a01n02
  952. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.23 2
  953. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.24 (null)
  954. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=24
  955. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: <nvpair id="status-2-fail-count-drbd.stop_0" name="fail-count-drbd#stop_0" value="INFINITY"/>
  956. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/18, version=0.13.24)
  957. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.24 2
  958. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.25 (null)
  959. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=25
  960. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: <nvpair id="status-2-last-failure-drbd.stop_0" name="last-failure-drbd#stop_0" value="1594763313"/>
  961. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/19, version=0.13.25)
  962. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:28 exit-code:0 exec-time:0ms queue-time:0ms
  963. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=0
  964. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=6:8:0:11323fbf-740e-4c7f-8ee5-4d98173fee16 op=virsh_node1_stop_0
  965. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/38)
  966. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2396817f-4776-4134-be53-572580788be0: rc=31, reply=1, notify=0
  967. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) info: executing - rsc:virsh_node1 action:stop call_id:31
  968. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) info: Result of monitor operation for virsh_node1 on el8-a01n01: Cancelled | call=28 key=virsh_node1_monitor_60000 confirmed=true
  969. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.25 2
  970. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-fenced [26605] (can_fence_host_with_device) notice: virsh_node1 is not eligible to fence (reboot) el8-a01n02: static-list
  971. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-fenced [26605] (can_fence_host_with_device) notice: virsh_node2 is eligible to fence (reboot) el8-a01n02: static-list
  972. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.26 (null)
  973. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=26
  974. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_stop_0, @operation=stop, @transition-key=6:8:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @transition-magic=-1:193;6:8:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594763313, @last-run=1594763313, @exec-time=0
  975. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) info: finished - rsc:virsh_node1 action:stop call_id:31 exit-code:0 exec-time:2ms queue-time:0ms
  976. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/38, version=0.13.26)
  977. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (cancel_recurring_action) info: Cancelling systemd operation hypervisor_status_60000
  978. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:22 exit-code:0 exec-time:0ms queue-time:0ms
  979. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=0
  980. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-fenced [26605] (can_fence_host_with_device) notice: virsh_node1 is not eligible to fence (reboot) el8-a01n02: static-list
  981. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-fenced [26605] (can_fence_host_with_device) notice: virsh_node2 is eligible to fence (reboot) el8-a01n02: static-list
  982. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-fenced [26605] (stonith_fence_get_devices_cb) info: Found 1 matching devices for 'el8-a01n02'
  983. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=7:8:0:11323fbf-740e-4c7f-8ee5-4d98173fee16 op=hypervisor_stop_0
  984. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/39)
  985. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2396817f-4776-4134-be53-572580788be0: rc=33, reply=1, notify=0
  986. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) info: executing - rsc:hypervisor action:stop call_id:33
  987. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit libvirtd named 'hypervisor'
  988. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.26 2
  989. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.27 (null)
  990. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=27
  991. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=7:8:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @transition-magic=-1:193;7:8:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594763313, @last-run=1594763313, @exec-time=0, @que
  992. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec_with_unit) debug: Calling StopUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice
  993. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) notice: Result of stop operation for virsh_node1 on el8-a01n01: 0 (ok) | call=31 key=virsh_node1_stop_0 confirmed=true cib-update=40
  994. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) info: Result of monitor operation for hypervisor on el8-a01n01: Cancelled | call=22 key=hypervisor_monitor_60000 confirmed=true
  995. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/39, version=0.13.27)
  996. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/40)
  997. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.27 2
  998. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.28 (null)
  999. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=28
  1000. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:0;6:8:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=31, @rc-code=0, @op-status=0, @exec-time=2
  1001. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/40, version=0.13.28)
  1002. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/1299
  1003. Jul 14 21:48:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: hypervisor stop may still be in progress: re-scheduling (elapsed=192ms, remaining=99808ms, start_delay=2000ms)
  1004. Jul 14 21:48:35 el8-a01n01.digimer.ca pacemaker-fenced [26605] (log_operation) notice: Operation 'reboot' [54242] (call 3 from pacemaker-controld.1613) for host 'el8-a01n02' with device 'virsh_node2' returned: 0 (OK)
  1005. Jul 14 21:48:35 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:33
  1006. Jul 14 21:48:35 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  1007. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1008. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: hypervisor stop is now complete (elapsed=2200ms, remaining=97800ms): not running (7)
  1009. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:33 exit-code:0 exec-time:2200ms queue-time:0ms
  1010. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) notice: Result of stop operation for hypervisor on el8-a01n01: 0 (ok) | call=33 key=hypervisor_stop_0 confirmed=true cib-update=41
  1011. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/41)
  1012. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (pcmk_quorum_notification) info: Quorum retained | membership=157 members=1
  1013. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now lost | nodeid=2 previous=member source=crm_reap_unseen_nodes
  1014. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (peer_update_callback) info: Cluster node el8-a01n02 is now lost (was member)
  1015. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemakerd [26603] (pcmk_quorum_notification) info: Quorum retained | membership=157 members=1
  1016. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemakerd [26603] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now lost | nodeid=2 previous=member source=crm_reap_unseen_nodes
  1017. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemakerd [26603] (pcmk_cpg_membership) info: Group pacemakerd event 2: el8-a01n02 (node 2 pid 1607) left via cluster exit
  1018. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemakerd [26603] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now offline
  1019. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemakerd [26603] (pcmk_cpg_membership) info: Group pacemakerd event 2: el8-a01n01 (node 1 pid 26603) is member
  1020. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (pcmk_cpg_membership) info: Group attrd event 1: el8-a01n02 (node 2 pid 1611) left via cluster exit
  1021. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (reap_dead_nodes) warning: Our DC node (el8-a01n02) left the cluster
  1022. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_state_transition) notice: State transition S_NOT_DC -> S_ELECTION | input=I_ELECTION cause=C_FSA_INTERNAL origin=reap_dead_nodes
  1023. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (update_dc) info: Unset DC. Was el8-a01n02
  1024. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (pcmk_cpg_membership) info: Group cib event 2: el8-a01n02 (node 2 pid 1608) left via cluster exit
  1025. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now offline
  1026. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (pcmk_cpg_membership) info: Group crmd event 1: el8-a01n02 (node 2 pid 1613) left via cluster exit
  1027. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-fenced [26605] (pcmk_cpg_membership) info: Group stonith-ng event 2: el8-a01n02 (node 2 pid 1609) left via cluster exit
  1028. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-fenced [26605] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now offline
  1029. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now offline
  1030. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (peer_update_callback) info: Node el8-a01n02 is no longer a peer | DC=<none> old=0x4000000 new=0x0000000
  1031. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now offline
  1032. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-fenced [26605] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now lost | nodeid=2 previous=member source=crm_update_peer_proc
  1033. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (controld_delete_node_state) info: Deleting transient attributes for node el8-a01n02 (via CIB call 42) | xpath=//node_state[@uname='el8-a01n02']/transient_attributes
  1034. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (pcmk_cpg_membership) info: Group crmd event 1: el8-a01n01 (node 1 pid 26609) is member
  1035. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-fenced [26605] (crm_reap_dead_member) info: Removing node with name el8-a01n02 and id 2 from membership cache
  1036. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-fenced [26605] (reap_crm_member) notice: Purged 1 peer with id=2 and/or uname=el8-a01n02 from the membership cache
  1037. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_remove_voter) notice: Lost attribute writer el8-a01n02
  1038. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-fenced [26605] (pcmk_cpg_membership) info: Group stonith-ng event 2: el8-a01n01 (node 1 pid 26605) is member
  1039. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_start_election_if_needed) info: Starting an election to determine the writer
  1040. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now lost | nodeid=2 previous=member source=crm_update_peer_proc
  1041. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (election_check) info: election-DC won by local node
  1042. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_log) info: Input I_ELECTION_DC received in state S_ELECTION from election_win_cb
  1043. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (crm_reap_dead_member) info: Removing node with name el8-a01n02 and id 2 from membership cache
  1044. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_state_transition) notice: State transition S_ELECTION -> S_INTEGRATION | input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=election_win_cb
  1045. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-fenced [26605] (remote_op_done) notice: Operation 'reboot' targeting el8-a01n02 on el8-a01n01 for pacemaker-controld.1613@el8-a01n02.5d01b7f3: OK
  1046. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_te_control) info: Registering TE UUID: a146f13d-4f30-4ad9-80a0-d1ab28984e52
  1047. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now lost | nodeid=2 previous=member source=crm_update_peer_proc
  1048. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_peer_remove) notice: Removing all el8-a01n02 attributes for peer loss
  1049. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (crm_reap_dead_member) info: Removing node with name el8-a01n02 and id 2 from membership cache
  1050. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (reap_crm_member) notice: Purged 1 peer with id=2 and/or uname=el8-a01n02 from the membership cache
  1051. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (pcmk_cpg_membership) info: Group attrd event 1: el8-a01n01 (node 1 pid 26607) is member
  1052. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (election_check) info: election-attrd won by local node
  1053. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_declare_winner) notice: Recorded local node as attribute writer (was unset)
  1054. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (reap_crm_member) notice: Purged 1 peer with id=2 and/or uname=el8-a01n02 from the membership cache
  1055. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (pcmk_cpg_membership) info: Group cib event 2: el8-a01n01 (node 1 pid 26604) is member
  1056. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.28 2
  1057. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.29 (null)
  1058. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=29
  1059. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;7:8:0:11323fbf-740e-4c7f-8ee5-4d98173fee16, @call-id=33, @rc-code=0, @op-status=0, @exec-time=2200
  1060. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/41, version=0.13.29)
  1061. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n02']/transient_attributes to all (origin=local/crmd/42)
  1062. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (write_attribute) info: Sent CIB request 4 with 1 change for last-failure-virsh_node1#start_0 (id n/a, set n/a)
  1063. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (write_attribute) info: Processed 1 private change for #attrd-protocol, id=n/a, set=n/a
  1064. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (write_attribute) info: Sent CIB request 5 with 1 change for fail-count-virsh_node1#start_0 (id n/a, set n/a)
  1065. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (write_attribute) info: Sent CIB request 6 with 1 change for last-failure-virsh_node2#start_0 (id n/a, set n/a)
  1066. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (write_attribute) info: Sent CIB request 7 with 1 change for fail-count-virsh_node2#start_0 (id n/a, set n/a)
  1067. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (write_attribute) info: Sent CIB request 8 with 1 change for shutdown (id n/a, set n/a)
  1068. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.29 2
  1069. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.30 15e4e33b23afe6fe44c21bd18c1de8a7
  1070. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/transient_attributes[@id='2']
  1071. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=30
  1072. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/transient_attributes: OK (rc=0, origin=el8-a01n01/crmd/42, version=0.13.30)
  1073. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/4)
  1074. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (set_graph_functions) info: Setting custom graph functions
  1075. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/5)
  1076. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/6)
  1077. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/7)
  1078. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/8)
  1079. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.30 2
  1080. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.31 (null)
  1081. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=31
  1082. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/attrd/4, version=0.13.31)
  1083. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.31 2
  1084. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.32 (null)
  1085. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=32
  1086. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/attrd/5, version=0.13.32)
  1087. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.32 2
  1088. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.33 (null)
  1089. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=33
  1090. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/attrd/6, version=0.13.33)
  1091. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.33 2
  1092. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.34 (null)
  1093. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=34
  1094. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/attrd/7, version=0.13.34)
  1095. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/attrd/8, version=0.13.34)
  1096. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_cib_callback) info: CIB update 4 result for last-failure-virsh_node1#start_0: OK | rc=0
  1097. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_cib_callback) info: * last-failure-virsh_node1#start_0[el8-a01n01]=(null)
  1098. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_cib_callback) info: CIB update 5 result for fail-count-virsh_node1#start_0: OK | rc=0
  1099. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_cib_callback) info: * fail-count-virsh_node1#start_0[el8-a01n01]=(null)
  1100. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_cib_callback) info: CIB update 6 result for last-failure-virsh_node2#start_0: OK | rc=0
  1101. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_cib_callback) info: * last-failure-virsh_node2#start_0[el8-a01n01]=(null)
  1102. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_cib_callback) info: CIB update 7 result for fail-count-virsh_node2#start_0: OK | rc=0
  1103. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_cib_callback) info: * fail-count-virsh_node2#start_0[el8-a01n01]=(null)
  1104. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_cib_callback) info: CIB update 8 result for shutdown: OK | rc=0
  1105. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-attrd [26607] (attrd_cib_callback) info: * shutdown[el8-a01n01]=1594763213
  1106. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_dc_takeover) info: Taking over DC status for this partition
  1107. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_readwrite) info: We are now in R/W mode
  1108. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_master operation for section 'all': OK (rc=0, origin=local/crmd/43, version=0.13.34)
  1109. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/44)
  1110. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n01/crmd/44, version=0.13.34)
  1111. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/46)
  1112. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n01/crmd/46, version=0.13.34)
  1113. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/48)
  1114. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (join_make_offer) info: Making join-1 offers based on membership event 157
  1115. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (join_make_offer) info: Sending join-1 offer to el8-a01n01
  1116. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (join_make_offer) info: Not making join-1 offer to inactive node el8-a01n02
  1117. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_dc_join_offer_all) info: Waiting on join-1 requests from 1 outstanding node
  1118. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (tengine_stonith_notify) notice: Peer el8-a01n02 was terminated (reboot) by el8-a01n01 on behalf of pacemaker-controld.1613: OK | initiator=el8-a01n02 ref=5d01b7f3-0da6-4d2c-88ab-c38adf182303
  1119. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n01/crmd/48, version=0.13.34)
  1120. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (controld_delete_node_state) info: Deleting all state for node el8-a01n02 (via CIB call 53) | xpath=//node_state[@uname='el8-a01n02']/*
  1121. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (tengine_stonith_notify) info: External fencing operation from pacemaker-controld.1613 fenced el8-a01n02
  1122. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (abort_transition_graph) info: Transition -1 aborted: External Fencing Operation | source=tengine_stonith_notify:542 complete=true
  1123. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (update_dc) info: Set DC to el8-a01n01 (3.3.0)
  1124. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_update_peer_expected) info: update_dc: Node el8-a01n01[1] - expected state is now member (was down)
  1125. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_update_peer_expected) info: update_dc_expected: Node el8-a01n01[1] - expected state is now down (was member)
  1126. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/50)
  1127. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/52)
  1128. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n02']/* to all (origin=local/crmd/53)
  1129. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n01/crmd/50, version=0.13.34)
  1130. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.34 2
  1131. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.35 (null)
  1132. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=35
  1133. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=send_stonith_update, @join=down, @expected=down
  1134. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/52, version=0.13.35)
  1135. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.35 2
  1136. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.36 (null)
  1137. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/lrm[@id='2']
  1138. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=36
  1139. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/*: OK (rc=0, origin=el8-a01n01/crmd/53, version=0.13.36)
  1140. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (cib_fencing_updated) info: Fencing update 52 for el8-a01n02: complete
  1141. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_update_peer_expected) info: do_dc_join_filter_offer: Node el8-a01n01[1] - expected state is now member (was down)
  1142. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_state_transition) info: State transition S_INTEGRATION -> S_FINALIZE_JOIN | input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state
  1143. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (crm_update_peer_expected) info: update_dc_expected: Node el8-a01n01[1] - expected state is now down (was member)
  1144. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (controld_delete_node_state) info: Deleting resource history for node el8-a01n01 (via CIB call 57) | xpath=//node_state[@uname='el8-a01n01']/lrm
  1145. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_replace) info: Digest matched on replace from el8-a01n01: 6209e282955acce8785d7ed4c8344fd2
  1146. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_replace) info: Replaced 0.13.36 with 0.13.36 from el8-a01n01
  1147. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=el8-a01n01/crmd/55, version=0.13.36)
  1148. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/56)
  1149. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n01']/lrm to all (origin=local/crmd/57)
  1150. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/58)
  1151. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/56, version=0.13.36)
  1152. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.36 2
  1153. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.37 (null)
  1154. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: -- /cib/status/node_state[@id='1']/lrm[@id='1']
  1155. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=37
  1156. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/lrm: OK (rc=0, origin=el8-a01n01/crmd/57, version=0.13.37)
  1157. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.37 2
  1158. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.38 (null)
  1159. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=38
  1160. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_lrm_query_internal
  1161. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']: <lrm id="1"/>
  1162. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_resources>
  1163. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_resource id="virsh_node1" type="fence_virsh" class="stonith">
  1164. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node1_last_0" operation_key="virsh_node1_stop_0" operation="stop" crm-debug-origin="build_active_RAs" crm_feature_set="3.3.0" transition-key="6:8:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="0:0;6:8:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n01" call-id="31" rc-code="0" op-status="0" interval="0" last-rc-change="
  1165. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </lrm_resource>
  1166. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_resource id="virsh_node2" type="fence_virsh" class="stonith">
  1167. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node2_last_0" operation_key="virsh_node2_stop_0" operation="stop" crm-debug-origin="build_active_RAs" crm_feature_set="3.3.0" transition-key="3:4:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="0:0;3:4:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n01" call-id="29" rc-code="0" op-status="0" interval="0" last-rc-change="
  1168. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </lrm_resource>
  1169. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_resource id="drbd" type="drbd" class="systemd">
  1170. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_rsc_op id="drbd_last_0" operation_key="drbd_start_0" operation="start" crm-debug-origin="build_active_RAs" crm_feature_set="3.3.0" transition-key="21:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="0:0;21:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n01" call-id="23" rc-code="0" op-status="0" interval="0" last-rc-change="1594762908
  1171. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_rsc_op id="drbd_monitor_60000" operation_key="drbd_monitor_60000" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.3.0" transition-key="22:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="0:0;22:0:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n01" call-id="24" rc-code="0" op-status="0" interval="60000" last-rc
  1172. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </lrm_resource>
  1173. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_resource id="hypervisor" type="libvirtd" class="systemd">
  1174. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ <lrm_rsc_op id="hypervisor_last_0" operation_key="hypervisor_stop_0" operation="stop" crm-debug-origin="build_active_RAs" crm_feature_set="3.3.0" transition-key="7:8:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" transition-magic="0:0;7:8:0:11323fbf-740e-4c7f-8ee5-4d98173fee16" exit-reason="" on_node="el8-a01n01" call-id="33" rc-code="0" op-status="0" interval="0" last-rc-change="15
  1175. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </lrm_resource>
  1176. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </lrm_resources>
  1177. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: ++ </lrm>
  1178. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/58, version=0.13.38)
  1179. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_state_transition) info: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE | input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state
  1180. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-controld [26609] (abort_transition_graph) info: Transition -1 aborted: Peer Cancelled | source=do_te_invoke:129 complete=true
  1181. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/61)
  1182. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/62)
  1183. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/63)
  1184. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-14.raw
  1185. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/61, version=0.13.38)
  1186. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.38 2
  1187. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.39 (null)
  1188. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=39
  1189. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition, @expected=down
  1190. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @in_ccm=false, @crmd=offline, @crm-debug-origin=do_state_transition
  1191. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/62, version=0.13.39)
  1192. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.39 2
  1193. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.40 (null)
  1194. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=40, @dc-uuid=1
  1195. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n01/crmd/63, version=0.13.40)
  1196. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_file_write_with_digest) info: Wrote version 0.13.0 of the CIB to disk (digest: 362bd359cce467645aa13bd5b0e4baac)
  1197. Jul 14 21:48:36 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.ujR6w3 (digest: /var/lib/pacemaker/cib/cib.CRJACB)
  1198. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (determine_online_status) info: Node el8-a01n01 is shutting down
  1199. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (unpack_node_loop) info: Node 1 is already processed
  1200. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (unpack_node_loop) info: Node 2 is already processed
  1201. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (unpack_node_loop) info: Node 1 is already processed
  1202. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (unpack_node_loop) info: Node 2 is already processed
  1203. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (common_print) info: virsh_node1 (stonith:fence_virsh): Stopped
  1204. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (common_print) info: virsh_node2 (stonith:fence_virsh): Stopped
  1205. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  1206. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (short_print) info: Stopped: [ el8-a01n01 el8-a01n02 ]
  1207. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (clone_print) info: Clone Set: drbd-clone [drbd]
  1208. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (short_print) info: Started: [ el8-a01n01 ]
  1209. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (short_print) info: Stopped: [ el8-a01n02 ]
  1210. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (native_color) info: Resource virsh_node1 cannot run anywhere
  1211. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (native_color) info: Resource virsh_node2 cannot run anywhere
  1212. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (native_color) info: Resource hypervisor:0 cannot run anywhere
  1213. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (native_color) info: Resource hypervisor:1 cannot run anywhere
  1214. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (native_color) info: Resource drbd:0 cannot run anywhere
  1215. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (native_color) info: Resource drbd:1 cannot run anywhere
  1216. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (sched_shutdown_op) notice: Scheduling shutdown of node el8-a01n01
  1217. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (LogNodeActions) notice: * Shutdown el8-a01n01
  1218. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (LogActions) info: Leave virsh_node1 (Stopped)
  1219. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (LogActions) info: Leave virsh_node2 (Stopped)
  1220. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (LogActions) info: Leave hypervisor:0 (Stopped)
  1221. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (LogActions) info: Leave hypervisor:1 (Stopped)
  1222. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (LogAction) notice: * Stop drbd:0 ( el8-a01n01 ) due to node availability
  1223. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (LogActions) info: Leave drbd:1 (Stopped)
  1224. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-schedulerd[26608] (pcmk__log_transition_summary) notice: Calculated transition 0, saving inputs in /var/lib/pacemaker/pengine/pe-input-8.bz2
  1225. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response
  1226. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_te_invoke) info: Processing graph 0 (ref=pe_calc-dc-1594763317-15) derived from /var/lib/pacemaker/pengine/pe-input-8.bz2
  1227. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-controld [26609] (te_rsc_command) notice: Initiating stop operation drbd_stop_0 locally on el8-a01n01 | action 6
  1228. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-execd [26606] (cancel_recurring_action) info: Cancelling systemd operation drbd_status_60000
  1229. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_finished) debug: finished - rsc:drbd action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  1230. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from 2396817f-4776-4134-be53-572580788be0: rc=0, reply=1, notify=0
  1231. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-controld [26609] (do_lrm_rsc_op) info: Performing key=6:0:0:a146f13d-4f30-4ad9-80a0-d1ab28984e52 op=drbd_stop_0
  1232. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-execd [26606] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from 2396817f-4776-4134-be53-572580788be0: rc=35, reply=1, notify=0
  1233. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) info: executing - rsc:drbd action:stop call_id:35
  1234. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit drbd named 'drbd'
  1235. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/66)
  1236. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: --- 0.13.40 2
  1237. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: Diff: +++ 0.13.41 (null)
  1238. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib: @num_updates=41
  1239. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource
  1240. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_stop_0, @operation=stop, @crm-debug-origin=do_update_resource, @transition-key=6:0:0:a146f13d-4f30-4ad9-80a0-d1ab28984e52, @transition-magic=-1:193;6:0:0:a146f13d-4f30-4ad9-80a0-d1ab28984e52, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594763317, @last-run=1594763317
  1241. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/66, version=0.13.41)
  1242. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec_with_unit) debug: Calling StopUnit for drbd: /org/freedesktop/systemd1/unit/drbd_2eservice
  1243. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-controld [26609] (process_lrm_event) info: Result of monitor operation for drbd on el8-a01n01: Cancelled | call=24 key=drbd_monitor_60000 confirmed=true
  1244. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/1300
  1245. Jul 14 21:48:37 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=201ms, remaining=99799ms, start_delay=2000ms)
  1246. Jul 14 21:48:39 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1247. Jul 14 21:48:39 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1248. Jul 14 21:48:39 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1249. Jul 14 21:48:39 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=2207ms, remaining=97793ms, start_delay=2000ms)
  1250. Jul 14 21:48:41 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1251. Jul 14 21:48:41 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1252. Jul 14 21:48:41 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1253. Jul 14 21:48:41 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=4212ms, remaining=95788ms, start_delay=2000ms)
  1254. Jul 14 21:48:42 el8-a01n01.digimer.ca pacemaker-based [26604] (cib_process_ping) info: Reporting our current digest to el8-a01n01: 03a69144313c40f4bba676f411a9bdae for 0.13.41 (0x55db1c3afce0 0)
  1255. Jul 14 21:48:43 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1256. Jul 14 21:48:43 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1257. Jul 14 21:48:43 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1258. Jul 14 21:48:43 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=6217ms, remaining=93783ms, start_delay=2000ms)
  1259. Jul 14 21:48:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1260. Jul 14 21:48:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1261. Jul 14 21:48:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1262. Jul 14 21:48:45 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=8220ms, remaining=91780ms, start_delay=2000ms)
  1263. Jul 14 21:48:47 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1264. Jul 14 21:48:47 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1265. Jul 14 21:48:47 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1266. Jul 14 21:48:47 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=10223ms, remaining=89777ms, start_delay=2000ms)
  1267. Jul 14 21:48:49 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1268. Jul 14 21:48:49 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1269. Jul 14 21:48:49 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1270. Jul 14 21:48:49 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=12228ms, remaining=87772ms, start_delay=2000ms)
  1271. Jul 14 21:48:51 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1272. Jul 14 21:48:51 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1273. Jul 14 21:48:51 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1274. Jul 14 21:48:51 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=14230ms, remaining=85770ms, start_delay=2000ms)
  1275. Jul 14 21:48:53 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1276. Jul 14 21:48:53 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1277. Jul 14 21:48:53 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1278. Jul 14 21:48:53 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=16238ms, remaining=83762ms, start_delay=2000ms)
  1279. Jul 14 21:48:55 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1280. Jul 14 21:48:55 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1281. Jul 14 21:48:55 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1282. Jul 14 21:48:55 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=18242ms, remaining=81758ms, start_delay=2000ms)
  1283. Jul 14 21:48:57 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1284. Jul 14 21:48:57 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1285. Jul 14 21:48:57 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1286. Jul 14 21:48:57 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=20245ms, remaining=79755ms, start_delay=2000ms)
  1287. Jul 14 21:48:59 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1288. Jul 14 21:48:59 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1289. Jul 14 21:48:59 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1290. Jul 14 21:48:59 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=22248ms, remaining=77752ms, start_delay=2000ms)
  1291. Jul 14 21:49:01 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1292. Jul 14 21:49:01 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1293. Jul 14 21:49:01 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1294. Jul 14 21:49:01 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=24257ms, remaining=75743ms, start_delay=2000ms)
  1295. Jul 14 21:49:03 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1296. Jul 14 21:49:03 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1297. Jul 14 21:49:03 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1298. Jul 14 21:49:03 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=26261ms, remaining=73739ms, start_delay=2000ms)
  1299. Jul 14 21:49:05 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1300. Jul 14 21:49:05 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1301. Jul 14 21:49:05 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1302. Jul 14 21:49:05 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=28263ms, remaining=71737ms, start_delay=2000ms)
  1303. Jul 14 21:49:07 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1304. Jul 14 21:49:07 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1305. Jul 14 21:49:07 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1306. Jul 14 21:49:07 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=30270ms, remaining=69730ms, start_delay=2000ms)
  1307. Jul 14 21:49:09 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1308. Jul 14 21:49:09 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1309. Jul 14 21:49:09 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1310. Jul 14 21:49:09 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=32273ms, remaining=67727ms, start_delay=2000ms)
  1311. Jul 14 21:49:11 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1312. Jul 14 21:49:11 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1313. Jul 14 21:49:11 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1314. Jul 14 21:49:11 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=34283ms, remaining=65717ms, start_delay=2000ms)
  1315. Jul 14 21:49:13 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1316. Jul 14 21:49:13 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1317. Jul 14 21:49:13 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1318. Jul 14 21:49:13 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=36287ms, remaining=63713ms, start_delay=2000ms)
  1319. Jul 14 21:49:15 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1320. Jul 14 21:49:15 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1321. Jul 14 21:49:15 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1322. Jul 14 21:49:15 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=38290ms, remaining=61710ms, start_delay=2000ms)
  1323. Jul 14 21:49:17 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1324. Jul 14 21:49:17 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1325. Jul 14 21:49:17 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1326. Jul 14 21:49:17 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=40294ms, remaining=59706ms, start_delay=2000ms)
  1327. Jul 14 21:49:19 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1328. Jul 14 21:49:19 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1329. Jul 14 21:49:19 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1330. Jul 14 21:49:19 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=42304ms, remaining=57696ms, start_delay=2000ms)
  1331. Jul 14 21:49:21 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1332. Jul 14 21:49:21 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1333. Jul 14 21:49:21 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1334. Jul 14 21:49:21 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=44309ms, remaining=55691ms, start_delay=2000ms)
  1335. Jul 14 21:49:23 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1336. Jul 14 21:49:23 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1337. Jul 14 21:49:23 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1338. Jul 14 21:49:23 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=46312ms, remaining=53688ms, start_delay=2000ms)
  1339. Jul 14 21:49:25 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1340. Jul 14 21:49:25 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1341. Jul 14 21:49:25 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1342. Jul 14 21:49:25 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=48315ms, remaining=51685ms, start_delay=2000ms)
  1343. Jul 14 21:49:27 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1344. Jul 14 21:49:27 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1345. Jul 14 21:49:27 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1346. Jul 14 21:49:27 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=50320ms, remaining=49680ms, start_delay=2000ms)
  1347. Jul 14 21:49:29 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1348. Jul 14 21:49:29 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1349. Jul 14 21:49:29 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1350. Jul 14 21:49:29 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=52323ms, remaining=47677ms, start_delay=2000ms)
  1351. Jul 14 21:49:31 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1352. Jul 14 21:49:31 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1353. Jul 14 21:49:31 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1354. Jul 14 21:49:31 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=54326ms, remaining=45674ms, start_delay=2000ms)
  1355. Jul 14 21:49:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1356. Jul 14 21:49:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1357. Jul 14 21:49:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1358. Jul 14 21:49:33 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=56329ms, remaining=43671ms, start_delay=2000ms)
  1359. Jul 14 21:49:35 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1360. Jul 14 21:49:35 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1361. Jul 14 21:49:35 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1362. Jul 14 21:49:35 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=58332ms, remaining=41668ms, start_delay=2000ms)
  1363. Jul 14 21:49:37 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1364. Jul 14 21:49:37 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1365. Jul 14 21:49:37 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1366. Jul 14 21:49:37 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=60337ms, remaining=39663ms, start_delay=2000ms)
  1367. Jul 14 21:49:39 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1368. Jul 14 21:49:39 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1369. Jul 14 21:49:39 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1370. Jul 14 21:49:39 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=62340ms, remaining=37660ms, start_delay=2000ms)
  1371. Jul 14 21:49:41 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1372. Jul 14 21:49:41 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1373. Jul 14 21:49:41 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1374. Jul 14 21:49:41 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=64343ms, remaining=35657ms, start_delay=2000ms)
  1375. Jul 14 21:49:44 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1376. Jul 14 21:49:44 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1377. Jul 14 21:49:44 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1378. Jul 14 21:49:44 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=66359ms, remaining=33641ms, start_delay=2000ms)
  1379. Jul 14 21:49:46 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1380. Jul 14 21:49:46 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1381. Jul 14 21:49:46 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1382. Jul 14 21:49:46 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=68362ms, remaining=31638ms, start_delay=2000ms)
  1383. Jul 14 21:49:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1384. Jul 14 21:49:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1385. Jul 14 21:49:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1386. Jul 14 21:49:48 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=70365ms, remaining=29635ms, start_delay=2000ms)
  1387. Jul 14 21:49:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1388. Jul 14 21:49:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1389. Jul 14 21:49:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1390. Jul 14 21:49:50 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=72368ms, remaining=27632ms, start_delay=2000ms)
  1391. Jul 14 21:49:52 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1392. Jul 14 21:49:52 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1393. Jul 14 21:49:52 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1394. Jul 14 21:49:52 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=74372ms, remaining=25628ms, start_delay=2000ms)
  1395. Jul 14 21:49:54 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1396. Jul 14 21:49:54 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1397. Jul 14 21:49:54 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1398. Jul 14 21:49:54 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=76374ms, remaining=23626ms, start_delay=2000ms)
  1399. Jul 14 21:49:56 el8-a01n01.digimer.ca pacemaker-execd [26606] (log_execute) debug: executing - rsc:drbd action:monitor call_id:35
  1400. Jul 14 21:49:56 el8-a01n01.digimer.ca pacemaker-execd [26606] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1401. Jul 14 21:49:56 el8-a01n01.digimer.ca pacemaker-execd [26606] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1402. Jul 14 21:49:56 el8-a01n01.digimer.ca pacemaker-execd [26606] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=78378ms, remaining=21622ms, start_delay=2000ms)
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!