API tools faq
paste
digimer

Untitled

digimer
Jul 14th, 2020
150
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 241.61 KB | None | 0 0
raw download clone embed print report
  1. [root@el8-a01n01 ~]# cat /var/log/pacemaker/el8-a01n01/pacemaker.log.drbd-down-up-to-crash
  2. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  3. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (get_cluster_type) info: Detected an active 'corosync' cluster
  4. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (mcp_read_config) info: Reading configure for stack: corosync
  5. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (main) notice: Starting Pacemaker 2.0.3-5.el8_2.1 | build=4b1f869f0f features: generated-manpages agent-manpages ncurses libqb-logging libqb-ipc systemd nagios corosync-native atomic-attrd acls
  6. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (main) info: Maximum core file size is: 18446744073709551615
  7. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (qb_ipcs_us_publish) info: server name: pacemakerd
  8. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (pcmk__ipc_is_authentic_process_active) info: Could not connect to lrmd IPC: Connection refused
  9. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (pcmk__ipc_is_authentic_process_active) info: Could not connect to cib_ro IPC: Connection refused
  10. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (pcmk__ipc_is_authentic_process_active) info: Could not connect to crmd IPC: Connection refused
  11. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (pcmk__ipc_is_authentic_process_active) info: Could not connect to attrd IPC: Connection refused
  12. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (pcmk__ipc_is_authentic_process_active) info: Could not connect to pengine IPC: Connection refused
  13. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (pcmk__ipc_is_authentic_process_active) info: Could not connect to stonith-ng IPC: Connection refused
  14. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (crm_get_peer) info: Created entry 0dbc95c7-f89b-471a-8607-0f48f89e06ac/0x564a746618c0 for node el8-a01n01/1 (1 total)
  15. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  16. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (crm_get_peer) info: Node 1 has uuid 1
  17. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online
  18. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (cluster_connect_quorum) warning: Quorum lost
  19. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (crm_get_peer) info: Created entry ac6b79a3-392f-40d9-833e-d3bceee892f4/0x564a74663860 for node el8-a01n02/2 (2 total)
  20. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  21. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (crm_get_peer) info: Node 2 has uuid 2
  22. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (start_child) info: Using uid=189 and group=189 for process pacemaker-based
  23. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (start_child) info: Forked child 32267 for process pacemaker-based
  24. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (start_child) info: Forked child 32268 for process pacemaker-fenced
  25. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (start_child) info: Forked child 32269 for process pacemaker-execd
  26. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (start_child) info: Using uid=189 and group=189 for process pacemaker-attrd
  27. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (start_child) info: Forked child 32270 for process pacemaker-attrd
  28. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (start_child) info: Using uid=189 and group=189 for process pacemaker-schedulerd
  29. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (start_child) info: Forked child 32271 for process pacemaker-schedulerd
  30. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (start_child) info: Using uid=189 and group=189 for process pacemaker-controld
  31. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (start_child) info: Forked child 32273 for process pacemaker-controld
  32. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (main) notice: Pacemaker daemon successfully started and accepting connections
  33. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (pcmk_quorum_notification) info: Quorum still lost | membership=171 members=1
  34. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification
  35. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (crm_reap_unseen_nodes) info: State of node el8-a01n02[2] is still unknown
  36. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (pcmk_cpg_membership) info: Group pacemakerd event 0: node 1 pid 32266 joined via cpg_join
  37. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemakerd [32266] (pcmk_cpg_membership) info: Group pacemakerd event 0: el8-a01n01 (node 1 pid 32266) is member
  38. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-execd [32269] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  39. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-execd [32269] (main) notice: Starting Pacemaker local executor
  40. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-execd [32269] (qb_ipcs_us_publish) info: server name: lrmd
  41. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-execd [32269] (main) notice: Pacemaker local executor successfully started and accepting connections
  42. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-fenced [32268] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  43. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-fenced [32268] (main) notice: Starting Pacemaker fencer
  44. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-fenced [32268] (get_cluster_type) info: Verifying cluster type: 'corosync'
  45. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-fenced [32268] (get_cluster_type) info: Assuming an active 'corosync' cluster
  46. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-fenced [32268] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  47. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  48. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (main) notice: Starting Pacemaker CIB manager
  49. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (get_cluster_type) info: Verifying cluster type: 'corosync'
  50. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (get_cluster_type) info: Assuming an active 'corosync' cluster
  51. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-fenced [32268] (crm_get_peer) info: Created entry 7351d875-2fce-4558-86b8-d280d8bde902/0x555b7d3c4d60 for node el8-a01n01/1 (1 total)
  52. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-fenced [32268] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  53. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (retrieveCib) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.xml (digest: /var/lib/pacemaker/cib/cib.xml.sig)
  54. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-attrd [32270] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  55. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-attrd [32270] (main) notice: Starting Pacemaker node attribute manager
  56. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  57. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-controld [32273] (main) notice: Starting Pacemaker controller
  58. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_log) info: Input I_STARTUP received in state S_STARTING from crmd_init
  59. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores
  60. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (main) notice: Starting Pacemaker scheduler
  61. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (qb_ipcs_us_publish) info: server name: pengine
  62. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (main) notice: Pacemaker scheduler successfully started and accepting connections
  63. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-fenced [32268] (crm_get_peer) info: Node 1 has uuid 1
  64. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-fenced [32268] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online
  65. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-fenced [32268] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  66. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-fenced [32268] (init_cs_connection_once) info: Connection to 'corosync': established
  67. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  68. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (crm_get_peer) info: Created entry f226808d-96b6-4acd-93c2-315848293505/0x55db3e137400 for node el8-a01n01/1 (1 total)
  69. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  70. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (crm_get_peer) info: Node 1 has uuid 1
  71. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online
  72. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  73. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (init_cs_connection_once) info: Connection to 'corosync': established
  74. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (qb_ipcs_us_publish) info: server name: cib_ro
  75. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (qb_ipcs_us_publish) info: server name: cib_rw
  76. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (qb_ipcs_us_publish) info: server name: cib_shm
  77. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (main) notice: Pacemaker CIB manager successfully started and accepting connections
  78. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (pcmk_cpg_membership) info: Group cib event 0: node 1 pid 32267 joined via cpg_join
  79. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (pcmk_cpg_membership) info: Group cib event 0: el8-a01n01 (node 1 pid 32267) is member
  80. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-17.raw
  81. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_file_write_with_digest) info: Wrote version 0.13.0 of the CIB to disk (digest: 721386e5982712a0d17a05527f015cab)
  82. Jul 14 22:08:15 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.UA6fqz (digest: /var/lib/pacemaker/cib/cib.PHKShp)
  83. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemakerd [32266] (pcmk_cpg_membership) info: Group pacemakerd event 1: node 2 pid 1814 joined via cluster join
  84. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemakerd [32266] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n01 (node 1 pid 32266) is member
  85. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemakerd [32266] (pcmk_cpg_membership) info: Group pacemakerd event 1: el8-a01n02 (node 2 pid 1814) is member
  86. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemakerd [32266] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online
  87. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemakerd [32266] (crm_cs_flush) info: Sent 0 CPG messages (1 remaining, last=7): Try again (6)
  88. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemakerd [32266] (pcmk_quorum_notification) notice: Quorum acquired | membership=175 members=2
  89. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemakerd [32266] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification
  90. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-based [32267] (pcmk_cpg_membership) info: Group cib event 1: node 2 pid 1815 joined via cluster join
  91. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-based [32267] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n01 (node 1 pid 32267) is member
  92. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-based [32267] (crm_get_peer) info: Created entry d77f39fc-5606-44c4-9222-a71b2ecc7003/0x55db3e13ca40 for node el8-a01n02/2 (2 total)
  93. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-based [32267] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  94. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-based [32267] (crm_get_peer) info: Node 2 has uuid 2
  95. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-based [32267] (pcmk_cpg_membership) info: Group cib event 1: el8-a01n02 (node 2 pid 1815) is member
  96. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-based [32267] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online
  97. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-based [32267] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  98. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/transient_attributes: OK (rc=0, origin=el8-a01n02/attrd/2, version=0.13.0)
  99. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n02/crmd/4, version=0.13.0)
  100. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (get_cluster_type) info: Verifying cluster type: 'corosync'
  101. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (get_cluster_type) info: Assuming an active 'corosync' cluster
  102. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  103. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (main) info: CIB connection active
  104. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (get_cluster_type) info: Verifying cluster type: 'corosync'
  105. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (get_cluster_type) info: Assuming an active 'corosync' cluster
  106. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (crm_cluster_connect) notice: Connecting to cluster infrastructure: corosync
  107. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_get_peer) info: Created entry 2b8ecfae-8c19-437c-a944-009ce327fd71/0x563312e87eb0 for node el8-a01n01/1 (1 total)
  108. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  109. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (peer_update_callback) info: Cluster node el8-a01n01 is now in unknown state
  110. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (crm_get_peer) info: Created entry 1327755a-a2a5-437e-a2aa-673f834532ac/0x5650d843ea20 for node el8-a01n01/1 (1 total)
  111. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (crm_get_peer) info: Node 1 is now known as el8-a01n01
  112. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_get_peer) info: Node 1 has uuid 1
  113. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online
  114. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (peer_update_callback) info: Node el8-a01n01 is now a peer | DC=<none> old=0x0000000 new=0x4000000
  115. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds
  116. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (init_cs_connection_once) info: Connection to 'corosync': established
  117. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (crm_get_peer) info: Node 1 has uuid 1
  118. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (crm_update_peer_proc) info: cluster_connect_cpg: Node el8-a01n01[1] - corosync-cpg is now online
  119. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc
  120. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (init_cs_connection_once) info: Connection to 'corosync': established
  121. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (setup_cib) info: Watching for stonith topology changes
  122. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (qb_ipcs_us_publish) info: server name: stonith-ng
  123. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (main) notice: Pacemaker fencer successfully started and accepting connections
  124. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (pcmk_cpg_membership) info: Group stonith-ng event 0: node 1 pid 32268 joined via cpg_join
  125. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (pcmk_cpg_membership) info: Group stonith-ng event 0: el8-a01n01 (node 1 pid 32268) is member
  126. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (init_cib_cache_cb) info: Updating device list from the cib: init
  127. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (cib_devices_update) info: Updating devices to version 0.13.0
  128. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (main) info: Cluster connection active
  129. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (attrd_erase_attrs) info: Clearing transient attributes from CIB | xpath=//node_state[@uname='el8-a01n01']/transient_attributes
  130. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n01']/transient_attributes to all (origin=local/attrd/2)
  131. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (cluster_connect_quorum) notice: Quorum acquired
  132. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/transient_attributes: OK (rc=0, origin=el8-a01n01/attrd/2, version=0.13.0)
  133. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (qb_ipcs_us_publish) info: server name: attrd
  134. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (main) notice: Pacemaker node attribute manager successfully started and accepting connections
  135. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (pcmk_cpg_membership) info: Group attrd event 0: node 1 pid 32270 joined via cpg_join
  136. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (pcmk_cpg_membership) info: Group attrd event 0: el8-a01n01 (node 1 pid 32270) is member
  137. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_get_peer) info: Created entry 8c5373b7-f283-4dbc-a37d-cd18f862b0ca/0x563312e8a790 for node el8-a01n02/2 (2 total)
  138. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  139. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (peer_update_callback) info: Cluster node el8-a01n02 is now in unknown state
  140. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_get_peer) info: Node 2 has uuid 2
  141. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (crm_get_peer) info: Created entry 0852aa7d-2066-4b25-be7f-05aacf7d0bf2/0x5650d8445060 for node el8-a01n02/2 (2 total)
  142. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  143. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (crm_get_peer) info: Node 2 has uuid 2
  144. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (pcmk_cpg_membership) info: Group attrd event 0: el8-a01n02 (node 2 pid 1818) is member
  145. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online
  146. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  147. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (attrd_check_for_new_writer) notice: Recorded new attribute writer: el8-a01n02 (was unset)
  148. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (attrd_peer_message) info: Processing sync-response from el8-a01n02
  149. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n02]: (unset) -> 2 | from el8-a01n02
  150. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-attrd [32270] (attrd_peer_update) notice: Setting #attrd-protocol[el8-a01n01]: (unset) -> 2 | from el8-a01n01
  151. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_ha_control) info: Connected to the cluster
  152. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/4)
  153. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (lrmd_ipc_connect) info: Connecting to executor
  154. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (crm_client_new) debug: Connecting 0x55c354d31fb0 for uid=189 gid=189 pid=32273 id=ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1
  155. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (handle_new_connection) debug: IPC credentials authenticated (/dev/shm/qb-32269-32273-9-agJx1C/qb)
  156. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (qb_ipcs_shm_connect) debug: connecting to client [32273]
  157. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  158. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  159. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  160. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/4, version=0.13.0)
  161. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed register operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=0
  162. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_control) info: Connection to the executor established
  163. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_started) info: Delaying start, no membership data (0000000000100000)
  164. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_started) info: Delaying start, no membership data (0000000000100000)
  165. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (pcmk_quorum_notification) info: Quorum retained | membership=175 members=2
  166. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_update_peer_state_iter) notice: Node el8-a01n01 state is now member | nodeid=1 previous=unknown source=pcmk_quorum_notification
  167. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (peer_update_callback) info: Cluster node el8-a01n01 is now member (was in unknown state)
  168. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=pcmk_quorum_notification
  169. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (peer_update_callback) info: Cluster node el8-a01n02 is now member (was in unknown state)
  170. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_started) info: Delaying start, Config not read (0000000000000040)
  171. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (qb_ipcs_us_publish) info: server name: crmd
  172. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_started) notice: Pacemaker controller successfully started and accepting connections
  173. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_log) info: Input I_PENDING received in state S_STARTING from do_started
  174. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_state_transition) notice: State transition S_STARTING -> S_PENDING | input=I_PENDING cause=C_FSA_INTERNAL origin=do_started
  175. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 32334 exited with rc=0
  176. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (stonith_device_register) notice: Added 'virsh_node1' to the device list (1 active devices)
  177. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (stonith_device_register) notice: Added 'virsh_node2' to the device list (2 active devices)
  178. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (pcmk_cpg_membership) info: Group stonith-ng event 1: node 2 pid 1816 joined via cluster join
  179. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n01 (node 1 pid 32268) is member
  180. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (crm_get_peer) info: Created entry 5c118c0e-539c-4de8-a7b4-b7b7d2871e87/0x555b7d4e6d80 for node el8-a01n02/2 (2 total)
  181. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (crm_get_peer) info: Node 2 is now known as el8-a01n02
  182. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (crm_get_peer) info: Node 2 has uuid 2
  183. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (pcmk_cpg_membership) info: Group stonith-ng event 1: el8-a01n02 (node 2 pid 1816) is member
  184. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online
  185. Jul 14 22:08:16 el8-a01n01.digimer.ca pacemaker-fenced [32268] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc
  186. Jul 14 22:08:17 el8-a01n01.digimer.ca pacemaker-controld [32273] (pcmk_cpg_membership) info: Group crmd event 0: node 1 pid 32273 joined via cpg_join
  187. Jul 14 22:08:17 el8-a01n01.digimer.ca pacemaker-controld [32273] (pcmk_cpg_membership) info: Group crmd event 0: el8-a01n01 (node 1 pid 32273) is member
  188. Jul 14 22:08:17 el8-a01n01.digimer.ca pacemaker-controld [32273] (pcmk_cpg_membership) info: Group crmd event 0: el8-a01n02 (node 2 pid 1820) is member
  189. Jul 14 22:08:17 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now online
  190. Jul 14 22:08:17 el8-a01n01.digimer.ca pacemaker-controld [32273] (peer_update_callback) info: Node el8-a01n02 is now a peer | DC=<none> old=0x0000000 new=0x4000000
  191. Jul 14 22:08:17 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 5 seconds
  192. Jul 14 22:08:17 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 30 seconds
  193. Jul 14 22:08:17 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_connect_stonith) notice: Fencer successfully connected
  194. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (election_count_vote) info: election-DC round 1 (owner node ID 2) pass: vote from el8-a01n02 (Uptime)
  195. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_state_transition) info: State transition S_PENDING -> S_ELECTION | input=I_ELECTION cause=C_FSA_INTERNAL origin=do_election_count_vote
  196. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (election_check) info: election-DC won by local node
  197. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_log) info: Input I_ELECTION_DC received in state S_ELECTION from election_win_cb
  198. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_state_transition) notice: State transition S_ELECTION -> S_INTEGRATION | input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=election_win_cb
  199. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_te_control) info: Registering TE UUID: ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af
  200. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (set_graph_functions) info: Setting custom graph functions
  201. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_dc_takeover) info: Taking over DC status for this partition
  202. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_readwrite) info: We are now in R/W mode
  203. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_master operation for section 'all': OK (rc=0, origin=local/crmd/7, version=0.13.0)
  204. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/8)
  205. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/10)
  206. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n01/crmd/8, version=0.13.0)
  207. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n01/crmd/10, version=0.13.0)
  208. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/12)
  209. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (join_make_offer) info: Making join-1 offers based on membership event 175
  210. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (join_make_offer) info: Sending join-1 offer to el8-a01n02
  211. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (join_make_offer) info: Sending join-1 offer to el8-a01n01
  212. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_dc_join_offer_all) info: Waiting on join-1 requests from 2 outstanding nodes
  213. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n01/crmd/12, version=0.13.0)
  214. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section crm_config to all (origin=local/crmd/14)
  215. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (update_dc) info: Set DC to el8-a01n01 (3.3.0)
  216. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_update_peer_expected) info: update_dc: Node el8-a01n01[1] - expected state is now member (was (null))
  217. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section crm_config: OK (rc=0, origin=el8-a01n01/crmd/14, version=0.13.0)
  218. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_update_peer_expected) info: do_dc_join_filter_offer: Node el8-a01n02[2] - expected state is now member (was (null))
  219. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_state_transition) info: State transition S_INTEGRATION -> S_FINALIZE_JOIN | input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state
  220. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_replace) info: Digest matched on replace from el8-a01n01: e66cd993ae6725062fe95afbc6bd62d5
  221. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_replace) info: Replaced 0.13.0 with 0.13.0 from el8-a01n01
  222. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (controld_delete_node_state) info: Deleting resource history for node el8-a01n01 (via CIB call 20) | xpath=//node_state[@uname='el8-a01n01']/lrm
  223. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (controld_delete_node_state) info: Deleting resource history for node el8-a01n02 (via CIB call 22) | xpath=//node_state[@uname='el8-a01n02']/lrm
  224. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_replace operation for section 'all': OK (rc=0, origin=el8-a01n01/crmd/17, version=0.13.0)
  225. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/18)
  226. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/19)
  227. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n01']/lrm to all (origin=local/crmd/20)
  228. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/21)
  229. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n02']/lrm to all (origin=local/crmd/22)
  230. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/18, version=0.13.0)
  231. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/19, version=0.13.0)
  232. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n01']/lrm: OK (rc=0, origin=el8-a01n01/crmd/20, version=0.13.0)
  233. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.0 2
  234. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.1 (null)
  235. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=1
  236. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status: <node_state id="1" uname="el8-a01n01" in_ccm="true" crmd="online" crm-debug-origin="do_lrm_query_internal"/>
  237. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ <lrm id="1">
  238. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ <lrm_resources/>
  239. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ </lrm>
  240. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ </node_state>
  241. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/21, version=0.13.1)
  242. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_state_transition) info: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE | input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state
  243. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-controld [32273] (abort_transition_graph) info: Transition -1 aborted: Peer Cancelled | source=do_te_invoke:129 complete=true
  244. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/lrm: OK (rc=0, origin=el8-a01n01/crmd/22, version=0.13.1)
  245. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/23)
  246. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/26)
  247. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/27)
  248. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.1 2
  249. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.2 (null)
  250. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=2
  251. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status: <node_state id="2" uname="el8-a01n02" in_ccm="true" crmd="online" crm-debug-origin="do_lrm_query_internal"/>
  252. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ <lrm id="2">
  253. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ <lrm_resources/>
  254. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ </lrm>
  255. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ </node_state>
  256. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/23, version=0.13.2)
  257. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/26, version=0.13.2)
  258. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.2 2
  259. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.3 (null)
  260. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=3
  261. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition, @join=member, @expected=member
  262. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition, @join=member, @expected=member
  263. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/27, version=0.13.3)
  264. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section cib to all (origin=local/crmd/28)
  265. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.3 2
  266. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.4 (null)
  267. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=4, @dc-uuid=1
  268. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section cib: OK (rc=0, origin=el8-a01n01/crmd/28, version=0.13.4)
  269. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-18.raw
  270. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_file_write_with_digest) info: Wrote version 0.13.0 of the CIB to disk (digest: 96d8141ab40cd17f4dfe3f1be9e92394)
  271. Jul 14 22:08:37 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.n4El8S (digest: /var/lib/pacemaker/cib/cib.EadY4L)
  272. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status_fencing) info: Node el8-a01n01 is active
  273. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status) info: Node el8-a01n01 is online
  274. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status_fencing) info: Node el8-a01n02 is active
  275. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status) info: Node el8-a01n02 is online
  276. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 1 is already processed
  277. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 2 is already processed
  278. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 1 is already processed
  279. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 2 is already processed
  280. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (common_print) info: virsh_node1 (stonith:fence_virsh): Stopped
  281. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (common_print) info: virsh_node2 (stonith:fence_virsh): Stopped
  282. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  283. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (short_print) info: Stopped: [ el8-a01n01 el8-a01n02 ]
  284. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (clone_print) info: Clone Set: drbd-clone [drbd]
  285. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (short_print) info: Stopped: [ el8-a01n01 el8-a01n02 ]
  286. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (RecurringOp) info: Start recurring monitor (60s) for virsh_node1 on el8-a01n01
  287. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (RecurringOp) info: Start recurring monitor (60s) for virsh_node2 on el8-a01n02
  288. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (RecurringOp) info: Start recurring monitor (60s) for hypervisor:0 on el8-a01n01
  289. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (RecurringOp) info: Start recurring monitor (60s) for hypervisor:1 on el8-a01n02
  290. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (RecurringOp) info: Start recurring monitor (60s) for drbd:0 on el8-a01n01
  291. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (RecurringOp) info: Start recurring monitor (60s) for drbd:1 on el8-a01n02
  292. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Start virsh_node1 ( el8-a01n01 )
  293. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Start virsh_node2 ( el8-a01n02 )
  294. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Start hypervisor:0 ( el8-a01n01 )
  295. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Start hypervisor:1 ( el8-a01n02 )
  296. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Start drbd:0 ( el8-a01n01 )
  297. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Start drbd:1 ( el8-a01n02 )
  298. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (pcmk__log_transition_summary) notice: Calculated transition 0, saving inputs in /var/lib/pacemaker/pengine/pe-input-14.bz2
  299. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response
  300. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_te_invoke) info: Processing graph 0 (ref=pe_calc-dc-1594764518-9) derived from /var/lib/pacemaker/pengine/pe-input-14.bz2
  301. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating monitor operation virsh_node1_monitor_0 on el8-a01n02 | action 5
  302. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating monitor operation virsh_node1_monitor_0 locally on el8-a01n01 | action 1
  303. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node1' not in cache
  304. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=0
  305. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node1'
  306. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=1
  307. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=0
  308. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=1:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=virsh_node1_monitor_0
  309. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.4 2
  310. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.5 (null)
  311. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=5
  312. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_update_resource
  313. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="virsh_node1" type="fence_virsh" class="stonith"/>
  314. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node1_last_0" operation_key="virsh_node1_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="5:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" transition-magic="-1:193;5:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-stat
  315. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ </lrm_resource>
  316. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/9, version=0.13.5)
  317. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.5 2
  318. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.6 (null)
  319. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=6
  320. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;5:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=5, @rc-code=7, @op-status=0, @exec-time=8
  321. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/10, version=0.13.6)
  322. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (action_synced_wait) info: Managed fence_virsh_metadata_1 process 33668 exited with rc=0
  323. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/31)
  324. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=5, reply=1, notify=0
  325. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:5
  326. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (stonith_api_signon) debug: Attempting fencer connection by pacemaker-execd with mainloop
  327. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating monitor operation virsh_node2_monitor_0 on el8-a01n02 | action 6
  328. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.6 2
  329. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.7 (null)
  330. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=7
  331. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource
  332. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="virsh_node1" type="fence_virsh" class="stonith"/>
  333. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node1_last_0" operation_key="virsh_node1_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="1:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" transition-magic="-1:193;1:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-stat
  334. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ </lrm_resource>
  335. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating monitor operation virsh_node2_monitor_0 locally on el8-a01n01 | action 2
  336. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/31, version=0.13.7)
  337. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  338. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  339. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792
  340. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.7 2
  341. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.8 (null)
  342. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=8
  343. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="virsh_node2" type="fence_virsh" class="stonith"/>
  344. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node2_last_0" operation_key="virsh_node2_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="6:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" transition-magic="-1:193;6:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-stat
  345. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ </lrm_resource>
  346. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/11, version=0.13.8)
  347. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (stonith_api_signon) debug: Connection to fencer by pacemaker-execd succeeded (registration token: 27f0072c-422a-429c-b1a5-c89ea7ca3414)
  348. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.8 2
  349. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.9 (null)
  350. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=9
  351. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;6:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=9, @rc-code=7, @op-status=0
  352. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/12, version=0.13.9)
  353. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:5 exit-code:7 exec-time:8ms queue-time:1ms
  354. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_get_rsc_info) info: Agent information for 'virsh_node2' not in cache
  355. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=0
  356. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_rsc_register) info: Cached agent information for 'virsh_node2'
  357. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=1
  358. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=0
  359. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=2:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=virsh_node2_monitor_0
  360. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/32)
  361. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=9, reply=1, notify=0
  362. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:virsh_node2 action:monitor call_id:9
  363. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:virsh_node2 action:monitor call_id:9 exit-code:7 exec-time:0ms queue-time:1ms
  364. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.9 2
  365. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.10 (null)
  366. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=10
  367. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="virsh_node2" type="fence_virsh" class="stonith"/>
  368. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ <lrm_rsc_op id="virsh_node2_last_0" operation_key="virsh_node2_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="2:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" transition-magic="-1:193;2:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-stat
  369. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ </lrm_resource>
  370. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 5 (virsh_node1_monitor_0 on el8-a01n02) confirmed: not running | rc=7 call-id=5
  371. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/32, version=0.13.10)
  372. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 6 (virsh_node2_monitor_0 on el8-a01n02) confirmed: not running | rc=7 call-id=9
  373. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) notice: Result of probe operation for virsh_node1 on el8-a01n01: 7 (not running) | call=5 key=virsh_node1_monitor_0 confirmed=true cib-update=33
  374. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/33)
  375. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.10 2
  376. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.11 (null)
  377. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=11
  378. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:7;1:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=5, @rc-code=7, @op-status=0, @exec-time=8, @queue-time=1
  379. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/33, version=0.13.11)
  380. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) notice: Result of probe operation for virsh_node2 on el8-a01n01: 7 (not running) | call=9 key=virsh_node2_monitor_0 confirmed=true cib-update=34
  381. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/34)
  382. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.11 2
  383. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.12 (null)
  384. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=12
  385. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:7;2:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=9, @rc-code=7, @op-status=0, @queue-time=1
  386. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 1 (virsh_node1_monitor_0 on el8-a01n01) confirmed: not running | rc=7 call-id=5
  387. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/34, version=0.13.12)
  388. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 2 (virsh_node2_monitor_0 on el8-a01n01) confirmed: not running | rc=7 call-id=9
  389. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating start operation virsh_node1_start_0 locally on el8-a01n01 | action 9
  390. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=9:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=virsh_node1_start_0
  391. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/35)
  392. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.12 2
  393. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.13 (null)
  394. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=13
  395. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_start_0, @operation=start, @transition-key=9:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;9:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0, @queue-time=0
  396. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/35, version=0.13.13)
  397. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=10, reply=1, notify=0
  398. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) info: executing - rsc:virsh_node1 action:start call_id:10
  399. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating start operation virsh_node2_start_0 on el8-a01n02 | action 11
  400. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating monitor operation hypervisor:0_monitor_0 locally on el8-a01n01 | action 3
  401. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.13 2
  402. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.14 (null)
  403. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=14
  404. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=11:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;11:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1
  405. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/13, version=0.13.14)
  406. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor' not in cache
  407. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=0
  408. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_get_rsc_info) info: Agent information for 'hypervisor:0' not in cache
  409. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=0
  410. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_rsc_register) info: Cached agent information for 'hypervisor'
  411. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=1
  412. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=0
  413. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=3:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=hypervisor_monitor_0
  414. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/36)
  415. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.14 2
  416. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.15 (null)
  417. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=15
  418. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="hypervisor" type="libvirtd" class="systemd"/>
  419. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ <lrm_rsc_op id="hypervisor_last_0" operation_key="hypervisor_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="3:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" transition-magic="-1:193;3:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-status
  420. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ </lrm_resource>
  421. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/36, version=0.13.15)
  422. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=15, reply=1, notify=0
  423. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:15
  424. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating monitor operation hypervisor:1_monitor_0 on el8-a01n02 | action 7
  425. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  426. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.15 2
  427. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.16 (null)
  428. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=16
  429. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="hypervisor" type="libvirtd" class="systemd"/>
  430. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ <lrm_rsc_op id="hypervisor_last_0" operation_key="hypervisor_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="7:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" transition-magic="-1:193;7:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-status
  431. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ </lrm_resource>
  432. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/14, version=0.13.16)
  433. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  434. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.16 2
  435. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.17 (null)
  436. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=17
  437. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;7:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=15, @rc-code=7, @op-status=0, @exec-time=35
  438. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/15, version=0.13.17)
  439. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 7 (hypervisor_monitor_0 on el8-a01n02) confirmed: not running | rc=7 call-id=15
  440. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating monitor operation drbd:1_monitor_0 on el8-a01n02 | action 8
  441. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:15 exit-code:7 exec-time:45ms queue-time:0ms
  442. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) notice: Result of probe operation for hypervisor on el8-a01n01: 7 (not running) | call=15 key=hypervisor_monitor_0 confirmed=true cib-update=37
  443. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/37)
  444. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.17 2
  445. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.18 (null)
  446. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=18
  447. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:7;3:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=15, @rc-code=7, @op-status=0, @exec-time=45
  448. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/37, version=0.13.18)
  449. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 3 (hypervisor_monitor_0 on el8-a01n01) confirmed: not running | rc=7 call-id=15
  450. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating monitor operation drbd:0_monitor_0 locally on el8-a01n01 | action 4
  451. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_get_rsc_info) info: Agent information for 'drbd' not in cache
  452. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.18 2
  453. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.19 (null)
  454. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=19
  455. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: <lrm_resource id="drbd" type="drbd" class="systemd"/>
  456. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ <lrm_rsc_op id="drbd_last_0" operation_key="drbd_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="8:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" transition-magic="-1:193;8:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" exit-reason="" on_node="el8-a01n02" call-id="-1" rc-code="193" op-status="-1" interv
  457. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ </lrm_resource>
  458. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=0
  459. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/16, version=0.13.19)
  460. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_get_rsc_info) info: Agent information for 'drbd:0' not in cache
  461. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=0
  462. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_rsc_register) info: Cached agent information for 'drbd'
  463. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_register operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=1
  464. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_info operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=0
  465. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=4:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=drbd_monitor_0
  466. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/38)
  467. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=20, reply=1, notify=0
  468. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:20
  469. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  470. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.19 2
  471. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.20 (null)
  472. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=20
  473. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources: <lrm_resource id="drbd" type="drbd" class="systemd"/>
  474. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ <lrm_rsc_op id="drbd_last_0" operation_key="drbd_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="4:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" transition-magic="-1:193;4:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" exit-reason="" on_node="el8-a01n01" call-id="-1" rc-code="193" op-status="-1" interv
  475. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ </lrm_resource>
  476. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/38, version=0.13.20)
  477. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  478. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.20 2
  479. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.21 (null)
  480. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=21
  481. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;8:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=20, @rc-code=7, @op-status=0, @exec-time=11
  482. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/17, version=0.13.21)
  483. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 8 (drbd_monitor_0 on el8-a01n02) confirmed: not running | rc=7 call-id=20
  484. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating start operation hypervisor:1_start_0 on el8-a01n02 | action 15
  485. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:drbd action:monitor call_id:20 exit-code:7 exec-time:11ms queue-time:0ms
  486. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) notice: Result of probe operation for drbd on el8-a01n01: 7 (not running) | call=20 key=drbd_monitor_0 confirmed=true cib-update=39
  487. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.21 2
  488. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.22 (null)
  489. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=22
  490. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=15:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;15:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0
  491. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/18, version=0.13.22)
  492. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/39)
  493. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.22 2
  494. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.23 (null)
  495. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=23
  496. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:7;4:0:7:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=20, @rc-code=7, @op-status=0, @exec-time=11
  497. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/39, version=0.13.23)
  498. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 4 (drbd_monitor_0 on el8-a01n01) confirmed: not running | rc=7 call-id=20
  499. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating start operation hypervisor:0_start_0 locally on el8-a01n01 | action 13
  500. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=13:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=hypervisor_start_0
  501. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/40)
  502. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=21, reply=1, notify=0
  503. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) info: executing - rsc:hypervisor action:start call_id:21
  504. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous start op on systemd unit libvirtd named 'hypervisor'
  505. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.23 2
  506. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.24 (null)
  507. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=24
  508. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_start_0, @operation=start, @transition-key=13:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;13:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @exec-time=0
  509. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/40, version=0.13.24)
  510. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec_with_unit) debug: Calling StartUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice
  511. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_exec_result) info: Call to start passed: /org/freedesktop/systemd1/job/1743
  512. Jul 14 22:08:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: hypervisor start may still be in progress: re-scheduling (elapsed=241ms, remaining=99759ms, start_delay=2000ms)
  513. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.24 2
  514. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.25 (null)
  515. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=25
  516. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;11:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=10, @rc-code=0, @op-status=0, @exec-time=809
  517. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/19, version=0.13.25)
  518. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 11 (virsh_node2_start_0 on el8-a01n02) confirmed: ok | rc=0 call-id=10
  519. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating monitor operation virsh_node2_monitor_60000 on el8-a01n02 | action 12
  520. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.25 2
  521. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.26 (null)
  522. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=26
  523. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']: <lrm_rsc_op id="virsh_node2_monitor_60000" operation_key="virsh_node2_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="12:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" transition-magic="-1:193;12:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" exit-reason="" on_node="el8
  524. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/20, version=0.13.26)
  525. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) info: finished - rsc:virsh_node1 action:start call_id:10 exit-code:0 exec-time:896ms queue-time:0ms
  526. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) notice: Result of start operation for virsh_node1 on el8-a01n01: 0 (ok) | call=10 key=virsh_node1_start_0 confirmed=true cib-update=41
  527. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/41)
  528. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.26 2
  529. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.27 (null)
  530. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=27
  531. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:0;9:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=10, @rc-code=0, @op-status=0, @exec-time=896
  532. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/41, version=0.13.27)
  533. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 9 (virsh_node1_start_0 on el8-a01n01) confirmed: ok | rc=0 call-id=10
  534. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating monitor operation virsh_node1_monitor_60000 locally on el8-a01n01 | action 10
  535. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=10:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=virsh_node1_monitor_60000
  536. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=22, reply=1, notify=0
  537. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:22
  538. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/42)
  539. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.27 2
  540. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.28 (null)
  541. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=28
  542. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']: <lrm_rsc_op id="virsh_node1_monitor_60000" operation_key="virsh_node1_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="10:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" transition-magic="-1:193;10:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" exit-reason="" on_node="el8
  543. Jul 14 22:08:39 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/42, version=0.13.28)
  544. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.28 2
  545. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.29 (null)
  546. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=29
  547. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_monitor_60000']: @transition-magic=0:0;12:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=22, @rc-code=0, @op-status=0, @exec-time=932
  548. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/21, version=0.13.29)
  549. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 12 (virsh_node2_monitor_60000 on el8-a01n02) confirmed: ok | rc=0 call-id=22
  550. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating start operation drbd:1_start_0 on el8-a01n02 | action 23
  551. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.29 2
  552. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.30 (null)
  553. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=30
  554. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_start_0, @operation=start, @transition-key=23:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;23:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594764520, @last-run=1594764520, @exec-time=0
  555. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/22, version=0.13.30)
  556. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:22 exit-code:0 exec-time:1164ms queue-time:0ms
  557. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) notice: Result of monitor operation for virsh_node1 on el8-a01n01: 0 (ok) | call=22 key=virsh_node1_monitor_60000 confirmed=false cib-update=43
  558. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/43)
  559. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.30 2
  560. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.31 (null)
  561. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=31
  562. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_monitor_60000']: @transition-magic=0:0;10:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=22, @rc-code=0, @op-status=0, @exec-time=1164
  563. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/43, version=0.13.31)
  564. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 10 (virsh_node1_monitor_60000 on el8-a01n01) confirmed: ok | rc=0 call-id=22
  565. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating start operation drbd:0_start_0 locally on el8-a01n01 | action 21
  566. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=21:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=drbd_start_0
  567. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=23, reply=1, notify=0
  568. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) info: executing - rsc:drbd action:start call_id:23
  569. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous start op on systemd unit drbd named 'drbd'
  570. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/44)
  571. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.31 2
  572. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.32 (null)
  573. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=32
  574. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_start_0, @operation=start, @transition-key=21:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;21:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594764520, @last-run=1594764520, @exec-time=0
  575. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/44, version=0.13.32)
  576. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec_with_unit) debug: Calling StartUnit for drbd: /org/freedesktop/systemd1/unit/drbd_2eservice
  577. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.32 2
  578. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.33 (null)
  579. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=33
  580. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;15:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=21, @rc-code=0, @op-status=0, @exec-time=2224
  581. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/23, version=0.13.33)
  582. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 15 (hypervisor_start_0 on el8-a01n02) confirmed: ok | rc=0 call-id=21
  583. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating monitor operation hypervisor:1_monitor_60000 on el8-a01n02 | action 16
  584. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.33 2
  585. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.34 (null)
  586. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=34
  587. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']: <lrm_rsc_op id="hypervisor_monitor_60000" operation_key="hypervisor_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="16:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" transition-magic="-1:193;16:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" exit-reason="" on_node="el8-a0
  588. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/24, version=0.13.34)
  589. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.34 2
  590. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.35 (null)
  591. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=35
  592. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_monitor_60000']: @transition-magic=0:0;16:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=24, @rc-code=0, @op-status=0, @exec-time=6
  593. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/25, version=0.13.35)
  594. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 16 (hypervisor_monitor_60000 on el8-a01n02) confirmed: ok | rc=0 call-id=24
  595. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:21
  596. Jul 14 22:08:40 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  597. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_exec_result) info: Call to start passed: /org/freedesktop/systemd1/job/1827
  598. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd start may still be in progress: re-scheduling (elapsed=362ms, remaining=99638ms, start_delay=2000ms)
  599. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  600. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: hypervisor start is now complete (elapsed=2385ms, remaining=97615ms): ok (0)
  601. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:21 exit-code:0 exec-time:2385ms queue-time:0ms
  602. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) notice: Result of start operation for hypervisor on el8-a01n01: 0 (ok) | call=21 key=hypervisor_start_0 confirmed=true cib-update=45
  603. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/45)
  604. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.35 2
  605. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.36 (null)
  606. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=36
  607. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;13:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=21, @rc-code=0, @op-status=0, @exec-time=2385
  608. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/45, version=0.13.36)
  609. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 13 (hypervisor_start_0 on el8-a01n01) confirmed: ok | rc=0 call-id=21
  610. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating monitor operation hypervisor:0_monitor_60000 locally on el8-a01n01 | action 14
  611. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=14:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=hypervisor_monitor_60000
  612. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=24, reply=1, notify=0
  613. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:24
  614. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  615. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/46)
  616. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.36 2
  617. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.37 (null)
  618. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=37
  619. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']: <lrm_rsc_op id="hypervisor_monitor_60000" operation_key="hypervisor_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="14:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" transition-magic="-1:193;14:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" exit-reason="" on_node="el8-a0
  620. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/46, version=0.13.37)
  621. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  622. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:12ms queue-time:0ms
  623. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) notice: Result of monitor operation for hypervisor on el8-a01n01: 0 (ok) | call=24 key=hypervisor_monitor_60000 confirmed=false cib-update=47
  624. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/47)
  625. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.37 2
  626. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.38 (null)
  627. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=38
  628. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_monitor_60000']: @transition-magic=0:0;14:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=24, @rc-code=0, @op-status=0, @exec-time=12
  629. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 14 (hypervisor_monitor_60000 on el8-a01n01) confirmed: ok | rc=0 call-id=24
  630. Jul 14 22:08:41 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/47, version=0.13.38)
  631. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.38 2
  632. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.39 (null)
  633. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=39
  634. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;23:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=23, @rc-code=0, @op-status=0, @exec-time=2489
  635. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/26, version=0.13.39)
  636. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 23 (drbd_start_0 on el8-a01n02) confirmed: ok | rc=0 call-id=23
  637. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating monitor operation drbd:1_monitor_60000 on el8-a01n02 | action 24
  638. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.39 2
  639. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.40 (null)
  640. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=40
  641. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']: <lrm_rsc_op id="drbd_monitor_60000" operation_key="drbd_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="24:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" transition-magic="-1:193;24:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" exit-reason="" on_node="el8-a01n02" call-id="-1"
  642. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/27, version=0.13.40)
  643. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.40 2
  644. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.41 (null)
  645. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=41
  646. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_monitor_60000']: @transition-magic=0:0;24:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=25, @rc-code=0, @op-status=0, @exec-time=4
  647. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/28, version=0.13.41)
  648. Jul 14 22:08:42 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 24 (drbd_monitor_60000 on el8-a01n02) confirmed: ok | rc=0 call-id=25
  649. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:23
  650. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  651. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  652. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd start is now complete (elapsed=2370ms, remaining=97630ms): ok (0)
  653. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:drbd action:monitor call_id:23 exit-code:0 exec-time:2370ms queue-time:1ms
  654. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) notice: Result of start operation for drbd on el8-a01n01: 0 (ok) | call=23 key=drbd_start_0 confirmed=true cib-update=48
  655. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/48)
  656. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.41 2
  657. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.42 (null)
  658. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=42
  659. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;21:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=23, @rc-code=0, @op-status=0, @exec-time=2370, @queue-time=1
  660. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/48, version=0.13.42)
  661. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 21 (drbd_start_0 on el8-a01n01) confirmed: ok | rc=0 call-id=23
  662. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating monitor operation drbd:0_monitor_60000 locally on el8-a01n01 | action 22
  663. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=22:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=drbd_monitor_60000
  664. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=25, reply=1, notify=0
  665. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:25
  666. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  667. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  668. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:drbd action:monitor call_id:25 exit-code:0 exec-time:4ms queue-time:0ms
  669. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/49)
  670. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) notice: Result of monitor operation for drbd on el8-a01n01: 0 (ok) | call=25 key=drbd_monitor_60000 confirmed=false cib-update=50
  671. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.42 2
  672. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.43 (null)
  673. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=43
  674. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']: <lrm_rsc_op id="drbd_monitor_60000" operation_key="drbd_monitor_60000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="22:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" transition-magic="-1:193;22:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" exit-reason="" on_node="el8-a01n01" call-id="-1"
  675. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/49, version=0.13.43)
  676. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/50)
  677. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.43 2
  678. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.13.44 (null)
  679. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=44
  680. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_monitor_60000']: @transition-magic=0:0;22:0:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=25, @rc-code=0, @op-status=0, @exec-time=4
  681. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/50, version=0.13.44)
  682. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 0 action 22 (drbd_monitor_60000 on el8-a01n01) confirmed: ok | rc=0 call-id=25
  683. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-controld [32273] (run_graph) notice: Transition 0 (Complete=24, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-14.bz2): Complete
  684. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_log) info: Input I_TE_SUCCESS received in state S_TRANSITION_ENGINE from notify_crmd
  685. Jul 14 22:08:43 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_state_transition) notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd
  686. Jul 14 22:08:46 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_procfs_pid_of) info: Found pacemaker-based active as process 32267
  687. Jul 14 22:08:46 el8-a01n01.digimer.ca pacemaker-controld [32273] (throttle_send_command) info: New throttle mode: negligible load (was undetermined)
  688. Jul 14 22:08:48 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_ping) info: Reporting our current digest to el8-a01n01: 0a257520f27bc31c4e11ed12cf711cab for 0.13.44 (0x55db3dd56bd0 0)
  689. Jul 14 22:09:40 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:22
  690. Jul 14 22:09:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (recurring_action_timer) debug: Scheduling another invocation of hypervisor_status_60000
  691. Jul 14 22:09:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  692. Jul 14 22:09:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  693. Jul 14 22:09:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  694. Jul 14 22:09:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:22 exit-code:0 exec-time:600ms queue-time:0ms
  695. Jul 14 22:09:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (recurring_action_timer) debug: Scheduling another invocation of drbd_status_60000
  696. Jul 14 22:09:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  697. Jul 14 22:09:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  698. Jul 14 22:09:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:drbd action:monitor call_id:25 exit-code:0 exec-time:0ms queue-time:0ms
  699. Jul 14 22:10:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (recurring_action_timer) debug: Scheduling another invocation of hypervisor_status_60000
  700. Jul 14 22:10:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  701. Jul 14 22:10:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  702. Jul 14 22:10:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  703. Jul 14 22:10:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:22
  704. Jul 14 22:10:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:22 exit-code:0 exec-time:617ms queue-time:0ms
  705. Jul 14 22:10:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (recurring_action_timer) debug: Scheduling another invocation of drbd_status_60000
  706. Jul 14 22:10:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  707. Jul 14 22:10:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  708. Jul 14 22:10:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:drbd action:monitor call_id:25 exit-code:0 exec-time:0ms queue-time:0ms
  709. Jul 14 22:11:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (recurring_action_timer) debug: Scheduling another invocation of hypervisor_status_60000
  710. Jul 14 22:11:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  711. Jul 14 22:11:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  712. Jul 14 22:11:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  713. Jul 14 22:11:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:22
  714. Jul 14 22:11:42 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:22 exit-code:0 exec-time:606ms queue-time:0ms
  715. Jul 14 22:11:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (recurring_action_timer) debug: Scheduling another invocation of drbd_status_60000
  716. Jul 14 22:11:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  717. Jul 14 22:11:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  718. Jul 14 22:11:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:drbd action:monitor call_id:25 exit-code:0 exec-time:0ms queue-time:0ms
  719. Jul 14 22:12:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (recurring_action_timer) debug: Scheduling another invocation of hypervisor_status_60000
  720. Jul 14 22:12:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  721. Jul 14 22:12:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  722. Jul 14 22:12:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  723. Jul 14 22:12:42 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:22
  724. Jul 14 22:12:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (recurring_action_timer) debug: Scheduling another invocation of drbd_status_60000
  725. Jul 14 22:12:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  726. Jul 14 22:12:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  727. Jul 14 22:12:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:drbd action:monitor call_id:25 exit-code:0 exec-time:0ms queue-time:0ms
  728. Jul 14 22:12:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:22 exit-code:0 exec-time:692ms queue-time:0ms
  729. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_apply_diff operation for section 'all' to all (origin=local/cibadmin/2)
  730. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.13.44 2
  731. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.14.0 1bd192509acea0b190a7fd1f9680c252
  732. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @epoch=14, @num_updates=0
  733. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/configuration/resources/clone[@id='drbd-clone']/primitive[@id='drbd']: <meta_attributes id="drbd-meta_attributes"/>
  734. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ <nvpair id="drbd-meta_attributes-target-role" name="target-role" value="Stopped"/>
  735. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ </meta_attributes>
  736. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.14.0)
  737. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-controld [32273] (abort_transition_graph) info: Transition 0 aborted by meta_attributes.drbd-meta_attributes 'create': Configuration change | cib=0.14.0 source=te_update_diff_v2:490 path=/cib/configuration/resources/clone[@id='drbd-clone']/primitive[@id='drbd'] complete=true
  738. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_state_transition) notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph
  739. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-fenced [32268] (update_cib_stonith_devices_v2) info: Updating device list from the cib: create primitive[@id='drbd']
  740. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-fenced [32268] (cib_devices_update) info: Updating devices to version 0.14.0
  741. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status_fencing) info: Node el8-a01n01 is active
  742. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status) info: Node el8-a01n01 is online
  743. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status_fencing) info: Node el8-a01n02 is active
  744. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status) info: Node el8-a01n02 is online
  745. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 1 is already processed
  746. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 2 is already processed
  747. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 1 is already processed
  748. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 2 is already processed
  749. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (common_print) info: virsh_node1 (stonith:fence_virsh): Started el8-a01n01
  750. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (common_print) info: virsh_node2 (stonith:fence_virsh): Started el8-a01n02
  751. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  752. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (short_print) info: Started: [ el8-a01n01 el8-a01n02 ]
  753. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (clone_print) info: Clone Set: drbd-clone [drbd]
  754. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (short_print) info: Started: [ el8-a01n01 el8-a01n02 ]
  755. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_color) info: Resource drbd:0 cannot run anywhere
  756. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_color) info: Resource drbd:1 cannot run anywhere
  757. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogActions) info: Leave virsh_node1 (Started el8-a01n01)
  758. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogActions) info: Leave virsh_node2 (Started el8-a01n02)
  759. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogActions) info: Leave hypervisor:0 (Started el8-a01n01)
  760. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogActions) info: Leave hypervisor:1 (Started el8-a01n02)
  761. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Stop drbd:0 ( el8-a01n01 ) due to node availability
  762. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Stop drbd:1 ( el8-a01n02 ) due to node availability
  763. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (pcmk__log_transition_summary) notice: Calculated transition 1, saving inputs in /var/lib/pacemaker/pengine/pe-input-15.bz2
  764. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response
  765. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_te_invoke) info: Processing graph 1 (ref=pe_calc-dc-1594764783-31) derived from /var/lib/pacemaker/pengine/pe-input-15.bz2
  766. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating stop operation drbd_stop_0 locally on el8-a01n01 | action 19
  767. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-execd [32269] (cancel_recurring_action) info: Cancelling systemd operation drbd_status_60000
  768. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:drbd action:monitor call_id:25 exit-code:0 exec-time:0ms queue-time:0ms
  769. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=0
  770. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=19:1:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=drbd_stop_0
  771. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/52)
  772. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=27, reply=1, notify=0
  773. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) info: executing - rsc:drbd action:stop call_id:27
  774. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit drbd named 'drbd'
  775. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating stop operation drbd_stop_0 on el8-a01n02 | action 20
  776. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) info: Result of monitor operation for drbd on el8-a01n01: Cancelled | call=25 key=drbd_monitor_60000 confirmed=true
  777. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec_with_unit) debug: Calling StopUnit for drbd: /org/freedesktop/systemd1/unit/drbd_2eservice
  778. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.14.0 2
  779. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.14.1 (null)
  780. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=1
  781. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_stop_0, @operation=stop, @transition-key=19:1:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;19:1:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594764783, @last-run=1594764783, @exec-time=0, @queue-time=0
  782. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/52, version=0.14.1)
  783. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.14.1 2
  784. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.14.2 (null)
  785. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=2
  786. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_stop_0, @operation=stop, @transition-key=20:1:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;20:1:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594764783, @last-run=1594764783, @exec-time=0
  787. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/29, version=0.14.2)
  788. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-19.raw
  789. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_file_write_with_digest) info: Wrote version 0.14.0 of the CIB to disk (digest: f961ab7ee329b0f7a9c5229a9f0d6bfb)
  790. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.Isb2IW (digest: /var/lib/pacemaker/cib/cib.U26y3v)
  791. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/1941
  792. Jul 14 22:13:03 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=239ms, remaining=99761ms, start_delay=2000ms)
  793. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.14.2 2
  794. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.14.3 (null)
  795. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=3
  796. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;20:1:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=27, @rc-code=0, @op-status=0, @exec-time=2199
  797. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 1 action 20 (drbd_stop_0 on el8-a01n02) confirmed: ok | rc=0 call-id=27
  798. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/30, version=0.14.3)
  799. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:27
  800. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  801. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  802. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop is now complete (elapsed=2246ms, remaining=97754ms): not running (7)
  803. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:drbd action:monitor call_id:27 exit-code:0 exec-time:2246ms queue-time:0ms
  804. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/53)
  805. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) notice: Result of stop operation for drbd on el8-a01n01: 0 (ok) | call=27 key=drbd_stop_0 confirmed=true cib-update=53
  806. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.14.3 2
  807. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.14.4 (null)
  808. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=4
  809. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;19:1:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=27, @rc-code=0, @op-status=0, @exec-time=2246
  810. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/53, version=0.14.4)
  811. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 1 action 19 (drbd_stop_0 on el8-a01n01) confirmed: ok | rc=0 call-id=27
  812. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-controld [32273] (run_graph) notice: Transition 1 (Complete=4, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-15.bz2): Complete
  813. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_log) info: Input I_TE_SUCCESS received in state S_TRANSITION_ENGINE from notify_crmd
  814. Jul 14 22:13:05 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_state_transition) notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd
  815. Jul 14 22:13:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_ping) info: Reporting our current digest to el8-a01n01: d6f800ec4340ab45e7c21c068a77d538 for 0.14.4 (0x55db3e38ea40 0)
  816. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_apply_diff operation for section 'all' to all (origin=local/cibadmin/2)
  817. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.14.4 2
  818. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.0 (null)
  819. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: -- /cib/configuration/resources/clone[@id='drbd-clone']/primitive[@id='drbd']/meta_attributes[@id='drbd-meta_attributes']/nvpair[@id='drbd-meta_attributes-target-role']
  820. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @epoch=15, @num_updates=0
  821. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=el8-a01n01/cibadmin/2, version=0.15.0)
  822. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-controld [32273] (abort_transition_graph) info: Transition 1 aborted by deletion of nvpair[@id='drbd-meta_attributes-target-role']: Configuration change | cib=0.15.0 source=te_update_diff_v2:490 path=/cib/configuration/resources/clone[@id='drbd-clone']/primitive[@id='drbd']/meta_attributes[@id='drbd-meta_attributes']/nvpair[@id='drbd-meta_attributes-target-role'] complete=true
  823. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_state_transition) notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph
  824. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-fenced [32268] (update_cib_stonith_devices_v2) info: Updating device list from the cib: (meta) attribute deleted from resource
  825. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-fenced [32268] (cib_devices_update) info: Updating devices to version 0.15.0
  826. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status_fencing) info: Node el8-a01n01 is active
  827. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status) info: Node el8-a01n01 is online
  828. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status_fencing) info: Node el8-a01n02 is active
  829. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status) info: Node el8-a01n02 is online
  830. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 1 is already processed
  831. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 2 is already processed
  832. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 1 is already processed
  833. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 2 is already processed
  834. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (common_print) info: virsh_node1 (stonith:fence_virsh): Started el8-a01n01
  835. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (common_print) info: virsh_node2 (stonith:fence_virsh): Started el8-a01n02
  836. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  837. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (short_print) info: Started: [ el8-a01n01 el8-a01n02 ]
  838. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (clone_print) info: Clone Set: drbd-clone [drbd]
  839. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (short_print) info: Stopped: [ el8-a01n01 el8-a01n02 ]
  840. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (RecurringOp) info: Start recurring monitor (60s) for drbd:0 on el8-a01n01
  841. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (RecurringOp) info: Start recurring monitor (60s) for drbd:1 on el8-a01n02
  842. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogActions) info: Leave virsh_node1 (Started el8-a01n01)
  843. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogActions) info: Leave virsh_node2 (Started el8-a01n02)
  844. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogActions) info: Leave hypervisor:0 (Started el8-a01n01)
  845. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogActions) info: Leave hypervisor:1 (Started el8-a01n02)
  846. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Start drbd:0 ( el8-a01n01 )
  847. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Start drbd:1 ( el8-a01n02 )
  848. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (pcmk__log_transition_summary) notice: Calculated transition 2, saving inputs in /var/lib/pacemaker/pengine/pe-input-16.bz2
  849. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_file_backup) info: Archived previous version as /var/lib/pacemaker/cib/cib-20.raw
  850. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response
  851. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_te_invoke) info: Processing graph 2 (ref=pe_calc-dc-1594764793-34) derived from /var/lib/pacemaker/pengine/pe-input-16.bz2
  852. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating start operation drbd_start_0 locally on el8-a01n01 | action 17
  853. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=17:2:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=drbd_start_0
  854. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/55)
  855. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_file_write_with_digest) info: Wrote version 0.15.0 of the CIB to disk (digest: 5bb60ad37a5f7727a233a50a2a1ce24b)
  856. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.0 2
  857. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.1 (null)
  858. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=1
  859. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_start_0, @operation=start, @transition-key=17:2:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;17:2:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594764793, @last-run=1594764793, @exec-time=0
  860. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/55, version=0.15.1)
  861. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_file_write_with_digest) info: Reading cluster configuration file /var/lib/pacemaker/cib/cib.vDPMVP (digest: /var/lib/pacemaker/cib/cib.NYCpSU)
  862. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=28, reply=1, notify=0
  863. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) info: executing - rsc:drbd action:start call_id:28
  864. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous start op on systemd unit drbd named 'drbd'
  865. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating start operation drbd:1_start_0 on el8-a01n02 | action 19
  866. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec_with_unit) debug: Calling StartUnit for drbd: /org/freedesktop/systemd1/unit/drbd_2eservice
  867. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.1 2
  868. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.2 (null)
  869. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=2
  870. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_start_0, @operation=start, @transition-key=19:2:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;19:2:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594764793, @last-run=1594764793, @exec-time=0
  871. Jul 14 22:13:13 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/31, version=0.15.2)
  872. Jul 14 22:13:14 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_exec_result) info: Call to start passed: /org/freedesktop/systemd1/job/1942
  873. Jul 14 22:13:14 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd start may still be in progress: re-scheduling (elapsed=239ms, remaining=99761ms, start_delay=2000ms)
  874. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.2 2
  875. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.3 (null)
  876. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=3
  877. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;19:2:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=28, @rc-code=0, @op-status=0, @exec-time=2215
  878. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 2 action 19 (drbd_start_0 on el8-a01n02) confirmed: ok | rc=0 call-id=28
  879. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating monitor operation drbd:1_monitor_60000 on el8-a01n02 | action 20
  880. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/32, version=0.15.3)
  881. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.3 2
  882. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.4 (null)
  883. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=4
  884. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_monitor_60000']: @transition-key=20:2:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;20:2:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594764796, @exec-time=0
  885. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/33, version=0.15.4)
  886. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.4 2
  887. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.5 (null)
  888. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=5
  889. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_monitor_60000']: @transition-magic=0:0;20:2:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=29, @rc-code=0, @op-status=0, @exec-time=3
  890. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/34, version=0.15.5)
  891. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 2 action 20 (drbd_monitor_60000 on el8-a01n02) confirmed: ok | rc=0 call-id=29
  892. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:28
  893. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  894. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  895. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd start is now complete (elapsed=2248ms, remaining=97752ms): ok (0)
  896. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:drbd action:monitor call_id:28 exit-code:0 exec-time:2248ms queue-time:0ms
  897. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) notice: Result of start operation for drbd on el8-a01n01: 0 (ok) | call=28 key=drbd_start_0 confirmed=true cib-update=56
  898. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/56)
  899. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.5 2
  900. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.6 (null)
  901. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=6
  902. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=0:0;17:2:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=28, @rc-code=0, @op-status=0, @exec-time=2248
  903. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/56, version=0.15.6)
  904. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 2 action 17 (drbd_start_0 on el8-a01n01) confirmed: ok | rc=0 call-id=28
  905. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating monitor operation drbd_monitor_60000 locally on el8-a01n01 | action 18
  906. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=18:2:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=drbd_monitor_60000
  907. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=29, reply=1, notify=0
  908. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:29
  909. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  910. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/57)
  911. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.6 2
  912. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.7 (null)
  913. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=7
  914. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_monitor_60000']: @transition-key=18:2:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;18:2:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594764796, @exec-time=0
  915. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/57, version=0.15.7)
  916. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  917. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:drbd action:monitor call_id:29 exit-code:0 exec-time:7ms queue-time:0ms
  918. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) notice: Result of monitor operation for drbd on el8-a01n01: 0 (ok) | call=29 key=drbd_monitor_60000 confirmed=false cib-update=58
  919. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/58)
  920. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.7 2
  921. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.8 (null)
  922. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=8
  923. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_monitor_60000']: @transition-magic=0:0;18:2:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=29, @rc-code=0, @op-status=0, @exec-time=7
  924. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/58, version=0.15.8)
  925. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 2 action 18 (drbd_monitor_60000 on el8-a01n01) confirmed: ok | rc=0 call-id=29
  926. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (run_graph) notice: Transition 2 (Complete=6, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-16.bz2): Complete
  927. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_log) info: Input I_TE_SUCCESS received in state S_TRANSITION_ENGINE from notify_crmd
  928. Jul 14 22:13:16 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_state_transition) notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd
  929. Jul 14 22:13:21 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_ping) info: Reporting our current digest to el8-a01n01: 5847f7496bbf44bdca749ca35de63ec1 for 0.15.8 (0x55db3dd5f7a0 0)
  930. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_update_peer_expected) info: handle_request: Node el8-a01n02[2] - expected state is now down (was member)
  931. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (handle_shutdown_request) info: Creating shutdown request for el8-a01n02 (state=S_IDLE)
  932. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-attrd [32270] (attrd_peer_update) notice: Setting shutdown[el8-a01n02]: (unset) -> 1594764807 | from el8-a01n01
  933. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.8 2
  934. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.9 (null)
  935. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=9
  936. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']: <transient_attributes id="2"/>
  937. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ <instance_attributes id="status-2">
  938. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ <nvpair id="status-2-shutdown" name="shutdown" value="1594764807"/>
  939. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ </instance_attributes>
  940. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ </transient_attributes>
  941. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/4, version=0.15.9)
  942. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (abort_transition_graph) info: Transition 2 aborted by transient_attributes.2 'create': Transient attribute change | cib=0.15.9 source=abort_unless_down:329 path=/cib/status/node_state[@id='2'] complete=true
  943. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_state_transition) notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph
  944. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status_fencing) info: Node el8-a01n01 is active
  945. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status) info: Node el8-a01n01 is online
  946. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status) info: Node el8-a01n02 is shutting down
  947. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 1 is already processed
  948. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 2 is already processed
  949. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 1 is already processed
  950. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 2 is already processed
  951. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (common_print) info: virsh_node1 (stonith:fence_virsh): Started el8-a01n01
  952. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (common_print) info: virsh_node2 (stonith:fence_virsh): Started el8-a01n02
  953. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  954. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (short_print) info: Started: [ el8-a01n01 el8-a01n02 ]
  955. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (clone_print) info: Clone Set: drbd-clone [drbd]
  956. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (short_print) info: Started: [ el8-a01n01 el8-a01n02 ]
  957. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_color) info: Resource hypervisor:1 cannot run anywhere
  958. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_color) info: Resource drbd:1 cannot run anywhere
  959. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (RecurringOp) info: Start recurring monitor (60s) for virsh_node2 on el8-a01n01
  960. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (sched_shutdown_op) notice: Scheduling shutdown of node el8-a01n02
  961. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogNodeActions) notice: * Shutdown el8-a01n02
  962. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogActions) info: Leave virsh_node1 (Started el8-a01n01)
  963. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Move virsh_node2 ( el8-a01n02 -> el8-a01n01 )
  964. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogActions) info: Leave hypervisor:0 (Started el8-a01n01)
  965. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Stop hypervisor:1 ( el8-a01n02 ) due to node availability
  966. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogActions) info: Leave drbd:0 (Started el8-a01n01)
  967. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Stop drbd:1 ( el8-a01n02 ) due to node availability
  968. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (pcmk__log_transition_summary) notice: Calculated transition 3, saving inputs in /var/lib/pacemaker/pengine/pe-input-17.bz2
  969. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response
  970. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_te_invoke) info: Processing graph 3 (ref=pe_calc-dc-1594764807-39) derived from /var/lib/pacemaker/pengine/pe-input-17.bz2
  971. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating stop operation virsh_node2_stop_0 on el8-a01n02 | action 9
  972. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating stop operation hypervisor_stop_0 on el8-a01n02 | action 14
  973. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating stop operation drbd_stop_0 on el8-a01n02 | action 21
  974. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.9 2
  975. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.10 (null)
  976. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=10
  977. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=9:3:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;9:3:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594764807, @last-run=1594764807, @exec-time=0
  978. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/35, version=0.15.10)
  979. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.10 2
  980. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.11 (null)
  981. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=11
  982. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=14:3:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;14:3:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594764807, @last-run=1594764807, @exec-time=0
  983. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/36, version=0.15.11)
  984. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.11 2
  985. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.12 (null)
  986. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=12
  987. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;9:3:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=31, @rc-code=0, @op-status=0, @exec-time=1
  988. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/37, version=0.15.12)
  989. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 3 action 9 (virsh_node2_stop_0 on el8-a01n02) confirmed: ok | rc=0 call-id=31
  990. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating start operation virsh_node2_start_0 locally on el8-a01n01 | action 10
  991. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=10:3:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=virsh_node2_start_0
  992. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=30, reply=1, notify=0
  993. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) info: executing - rsc:virsh_node2 action:start call_id:30
  994. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.12 2
  995. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.13 (null)
  996. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=13
  997. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_stop_0, @operation=stop, @transition-key=21:3:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;21:3:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594764807, @last-run=1594764807, @exec-time=0
  998. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/38, version=0.15.13)
  999. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/60)
  1000. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.13 2
  1001. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.14 (null)
  1002. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=14
  1003. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_start_0, @operation=start, @transition-key=10:3:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;10:3:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594764807, @last-run=1594764807, @queue-time
  1004. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/60, version=0.15.14)
  1005. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemakerd [32266] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  1006. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemakerd [32266] (pcmk_shutdown_worker) notice: Shutting down Pacemaker
  1007. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemakerd [32266] (stop_child) notice: Stopping pacemaker-controld | sent signal 15 to process 32273
  1008. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)
  1009. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_shutdown) notice: Shutting down cluster resource manager | limit=1200000ms
  1010. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_log) warning: Input I_SHUTDOWN received in state S_TRANSITION_ENGINE from crm_shutdown
  1011. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_state_transition) info: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE | input=I_SHUTDOWN cause=C_SHUTDOWN origin=crm_shutdown
  1012. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_shutdown_req) info: Sending shutdown request to all peers (DC is el8-a01n01)
  1013. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_update_peer_expected) info: handle_request: Node el8-a01n01[1] - expected state is now down (was member)
  1014. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (handle_shutdown_request) info: Creating shutdown request for el8-a01n01 (state=S_POLICY_ENGINE)
  1015. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-attrd [32270] (attrd_peer_update) notice: Setting shutdown[el8-a01n01]: (unset) -> 1594764807 | from el8-a01n01
  1016. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.14 2
  1017. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.15 (null)
  1018. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=15
  1019. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='1']: <transient_attributes id="1"/>
  1020. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ <instance_attributes id="status-1">
  1021. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ <nvpair id="status-1-shutdown" name="shutdown" value="1594764807"/>
  1022. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ </instance_attributes>
  1023. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ </transient_attributes>
  1024. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/5, version=0.15.15)
  1025. Jul 14 22:13:27 el8-a01n01.digimer.ca pacemaker-controld [32273] (abort_transition_graph) notice: Transition 3 aborted by transient_attributes.1 'create': Transient attribute change | cib=0.15.15 source=abort_unless_down:329 path=/cib/status/node_state[@id='1'] complete=false
  1026. Jul 14 22:13:28 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) info: finished - rsc:virsh_node2 action:start call_id:30 exit-code:0 exec-time:652ms queue-time:0ms
  1027. Jul 14 22:13:28 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) notice: Result of start operation for virsh_node2 on el8-a01n01: 0 (ok) | call=30 key=virsh_node2_start_0 confirmed=true cib-update=61
  1028. Jul 14 22:13:28 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/61)
  1029. Jul 14 22:13:28 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.15 2
  1030. Jul 14 22:13:28 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.16 (null)
  1031. Jul 14 22:13:28 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=16
  1032. Jul 14 22:13:28 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;10:3:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=30, @rc-code=0, @op-status=0, @exec-time=652
  1033. Jul 14 22:13:28 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 3 action 10 (virsh_node2_start_0 on el8-a01n01) confirmed: ok | rc=0 call-id=30
  1034. Jul 14 22:13:28 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/61, version=0.15.16)
  1035. Jul 14 22:13:29 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.16 2
  1036. Jul 14 22:13:29 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.17 (null)
  1037. Jul 14 22:13:29 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=17
  1038. Jul 14 22:13:29 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;14:3:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=33, @rc-code=0, @op-status=0, @exec-time=2219
  1039. Jul 14 22:13:29 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/39, version=0.15.17)
  1040. Jul 14 22:13:29 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 3 action 14 (hypervisor_stop_0 on el8-a01n02) confirmed: ok | rc=0 call-id=33
  1041. Jul 14 22:13:34 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_ping) info: Reporting our current digest to el8-a01n01: a76984cee9df38db885f8565e9ac76c7 for 0.15.17 (0x55db3dd5f7a0 0)
  1042. Jul 14 22:13:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (recurring_action_timer) debug: Scheduling another invocation of hypervisor_status_60000
  1043. Jul 14 22:13:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  1044. Jul 14 22:13:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1045. Jul 14 22:13:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  1046. Jul 14 22:13:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:22
  1047. Jul 14 22:13:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:22 exit-code:0 exec-time:664ms queue-time:0ms
  1048. Jul 14 22:14:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (recurring_action_timer) debug: Scheduling another invocation of drbd_status_60000
  1049. Jul 14 22:14:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1050. Jul 14 22:14:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1051. Jul 14 22:14:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:drbd action:monitor call_id:29 exit-code:0 exec-time:0ms queue-time:0ms
  1052. Jul 14 22:14:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (recurring_action_timer) debug: Scheduling another invocation of hypervisor_status_60000
  1053. Jul 14 22:14:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  1054. Jul 14 22:14:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1055. Jul 14 22:14:41 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  1056. Jul 14 22:14:43 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:virsh_node1 action:monitor call_id:22
  1057. Jul 14 22:14:44 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:22 exit-code:0 exec-time:639ms queue-time:0ms
  1058. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.17 2
  1059. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.18 (null)
  1060. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=18
  1061. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @transition-magic=2:198;21:3:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=35, @rc-code=198, @op-status=2, @exec-time=99990
  1062. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='drbd']: <lrm_rsc_op id="drbd_last_failure_0" operation_key="drbd_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.3.0" transition-key="21:3:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" transition-magic="2:198;21:3:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af" exit-reason="" on_node="el8-a01n02" call-id="35" rc-code="
  1063. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (abort_transition_graph) info: Transition 3 aborted by operation drbd_stop_0 'modify' on el8-a01n02: Event failed | magic=2:198;21:3:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af cib=0.15.18 source=process_graph_event:453 complete=false
  1064. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (update_failcount) info: Updating failcount for drbd on el8-a01n02 after failed stop: rc=198 (update=INFINITY, time=1594764907)
  1065. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/crmd/40, version=0.15.18)
  1066. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) notice: Transition 3 action 21 (drbd_stop_0 on el8-a01n02): expected 'ok' but got 'OCF_TIMEOUT' | target-rc=0 rc=198 call-id=35
  1067. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (run_graph) notice: Transition 3 (Complete=8, Pending=0, Fired=0, Skipped=1, Incomplete=2, Source=/var/lib/pacemaker/pengine/pe-input-17.bz2): Stopped
  1068. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-attrd [32270] (attrd_peer_update) notice: Setting fail-count-drbd#stop_0[el8-a01n02]: (unset) -> INFINITY | from el8-a01n01
  1069. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-attrd [32270] (attrd_peer_update) notice: Setting last-failure-drbd#stop_0[el8-a01n02]: (unset) -> 1594764907 | from el8-a01n01
  1070. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.18 2
  1071. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.19 (null)
  1072. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=19
  1073. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: <nvpair id="status-2-fail-count-drbd.stop_0" name="fail-count-drbd#stop_0" value="INFINITY"/>
  1074. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/6, version=0.15.19)
  1075. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.19 2
  1076. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.20 (null)
  1077. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=20
  1078. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: <nvpair id="status-2-last-failure-drbd.stop_0" name="last-failure-drbd#stop_0" value="1594764907"/>
  1079. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n02/attrd/7, version=0.15.20)
  1080. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (abort_transition_graph) info: Transition 3 aborted by status-2-fail-count-drbd.stop_0 doing create fail-count-drbd#stop_0=INFINITY: Transient attribute change | cib=0.15.19 source=abort_unless_down:329 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'] complete=true
  1081. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (abort_transition_graph) info: Transition 3 aborted by status-2-last-failure-drbd.stop_0 doing create last-failure-drbd#stop_0=1594764907: Transient attribute change | cib=0.15.20 source=abort_unless_down:329 path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'] complete=true
  1082. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status) info: Node el8-a01n01 is shutting down
  1083. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status) info: Node el8-a01n02 is shutting down
  1084. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_rsc_op_failure) warning: Unexpected result (OCF_TIMEOUT) was recorded for stop of drbd:1 on el8-a01n02 at Jul 14 22:13:27 2020 | rc=198 id=drbd_last_failure_0
  1085. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_rsc_op_failure) warning: Unexpected result (OCF_TIMEOUT) was recorded for stop of drbd:1 on el8-a01n02 at Jul 14 22:13:27 2020 | rc=198 id=drbd_last_0
  1086. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (pe_fence_node) warning: Cluster node el8-a01n02 will be fenced: drbd:1 failed there
  1087. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 1 is already processed
  1088. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 2 is already processed
  1089. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 1 is already processed
  1090. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 2 is already processed
  1091. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (common_print) info: virsh_node1 (stonith:fence_virsh): Started el8-a01n01
  1092. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (common_print) info: virsh_node2 (stonith:fence_virsh): Started el8-a01n01
  1093. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  1094. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (short_print) info: Started: [ el8-a01n01 ]
  1095. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (short_print) info: Stopped: [ el8-a01n02 ]
  1096. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (clone_print) info: Clone Set: drbd-clone [drbd]
  1097. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (common_print) info: drbd (systemd:drbd): FAILED el8-a01n02
  1098. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (short_print) info: Started: [ el8-a01n01 ]
  1099. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_color) info: Resource virsh_node1 cannot run anywhere
  1100. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_color) info: Resource virsh_node2 cannot run anywhere
  1101. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_color) info: Resource hypervisor:0 cannot run anywhere
  1102. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_color) info: Resource hypervisor:1 cannot run anywhere
  1103. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_color) info: Resource drbd:0 cannot run anywhere
  1104. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_color) info: Resource drbd:1 cannot run anywhere
  1105. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (sched_shutdown_op) notice: Scheduling shutdown of node el8-a01n01
  1106. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (stage6) warning: Scheduling Node el8-a01n02 for STONITH
  1107. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_stop_constraints) notice: Stop of failed resource drbd:1 is implicit after el8-a01n02 is fenced
  1108. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogNodeActions) notice: * Shutdown el8-a01n01
  1109. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogNodeActions) notice: * Fence (reboot) el8-a01n02 'drbd:1 failed there'
  1110. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Stop virsh_node1 ( el8-a01n01 ) due to node availability
  1111. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Stop virsh_node2 ( el8-a01n01 ) due to node availability
  1112. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Stop hypervisor:0 ( el8-a01n01 ) due to node availability
  1113. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogActions) info: Leave hypervisor:1 (Stopped)
  1114. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Stop drbd:0 ( el8-a01n01 ) due to node availability
  1115. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Stop drbd:1 ( el8-a01n02 ) due to node availability
  1116. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (pcmk__log_transition_summary) warning: Calculated transition 4 (with warnings), saving inputs in /var/lib/pacemaker/pengine/pe-warn-4.bz2
  1117. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (handle_response) info: pe_calc calculation pe_calc-dc-1594764907-45 is obsolete
  1118. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status) info: Node el8-a01n01 is shutting down
  1119. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (determine_online_status) info: Node el8-a01n02 is shutting down
  1120. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_rsc_op_failure) warning: Unexpected result (OCF_TIMEOUT) was recorded for stop of drbd:1 on el8-a01n02 at Jul 14 22:13:27 2020 | rc=198 id=drbd_last_failure_0
  1121. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_rsc_op_failure) warning: Unexpected result (OCF_TIMEOUT) was recorded for stop of drbd:1 on el8-a01n02 at Jul 14 22:13:27 2020 | rc=198 id=drbd_last_0
  1122. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (pe_fence_node) warning: Cluster node el8-a01n02 will be fenced: drbd:1 failed there
  1123. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 1 is already processed
  1124. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 2 is already processed
  1125. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 1 is already processed
  1126. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (unpack_node_loop) info: Node 2 is already processed
  1127. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (common_print) info: virsh_node1 (stonith:fence_virsh): Started el8-a01n01
  1128. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (common_print) info: virsh_node2 (stonith:fence_virsh): Started el8-a01n01
  1129. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (clone_print) info: Clone Set: hypervisor-clone [hypervisor]
  1130. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (short_print) info: Started: [ el8-a01n01 ]
  1131. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (short_print) info: Stopped: [ el8-a01n02 ]
  1132. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (clone_print) info: Clone Set: drbd-clone [drbd]
  1133. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (common_print) info: drbd (systemd:drbd): FAILED el8-a01n02
  1134. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (short_print) info: Started: [ el8-a01n01 ]
  1135. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (pe_get_failcount) info: drbd:0 has failed INFINITY times on el8-a01n02
  1136. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (check_migration_threshold) warning: Forcing drbd-clone away from el8-a01n02 after 1000000 failures (max=1000000)
  1137. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (pe_get_failcount) info: drbd:1 has failed INFINITY times on el8-a01n02
  1138. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (check_migration_threshold) warning: Forcing drbd-clone away from el8-a01n02 after 1000000 failures (max=1000000)
  1139. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_color) info: Resource virsh_node1 cannot run anywhere
  1140. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_color) info: Resource virsh_node2 cannot run anywhere
  1141. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_color) info: Resource hypervisor:0 cannot run anywhere
  1142. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_color) info: Resource hypervisor:1 cannot run anywhere
  1143. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_color) info: Resource drbd:0 cannot run anywhere
  1144. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_color) info: Resource drbd:1 cannot run anywhere
  1145. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (sched_shutdown_op) notice: Scheduling shutdown of node el8-a01n01
  1146. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (stage6) warning: Scheduling Node el8-a01n02 for STONITH
  1147. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (native_stop_constraints) notice: Stop of failed resource drbd:1 is implicit after el8-a01n02 is fenced
  1148. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogNodeActions) notice: * Shutdown el8-a01n01
  1149. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogNodeActions) notice: * Fence (reboot) el8-a01n02 'drbd:1 failed there'
  1150. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Stop virsh_node1 ( el8-a01n01 ) due to node availability
  1151. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Stop virsh_node2 ( el8-a01n01 ) due to node availability
  1152. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Stop hypervisor:0 ( el8-a01n01 ) due to node availability
  1153. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogActions) info: Leave hypervisor:1 (Stopped)
  1154. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Stop drbd:0 ( el8-a01n01 ) due to node availability
  1155. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (LogAction) notice: * Stop drbd:1 ( el8-a01n02 ) due to node availability
  1156. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-schedulerd[32271] (pcmk__log_transition_summary) warning: Calculated transition 5 (with warnings), saving inputs in /var/lib/pacemaker/pengine/pe-warn-5.bz2
  1157. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_state_transition) info: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response
  1158. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_te_invoke) info: Processing graph 5 (ref=pe_calc-dc-1594764907-46) derived from /var/lib/pacemaker/pengine/pe-warn-5.bz2
  1159. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating stop operation virsh_node1_stop_0 locally on el8-a01n01 | action 6
  1160. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:virsh_node1 action:monitor call_id:22 exit-code:0 exec-time:0ms queue-time:0ms
  1161. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=0
  1162. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=6:5:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=virsh_node1_stop_0
  1163. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=32, reply=1, notify=0
  1164. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) info: executing - rsc:virsh_node1 action:stop call_id:32
  1165. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/65)
  1166. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) info: finished - rsc:virsh_node1 action:stop call_id:32 exit-code:0 exec-time:0ms queue-time:0ms
  1167. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating stop operation virsh_node2_stop_0 locally on el8-a01n01 | action 7
  1168. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=7:5:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=virsh_node2_stop_0
  1169. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=33, reply=1, notify=0
  1170. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) info: executing - rsc:virsh_node2 action:stop call_id:33
  1171. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/66)
  1172. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) info: finished - rsc:virsh_node2 action:stop call_id:33 exit-code:0 exec-time:0ms queue-time:0ms
  1173. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_fence_node) notice: Requesting fencing (reboot) of node el8-a01n02 | action=5 timeout=60000
  1174. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.20 2
  1175. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.21 (null)
  1176. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=21
  1177. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @operation_key=virsh_node1_stop_0, @operation=stop, @transition-key=6:5:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;6:5:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594764907, @last-run=1594764907, @exec-time=0
  1178. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-fenced [32268] (handle_request) notice: Client pacemaker-controld.32273.16321daa wants to fence (reboot) 'el8-a01n02' with device '(any)'
  1179. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-fenced [32268] (initiate_remote_stonith_op) notice: Requesting peer fencing (reboot) targeting el8-a01n02 | id=f775778c-fc66-48cb-b74e-926c5586336e state=0
  1180. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/65, version=0.15.21)
  1181. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-fenced [32268] (can_fence_host_with_device) notice: virsh_node1 is not eligible to fence (reboot) el8-a01n02: static-list
  1182. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-fenced [32268] (can_fence_host_with_device) notice: virsh_node2 is eligible to fence (reboot) el8-a01n02: static-list
  1183. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.21 2
  1184. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.22 (null)
  1185. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=22
  1186. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @operation_key=virsh_node2_stop_0, @operation=stop, @transition-key=7:5:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;7:5:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594764907, @last-run=1594764907, @exec-time=0
  1187. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-fenced [32268] (process_remote_stonith_query) info: Query result 1 of 2 from el8-a01n01 for el8-a01n02/reboot (1 devices) f775778c-fc66-48cb-b74e-926c5586336e
  1188. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) info: Result of monitor operation for virsh_node1 on el8-a01n01: Cancelled | call=22 key=virsh_node1_monitor_60000 confirmed=true
  1189. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/66, version=0.15.22)
  1190. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-fenced [32268] (process_remote_stonith_query) info: Query result 2 of 2 from el8-a01n02 for el8-a01n02/reboot (0 devices) f775778c-fc66-48cb-b74e-926c5586336e
  1191. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-fenced [32268] (process_remote_stonith_query) info: All query replies have arrived, continuing (2 expected/2 received)
  1192. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-fenced [32268] (call_remote_stonith) info: Total timeout set to 60 for peer's fencing targeting el8-a01n02 for pacemaker-controld.32273|id=f775778c-fc66-48cb-b74e-926c5586336e
  1193. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-fenced [32268] (call_remote_stonith) notice: Requesting that el8-a01n01 perform 'reboot' action targeting el8-a01n02 | for client pacemaker-controld.32273 (72s, 0s)
  1194. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) notice: Result of stop operation for virsh_node1 on el8-a01n01: 0 (ok) | call=32 key=virsh_node1_stop_0 confirmed=true cib-update=67
  1195. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/67)
  1196. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-fenced [32268] (can_fence_host_with_device) notice: virsh_node1 is not eligible to fence (reboot) el8-a01n02: static-list
  1197. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-fenced [32268] (can_fence_host_with_device) notice: virsh_node2 is eligible to fence (reboot) el8-a01n02: static-list
  1198. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-fenced [32268] (stonith_fence_get_devices_cb) info: Found 1 matching devices for 'el8-a01n02'
  1199. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.22 2
  1200. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.23 (null)
  1201. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=23
  1202. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node1']/lrm_rsc_op[@id='virsh_node1_last_0']: @transition-magic=0:0;6:5:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=32, @rc-code=0, @op-status=0
  1203. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/67, version=0.15.23)
  1204. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/68)
  1205. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.23 2
  1206. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.24 (null)
  1207. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=24
  1208. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='virsh_node2']/lrm_rsc_op[@id='virsh_node2_last_0']: @transition-magic=0:0;7:5:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=33, @rc-code=0, @op-status=0
  1209. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/68, version=0.15.24)
  1210. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) notice: Result of stop operation for virsh_node2 on el8-a01n01: 0 (ok) | call=33 key=virsh_node2_stop_0 confirmed=true cib-update=68
  1211. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 5 action 6 (virsh_node1_stop_0 on el8-a01n01) confirmed: ok | rc=0 call-id=32
  1212. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 5 action 7 (virsh_node2_stop_0 on el8-a01n01) confirmed: ok | rc=0 call-id=33
  1213. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating stop operation hypervisor_stop_0 locally on el8-a01n01 | action 8
  1214. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-execd [32269] (cancel_recurring_action) info: Cancelling systemd operation hypervisor_status_60000
  1215. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:24 exit-code:0 exec-time:0ms queue-time:0ms
  1216. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=0
  1217. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=8:5:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=hypervisor_stop_0
  1218. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=35, reply=1, notify=0
  1219. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/69)
  1220. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) info: executing - rsc:hypervisor action:stop call_id:35
  1221. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit libvirtd named 'hypervisor'
  1222. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.24 2
  1223. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.25 (null)
  1224. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=25
  1225. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @operation_key=hypervisor_stop_0, @operation=stop, @transition-key=8:5:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;8:5:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594764907, @last-run=1594764907, @exec-time=0
  1226. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/69, version=0.15.25)
  1227. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec_with_unit) debug: Calling StopUnit for hypervisor: /org/freedesktop/systemd1/unit/libvirtd_2eservice
  1228. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) info: Result of monitor operation for hypervisor on el8-a01n01: Cancelled | call=24 key=hypervisor_monitor_60000 confirmed=true
  1229. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/2046
  1230. Jul 14 22:15:07 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: hypervisor stop may still be in progress: re-scheduling (elapsed=235ms, remaining=99765ms, start_delay=2000ms)
  1231. Jul 14 22:15:09 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:hypervisor action:monitor call_id:35
  1232. Jul 14 22:15:09 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit libvirtd named 'hypervisor'
  1233. Jul 14 22:15:09 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1234. Jul 14 22:15:09 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: hypervisor stop is now complete (elapsed=2244ms, remaining=97756ms): not running (7)
  1235. Jul 14 22:15:09 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:hypervisor action:monitor call_id:35 exit-code:0 exec-time:2244ms queue-time:0ms
  1236. Jul 14 22:15:09 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) notice: Result of stop operation for hypervisor on el8-a01n01: 0 (ok) | call=35 key=hypervisor_stop_0 confirmed=true cib-update=70
  1237. Jul 14 22:15:09 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/70)
  1238. Jul 14 22:15:09 el8-a01n01.digimer.ca pacemaker-fenced [32268] (log_operation) notice: Operation 'reboot' [60289] (call 2 from pacemaker-controld.32273) for host 'el8-a01n02' with device 'virsh_node2' returned: 0 (OK)
  1239. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (pcmk_quorum_notification) info: Quorum retained | membership=179 members=1
  1240. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now lost | nodeid=2 previous=member source=crm_reap_unseen_nodes
  1241. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (peer_update_callback) info: Cluster node el8-a01n02 is now lost (was member)
  1242. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (pcmk_cpg_membership) info: Group cib event 2: el8-a01n02 (node 2 pid 1815) left via cluster exit
  1243. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now offline
  1244. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now lost | nodeid=2 previous=member source=crm_update_peer_proc
  1245. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (crm_reap_dead_member) info: Removing node with name el8-a01n02 and id 2 from membership cache
  1246. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (reap_crm_member) notice: Purged 1 peer with id=2 and/or uname=el8-a01n02 from the membership cache
  1247. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-attrd [32270] (pcmk_cpg_membership) info: Group attrd event 1: el8-a01n02 (node 2 pid 1818) left via cluster exit
  1248. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (pcmk_cpg_membership) info: Group cib event 2: el8-a01n01 (node 1 pid 32267) is member
  1249. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-attrd [32270] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now offline
  1250. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-attrd [32270] (attrd_remove_voter) notice: Lost attribute writer el8-a01n02
  1251. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-attrd [32270] (attrd_start_election_if_needed) info: Starting an election to determine the writer
  1252. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemakerd [32266] (pcmk_quorum_notification) info: Quorum retained | membership=179 members=1
  1253. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemakerd [32266] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now lost | nodeid=2 previous=member source=crm_reap_unseen_nodes
  1254. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-attrd [32270] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now lost | nodeid=2 previous=member source=crm_update_peer_proc
  1255. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemakerd [32266] (pcmk_cpg_membership) info: Group pacemakerd event 2: el8-a01n02 (node 2 pid 1814) left via cluster exit
  1256. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemakerd [32266] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now offline
  1257. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-attrd [32270] (attrd_peer_remove) notice: Removing all el8-a01n02 attributes for peer loss
  1258. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemakerd [32266] (pcmk_cpg_membership) info: Group pacemakerd event 2: el8-a01n01 (node 1 pid 32266) is member
  1259. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-attrd [32270] (crm_reap_dead_member) info: Removing node with name el8-a01n02 and id 2 from membership cache
  1260. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-attrd [32270] (reap_crm_member) notice: Purged 1 peer with id=2 and/or uname=el8-a01n02 from the membership cache
  1261. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-attrd [32270] (pcmk_cpg_membership) info: Group attrd event 1: el8-a01n01 (node 1 pid 32270) is member
  1262. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-attrd [32270] (election_check) info: election-attrd won by local node
  1263. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-attrd [32270] (attrd_declare_winner) notice: Recorded local node as attribute writer (was unset)
  1264. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-attrd [32270] (write_attribute) info: Processed 1 private change for #attrd-protocol, id=n/a, set=n/a
  1265. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-attrd [32270] (write_attribute) info: Sent CIB request 4 with 1 change for shutdown (id n/a, set n/a)
  1266. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.25 2
  1267. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.26 (null)
  1268. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=26
  1269. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='hypervisor']/lrm_rsc_op[@id='hypervisor_last_0']: @transition-magic=0:0;8:5:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=35, @rc-code=0, @op-status=0, @exec-time=2244
  1270. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/70, version=0.15.26)
  1271. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/attrd/4)
  1272. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-fenced [32268] (pcmk_cpg_membership) info: Group stonith-ng event 2: el8-a01n02 (node 2 pid 1816) left via cluster exit
  1273. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-fenced [32268] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now offline
  1274. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_graph_event) info: Transition 5 action 8 (hypervisor_stop_0 on el8-a01n01) confirmed: ok | rc=0 call-id=35
  1275. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (pcmk_cpg_membership) info: Group crmd event 1: el8-a01n02 (node 2 pid 1820) left via cluster exit
  1276. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (crm_update_peer_proc) info: pcmk_cpg_membership: Node el8-a01n02[2] - corosync-cpg is now offline
  1277. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (peer_update_callback) info: Node el8-a01n02 is no longer a peer | DC=true old=0x4000000 new=0x0000000
  1278. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (controld_delete_node_state) info: Deleting transient attributes for node el8-a01n02 (via CIB call 76) | xpath=//node_state[@uname='el8-a01n02']/transient_attributes
  1279. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (pcmk_cpg_membership) info: Group crmd event 1: el8-a01n01 (node 1 pid 32273) is member
  1280. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/71)
  1281. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-fenced [32268] (crm_update_peer_state_iter) notice: Node el8-a01n02 state is now lost | nodeid=2 previous=member source=crm_update_peer_proc
  1282. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-fenced [32268] (crm_reap_dead_member) info: Removing node with name el8-a01n02 and id 2 from membership cache
  1283. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-fenced [32268] (reap_crm_member) notice: Purged 1 peer with id=2 and/or uname=el8-a01n02 from the membership cache
  1284. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-fenced [32268] (pcmk_cpg_membership) info: Group stonith-ng event 2: el8-a01n01 (node 1 pid 32268) is member
  1285. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/attrd/4, version=0.15.26)
  1286. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-attrd [32270] (attrd_cib_callback) info: CIB update 4 result for shutdown: OK | rc=0
  1287. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-attrd [32270] (attrd_cib_callback) info: * shutdown[el8-a01n01]=1594764807
  1288. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-fenced [32268] (remote_op_done) notice: Operation 'reboot' targeting el8-a01n02 on el8-a01n01 for pacemaker-controld.32273@el8-a01n01.f775778c: OK
  1289. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (tengine_stonith_callback) notice: Stonith operation 2/5:5:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af: OK (0)
  1290. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (tengine_stonith_callback) info: Stonith operation 2 for el8-a01n02 passed
  1291. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (controld_delete_node_state) info: Deleting all state for node el8-a01n02 (via CIB call 79) | xpath=//node_state[@uname='el8-a01n02']/*
  1292. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (tengine_stonith_notify) notice: Peer el8-a01n02 was terminated (reboot) by el8-a01n01 on behalf of pacemaker-controld.32273: OK | initiator=el8-a01n01 ref=f775778c-fc66-48cb-b74e-926c5586336e
  1293. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (controld_delete_node_state) info: Deleting all state for node el8-a01n02 (via CIB call 81) | xpath=//node_state[@uname='el8-a01n02']/*
  1294. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (te_rsc_command) notice: Initiating stop operation drbd_stop_0 locally on el8-a01n01 | action 13
  1295. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-execd [32269] (cancel_recurring_action) info: Cancelling systemd operation drbd_status_60000
  1296. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_finished) debug: finished - rsc:drbd action:monitor call_id:29 exit-code:0 exec-time:0ms queue-time:0ms
  1297. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_cancel operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=0, reply=1, notify=0
  1298. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.26 2
  1299. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.27 (null)
  1300. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=27
  1301. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=peer_update_callback
  1302. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/71, version=0.15.27)
  1303. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/74)
  1304. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (do_lrm_rsc_op) info: Performing key=13:5:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af op=drbd_stop_0
  1305. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-execd [32269] (process_lrmd_message) debug: Processed lrmd_rsc_exec operation from ee3d8d6e-17ff-4c5b-a03a-b568ee2dd1e1: rc=37, reply=1, notify=0
  1306. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) info: executing - rsc:drbd action:stop call_id:37
  1307. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous stop op on systemd unit drbd named 'drbd'
  1308. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec_with_unit) debug: Calling StopUnit for drbd: /org/freedesktop/systemd1/unit/drbd_2eservice
  1309. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (process_lrm_event) info: Result of monitor operation for drbd on el8-a01n01: Cancelled | call=29 key=drbd_monitor_60000 confirmed=true
  1310. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/75)
  1311. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n02']/transient_attributes to all (origin=local/crmd/76)
  1312. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/77)
  1313. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section nodes: OK (rc=0, origin=el8-a01n01/crmd/74, version=0.15.27)
  1314. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.27 2
  1315. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.28 (null)
  1316. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=28
  1317. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=post_cache_update, @expected=down
  1318. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @in_ccm=false, @crm-debug-origin=post_cache_update, @expected=down
  1319. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/75, version=0.15.28)
  1320. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.28 2
  1321. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.29 (null)
  1322. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/transient_attributes[@id='2']
  1323. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=29
  1324. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/transient_attributes: OK (rc=0, origin=el8-a01n01/crmd/76, version=0.15.29)
  1325. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.29 2
  1326. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.30 (null)
  1327. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=30
  1328. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crmd=offline, @crm-debug-origin=peer_update_callback
  1329. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/77, version=0.15.30)
  1330. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/78)
  1331. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n02']/* to all (origin=local/crmd/79)
  1332. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/80)
  1333. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_delete operation for section //node_state[@uname='el8-a01n02']/* to all (origin=local/crmd/81)
  1334. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Forwarding cib_modify operation for section status to all (origin=local/crmd/82)
  1335. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.30 2
  1336. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.31 (null)
  1337. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=31
  1338. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='2']: @crm-debug-origin=send_stonith_update, @join=down
  1339. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/78, version=0.15.31)
  1340. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.31 2
  1341. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.32 (null)
  1342. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: -- /cib/status/node_state[@id='2']/lrm[@id='2']
  1343. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=32
  1344. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/*: OK (rc=0, origin=el8-a01n01/crmd/79, version=0.15.32)
  1345. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/80, version=0.15.32)
  1346. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_delete operation for section //node_state[@uname='el8-a01n02']/*: OK (rc=0, origin=el8-a01n01/crmd/81, version=0.15.32)
  1347. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: --- 0.15.32 2
  1348. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: Diff: +++ 0.15.33 (null)
  1349. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib: @num_updates=33
  1350. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource
  1351. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='drbd']/lrm_rsc_op[@id='drbd_last_0']: @operation_key=drbd_stop_0, @operation=stop, @transition-key=13:5:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @transition-magic=-1:193;13:5:0:ae37b40f-84ce-4cb6-b3a6-7dd8a73a65af, @call-id=-1, @rc-code=193, @op-status=-1, @last-rc-change=1594764910, @last-run=1594764910, @exec-time=0
  1352. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_request) info: Completed cib_modify operation for section status: OK (rc=0, origin=el8-a01n01/crmd/82, version=0.15.33)
  1353. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (cib_fencing_updated) info: Fencing update 78 for el8-a01n02: complete
  1354. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-controld [32273] (cib_fencing_updated) info: Fencing update 80 for el8-a01n02: complete
  1355. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_exec_result) info: Call to stop passed: /org/freedesktop/systemd1/job/2047
  1356. Jul 14 22:15:10 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=197ms, remaining=99803ms, start_delay=2000ms)
  1357. Jul 14 22:15:12 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1358. Jul 14 22:15:12 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1359. Jul 14 22:15:12 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1360. Jul 14 22:15:12 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=2202ms, remaining=97798ms, start_delay=2000ms)
  1361. Jul 14 22:15:14 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1362. Jul 14 22:15:14 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1363. Jul 14 22:15:14 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1364. Jul 14 22:15:14 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=4243ms, remaining=95757ms, start_delay=2000ms)
  1365. Jul 14 22:15:15 el8-a01n01.digimer.ca pacemaker-based [32267] (cib_process_ping) info: Reporting our current digest to el8-a01n01: a943c9f319091238e9c520a05c99839d for 0.15.33 (0x55db3e3826c0 0)
  1366. Jul 14 22:15:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1367. Jul 14 22:15:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1368. Jul 14 22:15:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1369. Jul 14 22:15:16 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=6248ms, remaining=93752ms, start_delay=2000ms)
  1370. Jul 14 22:15:18 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1371. Jul 14 22:15:18 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1372. Jul 14 22:15:18 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1373. Jul 14 22:15:18 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=8253ms, remaining=91747ms, start_delay=2000ms)
  1374. Jul 14 22:15:20 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1375. Jul 14 22:15:20 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1376. Jul 14 22:15:20 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1377. Jul 14 22:15:20 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=10257ms, remaining=89743ms, start_delay=2000ms)
  1378. Jul 14 22:15:22 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1379. Jul 14 22:15:22 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1380. Jul 14 22:15:22 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1381. Jul 14 22:15:22 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=12264ms, remaining=87736ms, start_delay=2000ms)
  1382. Jul 14 22:15:24 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1383. Jul 14 22:15:24 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1384. Jul 14 22:15:24 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1385. Jul 14 22:15:24 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=14270ms, remaining=85730ms, start_delay=2000ms)
  1386. Jul 14 22:15:26 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1387. Jul 14 22:15:26 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1388. Jul 14 22:15:26 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1389. Jul 14 22:15:26 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=16277ms, remaining=83723ms, start_delay=2000ms)
  1390. Jul 14 22:15:28 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1391. Jul 14 22:15:28 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1392. Jul 14 22:15:28 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1393. Jul 14 22:15:28 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=18280ms, remaining=81720ms, start_delay=2000ms)
  1394. Jul 14 22:15:30 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1395. Jul 14 22:15:30 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1396. Jul 14 22:15:30 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1397. Jul 14 22:15:30 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=20293ms, remaining=79707ms, start_delay=2000ms)
  1398. Jul 14 22:15:32 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1399. Jul 14 22:15:32 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1400. Jul 14 22:15:32 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1401. Jul 14 22:15:32 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=22303ms, remaining=77697ms, start_delay=2000ms)
  1402. Jul 14 22:15:34 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1403. Jul 14 22:15:34 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1404. Jul 14 22:15:34 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1405. Jul 14 22:15:34 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=24317ms, remaining=75683ms, start_delay=2000ms)
  1406. Jul 14 22:15:36 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1407. Jul 14 22:15:36 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1408. Jul 14 22:15:36 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1409. Jul 14 22:15:36 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=26328ms, remaining=73672ms, start_delay=2000ms)
  1410. Jul 14 22:15:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1411. Jul 14 22:15:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1412. Jul 14 22:15:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1413. Jul 14 22:15:38 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=28357ms, remaining=71643ms, start_delay=2000ms)
  1414. Jul 14 22:15:40 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1415. Jul 14 22:15:40 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1416. Jul 14 22:15:40 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1417. Jul 14 22:15:40 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=30367ms, remaining=69633ms, start_delay=2000ms)
  1418. Jul 14 22:15:42 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1419. Jul 14 22:15:42 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1420. Jul 14 22:15:42 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1421. Jul 14 22:15:42 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=32372ms, remaining=67628ms, start_delay=2000ms)
  1422. Jul 14 22:15:44 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1423. Jul 14 22:15:44 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1424. Jul 14 22:15:44 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1425. Jul 14 22:15:44 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=34375ms, remaining=65625ms, start_delay=2000ms)
  1426. Jul 14 22:15:46 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1427. Jul 14 22:15:46 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1428. Jul 14 22:15:46 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1429. Jul 14 22:15:46 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=36383ms, remaining=63617ms, start_delay=2000ms)
  1430. Jul 14 22:15:48 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1431. Jul 14 22:15:48 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1432. Jul 14 22:15:48 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1433. Jul 14 22:15:48 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=38386ms, remaining=61614ms, start_delay=2000ms)
  1434. Jul 14 22:15:50 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1435. Jul 14 22:15:50 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1436. Jul 14 22:15:50 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1437. Jul 14 22:15:50 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=40393ms, remaining=59607ms, start_delay=2000ms)
  1438. Jul 14 22:15:52 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1439. Jul 14 22:15:52 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1440. Jul 14 22:15:52 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1441. Jul 14 22:15:52 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=42398ms, remaining=57602ms, start_delay=2000ms)
  1442. Jul 14 22:15:54 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1443. Jul 14 22:15:54 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1444. Jul 14 22:15:54 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1445. Jul 14 22:15:54 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=44408ms, remaining=55592ms, start_delay=2000ms)
  1446. Jul 14 22:15:56 el8-a01n01.digimer.ca pacemaker-execd [32269] (log_execute) debug: executing - rsc:drbd action:monitor call_id:37
  1447. Jul 14 22:15:56 el8-a01n01.digimer.ca pacemaker-execd [32269] (systemd_unit_exec) debug: Performing asynchronous status op on systemd unit drbd named 'drbd'
  1448. Jul 14 22:15:56 el8-a01n01.digimer.ca pacemaker-execd [32269] (pcmk_dbus_get_property) debug: Calling: GetAll on org.freedesktop.systemd1
  1449. Jul 14 22:15:56 el8-a01n01.digimer.ca pacemaker-execd [32269] (action_complete) debug: drbd stop may still be in progress: re-scheduling (elapsed=46411ms, remaining=53589ms, start_delay=2000ms)
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!