API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Sep 3rd, 2013
348
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 101.52 KB | None | 0 0
raw download clone embed print report
  1. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync notice [MAIN ] Corosync Cluster Engine ('2.3.1.1-2740'): started and ready to provide service.
  2. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync info [MAIN ] Corosync built-in features: pie relro bindnow
  3. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] waiting_trans_ack changed to 1
  4. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] Token Timeout (1000 ms) retransmit timeout (238 ms)
  5. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] token hold (180 ms) retransmits before loss (4 retrans)
  6. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] join (50 ms) send_join (0 ms) consensus (1200 ms) merge (200 ms)
  7. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] downcheck (1000 ms) fail to recv const (2500 msgs)
  8. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] seqno unchanged const (30 rotations) Maximum network MTU 1401
  9. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] window size per rotation (50 messages) maximum messages per rotation (17 messages)
  10. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] missed count const (5 messages)
  11. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] send threads (0 threads)
  12. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] RRP token expired timeout (238 ms)
  13. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] RRP token problem counter (2000 ms)
  14. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] RRP threshold (10 problem count)
  15. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] RRP multicast threshold (100 problem count)
  16. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] RRP automatic recovery check timeout (1000 ms)
  17. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] RRP mode set to none.
  18. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] heartbeat_failures_allowed (0)
  19. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] max_network_delay (50 ms)
  20. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] HeartBeat is Disabled. To enable set heartbeat_failures_allowed > 0
  21. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync notice [TOTEM ] Initializing transport (UDP/IP Multicast).
  22. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync notice [TOTEM ] Initializing transmit/receive security (NSS) crypto: none hash: none
  23. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] Receive multicast socket recv buffer size (320000 bytes).
  24. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] Transmit multicast socket send buffer size (320000 bytes).
  25. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] Local receive multicast loop socket recv buffer size (320000 bytes).
  26. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] Local transmit multicast loop socket send buffer size (320000 bytes).
  27. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync notice [TOTEM ] The network interface [10.76.157.19] is now up.
  28. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] Created or loaded sequence id 18.10.76.157.19 for this ring.
  29. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync notice [SERV ] Service engine loaded: corosync configuration map access [0]
  30. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [MAIN ] Initializing IPC on cmap [0]
  31. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [MAIN ] No configured qb.ipc_type. Using native ipc
  32. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync notice [SERV ] Service engine loaded: corosync configuration service [1]
  33. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [MAIN ] Initializing IPC on cfg [1]
  34. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [MAIN ] No configured qb.ipc_type. Using native ipc
  35. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync notice [SERV ] Service engine loaded: corosync cluster closed process group service v1.01 [2]
  36. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [MAIN ] Initializing IPC on cpg [2]
  37. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [MAIN ] No configured qb.ipc_type. Using native ipc
  38. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync notice [SERV ] Service engine loaded: corosync profile loading service [4]
  39. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [MAIN ] NOT Initializing IPC on pload [4]
  40. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync notice [QUORUM] Using quorum provider corosync_votequorum
  41. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] Reading configuration (runtime: 0)
  42. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] total_votes=1, expected_votes=3
  43. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] node 2 state=1, votes=1, expected=3
  44. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] flags: quorate: No Leaving: No WFA Status: No First: Yes Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No
  45. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync notice [SERV ] Service engine loaded: corosync vote quorum service v1.0 [5]
  46. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [MAIN ] Initializing IPC on votequorum [5]
  47. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [MAIN ] No configured qb.ipc_type. Using native ipc
  48. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync notice [SERV ] Service engine loaded: corosync cluster quorum service v0.1 [3]
  49. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [MAIN ] Initializing IPC on quorum [3]
  50. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [MAIN ] No configured qb.ipc_type. Using native ipc
  51. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] entering GATHER state from 15.
  52. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] Creating commit token because I am the rep.
  53. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] Saving state aru 0 high seq received 0
  54. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] Storing new sequence id for ring 1c
  55. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] entering COMMIT state.
  56. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] got commit token
  57. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] entering RECOVERY state.
  58. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] position [0] member 10.76.157.19:
  59. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] previous ring seq 18 rep 10.76.157.19
  60. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] aru 0 high delivered 0 received flag 1
  61. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] Did not need to originate any messages in recovery.
  62. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] got commit token
  63. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] Sending initial ORF token
  64. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 0, aru 0
  65. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] install seq 0 aru 0 high seq received 0
  66. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 1, aru 0
  67. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] install seq 0 aru 0 high seq received 0
  68. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 2, aru 0
  69. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] install seq 0 aru 0 high seq received 0
  70. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 3, aru 0
  71. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] install seq 0 aru 0 high seq received 0
  72. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] retrans flag count 4 token aru 0 install seq 0 aru 0 0
  73. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] Resetting old ring state
  74. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] recovery to regular 1-0
  75. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] waiting_trans_ack changed to 1
  76. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [MAIN ] Member joined: r(0) ip(10.76.157.19)
  77. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] entering OPERATIONAL state.
  78. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync notice [TOTEM ] A new membership (10.76.157.19:28) was formed. Members joined: 2
  79. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] got nodeinfo message from cluster node 2
  80. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] nodeinfo message[2]: votes: 1, expected: 3 flags: 8
  81. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] flags: quorate: No Leaving: No WFA Status: No First: Yes Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No
  82. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] total_votes=1, expected_votes=3
  83. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] node 2 state=1, votes=1, expected=3
  84. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [SYNC ] Committing synchronization for corosync configuration map access
  85. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [CMAP ] Single node sync -> no action
  86. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [CPG ] comparing: sender r(0) ip(10.76.157.19) ; members(old:0 left:0)
  87. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [CPG ] chosen downlist: sender r(0) ip(10.76.157.19) ; members(old:0 left:0)
  88. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [SYNC ] Committing synchronization for corosync cluster closed process group service v1.01
  89. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] flags: quorate: No Leaving: No WFA Status: No First: Yes Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No
  90. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] got nodeinfo message from cluster node 2
  91. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] nodeinfo message[2]: votes: 1, expected: 3 flags: 8
  92. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] flags: quorate: No Leaving: No WFA Status: No First: Yes Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No
  93. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] total_votes=1, expected_votes=3
  94. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] node 2 state=1, votes=1, expected=3
  95. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] got nodeinfo message from cluster node 2
  96. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] nodeinfo message[0]: votes: 0, expected: 0 flags: 0
  97. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [SYNC ] Committing synchronization for corosync vote quorum service v1.0
  98. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] total_votes=1, expected_votes=3
  99. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [VOTEQ ] node 2 state=1, votes=1, expected=3
  100. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync notice [QUORUM] Members[1]: 2
  101. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [QUORUM] sending quorum notification to (nil), length = 52
  102. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync notice [MAIN ] Completed service synchronization, ready to provide service.
  103. Sep 03 17:28:09 [22111] dev-cluster2-node4 corosync debug [TOTEM ] waiting_trans_ack changed to 0
  104. Sep 03 17:28:10 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22116]
  105. Sep 03 17:28:10 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  106. Sep 03 17:28:10 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  107. Sep 03 17:28:10 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  108. Sep 03 17:28:10 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  109. Sep 03 17:28:10 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22116-19)
  110. Sep 03 17:28:10 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22116-19) state:2
  111. Sep 03 17:28:10 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  112. Sep 03 17:28:10 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  113. Sep 03 17:28:10 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  114. Sep 03 17:28:10 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cfg-response-22112-22116-19-header
  115. Sep 03 17:28:10 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cfg-event-22112-22116-19-header
  116. Sep 03 17:28:10 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cfg-request-22112-22116-19-header
  117. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22133]
  118. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  119. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  120. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  121. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  122. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CMAP ] lib_init_fn: conn=0x7f4de0cb8d40
  123. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22133]
  124. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  125. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  126. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  127. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  128. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CMAP ] lib_init_fn: conn=0x7f4de0dbf670
  129. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22133-20)
  130. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22133-20) state:2
  131. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  132. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  133. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CMAP ] exit_fn for conn=0x7f4de0dbf670
  134. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  135. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-response-22112-22133-20-header
  136. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-event-22112-22133-20-header
  137. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-request-22112-22133-20-header
  138. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:934 ) notice: main: Starting Pacemaker 1.1.11-1.el6 (Build: 4f672bc): generated-manpages agent-manpages ascii-docs publican-docs ncurses libqb-logging libqb-ipc lha-fencing nagios corosync-native snmp
  139. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:944 ) info: main: Maximum core file size is: 18446744073709551615
  140. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22133-19)
  141. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22133-19) state:2
  142. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  143. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  144. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( ipc_setup.c:372 ) info: qb_ipcs_us_publish: server name: pacemakerd
  145. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CMAP ] exit_fn for conn=0x7f4de0cb8d40
  146. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  147. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-response-22112-22133-19-header
  148. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-event-22112-22133-19-header
  149. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-request-22112-22133-19-header
  150. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22133]
  151. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  152. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  153. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  154. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  155. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  156. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  157. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  158. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:142 ) debug: cluster_connect_cfg: Our nodeid: 2
  159. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22133]
  160. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  161. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  162. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  163. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  164. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  165. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  166. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  167. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CPG ] lib_init_fn: conn=0x7f4de0dbf670, cpd=0x7f4de0cba5f4
  168. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( cpg.c:110 ) debug: get_local_nodeid: Local nodeid is 2
  169. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (membership.c:280 ) info: crm_get_peer: Created entry 3e401a9b-00be-40eb-9e60-77bf36a55a9c/0x2487730 for node (null)/2 (1 total)
  170. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (membership.c:313 ) info: crm_get_peer: Node 2 has uuid 2
  171. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (membership.c:431 ) info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[2] - corosync-cpg is now online
  172. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:255 ) debug: cluster_connect_quorum: Configuring Pacemaker to obtain quorum from Corosync
  173. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22133]
  174. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  175. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  176. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  177. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  178. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  179. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  180. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  181. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QUORUM] lib_init_fn: conn=0x7f4de0ec10f0
  182. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QUORUM] got quorum_type request on 0x7f4de0ec10f0
  183. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CPG ] got procjoin message from cluster node 2 (r(0) ip(10.76.157.19) ) for pid 22133
  184. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QUORUM] got quorate request on 0x7f4de0ec10f0
  185. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:273 ) notice: cluster_connect_quorum: Quorum lost
  186. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QUORUM] got trackstart request on 0x7f4de0ec10f0
  187. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QUORUM] sending initial status to 0x7f4de0ec10f0
  188. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QUORUM] sending quorum notification to 0x7f4de0ec10f0, length = 52
  189. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22133]
  190. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  191. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  192. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  193. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  194. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  195. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  196. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:501 ) trace: corosync_initialize_nodelist: Initializing corosync nodelist
  197. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  198. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CMAP ] lib_init_fn: conn=0x7f4de0fc4f80
  199. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 0 from nodelist.node.0.nodeid
  200. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 1 from nodelist.node.1.nodeid
  201. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 2 from nodelist.node.2.nodeid
  202. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:104 ) trace: corosync_node_name: Searching for node name for 2 in nodelist.node.2 (null)
  203. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:108 ) trace: corosync_node_name: nodelist.node.2.ring0_addr = 10.76.157.19
  204. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:120 ) trace: corosync_node_name: nodelist.node.2.name = dev-cluster2-node4 1
  205. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:533 ) trace: corosync_initialize_nodelist: Initializing node[0] 0 = dev-cluster2-node4
  206. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (membership.c:280 ) info: crm_get_peer: Created entry 3552f6e0-73fa-4e15-ac91-572b2ba1ac6c/0x2688ce0 for node dev-cluster2-node4/0 (2 total)
  207. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( cluster.c:124 ) info: get_corosync_uuid: Node dev-cluster2-node4 is not yet known by corosync
  208. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (membership.c:316 ) info: crm_get_peer: Cannot obtain a UUID for node 0/dev-cluster2-node4
  209. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:96 ) trace: corosync_node_name: Checking 1 vs 0 from nodelist.node.0.nodeid
  210. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:96 ) trace: corosync_node_name: Checking 1 vs 1 from nodelist.node.1.nodeid
  211. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:104 ) trace: corosync_node_name: Searching for node name for 1 in nodelist.node.1 (null)
  212. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:108 ) trace: corosync_node_name: nodelist.node.1.ring0_addr = 10.76.157.18
  213. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:120 ) trace: corosync_node_name: nodelist.node.1.name = dev-cluster2-node3 1
  214. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:533 ) trace: corosync_initialize_nodelist: Initializing node[1] 1 = dev-cluster2-node3
  215. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (membership.c:280 ) info: crm_get_peer: Created entry 7472de30-ed5c-42a0-b1ed-28299a43c80b/0x26896e0 for node dev-cluster2-node3/1 (3 total)
  216. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (membership.c:285 ) info: crm_get_peer: Node 1 is now known as dev-cluster2-node3
  217. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (membership.c:313 ) info: crm_get_peer: Node 1 has uuid 1
  218. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 0 from nodelist.node.0.nodeid
  219. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 1 from nodelist.node.1.nodeid
  220. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 2 from nodelist.node.2.nodeid
  221. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:104 ) trace: corosync_node_name: Searching for node name for 2 in nodelist.node.2 (null)
  222. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:108 ) trace: corosync_node_name: nodelist.node.2.ring0_addr = 10.76.157.19
  223. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:120 ) trace: corosync_node_name: nodelist.node.2.name = dev-cluster2-node4 1
  224. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:533 ) trace: corosync_initialize_nodelist: Initializing node[2] 2 = dev-cluster2-node4
  225. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (membership.c:177 ) debug: crm_dump_peer_hash: crm_get_peer: Node 0/dev-cluster2-node4 = 0x2688ce0 - 3552f6e0-73fa-4e15-ac91-572b2ba1ac6c
  226. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (membership.c:177 ) debug: crm_dump_peer_hash: crm_get_peer: Node 2/(null) = 0x2487730 - 3e401a9b-00be-40eb-9e60-77bf36a55a9c
  227. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (membership.c:177 ) debug: crm_dump_peer_hash: crm_get_peer: Node 1/dev-cluster2-node3 = 0x26896e0 - 7472de30-ed5c-42a0-b1ed-28299a43c80b
  228. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (membership.c:269 ) info: crm_get_peer: Merging 0x2688ce0 into 0x2487730
  229. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (membership.c:285 ) info: crm_get_peer: Node 2 is now known as dev-cluster2-node4
  230. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( ipcc.c:378 ) debug: qb_ipcc_disconnect: qb_ipcc_disconnect()
  231. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-22112-22133-22-header
  232. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-22112-22133-22-header
  233. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-22112-22133-22-header
  234. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:67 ) trace: corosync_node_name: Initializing CMAP connection
  235. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22133-22)
  236. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22133-22) state:2
  237. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  238. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  239. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CMAP ] exit_fn for conn=0x7f4de0fc4f80
  240. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  241. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-response-22112-22133-22-header
  242. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-event-22112-22133-22-header
  243. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-request-22112-22133-22-header
  244. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22133]
  245. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  246. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  247. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  248. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  249. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  250. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  251. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  252. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CMAP ] lib_init_fn: conn=0x7f4de0fc27c0
  253. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 0 from nodelist.node.0.nodeid
  254. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 1 from nodelist.node.1.nodeid
  255. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 2 from nodelist.node.2.nodeid
  256. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:104 ) trace: corosync_node_name: Searching for node name for 2 in nodelist.node.2 (null)
  257. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:108 ) trace: corosync_node_name: nodelist.node.2.ring0_addr = 10.76.157.19
  258. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:120 ) trace: corosync_node_name: nodelist.node.2.name = dev-cluster2-node4 1
  259. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( ipcc.c:378 ) debug: qb_ipcc_disconnect: qb_ipcc_disconnect()
  260. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-22112-22133-22-header
  261. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-22112-22133-22-header
  262. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-22112-22133-22-header
  263. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22133-22)
  264. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22133-22) state:2
  265. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  266. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  267. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CMAP ] exit_fn for conn=0x7f4de0fc27c0
  268. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  269. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-response-22112-22133-22-header
  270. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-event-22112-22133-22-header
  271. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-request-22112-22133-22-header
  272. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:270 ) info: start_child: Using uid=495 and group=496 for process cib
  273. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:281 ) info: start_child: Forked child 22135 for process cib
  274. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:589 ) debug: update_node_processes: Node dev-cluster2-node4 now has process list: 00000000000000000000000000000100 (was 00000000000000000000000004000000)
  275. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:281 ) info: start_child: Forked child 22136 for process stonith-ng
  276. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:589 ) debug: update_node_processes: Node dev-cluster2-node4 now has process list: 00000000000000000000000000100100 (was 00000000000000000000000000000100)
  277. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:281 ) info: start_child: Forked child 22137 for process lrmd
  278. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:589 ) debug: update_node_processes: Node dev-cluster2-node4 now has process list: 00000000000000000000000000100110 (was 00000000000000000000000000100100)
  279. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:270 ) info: start_child: Using uid=495 and group=496 for process attrd
  280. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:281 ) info: start_child: Forked child 22138 for process attrd
  281. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:589 ) debug: update_node_processes: Node dev-cluster2-node4 now has process list: 00000000000000000000000000101110 (was 00000000000000000000000000100110)
  282. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:270 ) info: start_child: Using uid=495 and group=496 for process pengine
  283. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:281 ) info: start_child: Forked child 22139 for process pengine
  284. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:589 ) debug: update_node_processes: Node dev-cluster2-node4 now has process list: 00000000000000000000000000111110 (was 00000000000000000000000000101110)
  285. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:270 ) info: start_child: Using uid=495 and group=496 for process crmd
  286. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:281 ) info: start_child: Forked child 22140 for process crmd
  287. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:589 ) debug: update_node_processes: Node dev-cluster2-node4 now has process list: 00000000000000000000000000111310 (was 00000000000000000000000000111110)
  288. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:1026 ) info: main: Starting mainloop
  289. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:191 ) info: pcmk_quorum_notification: Membership 28: quorum still lost (1)
  290. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:210 ) debug: pcmk_quorum_notification: Member[0] 2
  291. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: (membership.c:488 ) notice: crm_update_peer_state: pcmk_quorum_notification: Node dev-cluster2-node4[2] - state is now member (was (null))
  292. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:223 ) trace: pcmk_quorum_notification: Reaping unseen nodes...
  293. Sep 03 17:28:15 [22133] dev-cluster2-node4 pacemakerd: ( corosync.c:229 ) info: pcmk_quorum_notification: State of node dev-cluster2-node3[1] is still unknown
  294. Sep 03 17:28:15 [22137] dev-cluster2-node4 lrmd: ( logging.c:761 ) info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/root
  295. Sep 03 17:28:15 [22137] dev-cluster2-node4 lrmd: ( ipc_setup.c:372 ) info: qb_ipcs_us_publish: server name: lrmd
  296. Sep 03 17:28:15 [22137] dev-cluster2-node4 lrmd: ( main.c:313 ) info: main: Starting
  297. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( logging.c:761 ) info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster
  298. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( cluster.c:531 ) info: get_cluster_type: Verifying cluster type: 'corosync'
  299. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( cluster.c:568 ) info: get_cluster_type: Assuming an active 'corosync' cluster
  300. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( io.c:259 ) info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.xml (digest: /var/lib/pacemaker/cib/cib.xml.sig)
  301. Sep 03 17:28:15 [22139] dev-cluster2-node4 pengine: ( logging.c:761 ) info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster
  302. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( xml.c:2545 ) info: validate_with_relaxng: Creating RNG parser context
  303. Sep 03 17:28:15 [22139] dev-cluster2-node4 pengine: ( main.c:171 ) debug: main: Init server comms
  304. Sep 03 17:28:15 [22139] dev-cluster2-node4 pengine: ( ipc_setup.c:372 ) info: qb_ipcs_us_publish: server name: pengine
  305. Sep 03 17:28:15 [22139] dev-cluster2-node4 pengine: ( main.c:179 ) info: main: Starting pengine
  306. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: ( logging.c:761 ) info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/root
  307. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: ( cluster.c:531 ) info: get_cluster_type: Verifying cluster type: 'corosync'
  308. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: ( cluster.c:568 ) info: get_cluster_type: Assuming an active 'corosync' cluster
  309. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: ( cluster.c:179 ) notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync
  310. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( logging.c:761 ) info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster
  311. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( attrd.c:542 ) info: main: Starting up
  312. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( cluster.c:531 ) info: get_cluster_type: Verifying cluster type: 'corosync'
  313. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( cluster.c:568 ) info: get_cluster_type: Assuming an active 'corosync' cluster
  314. Sep 03 17:28:15 [22140] dev-cluster2-node4 crmd: ( logging.c:761 ) info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster
  315. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( cluster.c:179 ) notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync
  316. Sep 03 17:28:15 [22140] dev-cluster2-node4 crmd: ( main.c:97 ) notice: main: CRM Git Version: 4f672bc
  317. Sep 03 17:28:15 [22140] dev-cluster2-node4 crmd: ( main.c:134 ) debug: crmd_init: Starting crmd
  318. Sep 03 17:28:15 [22140] dev-cluster2-node4 crmd: ( fsa.c:191 ) debug: s_crmd_fsa: Processing I_STARTUP: [ state=S_STARTING cause=C_STARTUP origin=crmd_init ]
  319. Sep 03 17:28:15 [22140] dev-cluster2-node4 crmd: ( misc.c:47 ) info: do_log: FSA: Input I_STARTUP from crmd_init() received in state S_STARTING
  320. Sep 03 17:28:15 [22140] dev-cluster2-node4 crmd: ( control.c:447 ) debug: do_startup: Registering Signal Handlers
  321. Sep 03 17:28:15 [22140] dev-cluster2-node4 crmd: ( control.c:454 ) debug: do_startup: Creating CIB and LRM objects
  322. Sep 03 17:28:15 [22140] dev-cluster2-node4 crmd: ( cluster.c:531 ) info: get_cluster_type: Verifying cluster type: 'corosync'
  323. Sep 03 17:28:15 [22140] dev-cluster2-node4 crmd: ( cluster.c:568 ) info: get_cluster_type: Assuming an active 'corosync' cluster
  324. Sep 03 17:28:15 [22140] dev-cluster2-node4 crmd: ( ipc.c:729 ) info: crm_ipc_connect: Could not establish cib_shm connection: Connection refused (111)
  325. Sep 03 17:28:15 [22140] dev-cluster2-node4 crmd: (cib_native.c:229 ) debug: cib_native_signon_raw: Connection unsuccessful (0 (nil))
  326. Sep 03 17:28:15 [22140] dev-cluster2-node4 crmd: (cib_native.c:272 ) debug: cib_native_signon_raw: Connection to CIB failed: Transport endpoint is not connected
  327. Sep 03 17:28:15 [22140] dev-cluster2-node4 crmd: (cib_native.c:282 ) debug: cib_native_signoff: Signing out of the CIB Service
  328. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22136]
  329. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  330. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( io.c:579 ) debug: activateCibXml: Triggering CIB write for start op
  331. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( main.c:586 ) info: startCib: CIB Initialization completed successfully
  332. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( cluster.c:179 ) notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync
  333. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  334. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  335. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  336. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  337. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  338. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  339. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CPG ] lib_init_fn: conn=0x7f4de0fc27c0, cpd=0x7f4de10c5154
  340. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: ( cpg.c:110 ) debug: get_local_nodeid: Local nodeid is 2
  341. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: (membership.c:280 ) info: crm_get_peer: Created entry a6b91dcb-7092-4648-87fe-b4f27fb26626/0x15f3e60 for node (null)/2 (1 total)
  342. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: (membership.c:313 ) info: crm_get_peer: Node 2 has uuid 2
  343. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: (membership.c:431 ) info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[2] - corosync-cpg is now online
  344. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: ( corosync.c:345 ) info: init_cs_connection_once: Connection to 'corosync': established
  345. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: ( corosync.c:67 ) trace: corosync_node_name: Initializing CMAP connection
  346. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22138]
  347. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  348. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  349. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  350. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  351. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  352. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  353. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  354. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CPG ] lib_init_fn: conn=0x7f4de0fc2d20, cpd=0x7f4de0fc32b4
  355. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( cpg.c:110 ) debug: get_local_nodeid: Local nodeid is 2
  356. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22135]
  357. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  358. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  359. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  360. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  361. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CPG ] lib_init_fn: conn=0x7f4de0fc46e0, cpd=0x7f4de0fc4d74
  362. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  363. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22136]
  364. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  365. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  366. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  367. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  368. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  369. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  370. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  371. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  372. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  373. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CMAP ] lib_init_fn: conn=0x7f4de12c7860
  374. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: (membership.c:280 ) info: crm_get_peer: Created entry 80782aba-5eda-4f6c-a5a5-6165856e89cb/0x24b5520 for node (null)/2 (1 total)
  375. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: (membership.c:313 ) info: crm_get_peer: Node 2 has uuid 2
  376. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: (membership.c:431 ) info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[2] - corosync-cpg is now online
  377. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( corosync.c:345 ) info: init_cs_connection_once: Connection to 'corosync': established
  378. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( corosync.c:67 ) trace: corosync_node_name: Initializing CMAP connection
  379. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 0 from nodelist.node.0.nodeid
  380. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( cpg.c:110 ) debug: get_local_nodeid: Local nodeid is 2
  381. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 1 from nodelist.node.1.nodeid
  382. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CPG ] got procjoin message from cluster node 2 (r(0) ip(10.76.157.19) ) for pid 22136
  383. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CPG ] got procjoin message from cluster node 2 (r(0) ip(10.76.157.19) ) for pid 22138
  384. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CPG ] got procjoin message from cluster node 2 (r(0) ip(10.76.157.19) ) for pid 22135
  385. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: (membership.c:280 ) info: crm_get_peer: Created entry e6645385-ce36-4a66-a407-f76c88ad78a8/0x919510 for node (null)/2 (1 total)
  386. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22138]
  387. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: (membership.c:313 ) info: crm_get_peer: Node 2 has uuid 2
  388. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: (membership.c:431 ) info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[2] - corosync-cpg is now online
  389. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( corosync.c:345 ) info: init_cs_connection_once: Connection to 'corosync': established
  390. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( corosync.c:67 ) trace: corosync_node_name: Initializing CMAP connection
  391. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  392. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  393. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  394. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  395. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  396. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  397. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  398. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CMAP ] lib_init_fn: conn=0x7f4de13cc180
  399. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 2 from nodelist.node.2.nodeid
  400. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: ( corosync.c:104 ) trace: corosync_node_name: Searching for node name for 2 in nodelist.node.2 (null)
  401. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 0 from nodelist.node.0.nodeid
  402. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22135]
  403. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  404. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  405. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  406. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  407. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CMAP ] lib_init_fn: conn=0x7f4de14ce380
  408. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  409. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  410. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  411. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 1 from nodelist.node.1.nodeid
  412. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: ( corosync.c:108 ) trace: corosync_node_name: nodelist.node.2.ring0_addr = 10.76.157.19
  413. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 2 from nodelist.node.2.nodeid
  414. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( corosync.c:104 ) trace: corosync_node_name: Searching for node name for 2 in nodelist.node.2 (null)
  415. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 0 from nodelist.node.0.nodeid
  416. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: ( corosync.c:120 ) trace: corosync_node_name: nodelist.node.2.name = dev-cluster2-node4 1
  417. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: ( ipcc.c:378 ) debug: qb_ipcc_disconnect: qb_ipcc_disconnect()
  418. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-22112-22136-25-header
  419. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-22112-22136-25-header
  420. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-22112-22136-25-header
  421. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: (membership.c:285 ) info: crm_get_peer: Node 2 is now known as dev-cluster2-node4
  422. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: ( ipc.c:729 ) info: crm_ipc_connect: Could not establish cib_rw connection: Connection refused (111)
  423. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: (cib_native.c:229 ) debug: cib_native_signon_raw: Connection unsuccessful (0 (nil))
  424. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: (cib_native.c:272 ) debug: cib_native_signon_raw: Connection to CIB failed: Transport endpoint is not connected
  425. Sep 03 17:28:15 [22136] dev-cluster2-node4 stonith-ng: (cib_native.c:282 ) debug: cib_native_signoff: Signing out of the CIB Service
  426. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( corosync.c:108 ) trace: corosync_node_name: nodelist.node.2.ring0_addr = 10.76.157.19
  427. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22136-25)
  428. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22136-25) state:2
  429. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  430. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  431. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CMAP ] exit_fn for conn=0x7f4de12c7860
  432. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  433. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-response-22112-22136-25-header
  434. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-event-22112-22136-25-header
  435. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-request-22112-22136-25-header
  436. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 1 from nodelist.node.1.nodeid
  437. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( corosync.c:120 ) trace: corosync_node_name: nodelist.node.2.name = dev-cluster2-node4 1
  438. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( ipcc.c:378 ) debug: qb_ipcc_disconnect: qb_ipcc_disconnect()
  439. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-22112-22138-26-header
  440. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-22112-22138-26-header
  441. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-22112-22138-26-header
  442. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: (membership.c:285 ) info: crm_get_peer: Node 2 is now known as dev-cluster2-node4
  443. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( attrd.c:574 ) info: main: Cluster connection active
  444. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( ipc_setup.c:372 ) info: qb_ipcs_us_publish: server name: attrd
  445. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( attrd.c:580 ) info: main: Accepting attribute updates
  446. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( attrd.c:594 ) notice: main: Starting mainloop...
  447. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( cpg.c:378 ) info: pcmk_cpg_membership: Joined[0.0] attrd.2
  448. Sep 03 17:28:15 [22138] dev-cluster2-node4 attrd: ( cpg.c:384 ) info: pcmk_cpg_membership: Member[0.0] attrd.2
  449. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 2 from nodelist.node.2.nodeid
  450. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22138-26)
  451. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22138-26) state:2
  452. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  453. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  454. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CMAP ] exit_fn for conn=0x7f4de13cc180
  455. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  456. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-response-22112-22138-26-header
  457. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-event-22112-22138-26-header
  458. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( corosync.c:104 ) trace: corosync_node_name: Searching for node name for 2 in nodelist.node.2 (null)
  459. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-request-22112-22138-26-header
  460. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( corosync.c:108 ) trace: corosync_node_name: nodelist.node.2.ring0_addr = 10.76.157.19
  461. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( corosync.c:120 ) trace: corosync_node_name: nodelist.node.2.name = dev-cluster2-node4 1
  462. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( ipcc.c:378 ) debug: qb_ipcc_disconnect: qb_ipcc_disconnect()
  463. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-22112-22135-27-header
  464. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22135-27)
  465. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-22112-22135-27-header
  466. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-22112-22135-27-header
  467. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22135-27) state:2
  468. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: (membership.c:285 ) info: crm_get_peer: Node 2 is now known as dev-cluster2-node4
  469. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  470. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( ipc_setup.c:372 ) info: qb_ipcs_us_publish: server name: cib_ro
  471. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  472. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [CMAP ] exit_fn for conn=0x7f4de14ce380
  473. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( ipc_setup.c:372 ) info: qb_ipcs_us_publish: server name: cib_rw
  474. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  475. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( ipc_setup.c:372 ) info: qb_ipcs_us_publish: server name: cib_shm
  476. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( main.c:550 ) info: cib_init: Starting cib mainloop
  477. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-response-22112-22135-27-header
  478. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( cpg.c:378 ) info: pcmk_cpg_membership: Joined[0.0] cib.2
  479. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( cpg.c:384 ) info: pcmk_cpg_membership: Member[0.0] cib.2
  480. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-event-22112-22135-27-header
  481. Sep 03 17:28:15 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-request-22112-22135-27-header
  482. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( io.c:716 ) info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-9.raw
  483. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( io.c:725 ) debug: write_cib_contents: Writing CIB to disk
  484. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( io.c:750 ) info: write_cib_contents: Wrote version 0.9.0 of the CIB to disk (digest: ad2847553804a57e70bdcad4e33c861b)
  485. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( io.c:758 ) debug: write_cib_contents: Wrote digest ad2847553804a57e70bdcad4e33c861b to disk
  486. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( io.c:259 ) info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.6ATX8r (digest: /var/lib/pacemaker/cib/cib.HIYBXR)
  487. Sep 03 17:28:15 [22135] dev-cluster2-node4 cib: ( io.c:763 ) debug: write_cib_contents: Activating /var/lib/pacemaker/cib/cib.6ATX8r
  488. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipc.c:307 ) info: crm_client_new: Connecting 0x919ee0 for uid=495 gid=496 pid=22140 id=f0433b31-f491-401d-8e75-c10b0d527356
  489. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipc_setup.c:476 ) debug: handle_new_connection: IPC credentials authenticated (22135-22140-10)
  490. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipc_shm.c:294 ) debug: qb_ipcs_shm_connect: connecting to client [22140]
  491. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:524288; real_size:524288; rb->word_size:131072
  492. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:524288; real_size:524288; rb->word_size:131072
  493. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:524288; real_size:524288; rb->word_size:131072
  494. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:524288; real_size:524288; rb->word_size:131072
  495. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:524288; real_size:524288; rb->word_size:131072
  496. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:524288; real_size:524288; rb->word_size:131072
  497. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (cib_native.c:268 ) debug: cib_native_signon_raw: Connection to CIB successful
  498. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( callbacks.c:189 ) debug: cib_common_callback_worker: Setting cib_refresh_notify callbacks for crmd (f0433b31-f491-401d-8e75-c10b0d527356): on
  499. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( callbacks.c:189 ) debug: cib_common_callback_worker: Setting cib_diff_notify callbacks for crmd (f0433b31-f491-401d-8e75-c10b0d527356): on
  500. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( cib.c:215 ) info: do_cib_control: CIB connection established
  501. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( cluster.c:179 ) notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync
  502. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( callbacks.c:766 ) info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/crmd/2, version=0.9.0)
  503. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22140]
  504. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  505. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  506. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  507. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  508. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  509. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  510. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  511. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CPG ] lib_init_fn: conn=0x7f4de15cef80, cpd=0x7f4de15ceab4
  512. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( cpg.c:110 ) debug: get_local_nodeid: Local nodeid is 2
  513. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (membership.c:280 ) info: crm_get_peer: Created entry 94e64d0b-09d8-43af-8923-a8836be81c55/0x1b2a630 for node (null)/2 (1 total)
  514. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (membership.c:313 ) info: crm_get_peer: Node 2 has uuid 2
  515. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (membership.c:431 ) info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[2] - corosync-cpg is now online
  516. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:345 ) info: init_cs_connection_once: Connection to 'corosync': established
  517. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:67 ) trace: corosync_node_name: Initializing CMAP connection
  518. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22140]
  519. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  520. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  521. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  522. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  523. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  524. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  525. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  526. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CMAP ] lib_init_fn: conn=0x7f4de13c86e0
  527. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 0 from nodelist.node.0.nodeid
  528. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CPG ] got procjoin message from cluster node 2 (r(0) ip(10.76.157.19) ) for pid 22140
  529. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 1 from nodelist.node.1.nodeid
  530. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 2 from nodelist.node.2.nodeid
  531. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:104 ) trace: corosync_node_name: Searching for node name for 2 in nodelist.node.2 (null)
  532. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:108 ) trace: corosync_node_name: nodelist.node.2.ring0_addr = 10.76.157.19
  533. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:120 ) trace: corosync_node_name: nodelist.node.2.name = dev-cluster2-node4 1
  534. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( ipcc.c:378 ) debug: qb_ipcc_disconnect: qb_ipcc_disconnect()
  535. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-22112-22140-26-header
  536. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-22112-22140-26-header
  537. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-22112-22140-26-header
  538. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (membership.c:285 ) info: crm_get_peer: Node 2 is now known as dev-cluster2-node4
  539. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( callbacks.c:118 ) info: peer_update_callback: dev-cluster2-node4 is now (null)
  540. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:255 ) debug: cluster_connect_quorum: Configuring Pacemaker to obtain quorum from Corosync
  541. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22140-26)
  542. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22140-26) state:2
  543. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  544. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  545. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CMAP ] exit_fn for conn=0x7f4de13c86e0
  546. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  547. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-response-22112-22140-26-header
  548. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-event-22112-22140-26-header
  549. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-request-22112-22140-26-header
  550. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22140]
  551. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  552. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  553. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  554. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  555. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  556. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  557. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  558. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QUORUM] lib_init_fn: conn=0x7f4de13c86e0
  559. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QUORUM] got quorum_type request on 0x7f4de13c86e0
  560. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QUORUM] got quorate request on 0x7f4de13c86e0
  561. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:273 ) notice: cluster_connect_quorum: Quorum lost
  562. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QUORUM] got trackstart request on 0x7f4de13c86e0
  563. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QUORUM] sending initial status to 0x7f4de13c86e0
  564. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QUORUM] sending quorum notification to 0x7f4de13c86e0, length = 52
  565. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] connecting to client [22140]
  566. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  567. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  568. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] shm size:1048576; real_size:1048576; rb->word_size:262144
  569. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  570. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  571. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:1048576; real_size:1048576; rb->word_size:262144
  572. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:501 ) trace: corosync_initialize_nodelist: Initializing corosync nodelist
  573. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created
  574. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CMAP ] lib_init_fn: conn=0x7f4de13c9870
  575. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 0 from nodelist.node.0.nodeid
  576. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 1 from nodelist.node.1.nodeid
  577. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 2 from nodelist.node.2.nodeid
  578. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:104 ) trace: corosync_node_name: Searching for node name for 2 in nodelist.node.2 (null)
  579. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:108 ) trace: corosync_node_name: nodelist.node.2.ring0_addr = 10.76.157.19
  580. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:120 ) trace: corosync_node_name: nodelist.node.2.name = dev-cluster2-node4 1
  581. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:525 ) crit: corosync_initialize_nodelist: Nodes 2 and 0 share the same name 'dev-cluster2-node4': shutting down
  582. Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( xml.c:2637 ) info: crm_xml_cleanup: Cleaning up memory from libxml2
  583. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipcs.c:736 ) debug: qb_ipcs_dispatch_connection_request: HUP conn (22135-22140-10)
  584. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipcs.c:578 ) debug: qb_ipcs_disconnect: qb_ipcs_disconnect(22135-22140-10) state:2
  585. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipc.c:341 ) info: crm_client_destroy: Destroying 0 events
  586. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:177 ) error: pcmk_child_exit: Child process crmd (22140) exited: Network is down (100)
  587. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:182 ) warning: pcmk_child_exit: Pacemaker child process crmd no longer wishes to be respawned. Shutting ourselves down.
  588. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: (ringbuffer.c:291 ) debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_shm-response-22135-22140-10-header
  589. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:589 ) debug: update_node_processes: Node dev-cluster2-node4 now has process list: 00000000000000000000000000111110 (was 00000000000000000000000000111310)
  590. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:371 ) notice: pcmk_shutdown_worker: Shuting down Pacemaker
  591. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:408 ) debug: pcmk_shutdown_worker: crmd confirmed stopped
  592. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: (ringbuffer.c:291 ) debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_shm-event-22135-22140-10-header
  593. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:210 ) notice: stop_child: Stopping pengine: Sent -15 to process 22139
  594. Sep 03 17:28:16 [22139] dev-cluster2-node4 pengine: ( mainloop.c:270 ) info: crm_signal_dispatch: Invoking handler for signal 15: Terminated
  595. Sep 03 17:28:16 [22139] dev-cluster2-node4 pengine: ( ipc_setup.c:425 ) info: qb_ipcs_us_withdraw: withdrawing server sockets
  596. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: (ringbuffer.c:291 ) debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_shm-request-22135-22140-10-header
  597. Sep 03 17:28:16 [22139] dev-cluster2-node4 pengine: ( ipcs.c:221 ) debug: qb_ipcs_unref: qb_ipcs_unref() - destroying
  598. Sep 03 17:28:16 [22139] dev-cluster2-node4 pengine: ( xml.c:2637 ) info: crm_xml_cleanup: Cleaning up memory from libxml2
  599. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22140-27)
  600. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22140-27) state:2
  601. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  602. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  603. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CMAP ] exit_fn for conn=0x7f4de13c9870
  604. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  605. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-response-22112-22140-27-header
  606. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:177 ) info: pcmk_child_exit: Child process pengine (22139) exited: OK (0)
  607. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:589 ) debug: update_node_processes: Node dev-cluster2-node4 now has process list: 00000000000000000000000000101110 (was 00000000000000000000000000111110)
  608. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:408 ) debug: pcmk_shutdown_worker: pengine confirmed stopped
  609. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:210 ) notice: stop_child: Stopping attrd: Sent -15 to process 22138
  610. Sep 03 17:28:16 [22138] dev-cluster2-node4 attrd: ( mainloop.c:270 ) info: crm_signal_dispatch: Invoking handler for signal 15: Terminated
  611. Sep 03 17:28:16 [22138] dev-cluster2-node4 attrd: ( attrd.c:178 ) info: attrd_shutdown: Exiting
  612. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-event-22112-22140-27-header
  613. Sep 03 17:28:16 [22138] dev-cluster2-node4 attrd: ( attrd.c:596 ) notice: main: Exiting...
  614. Sep 03 17:28:16 [22138] dev-cluster2-node4 attrd: ( ipc_setup.c:425 ) info: qb_ipcs_us_withdraw: withdrawing server sockets
  615. Sep 03 17:28:16 [22138] dev-cluster2-node4 attrd: ( ipcs.c:221 ) debug: qb_ipcs_unref: qb_ipcs_unref() - destroying
  616. Sep 03 17:28:16 [22138] dev-cluster2-node4 attrd: ( xml.c:2637 ) info: crm_xml_cleanup: Cleaning up memory from libxml2
  617. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cmap-request-22112-22140-27-header
  618. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22140-25)
  619. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22140-25) state:2
  620. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  621. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  622. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CPG ] exit_fn for conn=0x7f4de15cef80
  623. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  624. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:177 ) info: pcmk_child_exit: Child process attrd (22138) exited: OK (0)
  625. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:589 ) debug: update_node_processes: Node dev-cluster2-node4 now has process list: 00000000000000000000000000100110 (was 00000000000000000000000000101110)
  626. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cpg-response-22112-22140-25-header
  627. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:408 ) debug: pcmk_shutdown_worker: attrd confirmed stopped
  628. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:210 ) notice: stop_child: Stopping lrmd: Sent -15 to process 22137
  629. Sep 03 17:28:16 [22137] dev-cluster2-node4 lrmd: ( mainloop.c:270 ) info: crm_signal_dispatch: Invoking handler for signal 15: Terminated
  630. Sep 03 17:28:16 [22137] dev-cluster2-node4 lrmd: ( main.c:221 ) info: lrmd_shutdown: Terminating with 0 clients
  631. Sep 03 17:28:16 [22137] dev-cluster2-node4 lrmd: ( ipc_setup.c:425 ) info: qb_ipcs_us_withdraw: withdrawing server sockets
  632. Sep 03 17:28:16 [22137] dev-cluster2-node4 lrmd: ( ipcs.c:221 ) debug: qb_ipcs_unref: qb_ipcs_unref() - destroying
  633. Sep 03 17:28:16 [22137] dev-cluster2-node4 lrmd: ( xml.c:2637 ) info: crm_xml_cleanup: Cleaning up memory from libxml2
  634. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:177 ) info: pcmk_child_exit: Child process lrmd (22137) exited: OK (0)
  635. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:589 ) debug: update_node_processes: Node dev-cluster2-node4 now has process list: 00000000000000000000000000100100 (was 00000000000000000000000000100110)
  636. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:408 ) debug: pcmk_shutdown_worker: lrmd confirmed stopped
  637. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cpg-event-22112-22140-25-header
  638. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cpg-request-22112-22140-25-header
  639. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22140-26)
  640. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22140-26) state:2
  641. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  642. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  643. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QUORUM] lib_exit_fn: conn=0x7f4de13c86e0
  644. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  645. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-quorum-response-22112-22140-26-header
  646. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipc.c:307 ) info: crm_client_new: Connecting 0x99b390 for uid=0 gid=0 pid=22136 id=8c45a6ba-c01e-41da-b691-a91b244e55b0
  647. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipc_setup.c:476 ) debug: handle_new_connection: IPC credentials authenticated (22135-22136-10)
  648. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipc_shm.c:294 ) debug: qb_ipcs_shm_connect: connecting to client [22136]
  649. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:210 ) notice: stop_child: Stopping stonith-ng: Sent -15 to process 22136
  650. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:524288; real_size:524288; rb->word_size:131072
  651. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:524288; real_size:524288; rb->word_size:131072
  652. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:524288; real_size:524288; rb->word_size:131072
  653. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:524288; real_size:524288; rb->word_size:131072
  654. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:524288; real_size:524288; rb->word_size:131072
  655. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: (ringbuffer.c:227 ) debug: qb_rb_open_2: shm size:524288; real_size:524288; rb->word_size:131072
  656. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: (cib_native.c:268 ) debug: cib_native_signon_raw: Connection to CIB successful
  657. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( callbacks.c:189 ) debug: cib_common_callback_worker: Setting cib_diff_notify callbacks for crmd (8c45a6ba-c01e-41da-b691-a91b244e55b0): on
  658. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-quorum-event-22112-22140-26-header
  659. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: ( main.c:921 ) notice: setup_cib: Watching for stonith topology changes
  660. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: ( ipc_setup.c:372 ) info: qb_ipcs_us_publish: server name: stonith-ng
  661. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: ( main.c:1151 ) info: main: Starting stonith-ng mainloop
  662. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: ( mainloop.c:270 ) info: crm_signal_dispatch: Invoking handler for signal 15: Terminated
  663. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: ( main.c:833 ) info: stonith_shutdown: Terminating with 0 clients
  664. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: (cib_native.c:282 ) debug: cib_native_signoff: Signing out of the CIB Service
  665. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: ( ipcc.c:378 ) debug: qb_ipcc_disconnect: qb_ipcc_disconnect()
  666. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-22135-22136-10-header
  667. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-22135-22136-10-header
  668. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-22135-22136-10-header
  669. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-quorum-request-22112-22140-26-header
  670. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: ( main.c:846 ) info: cib_connection_destroy: Connection to the CIB closed.
  671. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: ( ipc_setup.c:425 ) info: qb_ipcs_us_withdraw: withdrawing server sockets
  672. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: ( ipcs.c:221 ) debug: qb_ipcs_unref: qb_ipcs_unref() - destroying
  673. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: ( main.c:1162 ) info: main: Done
  674. Sep 03 17:28:16 [22136] dev-cluster2-node4 stonith-ng: ( xml.c:2637 ) info: crm_xml_cleanup: Cleaning up memory from libxml2
  675. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CPG ] got procleave message from cluster node 2
  676. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22138-23)
  677. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22138-23) state:2
  678. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipcs.c:736 ) debug: qb_ipcs_dispatch_connection_request: HUP conn (22135-22136-10)
  679. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipcs.c:578 ) debug: qb_ipcs_disconnect: qb_ipcs_disconnect(22135-22136-10) state:2
  680. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipc.c:341 ) info: crm_client_destroy: Destroying 0 events
  681. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:177 ) info: pcmk_child_exit: Child process stonith-ng (22136) exited: OK (0)
  682. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: (ringbuffer.c:291 ) debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-22135-22136-10-header
  683. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:589 ) debug: update_node_processes: Node dev-cluster2-node4 now has process list: 00000000000000000000000000000100 (was 00000000000000000000000000100100)
  684. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:408 ) debug: pcmk_shutdown_worker: stonith-ng confirmed stopped
  685. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:210 ) notice: stop_child: Stopping cib: Sent -15 to process 22135
  686. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: (ringbuffer.c:291 ) debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-22135-22136-10-header
  687. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  688. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: (ringbuffer.c:291 ) debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-22135-22136-10-header
  689. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  690. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CPG ] exit_fn for conn=0x7f4de0fc2d20
  691. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( mainloop.c:270 ) info: crm_signal_dispatch: Invoking handler for signal 15: Terminated
  692. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( callbacks.c:1351 ) debug: cib_shutdown: Disconnecting 0 remote clients
  693. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  694. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( callbacks.c:1353 ) info: cib_shutdown: Disconnected 0 clients
  695. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( callbacks.c:1359 ) info: cib_shutdown: All clients disconnected (0)
  696. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( corosync.c:465 ) trace: crm_is_corosync_peer_active: dev-cluster2-node4: state=(null)
  697. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( callbacks.c:1408 ) info: terminate_cib: initiate_exit: Disconnecting from cluster infrastructure
  698. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( cluster.c:239 ) info: crm_cluster_disconnect: Disconnecting from cluster infrastructure: corosync
  699. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cpg-response-22112-22138-23-header
  700. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( corosync.c:142 ) notice: terminate_cs_connection: Disconnecting from Corosync
  701. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cpg-event-22112-22138-23-header
  702. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cpg-request-22112-22138-23-header
  703. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22136-22)
  704. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22136-22) state:2
  705. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  706. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  707. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CPG ] exit_fn for conn=0x7f4de0fc27c0
  708. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  709. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cpg-response-22112-22136-22-header
  710. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cpg-event-22112-22136-22-header
  711. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cpg-request-22112-22136-22-header
  712. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CPG ] got leave request on 0x7f4de0fc46e0
  713. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CPG ] got procleave message from cluster node 2
  714. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CPG ] got procleave message from cluster node 2
  715. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CPG ] got procleave message from cluster node 2
  716. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CPG ] cpg finalize for conn=0x7f4de0fc46e0
  717. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipcc.c:378 ) debug: qb_ipcc_disconnect: qb_ipcc_disconnect()
  718. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-22112-22135-24-header
  719. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22135-24)
  720. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-22112-22135-24-header
  721. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: (ringbuffer.c:294 ) debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-22112-22135-24-header
  722. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22135-24) state:2
  723. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( corosync.c:152 ) info: terminate_cs_connection: No Quorum connection
  724. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( cluster.c:244 ) info: crm_cluster_disconnect: Disconnected from corosync
  725. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( io.c:498 ) debug: uninitializeCib: Deallocating the CIB.
  726. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( io.c:502 ) debug: uninitializeCib: The CIB has been deallocated.
  727. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  728. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( callbacks.c:1414 ) info: terminate_cib: initiate_exit: Exiting from mainloop...
  729. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipc_setup.c:425 ) info: qb_ipcs_us_withdraw: withdrawing server sockets
  730. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipcs.c:221 ) debug: qb_ipcs_unref: qb_ipcs_unref() - destroying
  731. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipc_setup.c:425 ) info: qb_ipcs_us_withdraw: withdrawing server sockets
  732. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  733. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipcs.c:221 ) debug: qb_ipcs_unref: qb_ipcs_unref() - destroying
  734. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipc_setup.c:425 ) info: qb_ipcs_us_withdraw: withdrawing server sockets
  735. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipcs.c:221 ) debug: qb_ipcs_unref: qb_ipcs_unref() - destroying
  736. Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( xml.c:2637 ) info: crm_xml_cleanup: Cleaning up memory from libxml2
  737. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CPG ] exit_fn for conn=0x7f4de0fc46e0
  738. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  739. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cpg-response-22112-22135-24-header
  740. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cpg-event-22112-22135-24-header
  741. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cpg-request-22112-22135-24-header
  742. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:177 ) info: pcmk_child_exit: Child process cib (22135) exited: OK (0)
  743. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:408 ) debug: pcmk_shutdown_worker: cib confirmed stopped
  744. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:414 ) notice: pcmk_shutdown_worker: Shutdown complete
  745. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( pacemaker.c:418 ) notice: pcmk_shutdown_worker: Attempting to inhibit respawning after fatal error
  746. Sep 03 17:28:16 [22133] dev-cluster2-node4 pacemakerd: ( xml.c:2637 ) info: crm_xml_cleanup: Cleaning up memory from libxml2
  747. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22133-19)
  748. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22133-19) state:2
  749. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  750. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  751. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  752. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cfg-response-22112-22133-19-header
  753. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cfg-event-22112-22133-19-header
  754. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cfg-request-22112-22133-19-header
  755. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22133-20)
  756. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22133-20) state:2
  757. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  758. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  759. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CPG ] exit_fn for conn=0x7f4de0dbf670
  760. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  761. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cpg-response-22112-22133-20-header
  762. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cpg-event-22112-22133-20-header
  763. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-cpg-request-22112-22133-20-header
  764. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] HUP conn (22112-22133-21)
  765. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] qb_ipcs_disconnect(22112-22133-21) state:2
  766. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] epoll_ctl(del): Bad file descriptor (9)
  767. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_closed()
  768. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QUORUM] lib_exit_fn: conn=0x7f4de0ec10f0
  769. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] cs_ipcs_connection_destroyed()
  770. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-quorum-response-22112-22133-21-header
  771. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-quorum-event-22112-22133-21-header
  772. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [QB ] Free'ing ringbuffer: /dev/shm/qb-quorum-request-22112-22133-21-header
  773. Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CPG ] got procleave message from cluster node 2
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!