API tools faq
paste
Advertisement
amartin

corosync.log from storage0

amartin
Oct 31st, 2012
554
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 14.91 KB | None | 0 0
raw download clone embed print report
  1. Oct 31 22:42:42 [1815] storage0 lrmd: info: upstart_job_running: nmbd is running
  2. Oct 31 22:43:02 [1726] storage0 corosync notice [TOTEM ] A processor failed, forming new configuration.
  3. Oct 31 22:43:03 [1726] storage0 corosync notice [QUORUM] Members[2]: 402732042 -1811860470
  4. Oct 31 22:43:03 [2882] storage0 crmd: info: pcmk_quorum_notification: Membership 1841448: quorum retained (2)
  5. Oct 31 22:43:03 [2882] storage0 crmd: notice: corosync_mark_unseen_peer_dead: Node -1795083254/storage1 was not seen in the previous transition
  6. Oct 31 22:43:03 [2882] storage0 crmd: notice: crm_update_peer_state: corosync_mark_unseen_peer_dead: Node storage1[2499884042] - state is now lost
  7. Oct 31 22:43:03 [1726] storage0 corosync notice [TOTEM ] A processor joined or left the membership and a new membership (10.52.1.24:1841448) was formed.
  8. Oct 31 22:43:03 [2882] storage0 crmd: info: peer_update_callback: storage1 is now lost (was member)
  9. Oct 31 22:43:03 [2882] storage0 crmd: info: send_ais_text: Peer overloaded or membership in flux: Re-sending message (Attempt 1 of 20)
  10. Oct 31 22:43:03 [1726] storage0 corosync notice [MAIN ] Completed service synchronization, ready to provide service.
  11. Oct 31 22:43:03 [1814] storage0 stonith-ng: info: pcmk_cpg_membership: Left[3.0] stonith-ng.-1795083254
  12. Oct 31 22:43:03 [1814] storage0 stonith-ng: info: crm_update_peer_proc: pcmk_cpg_membership: Node storage1[-1795083254] - corosync-cpg is now offline
  13. Oct 31 22:43:03 [1812] storage0 cib: info: pcmk_cpg_membership: Left[1.0] cib.-1795083254
  14. Oct 31 22:43:03 [1812] storage0 cib: info: crm_update_peer_proc: pcmk_cpg_membership: Node storage1[-1795083254] - corosync-cpg is now offline
  15. Oct 31 22:43:03 [1812] storage0 cib: info: pcmk_cpg_membership: Member[1.0] cib.402732042
  16. Oct 31 22:43:03 [1812] storage0 cib: info: pcmk_cpg_membership: Member[1.1] cib.-1811860470
  17. Oct 31 22:43:03 [1814] storage0 stonith-ng: info: pcmk_cpg_membership: Member[3.0] stonith-ng.402732042
  18. Oct 31 22:43:03 [1814] storage0 stonith-ng: info: pcmk_cpg_membership: Member[3.1] stonith-ng.-1811860470
  19. Oct 31 22:43:04 [2882] storage0 crmd: info: pcmk_cpg_membership: Left[1.0] crmd.-1795083254
  20. Oct 31 22:43:04 [2882] storage0 crmd: info: crm_update_peer_proc: pcmk_cpg_membership: Node storage1[-1795083254] - corosync-cpg is now offline
  21. Oct 31 22:43:04 [2882] storage0 crmd: info: peer_update_callback: Client storage1/peer now has status [offline] (DC=storagequorum)
  22. Oct 31 22:43:04 [2882] storage0 crmd: info: pcmk_cpg_membership: Member[1.0] crmd.402732042
  23. Oct 31 22:43:04 [2882] storage0 crmd: info: pcmk_cpg_membership: Member[1.1] crmd.-1811860470
  24. Oct 31 22:43:05 [1814] storage0 stonith-ng: error: pcmk_cpg_dispatch: Connection to the CPG API failed: 2
  25. Oct 31 22:43:05 [1814] storage0 stonith-ng: error: stonith_peer_ais_destroy: AIS connection terminated
  26. Oct 31 22:43:05 [1814] storage0 stonith-ng: info: stonith_shutdown: Terminating with 2 clients
  27. Oct 31 22:43:05 [1812] storage0 cib: error: pcmk_cpg_dispatch: Connection to the CPG API failed: 2
  28. Oct 31 22:43:05 [1812] storage0 cib: error: cib_ais_destroy: Corosync connection lost! Exiting.
  29. Oct 31 22:43:05 [1814] storage0 stonith-ng: info: qb_ipcs_us_withdraw: withdrawing server sockets
  30. Oct 31 22:43:05 [2882] storage0 crmd: error: crm_ipc_read: Connection to stonith-ng failed
  31. Oct 31 22:43:05 [2882] storage0 crmd: error: mainloop_gio_callback: Connection to stonith-ng[0x1d21c80] closed (I/O condition=17)
  32. Oct 31 22:43:05 [1815] storage0 lrmd: error: crm_ipc_read: Connection to stonith-ng failed
  33. Oct 31 22:43:05 [2882] storage0 crmd: crit: tengine_stonith_connection_destroy: Fencing daemon connection failed
  34. Oct 31 22:43:05 [1812] storage0 cib: info: terminate_cib: cib_ais_destroy: Exiting fast...
  35. Oct 31 22:43:05 [1812] storage0 cib: info: qb_ipcs_us_withdraw: withdrawing server sockets
  36. Oct 31 22:43:05 [1816] storage0 attrd: error: pcmk_cpg_dispatch: Connection to the CPG API failed: 2
  37. Oct 31 22:43:05 [1816] storage0 attrd: crit: attrd_ais_destroy: Lost connection to Corosync service!
  38. Oct 31 22:43:05 [1816] storage0 attrd: notice: main: Exiting...
  39. Oct 31 22:43:05 [1816] storage0 attrd: notice: main: Disconnecting client 0x20ee410, pid=2882...
  40. Oct 31 22:43:05 [1816] storage0 attrd: error: attrd_cib_connection_destroy: Connection to the CIB terminated...
  41. Oct 31 22:43:05 [1812] storage0 cib: info: qb_ipcs_us_withdraw: withdrawing server sockets
  42. Oct 31 22:43:05 [1812] storage0 cib: info: qb_ipcs_us_withdraw: withdrawing server sockets
  43. Oct 31 22:43:05 [1814] storage0 stonith-ng: info: crm_xml_cleanup: Cleaning up memory from libxml2
  44. Oct 31 22:43:05 [1814] storage0 stonith-ng: info: main: Done
  45. Oct 31 22:43:05 [1810] storage0 pacemakerd: error: cfg_connection_destroy: Connection destroyed
  46. Oct 31 22:43:05 [1810] storage0 pacemakerd: error: cpg_connection_destroy: Connection destroyed
  47. Oct 31 22:43:05 [1810] storage0 pacemakerd: notice: pcmk_shutdown_worker: Shuting down Pacemaker
  48. Oct 31 22:43:05 [1810] storage0 pacemakerd: notice: stop_child: Stopping crmd: Sent -15 to process 2882
  49. Oct 31 22:43:05 [1810] storage0 pacemakerd: error: pcmk_child_exit: Child process attrd exited (pid=1816, rc=1)
  50. Oct 31 22:43:05 [1810] storage0 pacemakerd: error: send_cpg_message: Sending message via cpg FAILED: (rc=9) Bad handle
  51. Oct 31 22:43:05 [2882] storage0 crmd: info: crm_ipc_connect: Could not establish stonith-ng connection: Connection refused (111)
  52. Oct 31 22:43:05 [2882] storage0 crmd: error: te_connect_stonith: Sign-in failed: triggered a retry
  53. Oct 31 22:43:05 [2882] storage0 crmd: info: crm_signal_dispatch: Invoking handler for signal 15: Terminated
  54. Oct 31 22:43:05 [2882] storage0 crmd: notice: crm_shutdown: Requesting shutdown, upper limit is 1200000ms
  55. Oct 31 22:43:05 [2882] storage0 crmd: info: do_shutdown_req: Sending shutdown request to storagequorum
  56. Oct 31 22:43:05 [2882] storage0 crmd: error: send_ais_text: Sending message 26 via cpg: FAILED (rc=2): Library error: Transport endpoint is not connected (107)
  57. Oct 31 22:43:05 [2882] storage0 crmd: error: do_log: FSA: Input I_ERROR from do_shutdown_req() received in state S_NOT_DC
  58. Oct 31 22:43:05 [2882] storage0 crmd: notice: do_state_transition: State transition S_NOT_DC -> S_RECOVERY [ input=I_ERROR cause=C_FSA_INTERNAL origin=do_shutdown_req ]
  59. Oct 31 22:43:05 [2882] storage0 crmd: error: do_recover: Action A_RECOVER (0000000001000000) not supported
  60. Oct 31 22:43:05 [2882] storage0 crmd: error: do_log: FSA: Input I_TERMINATE from do_recover() received in state S_RECOVERY
  61. Oct 31 22:43:05 [2882] storage0 crmd: info: do_state_transition: State transition S_RECOVERY -> S_TERMINATE [ input=I_TERMINATE cause=C_FSA_INTERNAL origin=do_recover ]
  62. Oct 31 22:43:05 [2882] storage0 crmd: info: do_shutdown: Disconnecting STONITH...
  63. Oct 31 22:43:05 [1810] storage0 pacemakerd: info: pcmk_child_exit: Child process stonith-ng exited (pid=1814, rc=0)
  64. Oct 31 22:43:05 [1810] storage0 pacemakerd: error: send_cpg_message: Sending message via cpg FAILED: (rc=9) Bad handle
  65. Oct 31 22:43:05 [1810] storage0 pacemakerd: error: pcmk_child_exit: Child process cib exited (pid=1812, rc=64)
  66. Oct 31 22:43:05 [1810] storage0 pacemakerd: error: send_cpg_message: Sending message via cpg FAILED: (rc=9) Bad handle
  67. Oct 31 22:43:05 [1815] storage0 lrmd: error: mainloop_gio_callback: Connection to stonith-ng[0x232bec0] closed (I/O condition=17)
  68. Oct 31 22:43:06 [1815] storage0 lrmd: error: stonith_connection_destroy_cb: LRMD lost STONITH connection
  69. Oct 31 22:43:06 [1815] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_daemon_smbd_monitor_30000
  70. Oct 31 22:43:06 [1815] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_fs_bind_opt_monitor_20000
  71. Oct 31 22:43:06 [1815] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_ip_drives0_monitor_30000
  72. Oct 31 22:43:06 [1815] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_daemon_nmbd_monitor_30000
  73. Oct 31 22:43:06 [1815] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_ip_drives3_monitor_30000
  74. Oct 31 22:43:06 [1815] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_ip_drives2_monitor_30000
  75. Oct 31 22:43:06 [1815] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_sysadmin_notify_monitor_10000
  76. Oct 31 22:43:06 [1815] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_drbd_drives_monitor_10000
  77. Oct 31 22:43:06 [1815] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_exportfs_storage_monitor_30000
  78. Oct 31 22:43:06 [1815] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_ping_monitor_10000
  79. Oct 31 22:43:06 [1815] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_daemon_nfs-kernel-server_status_30000
  80. Oct 31 22:43:06 [1815] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_fs_drives_monitor_20000
  81. Oct 31 22:43:06 [1815] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_ip_drives1_monitor_30000
  82. Oct 31 22:43:06 [1815] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_ip_storage_monitor_30000
  83. Oct 31 22:43:06 [2882] storage0 crmd: error: verify_stopped: Resource p_ip_drives0 was active at shutdown. You may ignore this error if it is unmanaged.
  84. Oct 31 22:43:06 [2882] storage0 crmd: error: verify_stopped: Resource p_ip_drives1 was active at shutdown. You may ignore this error if it is unmanaged.
  85. Oct 31 22:43:06 [2882] storage0 crmd: error: verify_stopped: Resource p_ip_drives2 was active at shutdown. You may ignore this error if it is unmanaged.
  86. Oct 31 22:43:06 [2882] storage0 crmd: error: verify_stopped: Resource p_ip_drives3 was active at shutdown. You may ignore this error if it is unmanaged.
  87. Oct 31 22:43:06 [2882] storage0 crmd: error: verify_stopped: Resource p_daemon_nmbd was active at shutdown. You may ignore this error if it is unmanaged.
  88. Oct 31 22:43:06 [2882] storage0 crmd: error: verify_stopped: Resource p_daemon_smbd was active at shutdown. You may ignore this error if it is unmanaged.
  89. Oct 31 22:43:06 [2882] storage0 crmd: error: verify_stopped: Resource p_ip_storage was active at shutdown. You may ignore this error if it is unmanaged.
  90. Oct 31 22:43:06 [2882] storage0 crmd: error: verify_stopped: Resource p_exportfs_storage was active at shutdown. You may ignore this error if it is unmanaged.
  91. Oct 31 22:43:06 [2882] storage0 crmd: info: lrmd_api_disconnect: Disconnecting from lrmd service
  92. Oct 31 22:43:06 [1815] storage0 lrmd: info: lrmd_ipc_destroy: LRMD client disconnecting 0x2322150 - name: crmd id: a29e2c72-a7f6-4b42-9467-14927998d3ea
  93. Oct 31 22:43:06 [2882] storage0 crmd: info: lrmd_connection_destroy: connection destroyed
  94. Oct 31 22:43:06 [2882] storage0 crmd: info: lrm_connection_destroy: LRM Connection disconnected
  95. Oct 31 22:43:06 [2882] storage0 crmd: info: do_lrm_control: Disconnected from the LRM
  96. Oct 31 22:43:06 [2882] storage0 crmd: info: crm_cluster_disconnect: Disconnecting from cluster infrastructure: corosync
  97. Oct 31 22:43:06 [2882] storage0 crmd: notice: terminate_cs_connection: Disconnecting from Corosync
  98. Oct 31 22:43:06 [2882] storage0 crmd: info: crm_cluster_disconnect: Disconnected from corosync
  99. Oct 31 22:43:06 [2882] storage0 crmd: info: do_ha_control: Disconnected from the cluster
  100. Oct 31 22:43:06 [2882] storage0 crmd: info: do_cib_control: Disconnecting CIB
  101. Oct 31 22:43:06 [2882] storage0 crmd: notice: crm_ipc_send: Connection to cib_shm closed
  102. Oct 31 22:43:06 [2882] storage0 crmd: notice: crm_ipc_send: Connection to cib_shm closed
  103. Oct 31 22:43:06 [2882] storage0 crmd: error: cib_native_perform_op_delegate: Couldn't perform cib_slave operation (timeout=120s): -107: Transport endpoint is not connected (107)
  104. Oct 31 22:43:06 [2882] storage0 crmd: error: cib_native_perform_op_delegate: CIB disconnected
  105. Oct 31 22:43:06 [2882] storage0 crmd: info: crmd_cib_connection_destroy: Connection to the CIB terminated...
  106. Oct 31 22:43:06 [2882] storage0 crmd: info: qb_ipcs_us_withdraw: withdrawing server sockets
  107. Oct 31 22:43:06 [2882] storage0 crmd: info: do_exit: Performing A_EXIT_0 - gracefully exiting the CRMd
  108. Oct 31 22:43:06 [2882] storage0 crmd: error: do_exit: Could not recover from internal error
  109. Oct 31 22:43:06 [2882] storage0 crmd: info: do_exit: [crmd] stopped (2)
  110. Oct 31 22:43:06 [2882] storage0 crmd: info: free_mem: Dropping I_TERMINATE: [ state=S_TERMINATE cause=C_FSA_INTERNAL origin=do_stop ]
  111. Oct 31 22:43:06 [2882] storage0 crmd: info: lrmd_api_disconnect: Disconnecting from lrmd service
  112. Oct 31 22:43:06 [2882] storage0 crmd: info: crm_xml_cleanup: Cleaning up memory from libxml2
  113. Oct 31 22:43:06 [1810] storage0 pacemakerd: error: pcmk_child_exit: Child process crmd exited (pid=2882, rc=2)
  114. Oct 31 22:43:06 [1810] storage0 pacemakerd: error: send_cpg_message: Sending message via cpg FAILED: (rc=9) Bad handle
  115. Oct 31 22:43:06 [1810] storage0 pacemakerd: notice: stop_child: Stopping pengine: Sent -15 to process 1817
  116. Oct 31 22:43:06 [1810] storage0 pacemakerd: info: pcmk_child_exit: Child process pengine exited (pid=1817, rc=0)
  117. Oct 31 22:43:06 [1810] storage0 pacemakerd: error: send_cpg_message: Sending message via cpg FAILED: (rc=9) Bad handle
  118. Oct 31 22:43:06 [1810] storage0 pacemakerd: notice: stop_child: Stopping lrmd: Sent -15 to process 1815
  119. Oct 31 22:43:06 [1815] storage0 lrmd: info: crm_signal_dispatch: Invoking handler for signal 15: Terminated
  120. Oct 31 22:43:06 [1815] storage0 lrmd: info: lrmd_shutdown: Terminating with 0 clients
  121. Oct 31 22:43:06 [1815] storage0 lrmd: info: qb_ipcs_us_withdraw: withdrawing server sockets
  122. Oct 31 22:43:06 [1810] storage0 pacemakerd: info: pcmk_child_exit: Child process lrmd exited (pid=1815, rc=0)
  123. Oct 31 22:43:06 [1810] storage0 pacemakerd: error: send_cpg_message: Sending message via cpg FAILED: (rc=9) Bad handle
  124. Oct 31 22:43:06 [1810] storage0 pacemakerd: notice: pcmk_shutdown_worker: Shutdown complete
  125. Oct 31 22:43:06 [1810] storage0 pacemakerd: info: qb_ipcs_us_withdraw: withdrawing server sockets
  126. Oct 31 22:43:06 [1810] storage0 pacemakerd: info: main: Exiting pacemakerd
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!