API tools faq
paste
Advertisement
amartin

storage0 corosync log during migration

amartin
Dec 4th, 2012
213
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 66.59 KB | None | 0 0
raw download clone embed print report
  1. exportfs[10024]: 2012/12/03_08:29:52 INFO: Directory /mnt/storage is exported to 10.0.0.0/255.0.0.0 (started).
  2. exportfs[10024]: 2012/12/03_08:29:52 INFO: Directory /mnt/storage is exported to 10.0.0.0/255.0.0.0 (started).
  3. Dec 03 08:30:22 [8479] storage0 lrmd: info: get_first_instance: Result: /com/ubuntu/Upstart/jobs/smbd/_
  4. Dec 03 08:30:22 [8479] storage0 lrmd: info: upstart_job_property: Calling GetAll on /com/ubuntu/Upstart/jobs/smbd/_
  5. Dec 03 08:30:22 [8479] storage0 lrmd: info: upstart_job_property: Call to GetAll passed: type '(a{sv})' 1
  6. Dec 03 08:30:22 [8479] storage0 lrmd: info: upstart_job_property: Got value 'running' for /com/ubuntu/Upstart/jobs/smbd/_[state]
  7. Dec 03 08:30:22 [8479] storage0 lrmd: info: upstart_job_running: State of smbd: running
  8. Dec 03 08:30:22 [8479] storage0 lrmd: info: upstart_job_running: smbd is running
  9. Dec 03 08:30:22 [8479] storage0 lrmd: info: get_first_instance: Result: /com/ubuntu/Upstart/jobs/nmbd/_
  10. Dec 03 08:30:22 [8479] storage0 lrmd: info: upstart_job_property: Calling GetAll on /com/ubuntu/Upstart/jobs/nmbd/_
  11. Dec 03 08:30:22 [8479] storage0 lrmd: info: upstart_job_property: Call to GetAll passed: type '(a{sv})' 1
  12. Dec 03 08:30:22 [8479] storage0 lrmd: info: upstart_job_property: Got value 'running' for /com/ubuntu/Upstart/jobs/nmbd/_[state]
  13. Dec 03 08:30:22 [8479] storage0 lrmd: info: upstart_job_running: State of nmbd: running
  14. Dec 03 08:30:22 [8479] storage0 lrmd: info: upstart_job_running: nmbd is running
  15. exportfs[10446]: 2012/12/03_08:30:22 INFO: Directory /mnt/storage is exported to 10.0.0.0/255.0.0.0 (started).
  16. exportfs[10446]: 2012/12/03_08:30:22 INFO: Directory /mnt/storage is exported to 10.0.0.0/255.0.0.0 (started).
  17. Dec 03 08:30:52 [8479] storage0 lrmd: info: get_first_instance: Result: /com/ubuntu/Upstart/jobs/smbd/_
  18. Dec 03 08:30:52 [8479] storage0 lrmd: info: upstart_job_property: Calling GetAll on /com/ubuntu/Upstart/jobs/smbd/_
  19. Dec 03 08:30:52 [8479] storage0 lrmd: info: upstart_job_property: Call to GetAll passed: type '(a{sv})' 1
  20. Dec 03 08:30:52 [8479] storage0 lrmd: info: upstart_job_property: Got value 'running' for /com/ubuntu/Upstart/jobs/smbd/_[state]
  21. Dec 03 08:30:52 [8479] storage0 lrmd: info: upstart_job_running: State of smbd: running
  22. Dec 03 08:30:52 [8479] storage0 lrmd: info: upstart_job_running: smbd is running
  23. Dec 03 08:30:52 [8479] storage0 lrmd: info: get_first_instance: Result: /com/ubuntu/Upstart/jobs/nmbd/_
  24. Dec 03 08:30:52 [8479] storage0 lrmd: info: upstart_job_property: Calling GetAll on /com/ubuntu/Upstart/jobs/nmbd/_
  25. Dec 03 08:30:52 [8479] storage0 lrmd: info: upstart_job_property: Call to GetAll passed: type '(a{sv})' 1
  26. Dec 03 08:30:52 [8479] storage0 lrmd: info: upstart_job_property: Got value 'running' for /com/ubuntu/Upstart/jobs/nmbd/_[state]
  27. Dec 03 08:30:52 [8479] storage0 lrmd: info: upstart_job_running: State of nmbd: running
  28. Dec 03 08:30:52 [8479] storage0 lrmd: info: upstart_job_running: nmbd is running
  29. exportfs[10867]: 2012/12/03_08:30:52 INFO: Directory /mnt/storage is exported to 10.0.0.0/255.0.0.0 (started).
  30. exportfs[10867]: 2012/12/03_08:30:52 INFO: Directory /mnt/storage is exported to 10.0.0.0/255.0.0.0 (started).
  31. Dec 03 08:31:12 [8482] storage0 crmd: info: crm_timer_popped: PEngine Recheck Timer (I_PE_CALC) just popped (900000ms)
  32. Dec 03 08:31:12 [8482] storage0 crmd: notice: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED origin=crm_timer_popped ]
  33. Dec 03 08:31:12 [8482] storage0 crmd: info: do_state_transition: Progressed to state S_POLICY_ENGINE after C_TIMER_POPPED
  34. Dec 03 08:31:12 [8481] storage0 pengine: warning: unpack_rsc_op: Processing failed op start for p_sysadmin_notify:0 on storage1: unknown error (1)
  35. Dec 03 08:31:12 [8481] storage0 pengine: notice: unpack_rsc_op: Preventing p_daemon_smbd from re-starting on storagequorum: operation monitor failed 'not installed' (rc=5)
  36. Dec 03 08:31:12 [8481] storage0 pengine: notice: unpack_rsc_op: Preventing p_daemon_nmbd from re-starting on storagequorum: operation monitor failed 'not installed' (rc=5)
  37. Dec 03 08:31:12 [8481] storage0 pengine: notice: unpack_rsc_op: Operation monitor found resource p_drbd_drives:1 active in master mode on storage0
  38. Dec 03 08:31:12 [8481] storage0 pengine: warning: common_apply_stickiness: Forcing cl_sysadmin_notify away from storage1 after 1000000 failures (max=1000000)
  39. Dec 03 08:31:12 [8481] storage0 pengine: warning: common_apply_stickiness: Forcing cl_sysadmin_notify away from storage1 after 1000000 failures (max=1000000)
  40. Dec 03 08:31:12 [8481] storage0 pengine: warning: common_apply_stickiness: Forcing cl_sysadmin_notify away from storage1 after 1000000 failures (max=1000000)
  41. ###################################################################################################
  42. ## pengine process dies here
  43. ###################################################################################################
  44. Dec 03 08:31:16 [8474] storage0 pacemakerd: notice: pcmk_child_exit: Child process pengine terminated with signal 6 (pid=8481, core=128)
  45. Dec 03 08:31:16 [8474] storage0 pacemakerd: notice: pcmk_child_exit: Respawning failed child process: pengine
  46. Dec 03 08:31:16 [8474] storage0 pacemakerd: info: start_child: Forked child 11239 for process pengine
  47. Dec 03 08:31:16 [8482] storage0 crmd: error: crm_ipc_read: Connection to pengine failed
  48. Dec 03 08:31:16 [8482] storage0 crmd: error: mainloop_gio_callback: Connection to pengine[0x131bcd0] closed (I/O condition=25)
  49. Dec 03 08:31:16 [8482] storage0 crmd: crit: pe_ipc_destroy: Connection to the Policy Engine failed (pid=-1, uuid=740275ec-a8a4-4071-b5b2-3f98523502a6)
  50. Dec 03 08:31:17 [8482] storage0 crmd: notice: save_cib_contents: Saved CIB contents after PE crash to /var/lib/pacemaker/pengine/pe-core-740275ec-a8a4-4071-b5b2-3f98523502a6.bz2
  51. Dec 03 08:31:17 [8482] storage0 crmd: error: do_log: FSA: Input I_ERROR from save_cib_contents() received in state S_POLICY_ENGINE
  52. Dec 03 08:31:17 [8482] storage0 crmd: warning: do_state_transition: State transition S_POLICY_ENGINE -> S_RECOVERY [ input=I_ERROR cause=C_FSA_INTERNAL origin=save_cib_contents ]
  53. Dec 03 08:31:17 [8482] storage0 crmd: error: do_recover: Action A_RECOVER (0000000001000000) not supported
  54. Dec 03 08:31:17 [8482] storage0 crmd: warning: do_election_vote: Not voting in election, we're in state S_RECOVERY
  55. Dec 03 08:31:17 [8482] storage0 crmd: info: do_dc_release: DC role released
  56. Dec 03 08:31:17 [8482] storage0 crmd: info: do_te_control: Transitioner is now inactive
  57. Dec 03 08:31:17 [8482] storage0 crmd: error: do_log: FSA: Input I_TERMINATE from do_recover() received in state S_RECOVERY
  58. Dec 03 08:31:17 [8482] storage0 crmd: info: do_state_transition: State transition S_RECOVERY -> S_TERMINATE [ input=I_TERMINATE cause=C_FSA_INTERNAL origin=do_recover ]
  59. Dec 03 08:31:17 [8482] storage0 crmd: info: do_shutdown: Disconnecting STONITH...
  60. Dec 03 08:31:17 [8482] storage0 crmd: info: tengine_stonith_connection_destroy: Fencing daemon disconnected
  61. Dec 03 08:31:17 [8479] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_exportfs_storage_monitor_30000
  62. Dec 03 08:31:17 [8479] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_drbd_drives_monitor_10000
  63. Dec 03 08:31:17 [8479] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_sysadmin_notify_monitor_10000
  64. Dec 03 08:31:17 [8479] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_fs_drives_monitor_20000
  65. Dec 03 08:31:17 [8479] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_fs_bind_opt_monitor_20000
  66. Dec 03 08:31:17 [8479] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_daemon_nfs-kernel-server_status_30000
  67. Dec 03 08:31:17 [8479] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_daemon_smbd_monitor_30000
  68. Dec 03 08:31:17 [8479] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_ip_drives1_monitor_30000
  69. Dec 03 08:31:17 [8479] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_ping_monitor_10000
  70. Dec 03 08:31:17 [8479] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_ip_storage_monitor_30000
  71. Dec 03 08:31:17 [8479] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_ip_drives2_monitor_30000
  72. Dec 03 08:31:17 [8479] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_daemon_nmbd_monitor_30000
  73. Dec 03 08:31:17 [8479] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_ip_drives3_monitor_30000
  74. Dec 03 08:31:17 [8479] storage0 lrmd: info: cancel_recurring_action: Cancelling operation p_ip_drives0_monitor_30000
  75. Dec 03 08:31:17 [8482] storage0 crmd: error: verify_stopped: Resource stonithstorage1 was active at shutdown. You may ignore this error if it is unmanaged.
  76. Dec 03 08:31:17 [8482] storage0 crmd: error: verify_stopped: Resource stonithstoragequorum was active at shutdown. You may ignore this error if it is unmanaged.
  77. Dec 03 08:31:17 [8482] storage0 crmd: info: lrmd_api_disconnect: Disconnecting from lrmd service
  78. Dec 03 08:31:17 [8482] storage0 crmd: info: lrmd_connection_destroy: connection destroyed
  79. Dec 03 08:31:17 [8482] storage0 crmd: info: lrm_connection_destroy: LRM Connection disconnected
  80. Dec 03 08:31:17 [8482] storage0 crmd: info: do_lrm_control: Disconnected from the LRM
  81. Dec 03 08:31:17 [8479] storage0 lrmd: info: lrmd_ipc_destroy: LRMD client disconnecting 0x24dec00 - name: crmd id: 16e9947a-436b-4473-9695-85a071861603
  82. Dec 03 08:31:17 [8482] storage0 crmd: info: crm_cluster_disconnect: Disconnecting from cluster infrastructure: corosync
  83. Dec 03 08:31:17 [8482] storage0 crmd: notice: terminate_cs_connection: Disconnecting from Corosync
  84. Dec 03 08:31:17 [8482] storage0 crmd: info: crm_cluster_disconnect: Disconnected from corosync
  85. Dec 03 08:31:17 [8482] storage0 crmd: info: do_ha_control: Disconnected from the cluster
  86. Dec 03 08:31:17 [8482] storage0 crmd: info: do_cib_control: Disconnecting CIB
  87. Dec 03 08:31:17 [8482] storage0 crmd: info: crmd_cib_connection_destroy: Connection to the CIB terminated...
  88. Dec 03 08:31:17 [8482] storage0 crmd: info: qb_ipcs_us_withdraw: withdrawing server sockets
  89. Dec 03 08:31:17 [8482] storage0 crmd: info: do_exit: Performing A_EXIT_0 - gracefully exiting the CRMd
  90. Dec 03 08:31:17 [8482] storage0 crmd: error: do_exit: Could not recover from internal error
  91. Dec 03 08:31:17 [8482] storage0 crmd: info: do_exit: [crmd] stopped (2)
  92. Dec 03 08:31:17 [8482] storage0 crmd: info: free_mem: Dropping I_PENDING: [ state=S_TERMINATE cause=C_FSA_INTERNAL origin=do_election_vote ]
  93. Dec 03 08:31:17 [8482] storage0 crmd: info: free_mem: Dropping I_RELEASE_SUCCESS: [ state=S_TERMINATE cause=C_FSA_INTERNAL origin=do_dc_release ]
  94. Dec 03 08:31:17 [8482] storage0 crmd: info: free_mem: Dropping I_TERMINATE: [ state=S_TERMINATE cause=C_FSA_INTERNAL origin=do_stop ]
  95. Dec 03 08:31:17 [8482] storage0 crmd: info: lrmd_api_disconnect: Disconnecting from lrmd service
  96. Dec 03 08:31:17 [8482] storage0 crmd: info: crm_xml_cleanup: Cleaning up memory from libxml2
  97. Dec 03 08:31:17 [8476] storage0 cib: info: cib_process_readwrite: We are now in R/O mode
  98. Dec 03 08:31:17 [8476] storage0 cib: warning: qb_ipcs_event_sendv: new_event_notification (8476-8482-11): Broken pipe (32)
  99. Dec 03 08:31:17 [8476] storage0 cib: info: crm_ipcs_send: Event 7454 failed, size=162, to=0xf94c00[8482], queue=1, rc=-32: <cib-reply t="cib" cib_op="cib_slave" cib_callid="329" cib_clientid="ca8c870c-cc1f-4290-bc14-ca84a91e037d" cib_callopt="
  100. Dec 03 08:31:17 [8476] storage0 cib: warning: do_local_notify: A-Sync reply to crmd failed: No message of desired type
  101. Dec 03 08:31:17 [8474] storage0 pacemakerd: error: pcmk_child_exit: Child process crmd exited (pid=8482, rc=2)
  102. Dec 03 08:31:17 [8474] storage0 pacemakerd: notice: pcmk_child_exit: Respawning failed child process: crmd
  103. Dec 03 08:31:17 [8474] storage0 pacemakerd: info: start_child: Forked child 11240 for process crmd
  104. ###################################################################################################
  105. ## restarted crmd
  106. ###################################################################################################
  107. Dec 03 08:31:17 [8476] storage0 cib: info: cib_process_replace: Digest matched on replace from storage1: 392a7525fe6f467d4624d317c772de45
  108. Dec 03 08:31:17 [8476] storage0 cib: info: cib_process_replace: Replaced 0.233.125 with 0.233.125 from storage1
  109. Dec 03 08:31:17 [11240] storage0 crmd: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster
  110. Dec 03 08:31:17 [11240] storage0 crmd: notice: main: CRM Git Version: 1f8858c
  111. Dec 03 08:31:17 [11240] storage0 crmd: info: get_cluster_type: Cluster type is: 'corosync'
  112. Dec 03 08:31:17 [11240] storage0 crmd: info: do_cib_control: CIB connection established
  113. Dec 03 08:31:17 [11240] storage0 crmd: notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync
  114. Dec 03 08:31:17 [11240] storage0 crmd: info: crm_get_peer: Node <null> now has id: 2483106826
  115. Dec 03 08:31:17 [11240] storage0 crmd: info: crm_update_peer_proc: init_cpg_connection: Node (null)[-1811860470] - corosync-cpg is now online
  116. Dec 03 08:31:17 [11240] storage0 crmd: info: init_cs_connection_once: Connection to 'corosync': established
  117. Dec 03 08:31:17 [11240] storage0 crmd: info: crm_get_peer: Node 2483106826 is now known as storage0
  118. Dec 03 08:31:17 [11240] storage0 crmd: info: peer_update_callback: storage0 is now (null)
  119. Dec 03 08:31:17 [11240] storage0 crmd: info: crm_get_peer: Node 2483106826 has uuid 2483106826
  120. Dec 03 08:31:17 [11240] storage0 crmd: notice: init_quorum_connection: Quorum acquired
  121. Dec 03 08:31:17 [11240] storage0 crmd: info: do_ha_control: Connected to the cluster
  122. Dec 03 08:31:17 [11240] storage0 crmd: info: lrmd_api_connect: Connecting to lrmd
  123. Dec 03 08:31:17 [8479] storage0 lrmd: info: lrmd_ipc_accept: Accepting client connection: 0x2526f00 pid=11240 for uid=110 gid=0
  124. Dec 03 08:31:17 [11240] storage0 crmd: info: do_started: Delaying start, no membership data (0000000000100000)
  125. Dec 03 08:31:17 [11240] storage0 crmd: info: pcmk_quorum_notification: Membership 1842136: quorum retained (3)
  126. Dec 03 08:31:17 [11240] storage0 crmd: info: crm_get_peer: Node <null> now has id: 402732042
  127. Dec 03 08:31:17 [11240] storage0 crmd: info: pcmk_quorum_notification: Obtaining name for new node 402732042
  128. Dec 03 08:31:17 [8476] storage0 cib: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/3, version=0.233.131): OK (rc=0)
  129. Dec 03 08:31:17 [11240] storage0 crmd: notice: corosync_node_name: Inferred node name 'storagequorum' for nodeid 402732042 from DNS
  130. Dec 03 08:31:17 [11240] storage0 crmd: info: crm_get_peer: Node 402732042 is now known as storagequorum
  131. Dec 03 08:31:17 [11240] storage0 crmd: info: peer_update_callback: storagequorum is now (null)
  132. Dec 03 08:31:17 [11240] storage0 crmd: info: crm_get_peer: Node 402732042 has uuid 402732042
  133. Dec 03 08:31:17 [11240] storage0 crmd: notice: crm_update_peer_state: pcmk_quorum_notification: Node storagequorum[402732042] - state is now member
  134. Dec 03 08:31:17 [11240] storage0 crmd: info: peer_update_callback: storagequorum is now member (was (null))
  135. Dec 03 08:31:17 [11240] storage0 crmd: notice: crm_update_peer_state: pcmk_quorum_notification: Node storage0[2483106826] - state is now member
  136. Dec 03 08:31:17 [11240] storage0 crmd: info: peer_update_callback: storage0 is now member (was (null))
  137. Dec 03 08:31:17 [11240] storage0 crmd: info: crm_get_peer: Node <null> now has id: 2499884042
  138. Dec 03 08:31:17 [11240] storage0 crmd: info: pcmk_quorum_notification: Obtaining name for new node 2499884042
  139. Dec 03 08:31:17 [11240] storage0 crmd: notice: corosync_node_name: Inferred node name 'storage1' for nodeid 2499884042 from DNS
  140. Dec 03 08:31:17 [11240] storage0 crmd: info: crm_get_peer: Node 2499884042 is now known as storage1
  141. Dec 03 08:31:17 [11240] storage0 crmd: info: peer_update_callback: storage1 is now (null)
  142. Dec 03 08:31:17 [11240] storage0 crmd: info: crm_get_peer: Node 2499884042 has uuid 2499884042
  143. Dec 03 08:31:17 [11240] storage0 crmd: notice: crm_update_peer_state: pcmk_quorum_notification: Node storage1[2499884042] - state is now member
  144. Dec 03 08:31:17 [11240] storage0 crmd: info: peer_update_callback: storage1 is now member (was (null))
  145. Dec 03 08:31:17 [11240] storage0 crmd: info: do_started: Delaying start, Config not read (0000000000000040)
  146. Dec 03 08:31:17 [11240] storage0 crmd: info: qb_ipcs_us_publish: server name: crmd
  147. Dec 03 08:31:17 [11240] storage0 crmd: notice: do_started: The local CRM is operational
  148. Dec 03 08:31:17 [11240] storage0 crmd: info: do_state_transition: State transition S_STARTING -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_started ]
  149. Dec 03 08:31:18 [11240] storage0 crmd: info: pcmk_cpg_membership: Joined[0.0] crmd.-1811860470
  150. Dec 03 08:31:18 [11240] storage0 crmd: info: pcmk_cpg_membership: Member[0.0] crmd.402732042
  151. Dec 03 08:31:18 [11240] storage0 crmd: info: crm_update_peer_proc: pcmk_cpg_membership: Node storagequorum[402732042] - corosync-cpg is now online
  152. Dec 03 08:31:18 [11240] storage0 crmd: info: peer_update_callback: Client storagequorum/peer now has status [online] (DC=<null>)
  153. Dec 03 08:31:18 [11240] storage0 crmd: info: pcmk_cpg_membership: Member[0.1] crmd.-1811860470
  154. Dec 03 08:31:18 [11240] storage0 crmd: info: pcmk_cpg_membership: Member[0.2] crmd.-1795083254
  155. Dec 03 08:31:18 [11240] storage0 crmd: info: crm_update_peer_proc: pcmk_cpg_membership: Node storage1[-1795083254] - corosync-cpg is now online
  156. Dec 03 08:31:18 [11240] storage0 crmd: info: peer_update_callback: Client storage1/peer now has status [online] (DC=<null>)
  157. Dec 03 08:31:20 [11240] storage0 crmd: info: update_dc: Set DC to storage1 (3.0.6)
  158. Dec 03 08:31:20 [11240] storage0 crmd: info: erase_status_tag: Deleting xpath: //node_state[@uname='storage0']/transient_attributes
  159. Dec 03 08:31:20 [11240] storage0 crmd: info: update_attrd: Connecting to attrd... 5 retries remaining
  160. Dec 03 08:31:20 [11240] storage0 crmd: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
  161. Dec 03 08:31:20 [8480] storage0 attrd: notice: attrd_local_callback: Sending full refresh (origin=crmd)
  162. Dec 03 08:31:20 [8480] storage0 attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: p_ping (5000)
  163. Dec 03 08:31:20 [8480] storage0 attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_drbd_drives (10000)
  164. Dec 03 08:31:20 [8476] storage0 cib: info: cib_process_replace: Digest matched on replace from storage1: 954ec6b7c3b1707815fa6e109c9edab3
  165. Dec 03 08:31:20 [8476] storage0 cib: info: cib_process_replace: Replaced 0.233.132 with 0.233.132 from storage1
  166. Dec 03 08:31:20 [8480] storage0 attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
  167. Dec 03 08:31:20 [8480] storage0 attrd: warning: attrd_cib_callback: Update fail-count-p_sysadmin_notify=(null) failed: No such device or address
  168. Dec 03 08:31:20 [8480] storage0 attrd: warning: attrd_cib_callback: Update last-failure-p_sysadmin_notify=(null) failed: No such device or address
  169. Dec 03 08:31:20 [8480] storage0 attrd: warning: attrd_cib_callback: Update 144 for master-p_drbd_drives=10000 failed: No such device or address
  170. Dec 03 08:31:20 [8480] storage0 attrd: warning: attrd_cib_callback: Update 146 for probe_complete=true failed: No such device or address
  171. Dec 03 08:31:20 [8478] storage0 stonith-ng: info: stonith_device_action: Device stonithstorage0 not found
  172. Dec 03 08:31:20 [8478] storage0 stonith-ng: info: stonith_command: Processed st_execute from lrmd.8479: No such device (-19)
  173. Dec 03 08:31:20 [8480] storage0 attrd: warning: attrd_cib_callback: Update 148 for p_ping=5000 failed: No such device or address
  174. Dec 03 08:31:20 [8478] storage0 stonith-ng: info: stonith_command: Processed st_execute from lrmd.8479: Operation now in progress (-115)
  175. Dec 03 08:31:20 [8478] storage0 stonith-ng: info: stonith_action_create: Initiating action monitor for agent fence_legacy (target=(null))
  176. Dec 03 08:31:20 [8478] storage0 stonith-ng: info: stonith_command: Processed st_execute from lrmd.8479: Operation now in progress (-115)
  177. Dec 03 08:31:20 [8478] storage0 stonith-ng: info: stonith_action_create: Initiating action monitor for agent fence_legacy (target=(null))
  178. Dec 03 08:31:20 [8480] storage0 attrd: warning: attrd_cib_callback: Update 150 for p_ping=5000 failed: No such device or address
  179. Dec 03 08:31:20 [8480] storage0 attrd: warning: attrd_cib_callback: Update 152 for master-p_drbd_drives=10000 failed: No such device or address
  180. Dec 03 08:31:20 [8480] storage0 attrd: warning: attrd_cib_callback: Update 155 for probe_complete=true failed: No such device or address
  181. Dec 03 08:31:20 [8479] storage0 lrmd: info: get_first_instance: Result: /com/ubuntu/Upstart/jobs/smbd/_
  182. Dec 03 08:31:20 [8479] storage0 lrmd: info: upstart_job_property: Calling GetAll on /com/ubuntu/Upstart/jobs/smbd/_
  183. Dec 03 08:31:20 [8479] storage0 lrmd: info: upstart_job_property: Call to GetAll passed: type '(a{sv})' 1
  184. Dec 03 08:31:20 [8479] storage0 lrmd: info: upstart_job_property: Got value 'running' for /com/ubuntu/Upstart/jobs/smbd/_[state]
  185. Dec 03 08:31:20 [8479] storage0 lrmd: info: upstart_job_running: State of smbd: running
  186. Dec 03 08:31:20 [8479] storage0 lrmd: info: upstart_job_running: smbd is running
  187. Dec 03 08:31:20 [8479] storage0 lrmd: info: get_first_instance: Result: /com/ubuntu/Upstart/jobs/nmbd/_
  188. Dec 03 08:31:20 [8479] storage0 lrmd: info: upstart_job_property: Calling GetAll on /com/ubuntu/Upstart/jobs/nmbd/_
  189. Dec 03 08:31:20 [8479] storage0 lrmd: info: upstart_job_property: Call to GetAll passed: type '(a{sv})' 1
  190. Dec 03 08:31:20 [8479] storage0 lrmd: info: upstart_job_property: Got value 'running' for /com/ubuntu/Upstart/jobs/nmbd/_[state]
  191. Dec 03 08:31:20 [8479] storage0 lrmd: info: upstart_job_running: State of nmbd: running
  192. Dec 03 08:31:20 [8479] storage0 lrmd: info: upstart_job_running: nmbd is running
  193. exportfs[11250]: 2012/12/03_08:31:20 INFO: Directory /mnt/storage is exported to 10.0.0.0/255.0.0.0 (started).
  194. exportfs[11250]: 2012/12/03_08:31:20 INFO: Directory /mnt/storage is exported to 10.0.0.0/255.0.0.0 (started).
  195. Dec 03 08:31:20 [11240] storage0 crmd: info: process_lrm_event: LRM operation stonithstorage0_monitor_0 (call=197, rc=7, cib-update=9, confirmed=true) not running
  196. Dec 03 08:31:21 storage0 stonith: [11534]: info: external_run_cmd: '/usr/lib/stonith/plugins/external/webpowerswitch status' output: PING 192.168.7.200 (192.168.7.200) 56(84) bytes of data.
  197.  
  198. Dec 03 08:31:21 storage0 stonith: [11534]: info: external_run_cmd: '/usr/lib/stonith/plugins/external/webpowerswitch status' output: 64 bytes from 192.168.7.200: icmp_req=1 ttl=64 time=2.44 ms
  199.  
  200. Dec 03 08:31:21 storage0 stonith: [11534]: info: external_run_cmd: '/usr/lib/stonith/plugins/external/webpowerswitch status' output:
  201.  
  202. Dec 03 08:31:21 storage0 stonith: [11534]: info: external_run_cmd: '/usr/lib/stonith/plugins/external/webpowerswitch status' output: --- 192.168.7.200 ping statistics ---
  203.  
  204. Dec 03 08:31:21 storage0 stonith: [11534]: info: external_run_cmd: '/usr/lib/stonith/plugins/external/webpowerswitch status' output: 1 packets transmitted, 1 received, 0% packet loss, time 0ms
  205.  
  206. Dec 03 08:31:21 storage0 stonith: [11534]: info: external_run_cmd: '/usr/lib/stonith/plugins/external/webpowerswitch status' output: rtt min/avg/max/mdev = 2.443/2.443/2.443/0.000 ms
  207.  
  208. Dec 03 08:31:21 storage0 stonith: [11533]: info: external_run_cmd: '/usr/lib/stonith/plugins/external/webpowerswitch status' output: PING 192.168.7.101 (192.168.7.101) 56(84) bytes of data.
  209.  
  210. Dec 03 08:31:21 storage0 stonith: [11533]: info: external_run_cmd: '/usr/lib/stonith/plugins/external/webpowerswitch status' output: 64 bytes from 192.168.7.101: icmp_req=1 ttl=64 time=1.71 ms
  211.  
  212. Dec 03 08:31:21 storage0 stonith: [11533]: info: external_run_cmd: '/usr/lib/stonith/plugins/external/webpowerswitch status' output:
  213.  
  214. Dec 03 08:31:21 storage0 stonith: [11533]: info: external_run_cmd: '/usr/lib/stonith/plugins/external/webpowerswitch status' output: --- 192.168.7.101 ping statistics ---
  215.  
  216. Dec 03 08:31:21 storage0 stonith: [11533]: info: external_run_cmd: '/usr/lib/stonith/plugins/external/webpowerswitch status' output: 1 packets transmitted, 1 received, 0% packet loss, time 0ms
  217.  
  218. Dec 03 08:31:21 storage0 stonith: [11533]: info: external_run_cmd: '/usr/lib/stonith/plugins/external/webpowerswitch status' output: rtt min/avg/max/mdev = 1.710/1.710/1.710/0.000 ms
  219.  
  220. Dec 03 08:31:21 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_daemon_smbd_monitor_0 (call=209, rc=0, cib-update=10, confirmed=true) ok
  221. Dec 03 08:31:21 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_daemon_nmbd_monitor_0 (call=211, rc=0, cib-update=11, confirmed=true) ok
  222. Dec 03 08:31:22 storage0 stonith: [11534]: info: external/webpowerswitch device OK.
  223. Dec 03 08:31:22 [8478] storage0 stonith-ng: info: log_operation: stonithstorage1: Performing: stonith -t external/webpowerswitch -S
  224. Dec 03 08:31:22 [8478] storage0 stonith-ng: info: log_operation: stonithstorage1: success: 0
  225. Dec 03 08:31:22 storage0 stonith: [11533]: info: external/webpowerswitch device OK.
  226. Dec 03 08:31:22 [8478] storage0 stonith-ng: info: log_operation: stonithstoragequorum: Performing: stonith -t external/webpowerswitch -S
  227. Dec 03 08:31:22 [8478] storage0 stonith-ng: info: log_operation: stonithstoragequorum: success: 0
  228. Dec 03 08:31:22 [11240] storage0 crmd: info: services_os_action_execute: Managed Filesystem_meta-data_0 process 11554 exited with rc=0
  229. Dec 03 08:31:22 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_fs_drives_monitor_0 (call=203, rc=0, cib-update=12, confirmed=true) ok
  230. Dec 03 08:31:22 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_fs_bind_opt_monitor_0 (call=205, rc=0, cib-update=13, confirmed=true) ok
  231. Dec 03 08:31:23 [11240] storage0 crmd: info: services_os_action_execute: Managed IPaddr2_meta-data_0 process 11559 exited with rc=0
  232. Dec 03 08:31:23 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_ip_storage_monitor_0 (call=213, rc=0, cib-update=14, confirmed=true) ok
  233. Dec 03 08:31:23 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_ip_drives0_monitor_0 (call=215, rc=0, cib-update=15, confirmed=true) ok
  234. Dec 03 08:31:23 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_ip_drives1_monitor_0 (call=217, rc=0, cib-update=16, confirmed=true) ok
  235. Dec 03 08:31:23 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_ip_drives2_monitor_0 (call=219, rc=0, cib-update=17, confirmed=true) ok
  236. Dec 03 08:31:23 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_ip_drives3_monitor_0 (call=221, rc=0, cib-update=18, confirmed=true) ok
  237. Dec 03 08:31:24 [11240] storage0 crmd: info: services_os_action_execute: Managed MailTo_meta-data_0 process 11566 exited with rc=0
  238. Dec 03 08:31:24 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_sysadmin_notify_monitor_0 (call=229, rc=0, cib-update=19, confirmed=true) ok
  239. Dec 03 08:31:25 [11240] storage0 crmd: info: services_os_action_execute: Managed drbd_meta-data_0 process 11571 exited with rc=0
  240. Dec 03 08:31:25 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_drbd_drives_monitor_0 (call=223, rc=8, cib-update=20, confirmed=true) master
  241. Dec 03 08:31:25 [11240] storage0 crmd: info: process_lrm_event: Result:
  242. Dec 03 08:31:25 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_daemon_nfs-kernel-server_monitor_0 (call=227, rc=0, cib-update=21, confirmed=true) ok
  243. Dec 03 08:31:26 [11240] storage0 crmd: info: services_os_action_execute: Managed exportfs_meta-data_0 process 11580 exited with rc=0
  244. Dec 03 08:31:26 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_exportfs_storage_monitor_0 (call=207, rc=0, cib-update=22, confirmed=true) ok
  245. Dec 03 08:31:26 [11240] storage0 crmd: notice: process_lrm_event: LRM operation stonithstorage1_monitor_0 (call=199, rc=0, cib-update=23, confirmed=true) ok
  246. Dec 03 08:31:26 [11240] storage0 crmd: notice: process_lrm_event: LRM operation stonithstoragequorum_monitor_0 (call=201, rc=0, cib-update=24, confirmed=true) ok
  247. Dec 03 08:31:31 [11240] storage0 crmd: info: services_os_action_execute: Managed ping_meta-data_0 process 11592 exited with rc=0
  248. Dec 03 08:31:31 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_ping_monitor_0 (call=225, rc=0, cib-update=25, confirmed=true) ok
  249. Dec 03 08:31:31 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_sysadmin_notify_monitor_10000 (call=254, rc=0, cib-update=26, confirmed=false) ok
  250. Dec 03 08:31:31 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_drbd_drives_notify_0 (call=256, rc=0, cib-update=0, confirmed=true) ok
  251. Dec 03 08:31:31 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_daemon_nfs-kernel-server_monitor_30000 (call=252, rc=0, cib-update=27, confirmed=false) ok
  252. ###################################################################################################
  253. ## migrate resources to storage1
  254. ###################################################################################################
  255. IPaddr2[11598]: 2012/12/03_08:31:31 INFO: IP status = ok, IP_CIP=
  256. IPaddr2[11598]: 2012/12/03_08:31:31 INFO: IP status = ok, IP_CIP=
  257. Dec 03 08:31:31 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_ip_drives3_stop_0 (call=248, rc=0, cib-update=28, confirmed=true) ok
  258. IPaddr2[11682]: 2012/12/03_08:31:31 INFO: IP status = ok, IP_CIP=
  259. IPaddr2[11682]: 2012/12/03_08:31:31 INFO: IP status = ok, IP_CIP=
  260. Dec 03 08:31:31 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_ip_drives2_stop_0 (call=262, rc=0, cib-update=29, confirmed=true) ok
  261. IPaddr2[11720]: 2012/12/03_08:31:31 INFO: IP status = ok, IP_CIP=
  262. IPaddr2[11720]: 2012/12/03_08:31:31 INFO: IP status = ok, IP_CIP=
  263. Dec 03 08:31:31 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_ip_drives1_stop_0 (call=265, rc=0, cib-update=30, confirmed=true) ok
  264. IPaddr2[11756]: 2012/12/03_08:31:31 INFO: IP status = ok, IP_CIP=
  265. IPaddr2[11756]: 2012/12/03_08:31:31 INFO: IP status = ok, IP_CIP=
  266. Dec 03 08:31:31 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_ip_drives0_stop_0 (call=268, rc=0, cib-update=31, confirmed=true) ok
  267. IPaddr2[11792]: 2012/12/03_08:31:31 INFO: IP status = ok, IP_CIP=
  268. IPaddr2[11792]: 2012/12/03_08:31:31 INFO: IP status = ok, IP_CIP=
  269. Dec 03 08:31:31 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_ip_storage_stop_0 (call=271, rc=0, cib-update=32, confirmed=true) ok
  270. Dec 03 08:31:31 [8479] storage0 lrmd: error: crm_abort: crm_glib_handler: Forked child 11829 to record non-fatal assert at logging.c:60 : the GVariant format string `(o)' has a type of `(o)' but the given value has a type of `()'
  271. Dec 03 08:32:06 [8479] storage0 lrmd: error: crm_abort: crm_glib_handler: Forked child 11904 to record non-fatal assert at logging.c:60 : g_variant_get: assertion `valid_format_string (format_string, TRUE, value)' failed
  272. Dec 03 08:32:06 [8479] storage0 lrmd: info: upstart_job_exec_done: Call to stop passed: type '()' (null)
  273. Dec 03 08:32:06 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_daemon_nmbd_stop_0 (call=274, rc=0, cib-update=33, confirmed=true) ok
  274. Dec 03 08:32:06 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_ping_monitor_10000 (call=250, rc=0, cib-update=34, confirmed=false) ok
  275. Dec 03 08:32:06 [8479] storage0 lrmd: error: crm_abort: crm_glib_handler: Forked child 11912 to record non-fatal assert at logging.c:60 : the GVariant format string `(o)' has a type of `(o)' but the given value has a type of `()'
  276. Dec 03 08:32:06 [8479] storage0 lrmd: error: crm_abort: crm_glib_handler: Forked child 11914 to record non-fatal assert at logging.c:60 : g_variant_get: assertion `valid_format_string (format_string, TRUE, value)' failed
  277. Dec 03 08:32:06 [8479] storage0 lrmd: info: upstart_job_exec_done: Call to stop passed: type '()' (null)
  278. Dec 03 08:32:07 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_daemon_smbd_stop_0 (call=278, rc=0, cib-update=35, confirmed=true) ok
  279. exportfs[11916]: 2012/12/03_08:32:07 INFO: Directory /mnt/storage is exported to 10.0.0.0/255.0.0.0 (started).
  280. exportfs[11916]: 2012/12/03_08:32:07 INFO: Directory /mnt/storage is exported to 10.0.0.0/255.0.0.0 (started).
  281. exportfs[11916]: 2012/12/03_08:32:07 INFO: Un-exporting file system ...
  282. exportfs[11916]: 2012/12/03_08:32:07 INFO: Un-exporting file system ...
  283. exportfs[11916]: 2012/12/03_08:32:07 INFO: unexporting 10.0.0.0/255.0.0.0:/mnt/storage
  284. exportfs[11916]: 2012/12/03_08:32:07 INFO: unexporting 10.0.0.0/255.0.0.0:/mnt/storage
  285. exportfs[11916]: 2012/12/03_08:32:07 INFO: Un-exported file system
  286. exportfs[11916]: 2012/12/03_08:32:07 INFO: Un-exported file system
  287. Dec 03 08:32:07 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_exportfs_storage_stop_0 (call=281, rc=0, cib-update=36, confirmed=true) ok
  288. Filesystem[11944]: 2012/12/03_08:32:07 INFO: Running stop for /mnt/storage/home/shared/opt on /opt
  289. Filesystem[11944]: 2012/12/03_08:32:07 INFO: Running stop for /mnt/storage/home/shared/opt on /opt
  290. Filesystem[11944]: 2012/12/03_08:32:07 INFO: Trying to unmount /opt
  291. Filesystem[11944]: 2012/12/03_08:32:07 INFO: Trying to unmount /opt
  292. Filesystem[11944]: 2012/12/03_08:32:07 INFO: unmounted /opt successfully
  293. Filesystem[11944]: 2012/12/03_08:32:07 INFO: unmounted /opt successfully
  294. Dec 03 08:32:07 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_fs_bind_opt_stop_0 (call=284, rc=0, cib-update=37, confirmed=true) ok
  295. Filesystem[12009]: 2012/12/03_08:32:07 INFO: Running stop for /dev/drbd0 on /mnt/storage
  296. Filesystem[12009]: 2012/12/03_08:32:07 INFO: Running stop for /dev/drbd0 on /mnt/storage
  297. Filesystem[12009]: 2012/12/03_08:32:07 INFO: Trying to unmount /mnt/storage
  298. Filesystem[12009]: 2012/12/03_08:32:07 INFO: Trying to unmount /mnt/storage
  299. Filesystem[12009]: 2012/12/03_08:32:07 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  300. Filesystem[12009]: 2012/12/03_08:32:07 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  301. Filesystem[12009]: 2012/12/03_08:32:07 INFO: Some processes on /mnt/storage were signalled
  302. Filesystem[12009]: 2012/12/03_08:32:07 INFO: Some processes on /mnt/storage were signalled
  303. Filesystem[12009]: 2012/12/03_08:32:09 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  304. Filesystem[12009]: 2012/12/03_08:32:09 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  305. Filesystem[12009]: 2012/12/03_08:32:09 INFO: Some processes on /mnt/storage were signalled
  306. Filesystem[12009]: 2012/12/03_08:32:09 INFO: Some processes on /mnt/storage were signalled
  307. Filesystem[12009]: 2012/12/03_08:32:10 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  308. Filesystem[12009]: 2012/12/03_08:32:10 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  309. Filesystem[12009]: 2012/12/03_08:32:10 INFO: Some processes on /mnt/storage were signalled
  310. Filesystem[12009]: 2012/12/03_08:32:10 INFO: Some processes on /mnt/storage were signalled
  311. Filesystem[12009]: 2012/12/03_08:32:11 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  312. Filesystem[12009]: 2012/12/03_08:32:11 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  313. Filesystem[12009]: 2012/12/03_08:32:11 INFO: Some processes on /mnt/storage were signalled
  314. Filesystem[12009]: 2012/12/03_08:32:11 INFO: Some processes on /mnt/storage were signalled
  315. Filesystem[12009]: 2012/12/03_08:32:12 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  316. Filesystem[12009]: 2012/12/03_08:32:12 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  317. Filesystem[12009]: 2012/12/03_08:32:12 INFO: Some processes on /mnt/storage were signalled
  318. Filesystem[12009]: 2012/12/03_08:32:12 INFO: Some processes on /mnt/storage were signalled
  319. Filesystem[12009]: 2012/12/03_08:32:13 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  320. Filesystem[12009]: 2012/12/03_08:32:13 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  321. Filesystem[12009]: 2012/12/03_08:32:13 INFO: Some processes on /mnt/storage were signalled
  322. Filesystem[12009]: 2012/12/03_08:32:13 INFO: Some processes on /mnt/storage were signalled
  323. Filesystem[12009]: 2012/12/03_08:32:14 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  324. Filesystem[12009]: 2012/12/03_08:32:14 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  325. Filesystem[12009]: 2012/12/03_08:32:14 INFO: Some processes on /mnt/storage were signalled
  326. Filesystem[12009]: 2012/12/03_08:32:14 INFO: Some processes on /mnt/storage were signalled
  327. Filesystem[12009]: 2012/12/03_08:32:15 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  328. Filesystem[12009]: 2012/12/03_08:32:15 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  329. Filesystem[12009]: 2012/12/03_08:32:15 INFO: Some processes on /mnt/storage were signalled
  330. Filesystem[12009]: 2012/12/03_08:32:15 INFO: Some processes on /mnt/storage were signalled
  331. Filesystem[12009]: 2012/12/03_08:32:16 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  332. Filesystem[12009]: 2012/12/03_08:32:16 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  333. Filesystem[12009]: 2012/12/03_08:32:16 INFO: Some processes on /mnt/storage were signalled
  334. Filesystem[12009]: 2012/12/03_08:32:16 INFO: Some processes on /mnt/storage were signalled
  335. Filesystem[12009]: 2012/12/03_08:32:17 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  336. Filesystem[12009]: 2012/12/03_08:32:17 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  337. Filesystem[12009]: 2012/12/03_08:32:17 INFO: Some processes on /mnt/storage were signalled
  338. Filesystem[12009]: 2012/12/03_08:32:17 INFO: Some processes on /mnt/storage were signalled
  339. Filesystem[12009]: 2012/12/03_08:32:18 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  340. Filesystem[12009]: 2012/12/03_08:32:18 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  341. Filesystem[12009]: 2012/12/03_08:32:18 INFO: Some processes on /mnt/storage were signalled
  342. Filesystem[12009]: 2012/12/03_08:32:18 INFO: Some processes on /mnt/storage were signalled
  343. Filesystem[12009]: 2012/12/03_08:32:19 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  344. Filesystem[12009]: 2012/12/03_08:32:19 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  345. Filesystem[12009]: 2012/12/03_08:32:19 INFO: Some processes on /mnt/storage were signalled
  346. Filesystem[12009]: 2012/12/03_08:32:19 INFO: Some processes on /mnt/storage were signalled
  347. Filesystem[12009]: 2012/12/03_08:32:20 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  348. Filesystem[12009]: 2012/12/03_08:32:20 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  349. Filesystem[12009]: 2012/12/03_08:32:20 INFO: Some processes on /mnt/storage were signalled
  350. Filesystem[12009]: 2012/12/03_08:32:20 INFO: Some processes on /mnt/storage were signalled
  351. Filesystem[12009]: 2012/12/03_08:32:21 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  352. Filesystem[12009]: 2012/12/03_08:32:21 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  353. Filesystem[12009]: 2012/12/03_08:32:21 INFO: Some processes on /mnt/storage were signalled
  354. Filesystem[12009]: 2012/12/03_08:32:21 INFO: Some processes on /mnt/storage were signalled
  355. Filesystem[12009]: 2012/12/03_08:32:22 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  356. Filesystem[12009]: 2012/12/03_08:32:22 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  357. Filesystem[12009]: 2012/12/03_08:32:22 INFO: Some processes on /mnt/storage were signalled
  358. Filesystem[12009]: 2012/12/03_08:32:22 INFO: Some processes on /mnt/storage were signalled
  359. Filesystem[12009]: 2012/12/03_08:32:23 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  360. Filesystem[12009]: 2012/12/03_08:32:23 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  361. Filesystem[12009]: 2012/12/03_08:32:23 INFO: Some processes on /mnt/storage were signalled
  362. Filesystem[12009]: 2012/12/03_08:32:23 INFO: Some processes on /mnt/storage were signalled
  363. Filesystem[12009]: 2012/12/03_08:32:24 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  364. Filesystem[12009]: 2012/12/03_08:32:24 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  365. Filesystem[12009]: 2012/12/03_08:32:24 INFO: Some processes on /mnt/storage were signalled
  366. Filesystem[12009]: 2012/12/03_08:32:24 INFO: Some processes on /mnt/storage were signalled
  367. Filesystem[12009]: 2012/12/03_08:32:25 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  368. Filesystem[12009]: 2012/12/03_08:32:25 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  369. Filesystem[12009]: 2012/12/03_08:32:25 INFO: Some processes on /mnt/storage were signalled
  370. Filesystem[12009]: 2012/12/03_08:32:25 INFO: Some processes on /mnt/storage were signalled
  371. Filesystem[12009]: 2012/12/03_08:32:26 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  372. Filesystem[12009]: 2012/12/03_08:32:26 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  373. Filesystem[12009]: 2012/12/03_08:32:26 INFO: Some processes on /mnt/storage were signalled
  374. Filesystem[12009]: 2012/12/03_08:32:26 INFO: Some processes on /mnt/storage were signalled
  375. Filesystem[12009]: 2012/12/03_08:32:27 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  376. Filesystem[12009]: 2012/12/03_08:32:27 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  377. Filesystem[12009]: 2012/12/03_08:32:27 INFO: Some processes on /mnt/storage were signalled
  378. Filesystem[12009]: 2012/12/03_08:32:27 INFO: Some processes on /mnt/storage were signalled
  379. Filesystem[12009]: 2012/12/03_08:32:28 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  380. Filesystem[12009]: 2012/12/03_08:32:28 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  381. Filesystem[12009]: 2012/12/03_08:32:28 INFO: Some processes on /mnt/storage were signalled
  382. Filesystem[12009]: 2012/12/03_08:32:28 INFO: Some processes on /mnt/storage were signalled
  383. Filesystem[12009]: 2012/12/03_08:32:29 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  384. Filesystem[12009]: 2012/12/03_08:32:29 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  385. Filesystem[12009]: 2012/12/03_08:32:29 INFO: Some processes on /mnt/storage were signalled
  386. Filesystem[12009]: 2012/12/03_08:32:29 INFO: Some processes on /mnt/storage were signalled
  387. Filesystem[12009]: 2012/12/03_08:32:31 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  388. Filesystem[12009]: 2012/12/03_08:32:31 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  389. Filesystem[12009]: 2012/12/03_08:32:31 INFO: Some processes on /mnt/storage were signalled
  390. Filesystem[12009]: 2012/12/03_08:32:31 INFO: Some processes on /mnt/storage were signalled
  391. Filesystem[12009]: 2012/12/03_08:32:32 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  392. Filesystem[12009]: 2012/12/03_08:32:32 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  393. Filesystem[12009]: 2012/12/03_08:32:32 INFO: Some processes on /mnt/storage were signalled
  394. Filesystem[12009]: 2012/12/03_08:32:32 INFO: Some processes on /mnt/storage were signalled
  395. Filesystem[12009]: 2012/12/03_08:32:33 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  396. Filesystem[12009]: 2012/12/03_08:32:33 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  397. Filesystem[12009]: 2012/12/03_08:32:33 INFO: Some processes on /mnt/storage were signalled
  398. Filesystem[12009]: 2012/12/03_08:32:33 INFO: Some processes on /mnt/storage were signalled
  399. Filesystem[12009]: 2012/12/03_08:32:34 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  400. Filesystem[12009]: 2012/12/03_08:32:34 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  401. Filesystem[12009]: 2012/12/03_08:32:34 INFO: Some processes on /mnt/storage were signalled
  402. Filesystem[12009]: 2012/12/03_08:32:34 INFO: Some processes on /mnt/storage were signalled
  403. Filesystem[12009]: 2012/12/03_08:32:35 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  404. Filesystem[12009]: 2012/12/03_08:32:35 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  405. Filesystem[12009]: 2012/12/03_08:32:35 INFO: Some processes on /mnt/storage were signalled
  406. Filesystem[12009]: 2012/12/03_08:32:35 INFO: Some processes on /mnt/storage were signalled
  407. Filesystem[12009]: 2012/12/03_08:32:36 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  408. Filesystem[12009]: 2012/12/03_08:32:36 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  409. Filesystem[12009]: 2012/12/03_08:32:36 INFO: Some processes on /mnt/storage were signalled
  410. Filesystem[12009]: 2012/12/03_08:32:36 INFO: Some processes on /mnt/storage were signalled
  411. Filesystem[12009]: 2012/12/03_08:32:37 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  412. Filesystem[12009]: 2012/12/03_08:32:37 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  413. Filesystem[12009]: 2012/12/03_08:32:37 INFO: Some processes on /mnt/storage were signalled
  414. Filesystem[12009]: 2012/12/03_08:32:37 INFO: Some processes on /mnt/storage were signalled
  415. Filesystem[12009]: 2012/12/03_08:32:38 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  416. Filesystem[12009]: 2012/12/03_08:32:38 ERROR: Couldn't unmount /mnt/storage; trying cleanup with TERM
  417. Filesystem[12009]: 2012/12/03_08:32:38 INFO: Some processes on /mnt/storage were signalled
  418. Filesystem[12009]: 2012/12/03_08:32:38 INFO: Some processes on /mnt/storage were signalled
  419. Filesystem[12009]: 2012/12/03_08:32:39 ERROR: Couldn't unmount /mnt/storage; trying cleanup with KILL
  420. Filesystem[12009]: 2012/12/03_08:32:39 ERROR: Couldn't unmount /mnt/storage; trying cleanup with KILL
  421. Filesystem[12009]: 2012/12/03_08:32:39 INFO: Some processes on /mnt/storage were signalled
  422. Filesystem[12009]: 2012/12/03_08:32:39 INFO: Some processes on /mnt/storage were signalled
  423. Filesystem[12009]: 2012/12/03_08:32:42 INFO: unmounted /mnt/storage successfully
  424. Filesystem[12009]: 2012/12/03_08:32:42 INFO: unmounted /mnt/storage successfully
  425. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  426. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  427. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  428. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: cc ]
  429. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  430. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  431. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  432. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  433. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  434. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  435. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  436. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  437. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  438. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  439. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  440. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  441. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  442. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  443. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  444. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  445. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  446. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  447. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  448. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  449. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  450. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  451. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  452. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  453. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  454. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  455. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  456. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  457. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  458. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  459. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  460. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  461. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  462. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  463. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  464. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  465. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  466. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  467. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  468. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  469. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  470. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  471. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  472. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  473. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  474. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  475. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  476. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  477. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  478. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  479. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  480. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  481. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  482. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  483. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  484. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  485. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  486. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  487. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  488. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  489. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  490. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  491. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  492. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  493. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  494. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  495. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  496. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  497. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  498. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  499. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  500. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  501. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  502. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  503. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  504. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  505. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  506. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  507. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  508. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  509. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  510. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  511. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  512. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  513. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  514. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  515. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  516. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  517. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  518. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  519. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  520. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  521. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  522. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  523. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  524. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  525. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  526. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  527. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  528. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  529. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  530. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  531. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  532. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  533. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  534. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  535. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  536. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  537. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  538. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  539. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  540. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  541. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  542. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  543. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  544. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  545. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ umount: /mnt/storage: device is busy. ]
  546. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ (In some cases useful info about processes that use ]
  547. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ the device is found by lsof(8) or fuser(1)) ]
  548. Dec 03 08:32:42 [8479] storage0 lrmd: notice: operation_finished: p_fs_drives_stop_0:12009 [ /mnt/storage: c ]
  549. Dec 03 08:32:42 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_fs_drives_stop_0 (call=287, rc=0, cib-update=38, confirmed=true) ok
  550. Dec 03 08:32:42 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_drbd_drives_demote_0 (call=290, rc=0, cib-update=39, confirmed=true) ok
  551. Dec 03 08:32:42 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_drbd_drives_notify_0 (call=293, rc=0, cib-update=0, confirmed=true) ok
  552. Dec 03 08:32:42 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_drbd_drives_notify_0 (call=296, rc=0, cib-update=0, confirmed=true) ok
  553. Dec 03 08:32:42 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_drbd_drives_notify_0 (call=299, rc=0, cib-update=0, confirmed=true) ok
  554. Dec 03 08:32:42 [11240] storage0 crmd: notice: process_lrm_event: LRM operation p_drbd_drives_monitor_20000 (call=302, rc=0, cib-update=40, confirmed=false) ok
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!