In node1 

Feb  6 09:59:42 wsguardian1 corosync[2725]: cman killed by node 2 because we were killed by cman_tool or other application
Feb  6 09:59:42 wsguardian1 fenced[2784]: cluster is down, exiting
Feb  6 09:59:42 wsguardian1 gfs_controld[2859]: cluster is down, exiting
Feb  6 09:59:42 wsguardian1 fenced[2784]: daemon cpg_dispatch error 2
Feb  6 09:59:42 wsguardian1 gfs_controld[2859]: daemon cpg_dispatch error 2
Feb  6 09:59:42 wsguardian1 dlm_controld[2800]: cluster is down, exiting
Feb  6 09:59:42 wsguardian1 fenced[2784]: cpg_dispatch error 2
Feb  6 09:59:42 wsguardian1 dlm_controld[2800]: daemon cpg_dispatch error 2
Feb  6 09:59:44 wsguardian1 kernel: dlm: closing connection to node 2
Feb  6 09:59:44 wsguardian1 kernel: dlm: closing connection to node 1



Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_db: PingAck did not arrive in time.
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_db: peer( Primary -> Unknown ) conn( Connected -> NetworkFailure ) pdsk( UpToDate -> DUnknown ) susp( 0 -> 1 ) 
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_db: asender terminated
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_db: Terminating asender thread
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_db: Connection closed
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_db: conn( NetworkFailure -> Unconnected ) 
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_db: receiver terminated
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_db: Restarting receiver thread
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_db: receiver (re)started
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_db: conn( Unconnected -> WFConnection ) 
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_db: helper command: /sbin/drbdadm fence-peer wsg_db
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_config: PingAck did not arrive in time.
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_config: peer( Primary -> Unknown ) conn( Connected -> NetworkFailure ) pdsk( UpToDate -> DUnknown ) susp( 0 -> 1 ) 
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_config: asender terminated
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_config: Terminating asender thread
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_config: Connection closed
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_config: conn( NetworkFailure -> Unconnected ) 
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_config: receiver terminated
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_config: Restarting receiver thread
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_config: receiver (re)started
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_config: conn( Unconnected -> WFConnection ) 
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_config: helper command: /sbin/drbdadm fence-peer wsg_config
Feb  6 10:00:03 wsguardian1 rhcs_fence: 125; DEBUG: Checking if: [uname] is at: [/bin/uname]
Feb  6 10:00:03 wsguardian1 rhcs_fence: 156; DEBUG: Found!
Feb  6 10:00:03 wsguardian1 rhcs_fence: 125; DEBUG: Checking if: [fence_node] is at: [/usr/sbin/fence_node]
Feb  6 10:00:03 wsguardian1 rhcs_fence: 156; DEBUG: Found!
Feb  6 10:00:03 wsguardian1 rhcs_fence: 125; DEBUG: Checking if: [cman_tool] is at: [/usr/sbin/cman_tool]
Feb  6 10:00:03 wsguardian1 rhcs_fence: 156; DEBUG: Found!
Feb  6 10:00:03 wsguardian1 rhcs_fence: 74; Attempting to fence peer using RHCS from DRBD...
Feb  6 10:00:03 wsguardian1 rhcs_fence: 80; DEBUG: Environment variable: [DRBD_RESOURCE] == [wsg_db]
Feb  6 10:00:03 wsguardian1 rhcs_fence: 80; DEBUG: Environment variable: [DRBD_MINOR] == [0]
Feb  6 10:00:03 wsguardian1 rhcs_fence: 80; DEBUG: Environment variable: [DRBD_PEERS] == [wsguardian2]
Feb  6 10:00:03 wsguardian1 rhcs_fence: 454; DEBUG: shell call: [/usr/sbin/cman_tool status]
Feb  6 10:00:03 wsguardian1 rhcs_fence: 460; DEBUG: output: /usr/sbin/cman_tool: Cannot open connection to cman, is it running ?
Feb  6 10:00:03 wsguardian1 rhcs_fence: 469; DEBUG: Attempt to get local node name via 'cman_tool status' exited with: [256]
Feb  6 10:00:03 wsguardian1 rhcs_fence: 471; DEBUG: I am: []
Feb  6 10:00:03 wsguardian1 rhcs_fence: 474; Unable to find local node name.
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_db: helper command: /sbin/drbdadm fence-peer wsg_db exit code 1 (0x100)
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_db: fence-peer helper broken, returned 1
Feb  6 10:00:03 wsguardian1 rhcs_fence: 125; DEBUG: Checking if: [uname] is at: [/bin/uname]
Feb  6 10:00:03 wsguardian1 rhcs_fence: 156; DEBUG: Found!
Feb  6 10:00:03 wsguardian1 rhcs_fence: 125; DEBUG: Checking if: [fence_node] is at: [/usr/sbin/fence_node]
Feb  6 10:00:03 wsguardian1 rhcs_fence: 156; DEBUG: Found!
Feb  6 10:00:03 wsguardian1 rhcs_fence: 125; DEBUG: Checking if: [cman_tool] is at: [/usr/sbin/cman_tool]
Feb  6 10:00:03 wsguardian1 rhcs_fence: 156; DEBUG: Found!
Feb  6 10:00:03 wsguardian1 rhcs_fence: 74; Attempting to fence peer using RHCS from DRBD...
Feb  6 10:00:03 wsguardian1 rhcs_fence: 80; DEBUG: Environment variable: [DRBD_RESOURCE] == [wsg_config]
Feb  6 10:00:03 wsguardian1 rhcs_fence: 80; DEBUG: Environment variable: [DRBD_MINOR] == [1]
Feb  6 10:00:03 wsguardian1 rhcs_fence: 80; DEBUG: Environment variable: [DRBD_PEERS] == [wsguardian2]
Feb  6 10:00:03 wsguardian1 rhcs_fence: 454; DEBUG: shell call: [/usr/sbin/cman_tool status]
Feb  6 10:00:03 wsguardian1 rhcs_fence: 460; DEBUG: output: /usr/sbin/cman_tool: Cannot open connection to cman, is it running ?
Feb  6 10:00:03 wsguardian1 rhcs_fence: 469; DEBUG: Attempt to get local node name via 'cman_tool status' exited with: [256]
Feb  6 10:00:03 wsguardian1 rhcs_fence: 471; DEBUG: I am: []
Feb  6 10:00:03 wsguardian1 rhcs_fence: 474; Unable to find local node name.
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_config: helper command: /sbin/drbdadm fence-peer wsg_config exit code 1 (0x100)
Feb  6 10:00:03 wsguardian1 kernel: d-con wsg_config: fence-peer helper broken, returned 1



In node 2

Feb  6 09:59:52 wsguardian2 corosync[2668]:   [TOTEM ] A processor failed, forming new configuration.
Feb  6 09:59:54 wsguardian2 corosync[2668]:   [QUORUM] Members[1]: 2
Feb  6 09:59:54 wsguardian2 corosync[2668]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Feb  6 09:59:54 wsguardian2 corosync[2668]:   [CPG   ] chosen downlist: sender r(0) ip(192.168.253.2) ; members(old:2 left:1)
Feb  6 09:59:54 wsguardian2 corosync[2668]:   [MAIN  ] Completed service synchronization, ready to provide service.
Feb  6 09:59:54 wsguardian2 kernel: dlm: closing connection to node 1
Feb  6 09:59:54 wsguardian2 fenced[2727]: fencing node wsguardian1


The node 2 halt complety after i execute the command /usr/sbin/cman_tool kill -n wsguardian1 from this