API tools faq
paste
digimer

an_node01 service start and stop

digimer
Mar 26th, 2011
158
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 16.31 KB | None | 0 0
raw download clone embed print report
  1. Mar 26 13:52:50 an-node01 kernel: dlm: Using TCP for communications
  2. Mar 26 13:52:50 an-node01 kernel: dlm: got connection from 2
  3. Mar 26 13:52:50 an-node01 clurgmgrd[6436]: <notice> Resource Group Manager Starting
  4. Mar 26 13:52:50 an-node01 clurgmgrd[6436]: <info> Loading Service Data
  5. Mar 26 13:52:51 an-node01 clurgmgrd[6436]: <info> Initializing Services
  6. Mar 26 13:52:51 an-node01 clurgmgrd: [6436]: <err> stop: Could not match /dev/drbd_sh1_vg0/xen_shared with a real device
  7. Mar 26 13:52:51 an-node01 clurgmgrd[6436]: <notice> stop on clusterfs "gfs2_xen_shared" returned 2 (invalid argument(s))
  8. Mar 26 13:52:51 an-node01 clurgmgrd: [6436]: <info> Executing /etc/init.d/clvmd stop
  9. Mar 26 13:52:51 an-node01 clurgmgrd: [6436]: <err> stop: Could not match /dev/drbd_sh1_vg0/xen_shared with a real device
  10. Mar 26 13:52:51 an-node01 clurgmgrd[6436]: <notice> stop on clusterfs "gfs2_xen_shared" returned 2 (invalid argument(s))
  11. Mar 26 13:52:51 an-node01 clurgmgrd: [6436]: <info> Executing /etc/init.d/drbd stop
  12. Mar 26 13:52:51 an-node01 clurgmgrd: [6436]: <info> Executing /etc/init.d/clvmd stop
  13. Mar 26 13:52:51 an-node01 clurgmgrd: [6436]: <info> Executing /etc/init.d/drbd stop
  14. Mar 26 13:52:51 an-node01 clurgmgrd[6436]: <info> Services Initialized
  15. Mar 26 13:52:51 an-node01 clurgmgrd[6436]: <info> State change: Local UP
  16. Mar 26 13:52:51 an-node01 clurgmgrd[6436]: <info> State change: an-node02.alteeve.com UP
  17. Mar 26 13:53:12 an-node01 clurgmgrd[6436]: <notice> Starting disabled service service:an1_storage
  18. Mar 26 13:53:12 an-node01 clurgmgrd: [6436]: <info> Executing /etc/init.d/drbd start
  19. Mar 26 13:53:12 an-node01 kernel: drbd: initialized. Version: 8.3.8 (api:88/proto:86-94)
  20. Mar 26 13:53:12 an-node01 kernel: drbd: GIT-hash: d78846e52224fd00562f7c225bcc25b2d422321d build by [email protected], 2010-06-04 08:04:27
  21. Mar 26 13:53:12 an-node01 kernel: drbd: registered as block device major 147
  22. Mar 26 13:53:12 an-node01 kernel: drbd: minor_table @ 0xffff8800c2d9eb80
  23. Mar 26 13:53:12 an-node01 kernel: block drbd0: Starting worker thread (from cqueue/1 [140])
  24. Mar 26 13:53:12 an-node01 kernel: klogd 1.4.1, ---------- state change ----------
  25. Mar 26 13:53:12 an-node01 kernel: block drbd0: disk( Diskless -> Attaching )
  26. Mar 26 13:53:12 an-node01 kernel: block drbd0: Found 4 transactions (4 active extents) in activity log.
  27. Mar 26 13:53:12 an-node01 kernel: block drbd0: Method to ensure write ordering: barrier
  28. Mar 26 13:53:12 an-node01 kernel: block drbd0: max_segment_size ( = BIO size ) = 32768
  29. Mar 26 13:53:12 an-node01 kernel: block drbd0: drbd_bm_resize called with capacity == 19550368
  30. Mar 26 13:53:12 an-node01 kernel: block drbd0: resync bitmap: bits=2443796 words=38185
  31. Mar 26 13:53:12 an-node01 kernel: block drbd0: size = 9546 MB (9775184 KB)
  32. Mar 26 13:53:12 an-node01 kernel: block drbd0: recounting of set bits took additional 0 jiffies
  33. Mar 26 13:53:12 an-node01 kernel: block drbd0: 0 KB (0 bits) marked out-of-sync by on disk bit-map.
  34. Mar 26 13:53:12 an-node01 kernel: block drbd0: disk( Attaching -> Outdated )
  35. Mar 26 13:53:12 an-node01 kernel: block drbd1: Starting worker thread (from cqueue/0 [139])
  36. Mar 26 13:53:12 an-node01 kernel: block drbd1: disk( Diskless -> Attaching )
  37. Mar 26 13:53:12 an-node01 kernel: block drbd1: Found 4 transactions (192 active extents) in activity log.
  38. Mar 26 13:53:12 an-node01 kernel: block drbd1: Method to ensure write ordering: barrier
  39. Mar 26 13:53:12 an-node01 kernel: block drbd1: max_segment_size ( = BIO size ) = 32768
  40. Mar 26 13:53:12 an-node01 kernel: block drbd1: drbd_bm_resize called with capacity == 195328232
  41. Mar 26 13:53:12 an-node01 kernel: block drbd1: resync bitmap: bits=24416029 words=381501
  42. Mar 26 13:53:12 an-node01 kernel: block drbd1: size = 93 GB (97664116 KB)
  43. Mar 26 13:53:12 an-node01 kernel: block drbd1: recounting of set bits took additional 1 jiffies
  44. Mar 26 13:53:12 an-node01 kernel: block drbd1: 0 KB (0 bits) marked out-of-sync by on disk bit-map.
  45. Mar 26 13:53:12 an-node01 kernel: block drbd1: disk( Attaching -> Outdated )
  46. Mar 26 13:53:13 an-node01 kernel: block drbd2: Starting worker thread (from cqueue/1 [140])
  47. Mar 26 13:53:13 an-node01 kernel: block drbd2: disk( Diskless -> Attaching )
  48. Mar 26 13:53:13 an-node01 kernel: block drbd2: Found 2 transactions (2 active extents) in activity log.
  49. Mar 26 13:53:13 an-node01 kernel: block drbd2: Method to ensure write ordering: barrier
  50. Mar 26 13:53:13 an-node01 kernel: block drbd2: max_segment_size ( = BIO size ) = 32768
  51. Mar 26 13:53:13 an-node01 kernel: block drbd2: drbd_bm_resize called with capacity == 195328232
  52. Mar 26 13:53:13 an-node01 kernel: block drbd2: resync bitmap: bits=24416029 words=381501
  53. Mar 26 13:53:13 an-node01 kernel: block drbd2: size = 93 GB (97664116 KB)
  54. Mar 26 13:53:13 an-node01 kernel: block drbd2: recounting of set bits took additional 0 jiffies
  55. Mar 26 13:53:13 an-node01 kernel: block drbd2: 0 KB (0 bits) marked out-of-sync by on disk bit-map.
  56. Mar 26 13:53:13 an-node01 kernel: block drbd2: disk( Attaching -> Outdated )
  57. Mar 26 13:53:13 an-node01 kernel: block drbd0: conn( StandAlone -> Unconnected )
  58. Mar 26 13:53:13 an-node01 kernel: block drbd0: Starting receiver thread (from drbd0_worker [7002])
  59. Mar 26 13:53:13 an-node01 kernel: block drbd0: receiver (re)started
  60. Mar 26 13:53:13 an-node01 kernel: block drbd0: conn( Unconnected -> WFConnection )
  61. Mar 26 13:53:13 an-node01 kernel: block drbd1: conn( StandAlone -> Unconnected )
  62. Mar 26 13:53:13 an-node01 kernel: block drbd1: Starting receiver thread (from drbd1_worker [7015])
  63. Mar 26 13:53:13 an-node01 kernel: block drbd1: receiver (re)started
  64. Mar 26 13:53:13 an-node01 kernel: block drbd1: conn( Unconnected -> WFConnection )
  65. Mar 26 13:53:13 an-node01 kernel: block drbd2: conn( StandAlone -> Unconnected )
  66. Mar 26 13:53:13 an-node01 kernel: block drbd2: Starting receiver thread (from drbd2_worker [7033])
  67. Mar 26 13:53:13 an-node01 kernel: block drbd2: receiver (re)started
  68. Mar 26 13:53:13 an-node01 kernel: block drbd2: conn( Unconnected -> WFConnection )
  69. Mar 26 13:53:13 an-node01 kernel: block drbd0: Handshake successful: Agreed network protocol version 94
  70. Mar 26 13:53:13 an-node01 kernel: block drbd0: conn( WFConnection -> WFReportParams )
  71. Mar 26 13:53:13 an-node01 kernel: block drbd0: Starting asender thread (from drbd0_receiver [7046])
  72. Mar 26 13:53:13 an-node01 kernel: block drbd1: Handshake successful: Agreed network protocol version 94
  73. Mar 26 13:53:13 an-node01 kernel: block drbd1: conn( WFConnection -> WFReportParams )
  74. Mar 26 13:53:13 an-node01 kernel: block drbd1: Starting asender thread (from drbd1_receiver [7049])
  75. Mar 26 13:53:13 an-node01 kernel: block drbd1: data-integrity-alg: <not-used>
  76. Mar 26 13:53:13 an-node01 kernel: block drbd0: data-integrity-alg: <not-used>
  77. Mar 26 13:53:13 an-node01 kernel: block drbd0: drbd_sync_handshake:
  78. Mar 26 13:53:13 an-node01 kernel: block drbd0: self 89BB46E0E8ABF0DC:0000000000000000:C1E02A391DE46EEB:EE74FCBAA78B3DE5 bits:0 flags:0
  79. Mar 26 13:53:13 an-node01 kernel: block drbd0: peer 181FA55222ADFAE8:89BB46E0E8ABF0DD:C1E02A391DE46EEA:EE74FCBAA78B3DE5 bits:0 flags:0
  80. Mar 26 13:53:13 an-node01 kernel: block drbd0: uuid_compare()=-1 by rule 50
  81. Mar 26 13:53:13 an-node01 kernel: block drbd0: peer( Unknown -> Secondary ) conn( WFReportParams -> WFBitMapT ) pdsk( DUnknown -> UpToDate )
  82. Mar 26 13:53:13 an-node01 kernel: block drbd1: drbd_sync_handshake:
  83. Mar 26 13:53:13 an-node01 kernel: block drbd1: self B2F543153FED82C4:0000000000000000:35602E8D749C7F2D:8BD6DFADCCE2534B bits:0 flags:0
  84. Mar 26 13:53:13 an-node01 kernel: block drbd1: peer 9AF928B7B999B96E:B2F543153FED82C5:35602E8D749C7F2C:8BD6DFADCCE2534B bits:0 flags:0
  85. Mar 26 13:53:13 an-node01 kernel: block drbd1: uuid_compare()=-1 by rule 50
  86. Mar 26 13:53:13 an-node01 kernel: block drbd1: peer( Unknown -> Secondary ) conn( WFReportParams -> WFBitMapT ) pdsk( DUnknown -> UpToDate )
  87. Mar 26 13:53:13 an-node01 kernel: block drbd2: Handshake successful: Agreed network protocol version 94
  88. Mar 26 13:53:13 an-node01 kernel: block drbd2: conn( WFConnection -> WFReportParams )
  89. Mar 26 13:53:13 an-node01 kernel: block drbd2: Starting asender thread (from drbd2_receiver [7051])
  90. Mar 26 13:53:13 an-node01 kernel: block drbd2: data-integrity-alg: <not-used>
  91. Mar 26 13:53:13 an-node01 kernel: block drbd2: drbd_sync_handshake:
  92. Mar 26 13:53:13 an-node01 kernel: block drbd2: self 2826164183A66266:0000000000000000:78C298F940364E07:9A3F783210285E37 bits:0 flags:0
  93. Mar 26 13:53:13 an-node01 kernel: block drbd2: peer 0EED6DFC332304C8:2826164183A66267:78C298F940364E07:9A3F783210285E37 bits:0 flags:0
  94. Mar 26 13:53:13 an-node01 kernel: block drbd2: uuid_compare()=-1 by rule 50
  95. Mar 26 13:53:13 an-node01 kernel: block drbd2: peer( Unknown -> Secondary ) conn( WFReportParams -> WFBitMapT ) pdsk( DUnknown -> UpToDate )
  96. Mar 26 13:53:13 an-node01 kernel: block drbd0: conn( WFBitMapT -> WFSyncUUID )
  97. Mar 26 13:53:13 an-node01 kernel: block drbd0: helper command: /sbin/drbdadm before-resync-target minor-0
  98. Mar 26 13:53:13 an-node01 kernel: block drbd0: helper command: /sbin/drbdadm before-resync-target minor-0 exit code 0 (0x0)
  99. Mar 26 13:53:13 an-node01 kernel: block drbd0: conn( WFSyncUUID -> SyncTarget ) disk( Outdated -> Inconsistent )
  100. Mar 26 13:53:13 an-node01 kernel: block drbd0: Began resync as SyncTarget (will sync 0 KB [0 bits set]).
  101. Mar 26 13:53:13 an-node01 kernel: block drbd0: Resync done (total 1 sec; paused 0 sec; 0 K/sec)
  102. Mar 26 13:53:13 an-node01 kernel: block drbd0: conn( SyncTarget -> Connected ) disk( Inconsistent -> UpToDate )
  103. Mar 26 13:53:13 an-node01 kernel: block drbd0: helper command: /sbin/drbdadm after-resync-target minor-0
  104. Mar 26 13:53:13 an-node01 kernel: block drbd0: helper command: /sbin/drbdadm after-resync-target minor-0 exit code 0 (0x0)
  105. Mar 26 13:53:13 an-node01 kernel: block drbd0: Connected in w_make_resync_request
  106. Mar 26 13:53:13 an-node01 kernel: block drbd0: peer( Secondary -> Primary )
  107. Mar 26 13:53:14 an-node01 kernel: block drbd2: conn( WFBitMapT -> WFSyncUUID )
  108. Mar 26 13:53:14 an-node01 kernel: block drbd2: helper command: /sbin/drbdadm before-resync-target minor-2
  109. Mar 26 13:53:14 an-node01 kernel: block drbd2: helper command: /sbin/drbdadm before-resync-target minor-2 exit code 0 (0x0)
  110. Mar 26 13:53:14 an-node01 kernel: block drbd2: conn( WFSyncUUID -> SyncTarget ) disk( Outdated -> Inconsistent )
  111. Mar 26 13:53:14 an-node01 kernel: block drbd2: Began resync as SyncTarget (will sync 0 KB [0 bits set]).
  112. Mar 26 13:53:14 an-node01 kernel: block drbd2: Resync done (total 1 sec; paused 0 sec; 0 K/sec)
  113. Mar 26 13:53:14 an-node01 kernel: block drbd2: conn( SyncTarget -> Connected ) disk( Inconsistent -> UpToDate )
  114. Mar 26 13:53:14 an-node01 kernel: block drbd2: helper command: /sbin/drbdadm after-resync-target minor-2
  115. Mar 26 13:53:14 an-node01 kernel: block drbd2: helper command: /sbin/drbdadm after-resync-target minor-2 exit code 0 (0x0)
  116. Mar 26 13:53:14 an-node01 kernel: block drbd2: Connected in w_make_resync_request
  117. Mar 26 13:53:14 an-node01 kernel: block drbd0: role( Secondary -> Primary )
  118. Mar 26 13:53:14 an-node01 kernel: block drbd1: conn( WFBitMapT -> WFSyncUUID )
  119. Mar 26 13:53:14 an-node01 kernel: block drbd1: peer( Secondary -> Primary )
  120. Mar 26 13:53:14 an-node01 kernel: block drbd2: peer( Secondary -> Primary )
  121. Mar 26 13:53:14 an-node01 kernel: block drbd1: helper command: /sbin/drbdadm before-resync-target minor-1
  122. Mar 26 13:53:14 an-node01 kernel: block drbd1: role( Secondary -> Primary )
  123. Mar 26 13:53:14 an-node01 kernel: block drbd1: helper command: /sbin/drbdadm before-resync-target minor-1 exit code 0 (0x0)
  124. Mar 26 13:53:14 an-node01 kernel: block drbd1: conn( WFSyncUUID -> SyncTarget ) disk( Outdated -> Inconsistent )
  125. Mar 26 13:53:14 an-node01 kernel: block drbd1: Began resync as SyncTarget (will sync 0 KB [0 bits set]).
  126. Mar 26 13:53:14 an-node01 kernel: block drbd1: Resync done (total 1 sec; paused 0 sec; 0 K/sec)
  127. Mar 26 13:53:14 an-node01 kernel: block drbd1: conn( SyncTarget -> Connected ) disk( Inconsistent -> UpToDate )
  128. Mar 26 13:53:14 an-node01 kernel: block drbd1: helper command: /sbin/drbdadm after-resync-target minor-1
  129. Mar 26 13:53:14 an-node01 kernel: block drbd1: helper command: /sbin/drbdadm after-resync-target minor-1 exit code 0 (0x0)
  130. Mar 26 13:53:14 an-node01 kernel: block drbd2: role( Secondary -> Primary )
  131. Mar 26 13:53:14 an-node01 clurgmgrd: [6436]: <info> Executing /etc/init.d/clvmd start
  132. Mar 26 13:53:15 an-node01 clvmd: Cluster LVM daemon started - connected to CMAN
  133. Mar 26 13:53:16 an-node01 kernel: GFS2: fsid=: Trying to join cluster "lock_dlm", "an-cluster:xen_shared"
  134. Mar 26 13:53:16 an-node01 kernel: GFS2: fsid=an-cluster:xen_shared.1: Joined cluster. Now mounting FS...
  135. Mar 26 13:53:16 an-node01 kernel: GFS2: fsid=an-cluster:xen_shared.1: jid=1, already locked for use
  136. Mar 26 13:53:16 an-node01 kernel: GFS2: fsid=an-cluster:xen_shared.1: jid=1: Looking at journal...
  137. Mar 26 13:53:16 an-node01 kernel: GFS2: fsid=an-cluster:xen_shared.1: jid=1: Done
  138. Mar 26 13:53:16 an-node01 clurgmgrd[6436]: <notice> Service service:an1_storage started
  139.  
  140.  
  141. Mar 26 13:53:38 an-node01 clurgmgrd[6436]: <notice> Stopping service service:an1_storage
  142. Mar 26 13:53:39 an-node01 clurgmgrd: [6436]: <info> Executing /etc/init.d/clvmd stop
  143. Mar 26 13:53:39 an-node01 clurgmgrd: [6436]: <err> script:clvmd: stop of /etc/init.d/clvmd failed (returned 5)
  144. Mar 26 13:53:39 an-node01 clurgmgrd[6436]: <notice> stop on script "clvmd" returned 1 (generic error)
  145. Mar 26 13:53:39 an-node01 clurgmgrd: [6436]: <info> Executing /etc/init.d/drbd stop
  146. Mar 26 13:53:39 an-node01 kernel: block drbd0: State change failed: Device is held open by someone
  147. Mar 26 13:53:39 an-node01 kernel: block drbd0: state = { cs:Connected ro:Primary/Primary ds:UpToDate/UpToDate r--- }
  148. Mar 26 13:53:39 an-node01 kernel: block drbd0: wanted = { cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate r--- }
  149. Mar 26 13:53:39 an-node01 kernel: block drbd1: role( Primary -> Secondary )
  150. Mar 26 13:53:39 an-node01 kernel: block drbd1: Requested state change failed by peer: Refusing to be Primary while peer is not outdated (-7)
  151. Mar 26 13:53:39 an-node01 kernel: block drbd1: peer( Primary -> Unknown ) conn( Connected -> Disconnecting ) disk( UpToDate -> Outdated ) pdsk( UpToDate -> DUnknown )
  152. Mar 26 13:53:39 an-node01 kernel: block drbd1: short read expecting header on sock: r=-512
  153. Mar 26 13:53:39 an-node01 kernel: block drbd1: asender terminated
  154. Mar 26 13:53:39 an-node01 kernel: block drbd1: Terminating asender thread
  155. Mar 26 13:53:39 an-node01 kernel: block drbd1: Connection closed
  156. Mar 26 13:53:39 an-node01 kernel: block drbd1: conn( Disconnecting -> StandAlone )
  157. Mar 26 13:53:39 an-node01 kernel: block drbd1: receiver terminated
  158. Mar 26 13:53:39 an-node01 kernel: block drbd1: Terminating receiver thread
  159. Mar 26 13:53:39 an-node01 kernel: block drbd1: disk( Outdated -> Diskless )
  160. Mar 26 13:53:39 an-node01 kernel: block drbd1: drbd_bm_resize called with capacity == 0
  161. Mar 26 13:53:39 an-node01 kernel: block drbd1: worker terminated
  162. Mar 26 13:53:39 an-node01 kernel: block drbd1: Terminating worker thread
  163. Mar 26 13:53:39 an-node01 kernel: block drbd2: role( Primary -> Secondary )
  164. Mar 26 13:53:39 an-node01 kernel: block drbd2: Requested state change failed by peer: Refusing to be Primary while peer is not outdated (-7)
  165. Mar 26 13:53:39 an-node01 kernel: block drbd2: peer( Primary -> Unknown ) conn( Connected -> Disconnecting ) disk( UpToDate -> Outdated ) pdsk( UpToDate -> DUnknown )
  166. Mar 26 13:53:39 an-node01 kernel: block drbd2: short read expecting header on sock: r=-512
  167. Mar 26 13:53:39 an-node01 kernel: block drbd2: asender terminated
  168. Mar 26 13:53:39 an-node01 kernel: block drbd2: Terminating asender thread
  169. Mar 26 13:53:39 an-node01 kernel: block drbd2: Connection closed
  170. Mar 26 13:53:39 an-node01 kernel: block drbd2: conn( Disconnecting -> StandAlone )
  171. Mar 26 13:53:39 an-node01 kernel: block drbd2: receiver terminated
  172. Mar 26 13:53:39 an-node01 kernel: block drbd2: Terminating receiver thread
  173. Mar 26 13:53:39 an-node01 kernel: block drbd2: disk( Outdated -> Diskless )
  174. Mar 26 13:53:39 an-node01 kernel: block drbd2: drbd_bm_resize called with capacity == 0
  175. Mar 26 13:53:39 an-node01 kernel: block drbd2: worker terminated
  176. Mar 26 13:53:39 an-node01 kernel: block drbd2: Terminating worker thread
  177. Mar 26 13:53:39 an-node01 clurgmgrd[6436]: <crit> #12: RG service:an1_storage failed to stop; intervention required
  178. Mar 26 13:53:39 an-node01 clurgmgrd[6436]: <notice> Service service:an1_storage is failed
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!