API tools faq
paste
digimer

an_node02 service start and stop

digimer
Mar 26th, 2011
235
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 16.73 KB | None | 0 0
raw download clone embed print report
  1. Mar 26 13:49:44 an-node02 kernel: dlm: Using TCP for communications
  2. Mar 26 13:49:44 an-node02 clurgmgrd[6630]: <notice> Resource Group Manager Starting
  3. Mar 26 13:49:44 an-node02 clurgmgrd[6630]: <info> Loading Service Data
  4. Mar 26 13:49:45 an-node02 kernel: dlm: connecting to 1
  5. Mar 26 13:49:45 an-node02 clurgmgrd[6630]: <info> Initializing Services
  6. Mar 26 13:49:45 an-node02 clurgmgrd: [6630]: <err> stop: Could not match /dev/drbd_sh1_vg0/xen_shared with a real device
  7. Mar 26 13:49:45 an-node02 clurgmgrd[6630]: <notice> stop on clusterfs "gfs2_xen_shared" returned 2 (invalid argument(s))
  8. Mar 26 13:49:45 an-node02 clurgmgrd: [6630]: <err> stop: Could not match /dev/drbd_sh1_vg0/xen_shared with a real device
  9. Mar 26 13:49:45 an-node02 clurgmgrd[6630]: <notice> stop on clusterfs "gfs2_xen_shared" returned 2 (invalid argument(s))
  10. Mar 26 13:49:45 an-node02 clurgmgrd: [6630]: <info> Executing /etc/init.d/clvmd stop
  11. Mar 26 13:49:46 an-node02 clurgmgrd: [6630]: <info> Executing /etc/init.d/drbd stop
  12. Mar 26 13:49:46 an-node02 clurgmgrd: [6630]: <info> Executing /etc/init.d/clvmd stop
  13. Mar 26 13:49:46 an-node02 clurgmgrd: [6630]: <info> Executing /etc/init.d/drbd stop
  14. Mar 26 13:49:46 an-node02 clurgmgrd[6630]: <info> Services Initialized
  15. Mar 26 13:49:46 an-node02 clurgmgrd[6630]: <info> State change: Local UP
  16. Mar 26 13:49:46 an-node02 clurgmgrd[6630]: <info> State change: an-node01.alteeve.com UP
  17. Mar 26 13:50:06 an-node02 clurgmgrd[6630]: <notice> Starting disabled service service:an2_storage
  18. Mar 26 13:50:06 an-node02 clurgmgrd: [6630]: <info> Executing /etc/init.d/drbd start
  19. Mar 26 13:50:06 an-node02 kernel: drbd: initialized. Version: 8.3.8 (api:88/proto:86-94)
  20. Mar 26 13:50:06 an-node02 kernel: drbd: GIT-hash: d78846e52224fd00562f7c225bcc25b2d422321d build by [email protected], 2010-06-04 08:04:27
  21. Mar 26 13:50:06 an-node02 kernel: drbd: registered as block device major 147
  22. Mar 26 13:50:06 an-node02 kernel: drbd: minor_table @ 0xffff8800c060b980
  23. Mar 26 13:50:07 an-node02 kernel: block drbd0: Starting worker thread (from cqueue/0 [138])
  24. Mar 26 13:50:07 an-node02 kernel: block drbd0: disk( Diskless -> Attaching )
  25. Mar 26 13:50:07 an-node02 kernel: klogd 1.4.1, ---------- state change ----------
  26. Mar 26 13:50:07 an-node02 kernel: block drbd0: Found 4 transactions (6 active extents) in activity log.
  27. Mar 26 13:50:07 an-node02 kernel: block drbd0: Method to ensure write ordering: barrier
  28. Mar 26 13:50:07 an-node02 kernel: block drbd0: max_segment_size ( = BIO size ) = 32768
  29. Mar 26 13:50:07 an-node02 kernel: block drbd0: drbd_bm_resize called with capacity == 19550368
  30. Mar 26 13:50:07 an-node02 kernel: block drbd0: resync bitmap: bits=2443796 words=38185
  31. Mar 26 13:50:07 an-node02 kernel: block drbd0: size = 9546 MB (9775184 KB)
  32. Mar 26 13:50:07 an-node02 kernel: block drbd0: recounting of set bits took additional 0 jiffies
  33. Mar 26 13:50:07 an-node02 kernel: block drbd0: 0 KB (0 bits) marked out-of-sync by on disk bit-map.
  34. Mar 26 13:50:07 an-node02 kernel: block drbd0: disk( Attaching -> UpToDate ) pdsk( DUnknown -> Outdated )
  35. Mar 26 13:50:07 an-node02 kernel: block drbd1: Starting worker thread (from cqueue/1 [139])
  36. Mar 26 13:50:07 an-node02 kernel: block drbd1: disk( Diskless -> Attaching )
  37. Mar 26 13:50:07 an-node02 kernel: block drbd1: Found 4 transactions (136 active extents) in activity log.
  38. Mar 26 13:50:07 an-node02 kernel: block drbd1: Method to ensure write ordering: barrier
  39. Mar 26 13:50:07 an-node02 kernel: block drbd1: max_segment_size ( = BIO size ) = 32768
  40. Mar 26 13:50:07 an-node02 kernel: block drbd1: drbd_bm_resize called with capacity == 195328232
  41. Mar 26 13:50:07 an-node02 kernel: block drbd1: resync bitmap: bits=24416029 words=381501
  42. Mar 26 13:50:07 an-node02 kernel: block drbd1: size = 93 GB (97664116 KB)
  43. Mar 26 13:50:07 an-node02 kernel: block drbd1: recounting of set bits took additional 0 jiffies
  44. Mar 26 13:50:07 an-node02 kernel: block drbd1: 0 KB (0 bits) marked out-of-sync by on disk bit-map.
  45. Mar 26 13:50:07 an-node02 kernel: block drbd1: disk( Attaching -> UpToDate ) pdsk( DUnknown -> Outdated )
  46. Mar 26 13:50:07 an-node02 kernel: block drbd2: Starting worker thread (from cqueue/1 [139])
  47. Mar 26 13:50:07 an-node02 kernel: block drbd2: disk( Diskless -> Attaching )
  48. Mar 26 13:50:07 an-node02 kernel: block drbd2: Found 1 transactions (1 active extents) in activity log.
  49. Mar 26 13:50:07 an-node02 kernel: block drbd2: Method to ensure write ordering: barrier
  50. Mar 26 13:50:07 an-node02 kernel: block drbd2: max_segment_size ( = BIO size ) = 32768
  51. Mar 26 13:50:07 an-node02 kernel: block drbd2: drbd_bm_resize called with capacity == 195328232
  52. Mar 26 13:50:07 an-node02 kernel: block drbd2: resync bitmap: bits=24416029 words=381501
  53. Mar 26 13:50:07 an-node02 kernel: block drbd2: size = 93 GB (97664116 KB)
  54. Mar 26 13:50:07 an-node02 kernel: block drbd2: recounting of set bits took additional 1 jiffies
  55. Mar 26 13:50:07 an-node02 kernel: block drbd2: 0 KB (0 bits) marked out-of-sync by on disk bit-map.
  56. Mar 26 13:50:07 an-node02 kernel: block drbd2: disk( Attaching -> UpToDate ) pdsk( DUnknown -> Outdated )
  57. Mar 26 13:50:07 an-node02 kernel: block drbd0: conn( StandAlone -> Unconnected )
  58. Mar 26 13:50:07 an-node02 kernel: block drbd0: Starting receiver thread (from drbd0_worker [7195])
  59. Mar 26 13:50:07 an-node02 kernel: block drbd0: receiver (re)started
  60. Mar 26 13:50:07 an-node02 kernel: block drbd0: conn( Unconnected -> WFConnection )
  61. Mar 26 13:50:07 an-node02 kernel: block drbd1: conn( StandAlone -> Unconnected )
  62. Mar 26 13:50:07 an-node02 kernel: block drbd1: Starting receiver thread (from drbd1_worker [7208])
  63. Mar 26 13:50:07 an-node02 kernel: block drbd1: receiver (re)started
  64. Mar 26 13:50:07 an-node02 kernel: block drbd1: conn( Unconnected -> WFConnection )
  65. Mar 26 13:50:07 an-node02 kernel: block drbd2: conn( StandAlone -> Unconnected )
  66. Mar 26 13:50:07 an-node02 kernel: block drbd2: Starting receiver thread (from drbd2_worker [7223])
  67. Mar 26 13:50:07 an-node02 kernel: block drbd2: receiver (re)started
  68. Mar 26 13:50:07 an-node02 kernel: block drbd2: conn( Unconnected -> WFConnection )
  69. Mar 26 13:50:08 an-node02 kernel: block drbd1: Handshake successful: Agreed network protocol version 94
  70. Mar 26 13:50:08 an-node02 kernel: block drbd1: conn( WFConnection -> WFReportParams )
  71. Mar 26 13:50:08 an-node02 kernel: block drbd1: Starting asender thread (from drbd1_receiver [7241])
  72. Mar 26 13:50:08 an-node02 kernel: block drbd0: Handshake successful: Agreed network protocol version 94
  73. Mar 26 13:50:08 an-node02 kernel: block drbd0: conn( WFConnection -> WFReportParams )
  74. Mar 26 13:50:08 an-node02 kernel: block drbd0: Starting asender thread (from drbd0_receiver [7238])
  75. Mar 26 13:50:08 an-node02 kernel: block drbd0: data-integrity-alg: <not-used>
  76. Mar 26 13:50:08 an-node02 kernel: block drbd0: drbd_sync_handshake:
  77. Mar 26 13:50:08 an-node02 kernel: block drbd0: self 181FA55222ADFAE8:89BB46E0E8ABF0DD:C1E02A391DE46EEA:EE74FCBAA78B3DE5 bits:0 flags:0
  78. Mar 26 13:50:08 an-node02 kernel: block drbd0: peer 89BB46E0E8ABF0DC:0000000000000000:C1E02A391DE46EEB:EE74FCBAA78B3DE5 bits:0 flags:0
  79. Mar 26 13:50:08 an-node02 kernel: block drbd0: uuid_compare()=1 by rule 70
  80. Mar 26 13:50:08 an-node02 kernel: block drbd0: peer( Unknown -> Secondary ) conn( WFReportParams -> WFBitMapS )
  81. Mar 26 13:50:08 an-node02 kernel: block drbd1: data-integrity-alg: <not-used>
  82. Mar 26 13:50:08 an-node02 kernel: block drbd1: drbd_sync_handshake:
  83. Mar 26 13:50:08 an-node02 kernel: block drbd1: self 9AF928B7B999B96E:B2F543153FED82C5:35602E8D749C7F2C:8BD6DFADCCE2534B bits:0 flags:0
  84. Mar 26 13:50:08 an-node02 kernel: block drbd1: peer B2F543153FED82C4:0000000000000000:35602E8D749C7F2D:8BD6DFADCCE2534B bits:0 flags:0
  85. Mar 26 13:50:08 an-node02 kernel: block drbd1: uuid_compare()=1 by rule 70
  86. Mar 26 13:50:08 an-node02 kernel: block drbd1: peer( Unknown -> Secondary ) conn( WFReportParams -> WFBitMapS )
  87. Mar 26 13:50:08 an-node02 kernel: block drbd2: Handshake successful: Agreed network protocol version 94
  88. Mar 26 13:50:08 an-node02 kernel: block drbd2: conn( WFConnection -> WFReportParams )
  89. Mar 26 13:50:08 an-node02 kernel: block drbd2: Starting asender thread (from drbd2_receiver [7243])
  90. Mar 26 13:50:08 an-node02 kernel: block drbd2: data-integrity-alg: <not-used>
  91. Mar 26 13:50:08 an-node02 kernel: block drbd2: drbd_sync_handshake:
  92. Mar 26 13:50:08 an-node02 kernel: block drbd2: self 0EED6DFC332304C8:2826164183A66267:78C298F940364E07:9A3F783210285E37 bits:0 flags:0
  93. Mar 26 13:50:08 an-node02 kernel: block drbd2: peer 2826164183A66266:0000000000000000:78C298F940364E07:9A3F783210285E37 bits:0 flags:0
  94. Mar 26 13:50:08 an-node02 kernel: block drbd2: uuid_compare()=1 by rule 70
  95. Mar 26 13:50:08 an-node02 kernel: block drbd2: peer( Unknown -> Secondary ) conn( WFReportParams -> WFBitMapS )
  96. Mar 26 13:50:08 an-node02 kernel: block drbd0: conn( WFBitMapS -> SyncSource ) pdsk( Outdated -> Inconsistent )
  97. Mar 26 13:50:08 an-node02 kernel: block drbd0: Began resync as SyncSource (will sync 0 KB [0 bits set]).
  98. Mar 26 13:50:08 an-node02 kernel: block drbd0: Resync done (total 1 sec; paused 0 sec; 0 K/sec)
  99. Mar 26 13:50:08 an-node02 kernel: block drbd0: conn( SyncSource -> Connected ) pdsk( Inconsistent -> UpToDate )
  100. Mar 26 13:50:08 an-node02 kernel: block drbd0: role( Secondary -> Primary )
  101. Mar 26 13:50:08 an-node02 kernel: block drbd2: conn( WFBitMapS -> SyncSource ) pdsk( Outdated -> Inconsistent )
  102. Mar 26 13:50:08 an-node02 kernel: block drbd2: Began resync as SyncSource (will sync 0 KB [0 bits set]).
  103. Mar 26 13:50:08 an-node02 kernel: block drbd2: Resync done (total 1 sec; paused 0 sec; 0 K/sec)
  104. Mar 26 13:50:08 an-node02 kernel: block drbd2: conn( SyncSource -> Connected ) pdsk( Inconsistent -> UpToDate )
  105. Mar 26 13:50:09 an-node02 kernel: block drbd0: peer( Secondary -> Primary )
  106. Mar 26 13:50:09 an-node02 kernel: block drbd1: role( Secondary -> Primary )
  107. Mar 26 13:50:09 an-node02 kernel: block drbd2: role( Secondary -> Primary )
  108. Mar 26 13:50:09 an-node02 kernel: block drbd1: conn( WFBitMapS -> SyncSource ) pdsk( Outdated -> Inconsistent )
  109. Mar 26 13:50:09 an-node02 kernel: block drbd1: Began resync as SyncSource (will sync 0 KB [0 bits set]).
  110. Mar 26 13:50:09 an-node02 kernel: block drbd1: Resync done (total 1 sec; paused 0 sec; 0 K/sec)
  111. Mar 26 13:50:09 an-node02 kernel: block drbd1: conn( SyncSource -> Connected ) pdsk( Inconsistent -> UpToDate )
  112. Mar 26 13:50:09 an-node02 clurgmgrd: [6630]: <info> Executing /etc/init.d/clvmd start
  113. Mar 26 13:50:09 an-node02 kernel: block drbd1: peer( Secondary -> Primary )
  114. Mar 26 13:50:09 an-node02 kernel: block drbd2: peer( Secondary -> Primary )
  115. Mar 26 13:50:09 an-node02 clvmd: Cluster LVM daemon started - connected to CMAN
  116. Mar 26 13:50:09 an-node02 kernel: GFS2: fsid=: Trying to join cluster "lock_dlm", "an-cluster:xen_shared"
  117. Mar 26 13:50:09 an-node02 kernel: GFS2: fsid=an-cluster:xen_shared.0: Joined cluster. Now mounting FS...
  118. Mar 26 13:50:10 an-node02 kernel: GFS2: fsid=an-cluster:xen_shared.0: jid=0, already locked for use
  119. Mar 26 13:50:10 an-node02 kernel: GFS2: fsid=an-cluster:xen_shared.0: jid=0: Looking at journal...
  120. Mar 26 13:50:10 an-node02 kernel: GFS2: fsid=an-cluster:xen_shared.0: jid=0: Done
  121. Mar 26 13:50:10 an-node02 kernel: GFS2: fsid=an-cluster:xen_shared.0: jid=1: Trying to acquire journal lock...
  122. Mar 26 13:50:10 an-node02 kernel: GFS2: fsid=an-cluster:xen_shared.0: jid=1: Looking at journal...
  123. Mar 26 13:50:10 an-node02 kernel: GFS2: fsid=an-cluster:xen_shared.0: jid=1: Done
  124. Mar 26 13:50:10 an-node02 clurgmgrd[6630]: <notice> Service service:an2_storage started
  125.  
  126.  
  127. Mar 26 13:50:34 an-node02 kernel: block drbd1: peer( Primary -> Secondary )
  128. Mar 26 13:50:34 an-node02 kernel: block drbd1: State change failed: Refusing to be Primary while peer is not outdated
  129. Mar 26 13:50:34 an-node02 kernel: block drbd1: state = { cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate r--- }
  130. Mar 26 13:50:34 an-node02 kernel: block drbd1: wanted = { cs:TearDown ro:Primary/Unknown ds:UpToDate/DUnknown s--- }
  131. Mar 26 13:50:34 an-node02 kernel: block drbd1: peer( Secondary -> Unknown ) conn( Connected -> TearDown ) pdsk( UpToDate -> Outdated )
  132. Mar 26 13:50:34 an-node02 kernel: block drbd1: Creating new current UUID
  133. Mar 26 13:50:34 an-node02 kernel: block drbd1: meta connection shut down by peer.
  134. Mar 26 13:50:34 an-node02 kernel: block drbd1: asender terminated
  135. Mar 26 13:50:34 an-node02 kernel: block drbd1: Terminating asender thread
  136. Mar 26 13:50:34 an-node02 kernel: block drbd1: Connection closed
  137. Mar 26 13:50:34 an-node02 kernel: block drbd1: conn( TearDown -> Unconnected )
  138. Mar 26 13:50:34 an-node02 kernel: block drbd1: receiver terminated
  139. Mar 26 13:50:34 an-node02 kernel: block drbd1: Restarting receiver thread
  140. Mar 26 13:50:34 an-node02 kernel: block drbd1: receiver (re)started
  141. Mar 26 13:50:34 an-node02 kernel: block drbd1: conn( Unconnected -> WFConnection )
  142. Mar 26 13:50:34 an-node02 kernel: block drbd2: peer( Primary -> Secondary )
  143. Mar 26 13:50:34 an-node02 kernel: block drbd2: State change failed: Refusing to be Primary while peer is not outdated
  144. Mar 26 13:50:34 an-node02 kernel: block drbd2: state = { cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate r--- }
  145. Mar 26 13:50:34 an-node02 kernel: block drbd2: wanted = { cs:TearDown ro:Primary/Unknown ds:UpToDate/DUnknown s--- }
  146. Mar 26 13:50:34 an-node02 kernel: block drbd2: peer( Secondary -> Unknown ) conn( Connected -> TearDown ) pdsk( UpToDate -> Outdated )
  147. Mar 26 13:50:34 an-node02 kernel: block drbd2: Creating new current UUID
  148. Mar 26 13:50:34 an-node02 kernel: block drbd2: meta connection shut down by peer.
  149. Mar 26 13:50:34 an-node02 kernel: block drbd2: asender terminated
  150. Mar 26 13:50:34 an-node02 kernel: block drbd2: Terminating asender thread
  151. Mar 26 13:50:34 an-node02 clurgmgrd[6630]: <notice> Stopping service service:an2_storage
  152. Mar 26 13:50:34 an-node02 kernel: block drbd2: Connection closed
  153. Mar 26 13:50:34 an-node02 kernel: block drbd2: conn( TearDown -> Unconnected )
  154. Mar 26 13:50:34 an-node02 kernel: block drbd2: receiver terminated
  155. Mar 26 13:50:34 an-node02 kernel: block drbd2: Restarting receiver thread
  156. Mar 26 13:50:34 an-node02 kernel: block drbd2: receiver (re)started
  157. Mar 26 13:50:34 an-node02 kernel: block drbd2: conn( Unconnected -> WFConnection )
  158. Mar 26 13:50:34 an-node02 clurgmgrd: [6630]: <info> Executing /etc/init.d/clvmd stop
  159. Mar 26 13:50:34 an-node02 clurgmgrd: [6630]: <err> script:clvmd: stop of /etc/init.d/clvmd failed (returned 5)
  160. Mar 26 13:50:34 an-node02 clurgmgrd[6630]: <notice> stop on script "clvmd" returned 1 (generic error)
  161. Mar 26 13:50:34 an-node02 clurgmgrd: [6630]: <info> Executing /etc/init.d/drbd stop
  162. Mar 26 13:50:35 an-node02 kernel: block drbd0: State change failed: Device is held open by someone
  163. Mar 26 13:50:35 an-node02 kernel: block drbd0: state = { cs:Connected ro:Primary/Primary ds:UpToDate/UpToDate r--- }
  164. Mar 26 13:50:35 an-node02 kernel: block drbd0: wanted = { cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate r--- }
  165. Mar 26 13:50:35 an-node02 kernel: block drbd1: role( Primary -> Secondary )
  166. Mar 26 13:50:35 an-node02 kernel: block drbd1: conn( WFConnection -> Disconnecting )
  167. Mar 26 13:50:35 an-node02 kernel: block drbd1: Discarding network configuration.
  168. Mar 26 13:50:35 an-node02 kernel: block drbd1: Connection closed
  169. Mar 26 13:50:35 an-node02 kernel: block drbd1: conn( Disconnecting -> StandAlone )
  170. Mar 26 13:50:35 an-node02 kernel: block drbd1: receiver terminated
  171. Mar 26 13:50:35 an-node02 kernel: block drbd1: Terminating receiver thread
  172. Mar 26 13:50:35 an-node02 kernel: block drbd1: disk( UpToDate -> Diskless ) pdsk( Outdated -> DUnknown )
  173. Mar 26 13:50:35 an-node02 kernel: block drbd1: drbd_bm_resize called with capacity == 0
  174. Mar 26 13:50:35 an-node02 kernel: block drbd1: worker terminated
  175. Mar 26 13:50:35 an-node02 kernel: block drbd1: Terminating worker thread
  176. Mar 26 13:50:35 an-node02 kernel: block drbd2: role( Primary -> Secondary )
  177. Mar 26 13:50:35 an-node02 kernel: block drbd2: conn( WFConnection -> Disconnecting )
  178. Mar 26 13:50:35 an-node02 kernel: block drbd2: Discarding network configuration.
  179. Mar 26 13:50:35 an-node02 kernel: block drbd2: Connection closed
  180. Mar 26 13:50:35 an-node02 kernel: block drbd2: conn( Disconnecting -> StandAlone )
  181. Mar 26 13:50:35 an-node02 kernel: block drbd2: receiver terminated
  182. Mar 26 13:50:35 an-node02 kernel: block drbd2: Terminating receiver thread
  183. Mar 26 13:50:35 an-node02 kernel: block drbd2: disk( UpToDate -> Diskless ) pdsk( Outdated -> DUnknown )
  184. Mar 26 13:50:35 an-node02 kernel: block drbd2: drbd_bm_resize called with capacity == 0
  185. Mar 26 13:50:35 an-node02 kernel: block drbd2: worker terminated
  186. Mar 26 13:50:35 an-node02 kernel: block drbd2: Terminating worker thread
  187. Mar 26 13:50:35 an-node02 clurgmgrd[6630]: <crit> #12: RG service:an2_storage failed to stop; intervention required
  188. Mar 26 13:50:35 an-node02 clurgmgrd[6630]: <notice> Service service:an2_storage is failed
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!