Guest User

[CASE-7] node3 log

a guest
Feb 26th, 2016
68
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. Node3
  2.  
  3. Feb 26 18:26:32 drbd9-03 kernel: drbd r0 drbd9-01: conn( StandAlone -> Unconnected )
  4. Feb 26 18:26:32 drbd9-03 kernel: drbd r0 drbd9-01: Starting receiver thread (from drbd_w_r0 [2971])
  5. Feb 26 18:26:32 drbd9-03 kernel: drbd r0 drbd9-02: conn( StandAlone -> Unconnected )
  6. Feb 26 18:26:32 drbd9-03 kernel: drbd r0 drbd9-02: Starting receiver thread (from drbd_w_r0 [2971])
  7. Feb 26 18:26:32 drbd9-03 kernel: drbd r0 drbd9-01: conn( Unconnected -> Connecting )
  8. Feb 26 18:26:32 drbd9-03 kernel: drbd r0 drbd9-04: conn( StandAlone -> Unconnected )
  9. Feb 26 18:26:32 drbd9-03 kernel: drbd r0 drbd9-04: Starting receiver thread (from drbd_w_r0 [2971])
  10. Feb 26 18:26:32 drbd9-03 kernel: drbd r0 drbd9-02: conn( Unconnected -> Connecting )
  11. Feb 26 18:26:32 drbd9-03 kernel: drbd r0 drbd9-05: conn( StandAlone -> Unconnected )
  12. Feb 26 18:26:32 drbd9-03 kernel: drbd r0 drbd9-05: Starting receiver thread (from drbd_w_r0 [2971])
  13. Feb 26 18:26:32 drbd9-03 kernel: drbd r0 drbd9-04: conn( Unconnected -> Connecting )
  14. Feb 26 18:26:32 drbd9-03 kernel: drbd r0 drbd9-05: conn( Unconnected -> Connecting )
  15. Feb 26 18:26:36 drbd9-03 kernel: drbd r0 tcp:drbd9-01: Receiver busy; rejecting incoming connection
  16. Feb 26 18:26:41 drbd9-03 kernel: drbd r0 drbd9-02: Handshake successful: Agreed network protocol version 111
  17. Feb 26 18:26:41 drbd9-03 kernel: drbd r0 drbd9-02: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
  18. Feb 26 18:26:41 drbd9-03 kernel: drbd r0 drbd9-02: Starting ack_recv thread (from drbd_r_r0 [3062])
  19. Feb 26 18:26:41 drbd9-03 kernel: drbd r0 drbd9-02: Preparing remote state change 3687696482 (primary_nodes=0, weak_nodes=0)
  20. Feb 26 18:26:41 drbd9-03 kernel: drbd r0 drbd9-02: Committing remote state change 3687696482
  21. Feb 26 18:26:41 drbd9-03 kernel: drbd r0 drbd9-02: conn( Connecting -> Connected ) peer( Unknown -> Secondary )
  22. Feb 26 18:26:41 drbd9-03 kernel: drbd r0/0 drbd1: peer does not support WRITE_SAME
  23. Feb 26 18:26:41 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: unexpected
  24. Feb 26 18:26:41 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: In UUIDs from node 1 found equal UUID (351C9E61C6B06FB6) for nodes 3 4
  25. Feb 26 18:26:41 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: I have 2603DB27E7D1C272 for node_id=3
  26. Feb 26 18:26:41 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: I have 2603DB27E7D1C272 for node_id=4
  27. Feb 26 18:26:41 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: drbd_sync_handshake:
  28. Feb 26 18:26:41 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: self 15F5D43C9A0B7F26:0000000000000000:2603DB27E7D1C272:4CB7EEE2CCD6AD56 bits:0 flags:0
  29. Feb 26 18:26:41 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: peer 15F5D43C9A0B7F26:FFFFFFFFFFFFFFFF:351C9E61C6B06FB6:0000000000000000 bits:0 flags:20
  30. Feb 26 18:26:41 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: uuid_compare()=0 by rule 40
  31. Feb 26 18:26:41 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( DUnknown -> UpToDate ) repl( Off -> Established )
  32. Feb 26 18:26:42 drbd9-03 kernel: drbd r0 drbd9-01: Handshake successful: Agreed network protocol version 111
  33. Feb 26 18:26:42 drbd9-03 kernel: drbd r0 drbd9-01: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
  34. Feb 26 18:26:42 drbd9-03 kernel: drbd r0 drbd9-01: Starting ack_recv thread (from drbd_r_r0 [3060])
  35. Feb 26 18:26:42 drbd9-03 kernel: drbd r0 drbd9-01: sock was shut down by peer
  36. Feb 26 18:26:42 drbd9-03 kernel: drbd r0 drbd9-01: conn( Connecting -> BrokenPipe )
  37. Feb 26 18:26:42 drbd9-03 kernel: drbd r0 drbd9-01: ack_receiver terminated
  38. Feb 26 18:26:42 drbd9-03 kernel: drbd r0 drbd9-01: Terminating ack_recv thread
  39. Feb 26 18:26:42 drbd9-03 kernel: drbd r0 drbd9-01: Connection closed
  40. Feb 26 18:26:42 drbd9-03 kernel: drbd r0 drbd9-01: conn( BrokenPipe -> Unconnected )
  41. Feb 26 18:26:42 drbd9-03 kernel: drbd r0 drbd9-01: Restarting receiver thread
  42. Feb 26 18:26:42 drbd9-03 kernel: drbd r0 drbd9-01: conn( Unconnected -> Connecting )
  43. Feb 26 18:26:43 drbd9-03 kernel: drbd r0 drbd9-01: Handshake successful: Agreed network protocol version 111
  44. Feb 26 18:26:43 drbd9-03 kernel: drbd r0 drbd9-01: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
  45. Feb 26 18:26:43 drbd9-03 kernel: drbd r0 drbd9-01: Starting ack_recv thread (from drbd_r_r0 [3060])
  46. Feb 26 18:26:43 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.11
  47. Feb 26 18:26:43 drbd9-03 kernel: drbd r0 drbd9-01: meta connection shut down by peer.
  48. Feb 26 18:26:43 drbd9-03 kernel: drbd r0 drbd9-01: conn( Connecting -> NetworkFailure )
  49. Feb 26 18:26:43 drbd9-03 kernel: drbd r0 drbd9-01: ack_receiver terminated
  50. Feb 26 18:26:43 drbd9-03 kernel: drbd r0 drbd9-01: Terminating ack_recv thread
  51. Feb 26 18:26:43 drbd9-03 kernel: drbd r0 drbd9-01: Connection closed
  52. Feb 26 18:26:43 drbd9-03 kernel: drbd r0 drbd9-01: conn( NetworkFailure -> Unconnected )
  53. Feb 26 18:26:43 drbd9-03 kernel: drbd r0 drbd9-01: Restarting receiver thread
  54. Feb 26 18:26:43 drbd9-03 kernel: drbd r0 drbd9-01: conn( Unconnected -> Connecting )
  55. Feb 26 18:26:44 drbd9-03 kernel: drbd r0 drbd9-01: Handshake successful: Agreed network protocol version 111
  56. Feb 26 18:26:44 drbd9-03 kernel: drbd r0 drbd9-01: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
  57. Feb 26 18:26:44 drbd9-03 kernel: drbd r0 drbd9-01: Starting ack_recv thread (from drbd_r_r0 [3060])
  58. Feb 26 18:26:44 drbd9-03 kernel: drbd r0 drbd9-01: Preparing remote state change 2103542594 (primary_nodes=0, weak_nodes=0)
  59. Feb 26 18:26:44 drbd9-03 kernel: drbd r0 drbd9-01: Committing remote state change 2103542594
  60. Feb 26 18:26:44 drbd9-03 kernel: drbd r0 drbd9-01: conn( Connecting -> Connected ) peer( Unknown -> Secondary )
  61. Feb 26 18:26:44 drbd9-03 kernel: drbd r0/0 drbd1: peer does not support WRITE_SAME
  62. Feb 26 18:26:44 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: unexpected
  63. Feb 26 18:26:44 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: In UUIDs from node 0 found equal UUID (A9F0F625EF8BF07E) for nodes 3 4
  64. Feb 26 18:26:44 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: I have 2603DB27E7D1C272 for node_id=3
  65. Feb 26 18:26:44 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: I have 2603DB27E7D1C272 for node_id=4
  66. Feb 26 18:26:44 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: drbd_sync_handshake:
  67. Feb 26 18:26:44 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: self 15F5D43C9A0B7F26:0000000000000000:2603DB27E7D1C272:4CB7EEE2CCD6AD56 bits:0 flags:0
  68. Feb 26 18:26:44 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: peer 15F5D43C9A0B7F26:0000000000000000:351C9E61C6B06FB6:0000000000000000 bits:0 flags:120
  69. Feb 26 18:26:44 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: uuid_compare()=0 by rule 38
  70. Feb 26 18:26:44 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: pdsk( DUnknown -> UpToDate ) repl( Off -> Established )
  71. Feb 26 18:26:45 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2750423004 (2->0 496/16)
  72. Feb 26 18:26:45 drbd9-03 kernel: drbd r0: State change 2750423004: primary_nodes=0, weak_nodes=0
  73. Feb 26 18:26:45 drbd9-03 kernel: drbd r0: Committing cluster-wide state change 2750423004 (0ms)
  74. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-01: conn( Connected -> Disconnecting ) peer( Secondary -> Unknown )
  75. Feb 26 18:26:45 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
  76. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-01: ack_receiver terminated
  77. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-01: Terminating ack_recv thread
  78. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-01: Connection closed
  79. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-01: conn( Disconnecting -> StandAlone )
  80. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-01: Terminating receiver thread
  81. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.11
  82. Feb 26 18:26:45 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 620245561 (2->1 496/16)
  83. Feb 26 18:26:45 drbd9-03 kernel: drbd r0: State change 620245561: primary_nodes=0, weak_nodes=0
  84. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-02: Cluster is now split
  85. Feb 26 18:26:45 drbd9-03 kernel: drbd r0: Committing cluster-wide state change 620245561 (1ms)
  86. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-02: conn( Connected -> Disconnecting ) peer( Secondary -> Unknown )
  87. Feb 26 18:26:45 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
  88. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-02: ack_receiver terminated
  89. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-02: Terminating ack_recv thread
  90. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-02: Connection closed
  91. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-02: conn( Disconnecting -> StandAlone )
  92. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-02: Terminating receiver thread
  93. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-04: conn( Connecting -> Disconnecting )
  94. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-04: Connection closed
  95. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-04: conn( Disconnecting -> StandAlone )
  96. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-04: Terminating receiver thread
  97. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-05: conn( Connecting -> Disconnecting )
  98. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-05: Connection closed
  99. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-05: conn( Disconnecting -> StandAlone )
  100. Feb 26 18:26:45 drbd9-03 kernel: drbd r0 drbd9-05: Terminating receiver thread
  101. Feb 26 18:27:02 drbd9-03 kernel: drbd r0 drbd9-01: conn( StandAlone -> Unconnected )
  102. Feb 26 18:27:02 drbd9-03 kernel: drbd r0 drbd9-01: Starting receiver thread (from drbd_w_r0 [2971])
  103. Feb 26 18:27:02 drbd9-03 kernel: drbd r0 drbd9-02: conn( StandAlone -> Unconnected )
  104. Feb 26 18:27:02 drbd9-03 kernel: drbd r0 drbd9-02: Starting receiver thread (from drbd_w_r0 [2971])
  105. Feb 26 18:27:02 drbd9-03 kernel: drbd r0 drbd9-01: conn( Unconnected -> Connecting )
  106. Feb 26 18:27:02 drbd9-03 kernel: drbd r0 drbd9-04: conn( StandAlone -> Unconnected )
  107. Feb 26 18:27:02 drbd9-03 kernel: drbd r0 drbd9-04: Starting receiver thread (from drbd_w_r0 [2971])
  108. Feb 26 18:27:02 drbd9-03 kernel: drbd r0 drbd9-02: conn( Unconnected -> Connecting )
  109. Feb 26 18:27:02 drbd9-03 kernel: drbd r0 drbd9-05: conn( StandAlone -> Unconnected )
  110. Feb 26 18:27:02 drbd9-03 kernel: drbd r0 drbd9-05: Starting receiver thread (from drbd_w_r0 [2971])
  111. Feb 26 18:27:02 drbd9-03 kernel: drbd r0 drbd9-04: conn( Unconnected -> Connecting )
  112. Feb 26 18:27:02 drbd9-03 kernel: drbd r0 drbd9-05: conn( Unconnected -> Connecting )
  113. Feb 26 18:27:06 drbd9-03 kernel: drbd r0 tcp:drbd9-01: Receiver busy; rejecting incoming connection
  114. Feb 26 18:27:06 drbd9-03 kernel: drbd r0 tcp:drbd9-02: Receiver busy; rejecting incoming connection
  115. Feb 26 18:27:12 drbd9-03 kernel: drbd r0 drbd9-02: Handshake successful: Agreed network protocol version 111
  116. Feb 26 18:27:12 drbd9-03 kernel: drbd r0 drbd9-02: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
  117. Feb 26 18:27:12 drbd9-03 kernel: drbd r0 drbd9-02: Starting ack_recv thread (from drbd_r_r0 [3114])
  118. Feb 26 18:27:12 drbd9-03 kernel: drbd r0 drbd9-02: meta connection shut down by peer.
  119. Feb 26 18:27:12 drbd9-03 kernel: drbd r0 drbd9-02: conn( Connecting -> NetworkFailure )
  120. Feb 26 18:27:12 drbd9-03 kernel: drbd r0 drbd9-02: ack_receiver terminated
  121. Feb 26 18:27:12 drbd9-03 kernel: drbd r0 drbd9-02: Terminating ack_recv thread
  122. Feb 26 18:27:12 drbd9-03 kernel: drbd r0 drbd9-02: Connection closed
  123. Feb 26 18:27:12 drbd9-03 kernel: drbd r0 drbd9-02: conn( NetworkFailure -> Unconnected )
  124. Feb 26 18:27:12 drbd9-03 kernel: drbd r0 drbd9-02: Restarting receiver thread
  125. Feb 26 18:27:12 drbd9-03 kernel: drbd r0 drbd9-02: conn( Unconnected -> Connecting )
  126. Feb 26 18:27:13 drbd9-03 kernel: drbd r0 drbd9-02: Handshake successful: Agreed network protocol version 111
  127. Feb 26 18:27:13 drbd9-03 kernel: drbd r0 drbd9-02: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
  128. Feb 26 18:27:13 drbd9-03 kernel: drbd r0 drbd9-02: Starting ack_recv thread (from drbd_r_r0 [3114])
  129. Feb 26 18:27:13 drbd9-03 kernel: drbd r0 drbd9-02: meta connection shut down by peer.
  130. Feb 26 18:27:13 drbd9-03 kernel: drbd r0 drbd9-02: conn( Connecting -> NetworkFailure )
  131. Feb 26 18:27:13 drbd9-03 kernel: drbd r0 drbd9-02: ack_receiver terminated
  132. Feb 26 18:27:13 drbd9-03 kernel: drbd r0 drbd9-02: Terminating ack_recv thread
  133. Feb 26 18:27:13 drbd9-03 kernel: drbd r0 drbd9-02: sock was shut down by peer
  134. Feb 26 18:27:13 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  135. Feb 26 18:27:13 drbd9-03 kernel: drbd r0 drbd9-02: Connection closed
  136. Feb 26 18:27:13 drbd9-03 kernel: drbd r0 drbd9-02: conn( NetworkFailure -> Unconnected )
  137. Feb 26 18:27:13 drbd9-03 kernel: drbd r0 drbd9-02: Restarting receiver thread
  138. Feb 26 18:27:13 drbd9-03 kernel: drbd r0 drbd9-02: conn( Unconnected -> Connecting )
  139. Feb 26 18:27:14 drbd9-03 kernel: drbd r0 drbd9-02: Handshake successful: Agreed network protocol version 111
  140. Feb 26 18:27:14 drbd9-03 kernel: drbd r0 drbd9-02: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
  141. Feb 26 18:27:14 drbd9-03 kernel: drbd r0 drbd9-02: Starting ack_recv thread (from drbd_r_r0 [3114])
  142. Feb 26 18:27:14 drbd9-03 kernel: drbd r0 drbd9-02: Preparing remote state change 683403703 (primary_nodes=0, weak_nodes=0)
  143. Feb 26 18:27:14 drbd9-03 kernel: drbd r0 drbd9-02: Committing remote state change 683403703
  144. Feb 26 18:27:14 drbd9-03 kernel: drbd r0 drbd9-02: conn( Connecting -> Connected ) peer( Unknown -> Secondary )
  145. Feb 26 18:27:14 drbd9-03 kernel: drbd r0/0 drbd1: peer does not support WRITE_SAME
  146. Feb 26 18:27:14 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: unexpected
  147. Feb 26 18:27:14 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: In UUIDs from node 1 found equal UUID (351C9E61C6B06FB6) for nodes 3 4
  148. Feb 26 18:27:14 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: I have 2603DB27E7D1C272 for node_id=3
  149. Feb 26 18:27:14 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: I have 2603DB27E7D1C272 for node_id=4
  150. Feb 26 18:27:14 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: drbd_sync_handshake:
  151. Feb 26 18:27:14 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: self 15F5D43C9A0B7F26:0000000000000000:2603DB27E7D1C272:4CB7EEE2CCD6AD56 bits:0 flags:0
  152. Feb 26 18:27:14 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: peer 15F5D43C9A0B7F26:0000000000000000:351C9E61C6B06FB6:0000000000000000 bits:0 flags:120
  153. Feb 26 18:27:14 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: uuid_compare()=0 by rule 38
  154. Feb 26 18:27:14 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( DUnknown -> UpToDate ) repl( Off -> Established )
  155. Feb 26 18:27:15 drbd9-03 kernel: drbd r0 drbd9-01: Handshake successful: Agreed network protocol version 111
  156. Feb 26 18:27:15 drbd9-03 kernel: drbd r0 drbd9-01: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
  157. Feb 26 18:27:15 drbd9-03 kernel: drbd r0 drbd9-01: Starting ack_recv thread (from drbd_r_r0 [3112])
  158. Feb 26 18:27:15 drbd9-03 kernel: drbd r0 drbd9-01: sock was shut down by peer
  159. Feb 26 18:27:15 drbd9-03 kernel: drbd r0 drbd9-01: conn( Connecting -> BrokenPipe )
  160. Feb 26 18:27:15 drbd9-03 kernel: drbd r0 drbd9-01: ack_receiver terminated
  161. Feb 26 18:27:15 drbd9-03 kernel: drbd r0 drbd9-01: Terminating ack_recv thread
  162. Feb 26 18:27:15 drbd9-03 kernel: drbd r0 drbd9-01: Connection closed
  163. Feb 26 18:27:15 drbd9-03 kernel: drbd r0 drbd9-01: conn( BrokenPipe -> Unconnected )
  164. Feb 26 18:27:15 drbd9-03 kernel: drbd r0 drbd9-01: Restarting receiver thread
  165. Feb 26 18:27:15 drbd9-03 kernel: drbd r0 drbd9-01: conn( Unconnected -> Connecting )
  166. Feb 26 18:27:15 drbd9-03 kernel: drbd r0 tcp:drbd9-01: initial packet S crossed
  167. Feb 26 18:27:18 drbd9-03 kernel: drbd r0 tcp:drbd9-01: initial packet S crossed
  168. Feb 26 18:27:18 drbd9-03 kernel: drbd r0 drbd9-01: Handshake successful: Agreed network protocol version 111
  169. Feb 26 18:27:18 drbd9-03 kernel: drbd r0 drbd9-01: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
  170. Feb 26 18:27:18 drbd9-03 kernel: drbd r0 drbd9-01: Starting ack_recv thread (from drbd_r_r0 [3112])
  171. Feb 26 18:27:18 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.11
  172. Feb 26 18:27:18 drbd9-03 kernel: drbd r0 drbd9-01: meta connection shut down by peer.
  173. Feb 26 18:27:18 drbd9-03 kernel: drbd r0 drbd9-01: conn( Connecting -> NetworkFailure )
  174. Feb 26 18:27:18 drbd9-03 kernel: drbd r0 drbd9-01: ack_receiver terminated
  175. Feb 26 18:27:18 drbd9-03 kernel: drbd r0 drbd9-01: Terminating ack_recv thread
  176. Feb 26 18:27:18 drbd9-03 kernel: drbd r0 drbd9-01: Connection closed
  177. Feb 26 18:27:18 drbd9-03 kernel: drbd r0 drbd9-01: conn( NetworkFailure -> Unconnected )
  178. Feb 26 18:27:18 drbd9-03 kernel: drbd r0 drbd9-01: Restarting receiver thread
  179. Feb 26 18:27:18 drbd9-03 kernel: drbd r0 drbd9-01: conn( Unconnected -> Connecting )
  180. Feb 26 18:27:19 drbd9-03 kernel: drbd r0 drbd9-01: Handshake successful: Agreed network protocol version 111
  181. Feb 26 18:27:19 drbd9-03 kernel: drbd r0 drbd9-01: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
  182. Feb 26 18:27:19 drbd9-03 kernel: drbd r0 drbd9-01: Starting ack_recv thread (from drbd_r_r0 [3112])
  183. Feb 26 18:27:19 drbd9-03 kernel: drbd r0 drbd9-01: Preparing remote state change 4127012051 (primary_nodes=0, weak_nodes=0)
  184. Feb 26 18:27:19 drbd9-03 kernel: drbd r0 drbd9-01: Committing remote state change 4127012051
  185. Feb 26 18:27:19 drbd9-03 kernel: drbd r0 drbd9-01: conn( Connecting -> Connected ) peer( Unknown -> Secondary )
  186. Feb 26 18:27:19 drbd9-03 kernel: drbd r0/0 drbd1: peer does not support WRITE_SAME
  187. Feb 26 18:27:19 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: unexpected
  188. Feb 26 18:27:19 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: In UUIDs from node 0 found equal UUID (A9F0F625EF8BF07E) for nodes 3 4
  189. Feb 26 18:27:19 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: I have 2603DB27E7D1C272 for node_id=3
  190. Feb 26 18:27:19 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: I have 2603DB27E7D1C272 for node_id=4
  191. Feb 26 18:27:19 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: drbd_sync_handshake:
  192. Feb 26 18:27:19 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: self 15F5D43C9A0B7F26:0000000000000000:2603DB27E7D1C272:4CB7EEE2CCD6AD56 bits:0 flags:0
  193. Feb 26 18:27:19 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: peer 15F5D43C9A0B7F26:0000000000000000:351C9E61C6B06FB6:0000000000000000 bits:0 flags:120
  194. Feb 26 18:27:19 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: uuid_compare()=0 by rule 38
  195. Feb 26 18:27:19 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: pdsk( DUnknown -> UpToDate ) repl( Off -> Established )
  196. Feb 26 18:27:37 drbd9-03 kernel: drbd r0 drbd9-01: Preparing remote state change 208582347 (primary_nodes=0, weak_nodes=0)
  197. Feb 26 18:27:37 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: unexpected
  198. Feb 26 18:27:37 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: In UUIDs from node 1 found equal UUID (351C9E61C6B06FB6) for nodes 3 4
  199. Feb 26 18:27:37 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: I have 2603DB27E7D1C272 for node_id=3
  200. Feb 26 18:27:37 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: I have 2603DB27E7D1C272 for node_id=4
  201. Feb 26 18:27:37 drbd9-03 kernel: drbd r0 drbd9-01: Committing remote state change 208582347
  202. Feb 26 18:27:37 drbd9-03 kernel: drbd r0 drbd9-01: peer( Secondary -> Primary )
  203. Feb 26 18:28:06 drbd9-03 chronyd[887]: Selected source 211.233.84.186
  204. Feb 26 18:28:17 drbd9-03 kernel: drbd r0 drbd9-01: Preparing remote state change 329347875 (primary_nodes=0, weak_nodes=0)
  205. Feb 26 18:28:17 drbd9-03 kernel: drbd r0 drbd9-01: Committing remote state change 329347875
  206. Feb 26 18:28:17 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( UpToDate -> Outdated ) resync-susp( no -> peer )
  207. Feb 26 18:28:17 drbd9-03 kernel: drbd r0 drbd9-01: Preparing remote state change 2733829546 (primary_nodes=0, weak_nodes=0)
  208. Feb 26 18:28:17 drbd9-03 kernel: drbd r0 drbd9-01: Committing remote state change 2733829546
  209. Feb 26 18:28:17 drbd9-03 kernel: drbd r0/0 drbd1: disk( UpToDate -> Outdated )
  210. Feb 26 18:28:17 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-05: resync-susp( no -> connection dependency )
  211. Feb 26 18:28:17 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-04: resync-susp( no -> connection dependency )
  212. Feb 26 18:28:17 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: repl( Established -> PausedSyncT )
  213. Feb 26 18:28:17 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: repl( Established -> StartingSyncT )
  214. Feb 26 18:28:17 drbd9-03 drbdadm[3179]: Don't know which config file belongs to resource r0, trying default ones...
  215. Feb 26 18:28:17 drbd9-03 kernel: drbd r0/0 drbd1: bitmap WRITE of 153 pages took 9 ms
  216. Feb 26 18:28:17 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: helper command: /sbin/drbdadm before-resync-target
  217. Feb 26 18:28:17 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: helper command: /sbin/drbdadm before-resync-target exit code 0 (0x0)
  218. Feb 26 18:28:17 drbd9-03 kernel: drbd r0/0 drbd1: disk( Outdated -> Inconsistent )
  219. Feb 26 18:28:17 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: repl( StartingSyncT -> SyncTarget )
  220. Feb 26 18:28:17 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: Began resync as SyncTarget (will sync 4998328 KB [1249582 bits set]).
  221. Feb 26 18:29:28 drbd9-03 systemd: Starting Cleanup of Temporary Directories...
  222. Feb 26 18:29:28 drbd9-03 systemd: Started Cleanup of Temporary Directories.
  223. Feb 26 18:30:01 drbd9-03 systemd: Starting Session 3 of user root.
  224. Feb 26 18:30:01 drbd9-03 systemd: Started Session 3 of user root.
  225. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: Resync done (total 130 sec; paused 0 sec; 38448 K/sec)
  226. Feb 26 18:30:28 drbd9-03 drbdadm[3230]: Don't know which config file belongs to resource r0, trying default ones...
  227. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: updated UUIDs 15F5D43C9A0B7F26:0000000000000000:15F5D43C9A0B7F26:2603DB27E7D1C272
  228. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1: disk( Inconsistent -> UpToDate )
  229. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-05: pdsk( DUnknown -> Outdated ) resync-susp( connection dependency -> no )
  230. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-04: pdsk( DUnknown -> Outdated ) resync-susp( connection dependency -> no )
  231. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: repl( PausedSyncT -> Established )
  232. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: repl( SyncTarget -> Established )
  233. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: helper command: /sbin/drbdadm after-resync-target
  234. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: helper command: /sbin/drbdadm after-resync-target exit code 0 (0x0)
  235. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: unexpected
  236. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: In UUIDs from node 1 found equal UUID (351C9E61C6B06FB6) for nodes 3 4
  237. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: I have 15F5D43C9A0B7F26 for node_id=3
  238. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: I have 15F5D43C9A0B7F26 for node_id=4
  239. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( Outdated -> Inconsistent )
  240. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: unexpected
  241. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: In UUIDs from node 1 found equal UUID (351C9E61C6B06FB6) for nodes 3 4
  242. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: I have 15F5D43C9A0B7F26 for node_id=3
  243. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: I have 15F5D43C9A0B7F26 for node_id=4
  244. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: unexpected
  245. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: In UUIDs from node 1 found equal UUID (351C9E61C6B06FB6) for nodes 3 4
  246. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: I have 15F5D43C9A0B7F26 for node_id=3
  247. Feb 26 18:30:28 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: I have 15F5D43C9A0B7F26 for node_id=4
  248. Feb 26 18:30:31 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( Inconsistent -> UpToDate ) resync-susp( peer -> no )
  249. Feb 26 18:32:25 drbd9-03 chronyd[887]: Can't synchronise: no reachable sources
  250.  
  251.  
  252. Network disable!!!!!!!!!!!!!!!!!!!!!
  253.  
  254.  
  255. Feb 26 18:32:41 drbd9-03 kernel: drbd r0 drbd9-01: PingAck did not arrive in time.
  256. Feb 26 18:32:41 drbd9-03 kernel: drbd r0 drbd9-01: conn( Connected -> NetworkFailure ) peer( Primary -> Unknown )
  257. Feb 26 18:32:41 drbd9-03 kernel: drbd r0/0 drbd1: disk( UpToDate -> Consistent )
  258. Feb 26 18:32:41 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
  259. Feb 26 18:32:41 drbd9-03 kernel: drbd r0 drbd9-01: ack_receiver terminated
  260. Feb 26 18:32:41 drbd9-03 kernel: drbd r0 drbd9-01: Terminating ack_recv thread
  261. Feb 26 18:32:41 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( UpToDate -> Consistent )
  262. Feb 26 18:32:41 drbd9-03 kernel: drbd r0 drbd9-01: Connection closed
  263. Feb 26 18:32:41 drbd9-03 kernel: drbd r0 drbd9-01: conn( NetworkFailure -> Unconnected )
  264. Feb 26 18:32:41 drbd9-03 kernel: drbd r0 drbd9-01: Restarting receiver thread
  265. Feb 26 18:32:41 drbd9-03 kernel: drbd r0 drbd9-02: No reconciliation resync even though 'drbd9-01' disappeared. (o=0)
  266. Feb 26 18:32:41 drbd9-03 kernel: drbd r0 drbd9-02: Aborting local state change 2887028641 to yield to remote state change 2209710663.
  267. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2887028641 (2->-1 0/0)
  268. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2887028641 (1ms) rv = -19
  269. Feb 26 18:32:41 drbd9-03 kernel: drbd r0 drbd9-01: conn( Unconnected -> Connecting )
  270. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1380038023 (2->-1 0/0)
  271. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1380038023 (0ms) rv = -19
  272. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 284779277 (2->-1 0/0)
  273. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 284779277 (0ms) rv = -19
  274. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1231088145 (2->-1 0/0)
  275. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1231088145 (0ms) rv = -19
  276. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 320861430 (2->-1 0/0)
  277. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 320861430 (0ms) rv = -19
  278. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 51531563 (2->-1 0/0)
  279. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 51531563 (0ms) rv = -19
  280. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4024145528 (2->-1 0/0)
  281. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 4024145528 (0ms) rv = -19
  282. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3375454407 (2->-1 0/0)
  283. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3375454407 (0ms) rv = -19
  284. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 703589942 (2->-1 0/0)
  285. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 703589942 (0ms) rv = -19
  286. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2042019203 (2->-1 0/0)
  287. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2042019203 (1ms) rv = -19
  288. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1025523795 (2->-1 0/0)
  289. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1025523795 (0ms) rv = -19
  290. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2069922678 (2->-1 0/0)
  291. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2069922678 (0ms) rv = -19
  292. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 724148458 (2->-1 0/0)
  293. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 724148458 (0ms) rv = -19
  294. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3368180734 (2->-1 0/0)
  295. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3368180734 (0ms) rv = -19
  296. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 562168394 (2->-1 0/0)
  297. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 562168394 (0ms) rv = -19
  298. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 942841954 (2->-1 0/0)
  299. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 942841954 (0ms) rv = -19
  300. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 134699653 (2->-1 0/0)
  301. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 134699653 (0ms) rv = -19
  302. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 394756796 (2->-1 0/0)
  303. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 394756796 (1ms) rv = -19
  304. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 205856850 (2->-1 0/0)
  305. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 205856850 (0ms) rv = -19
  306. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 160380919 (2->-1 0/0)
  307. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 160380919 (0ms) rv = -19
  308. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1186455563 (2->-1 0/0)
  309. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1186455563 (0ms) rv = -19
  310. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2964883239 (2->-1 0/0)
  311. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2964883239 (0ms) rv = -19
  312. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3202301971 (2->-1 0/0)
  313. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3202301971 (0ms) rv = -19
  314. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4071255045 (2->-1 0/0)
  315. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 4071255045 (0ms) rv = -19
  316. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2262832108 (2->-1 0/0)
  317. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2262832108 (0ms) rv = -19
  318. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3077675474 (2->-1 0/0)
  319. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3077675474 (0ms) rv = -19
  320. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2239639422 (2->-1 0/0)
  321. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2239639422 (0ms) rv = -19
  322. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4121322242 (2->-1 0/0)
  323. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 4121322242 (0ms) rv = -19
  324. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 381323286 (2->-1 0/0)
  325. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 381323286 (0ms) rv = -19
  326. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4066450354 (2->-1 0/0)
  327. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 4066450354 (0ms) rv = -19
  328. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 367688867 (2->-1 0/0)
  329. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 367688867 (0ms) rv = -19
  330. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3478855915 (2->-1 0/0)
  331. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3478855915 (0ms) rv = -19
  332. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4191454852 (2->-1 0/0)
  333. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 4191454852 (0ms) rv = -19
  334. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1650702893 (2->-1 0/0)
  335. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1650702893 (0ms) rv = -19
  336. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3524186003 (2->-1 0/0)
  337. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3524186003 (0ms) rv = -19
  338. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1390769085 (2->-1 0/0)
  339. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1390769085 (0ms) rv = -19
  340. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 583019314 (2->-1 0/0)
  341. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 583019314 (0ms) rv = -19
  342. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2361531171 (2->-1 0/0)
  343. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2361531171 (0ms) rv = -19
  344. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2230396282 (2->-1 0/0)
  345. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2230396282 (2ms) rv = -19
  346. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 474832631 (2->-1 0/0)
  347. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 474832631 (0ms) rv = -19
  348. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 935237020 (2->-1 0/0)
  349. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 935237020 (0ms) rv = -19
  350. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 425046151 (2->-1 0/0)
  351. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 425046151 (0ms) rv = -19
  352. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4198095038 (2->-1 0/0)
  353. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 4198095038 (0ms) rv = -19
  354. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3670036661 (2->-1 0/0)
  355. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3670036661 (0ms) rv = -19
  356. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 783461194 (2->-1 0/0)
  357. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 783461194 (0ms) rv = -19
  358. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 480463785 (2->-1 0/0)
  359. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 480463785 (0ms) rv = -19
  360. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4066029001 (2->-1 0/0)
  361. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 4066029001 (0ms) rv = -19
  362. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 390969353 (2->-1 0/0)
  363. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 390969353 (0ms) rv = -19
  364. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3529986298 (2->-1 0/0)
  365. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3529986298 (0ms) rv = -19
  366. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3326334308 (2->-1 0/0)
  367. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3326334308 (0ms) rv = -19
  368. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1155526426 (2->-1 0/0)
  369. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1155526426 (0ms) rv = -19
  370. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2446204218 (2->-1 0/0)
  371. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2446204218 (0ms) rv = -19
  372. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 930308036 (2->-1 0/0)
  373. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 930308036 (0ms) rv = -19
  374. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1716810837 (2->-1 0/0)
  375. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1716810837 (0ms) rv = -19
  376. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2464570969 (2->-1 0/0)
  377. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2464570969 (0ms) rv = -19
  378. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 936481704 (2->-1 0/0)
  379. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 936481704 (0ms) rv = -19
  380. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 421583813 (2->-1 0/0)
  381. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 421583813 (0ms) rv = -19
  382. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4159669795 (2->-1 0/0)
  383. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 4159669795 (0ms) rv = -19
  384. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3444662429 (2->-1 0/0)
  385. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3444662429 (0ms) rv = -19
  386. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2342098425 (2->-1 0/0)
  387. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2342098425 (0ms) rv = -19
  388. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1771483728 (2->-1 0/0)
  389. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1771483728 (0ms) rv = -19
  390. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2073717554 (2->-1 0/0)
  391. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2073717554 (0ms) rv = -19
  392. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2053570765 (2->-1 0/0)
  393. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2053570765 (0ms) rv = -19
  394. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1301059454 (2->-1 0/0)
  395. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1301059454 (0ms) rv = -19
  396. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1327860695 (2->-1 0/0)
  397. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1327860695 (0ms) rv = -19
  398. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2325718518 (2->-1 0/0)
  399. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2325718518 (0ms) rv = -19
  400. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2251518774 (2->-1 0/0)
  401. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2251518774 (0ms) rv = -19
  402. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1303802844 (2->-1 0/0)
  403. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1303802844 (0ms) rv = -19
  404. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3435023042 (2->-1 0/0)
  405. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3435023042 (0ms) rv = -19
  406. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 992762480 (2->-1 0/0)
  407. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 992762480 (0ms) rv = -19
  408. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 628992183 (2->-1 0/0)
  409. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 628992183 (0ms) rv = -19
  410. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3497687090 (2->-1 0/0)
  411. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3497687090 (0ms) rv = -19
  412. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 642081453 (2->-1 0/0)
  413. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 642081453 (0ms) rv = -19
  414. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 752627710 (2->-1 0/0)
  415. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 752627710 (0ms) rv = -19
  416. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1943226365 (2->-1 0/0)
  417. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1943226365 (0ms) rv = -19
  418. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2165656411 (2->-1 0/0)
  419. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2165656411 (0ms) rv = -19
  420. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 905409409 (2->-1 0/0)
  421. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 905409409 (0ms) rv = -19
  422. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 609945593 (2->-1 0/0)
  423. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 609945593 (0ms) rv = -19
  424. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 95948096 (2->-1 0/0)
  425. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 95948096 (0ms) rv = -19
  426. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3301186778 (2->-1 0/0)
  427. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3301186778 (0ms) rv = -19
  428. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1389028230 (2->-1 0/0)
  429. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1389028230 (0ms) rv = -19
  430. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2271074096 (2->-1 0/0)
  431. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2271074096 (0ms) rv = -19
  432. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1118377091 (2->-1 0/0)
  433. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1118377091 (0ms) rv = -19
  434. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1502492202 (2->-1 0/0)
  435. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1502492202 (0ms) rv = -19
  436. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3408328426 (2->-1 0/0)
  437. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3408328426 (0ms) rv = -19
  438. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 773069168 (2->-1 0/0)
  439. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 773069168 (0ms) rv = -19
  440. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4090090794 (2->-1 0/0)
  441. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 4090090794 (0ms) rv = -19
  442. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1824186861 (2->-1 0/0)
  443. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1824186861 (0ms) rv = -19
  444. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1685839554 (2->-1 0/0)
  445. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1685839554 (0ms) rv = -19
  446. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2648441462 (2->-1 0/0)
  447. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2648441462 (0ms) rv = -19
  448. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2297954705 (2->-1 0/0)
  449. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2297954705 (0ms) rv = -19
  450. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3045658118 (2->-1 0/0)
  451. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3045658118 (0ms) rv = -19
  452. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2070512145 (2->-1 0/0)
  453. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2070512145 (0ms) rv = -19
  454. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3369071475 (2->-1 0/0)
  455. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3369071475 (0ms) rv = -19
  456. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3116085397 (2->-1 0/0)
  457. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3116085397 (0ms) rv = -19
  458. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1282966510 (2->-1 0/0)
  459. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1282966510 (0ms) rv = -19
  460. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1953027651 (2->-1 0/0)
  461. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1953027651 (0ms) rv = -19
  462. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2846268128 (2->-1 0/0)
  463. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2846268128 (0ms) rv = -19
  464. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3602219872 (2->-1 0/0)
  465. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3602219872 (0ms) rv = -19
  466. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2259527452 (2->-1 0/0)
  467. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2259527452 (0ms) rv = -19
  468. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2610606276 (2->-1 0/0)
  469. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2610606276 (0ms) rv = -19
  470. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1555972052 (2->-1 0/0)
  471. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1555972052 (0ms) rv = -19
  472. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 996174538 (2->-1 0/0)
  473. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 996174538 (0ms) rv = -19
  474. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1163653400 (2->-1 0/0)
  475. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1163653400 (0ms) rv = -19
  476. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2707499387 (2->-1 0/0)
  477. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2707499387 (0ms) rv = -19
  478. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3631119695 (2->-1 0/0)
  479. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3631119695 (0ms) rv = -19
  480. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2939595976 (2->-1 0/0)
  481. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2939595976 (0ms) rv = -19
  482. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 289164140 (2->-1 0/0)
  483. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 289164140 (0ms) rv = -19
  484. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3504425937 (2->-1 0/0)
  485. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3504425937 (0ms) rv = -19
  486. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1377988878 (2->-1 0/0)
  487. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1377988878 (0ms) rv = -19
  488. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 649230179 (2->-1 0/0)
  489. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 649230179 (5ms) rv = -19
  490. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2111210368 (2->-1 0/0)
  491. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2111210368 (0ms) rv = -19
  492. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1624833805 (2->-1 0/0)
  493. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1624833805 (0ms) rv = -19
  494. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2904400229 (2->-1 0/0)
  495. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2904400229 (0ms) rv = -19
  496. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 411175192 (2->-1 0/0)
  497. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 411175192 (0ms) rv = -19
  498. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2207574974 (2->-1 0/0)
  499. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2207574974 (0ms) rv = -19
  500. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2596576646 (2->-1 0/0)
  501. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2596576646 (0ms) rv = -19
  502. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3376775172 (2->-1 0/0)
  503. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3376775172 (0ms) rv = -19
  504. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1042142306 (2->-1 0/0)
  505. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1042142306 (0ms) rv = -19
  506. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3603439014 (2->-1 0/0)
  507. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3603439014 (0ms) rv = -19
  508. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3426026204 (2->-1 0/0)
  509. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3426026204 (0ms) rv = -19
  510. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3077705743 (2->-1 0/0)
  511. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3077705743 (0ms) rv = -19
  512. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 614967280 (2->-1 0/0)
  513. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 614967280 (0ms) rv = -19
  514. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1053254960 (2->-1 0/0)
  515. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1053254960 (0ms) rv = -19
  516. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3069703901 (2->-1 0/0)
  517. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3069703901 (0ms) rv = -19
  518. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 637557943 (2->-1 0/0)
  519. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 637557943 (0ms) rv = -19
  520. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1800606937 (2->-1 0/0)
  521. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1800606937 (0ms) rv = -19
  522. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3820699002 (2->-1 0/0)
  523. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3820699002 (0ms) rv = -19
  524. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2354233307 (2->-1 0/0)
  525. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2354233307 (0ms) rv = -19
  526. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1284489367 (2->-1 0/0)
  527. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1284489367 (0ms) rv = -19
  528. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 88143842 (2->-1 0/0)
  529. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 88143842 (0ms) rv = -19
  530. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2509261795 (2->-1 0/0)
  531. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2509261795 (0ms) rv = -19
  532. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 809005990 (2->-1 0/0)
  533. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 809005990 (0ms) rv = -19
  534. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3134443677 (2->-1 0/0)
  535. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3134443677 (0ms) rv = -19
  536. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2280903283 (2->-1 0/0)
  537. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2280903283 (0ms) rv = -19
  538. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2946151457 (2->-1 0/0)
  539. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2946151457 (0ms) rv = -19
  540. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 111655786 (2->-1 0/0)
  541. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 111655786 (0ms) rv = -19
  542. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 925027835 (2->-1 0/0)
  543. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 925027835 (0ms) rv = -19
  544. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 614172278 (2->-1 0/0)
  545. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 614172278 (0ms) rv = -19
  546. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 256094633 (2->-1 0/0)
  547. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 256094633 (0ms) rv = -19
  548. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1601710444 (2->-1 0/0)
  549. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1601710444 (0ms) rv = -19
  550. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2582397647 (2->-1 0/0)
  551. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2582397647 (0ms) rv = -19
  552. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2225272785 (2->-1 0/0)
  553. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2225272785 (0ms) rv = -19
  554. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3058545788 (2->-1 0/0)
  555. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3058545788 (0ms) rv = -19
  556. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2736530179 (2->-1 0/0)
  557. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2736530179 (0ms) rv = -19
  558. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1849134386 (2->-1 0/0)
  559. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1849134386 (0ms) rv = -19
  560. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1561393959 (2->-1 0/0)
  561. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1561393959 (0ms) rv = -19
  562. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2877771736 (2->-1 0/0)
  563. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2877771736 (0ms) rv = -19
  564. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 666002766 (2->-1 0/0)
  565. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 666002766 (0ms) rv = -19
  566. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3558106157 (2->-1 0/0)
  567. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3558106157 (0ms) rv = -19
  568. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 480004762 (2->-1 0/0)
  569. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 480004762 (0ms) rv = -19
  570. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 562403405 (2->-1 0/0)
  571. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 562403405 (0ms) rv = -19
  572. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1713889439 (2->-1 0/0)
  573. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1713889439 (0ms) rv = -19
  574. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 365894217 (2->-1 0/0)
  575. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 365894217 (0ms) rv = -19
  576. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2123462193 (2->-1 0/0)
  577. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2123462193 (0ms) rv = -19
  578. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1407318980 (2->-1 0/0)
  579. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1407318980 (0ms) rv = -19
  580. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1065239655 (2->-1 0/0)
  581. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1065239655 (0ms) rv = -19
  582. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1396893476 (2->-1 0/0)
  583. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1396893476 (0ms) rv = -19
  584. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1330760173 (2->-1 0/0)
  585. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1330760173 (0ms) rv = -19
  586. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1582144196 (2->-1 0/0)
  587. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1582144196 (0ms) rv = -19
  588. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3101426736 (2->-1 0/0)
  589. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3101426736 (0ms) rv = -19
  590. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 688684164 (2->-1 0/0)
  591. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 688684164 (0ms) rv = -19
  592. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1449855887 (2->-1 0/0)
  593. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1449855887 (0ms) rv = -19
  594. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2174500853 (2->-1 0/0)
  595. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2174500853 (0ms) rv = -19
  596. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3381735950 (2->-1 0/0)
  597. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3381735950 (0ms) rv = -19
  598. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2742670336 (2->-1 0/0)
  599. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2742670336 (0ms) rv = -19
  600. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 337173319 (2->-1 0/0)
  601. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 337173319 (0ms) rv = -19
  602. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 368370290 (2->-1 0/0)
  603. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 368370290 (0ms) rv = -19
  604. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2849944234 (2->-1 0/0)
  605. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2849944234 (0ms) rv = -19
  606. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 765722237 (2->-1 0/0)
  607. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 765722237 (0ms) rv = -19
  608. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3985188689 (2->-1 0/0)
  609. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3985188689 (0ms) rv = -19
  610. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1103872287 (2->-1 0/0)
  611. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1103872287 (0ms) rv = -19
  612. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4178498231 (2->-1 0/0)
  613. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 4178498231 (0ms) rv = -19
  614. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1472643893 (2->-1 0/0)
  615. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1472643893 (0ms) rv = -19
  616. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 730038440 (2->-1 0/0)
  617. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 730038440 (0ms) rv = -19
  618. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1741182814 (2->-1 0/0)
  619. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1741182814 (0ms) rv = -19
  620. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 454483589 (2->-1 0/0)
  621. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 454483589 (0ms) rv = -19
  622. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1734246941 (2->-1 0/0)
  623. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1734246941 (0ms) rv = -19
  624. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2091824435 (2->-1 0/0)
  625. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2091824435 (0ms) rv = -19
  626. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1591438634 (2->-1 0/0)
  627. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1591438634 (0ms) rv = -19
  628. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 185533743 (2->-1 0/0)
  629. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 185533743 (0ms) rv = -19
  630. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 929969654 (2->-1 0/0)
  631. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 929969654 (0ms) rv = -19
  632. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1462631908 (2->-1 0/0)
  633. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1462631908 (0ms) rv = -19
  634. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2036756723 (2->-1 0/0)
  635. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2036756723 (0ms) rv = -19
  636. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3097147518 (2->-1 0/0)
  637. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3097147518 (6ms) rv = -19
  638. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3072521432 (2->-1 0/0)
  639. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3072521432 (0ms) rv = -19
  640. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1258802512 (2->-1 0/0)
  641. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1258802512 (0ms) rv = -19
  642. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 117221436 (2->-1 0/0)
  643. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 117221436 (0ms) rv = -19
  644. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1068919959 (2->-1 0/0)
  645. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1068919959 (0ms) rv = -19
  646. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 513754338 (2->-1 0/0)
  647. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 513754338 (0ms) rv = -19
  648. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3389082869 (2->-1 0/0)
  649. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3389082869 (0ms) rv = -19
  650. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2593471089 (2->-1 0/0)
  651. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2593471089 (0ms) rv = -19
  652. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2229442842 (2->-1 0/0)
  653. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2229442842 (0ms) rv = -19
  654. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1461323336 (2->-1 0/0)
  655. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1461323336 (0ms) rv = -19
  656. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1310345875 (2->-1 0/0)
  657. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1310345875 (0ms) rv = -19
  658. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 32234710 (2->-1 0/0)
  659. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 32234710 (0ms) rv = -19
  660. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 912935068 (2->-1 0/0)
  661. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 912935068 (0ms) rv = -19
  662. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1081044637 (2->-1 0/0)
  663. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1081044637 (0ms) rv = -19
  664. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1870312858 (2->-1 0/0)
  665. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1870312858 (0ms) rv = -19
  666. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1917672677 (2->-1 0/0)
  667. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1917672677 (0ms) rv = -19
  668. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3886311575 (2->-1 0/0)
  669. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3886311575 (0ms) rv = -19
  670. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3348356676 (2->-1 0/0)
  671. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3348356676 (0ms) rv = -19
  672. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2846870569 (2->-1 0/0)
  673. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2846870569 (0ms) rv = -19
  674. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1378823074 (2->-1 0/0)
  675. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1378823074 (0ms) rv = -19
  676. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3101720573 (2->-1 0/0)
  677. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3101720573 (0ms) rv = -19
  678. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 955379876 (2->-1 0/0)
  679. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 955379876 (0ms) rv = -19
  680. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2481072681 (2->-1 0/0)
  681. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2481072681 (0ms) rv = -19
  682. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3422815603 (2->-1 0/0)
  683. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3422815603 (0ms) rv = -19
  684. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4174032324 (2->-1 0/0)
  685. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 4174032324 (0ms) rv = -19
  686. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4117329703 (2->-1 0/0)
  687. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 4117329703 (0ms) rv = -19
  688. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4204720172 (2->-1 0/0)
  689. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 4204720172 (0ms) rv = -19
  690. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4127168762 (2->-1 0/0)
  691. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 4127168762 (0ms) rv = -19
  692. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2961884908 (2->-1 0/0)
  693. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2961884908 (0ms) rv = -19
  694. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1777177039 (2->-1 0/0)
  695. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1777177039 (0ms) rv = -19
  696. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 471000821 (2->-1 0/0)
  697. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 471000821 (0ms) rv = -19
  698. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2331933597 (2->-1 0/0)
  699. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 2331933597 (0ms) rv = -19
  700. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1910935076 (2->-1 0/0)
  701. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1910935076 (0ms) rv = -19
  702. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 713019820 (2->-1 0/0)
  703. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 713019820 (0ms) rv = -19
  704. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 518750199 (2->-1 0/0)
  705. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 518750199 (0ms) rv = -19
  706. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3455178678 (2->-1 0/0)
  707. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3455178678 (0ms) rv = -19
  708. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 536075734 (2->-1 0/0)
  709. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 536075734 (0ms) rv = -19
  710. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1423197619 (2->-1 0/0)
  711. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1423197619 (0ms) rv = -19
  712. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3835494743 (2->-1 0/0)
  713. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3835494743 (0ms) rv = -19
  714. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1043266423 (2->-1 0/0)
  715. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1043266423 (0ms) rv = -19
  716. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1221820243 (2->-1 0/0)
  717. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 1221820243 (1ms) rv = -19
  718. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4157490588 (2->-1 0/0)
  719. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 4157490588 (0ms) rv = -19
  720. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3011411053 (2->-1 0/0)
  721. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3011411053 (0ms) rv = -19
  722. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3515696510 (2->-1 0/0)
  723. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Aborting cluster-wide state change 3515696510 (0ms) rv = -19
  724. Feb 26 18:32:41 drbd9-03 kernel: drbd r0 drbd9-02: Preparing remote state change 2209710663 (primary_nodes=0, weak_nodes=0)
  725. Feb 26 18:32:41 drbd9-03 kernel: drbd r0 drbd9-02: Committing remote state change 2209710663
  726. Feb 26 18:32:41 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( Consistent -> UpToDate )
  727. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2249192923 (2->-1 0/0)
  728. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: State change 2249192923: primary_nodes=0, weak_nodes=0
  729. Feb 26 18:32:41 drbd9-03 kernel: drbd r0: Committing cluster-wide state change 2249192923 (2ms)
  730. Feb 26 18:32:41 drbd9-03 kernel: drbd r0/0 drbd1: disk( Consistent -> UpToDate )
  731. Feb 26 18:33:00 drbd9-03 kernel: drbd r0 drbd9-01: Handshake successful: Agreed network protocol version 111
  732. Feb 26 18:33:00 drbd9-03 kernel: drbd r0 drbd9-01: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
  733. Feb 26 18:33:00 drbd9-03 kernel: drbd r0 drbd9-01: Starting ack_recv thread (from drbd_r_r0 [3112])
  734. Feb 26 18:33:00 drbd9-03 kernel: drbd r0 drbd9-01: Preparing remote state change 2265317532 (primary_nodes=0, weak_nodes=0)
  735. Feb 26 18:33:00 drbd9-03 kernel: drbd r0 drbd9-01: Committing remote state change 2265317532
  736. Feb 26 18:33:00 drbd9-03 kernel: drbd r0 drbd9-01: conn( Connecting -> Connected ) peer( Unknown -> Primary )
  737. Feb 26 18:33:00 drbd9-03 kernel: drbd r0/0 drbd1: peer does not support WRITE_SAME
  738. Feb 26 18:33:00 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: unexpected
  739. Feb 26 18:33:00 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: In UUIDs from node 0 found equal UUID (A9F0F625EF8BF07E) for nodes 3 4
  740. Feb 26 18:33:00 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: I have 15F5D43C9A0B7F26 for node_id=3
  741. Feb 26 18:33:00 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: I have 15F5D43C9A0B7F26 for node_id=4
  742. Feb 26 18:33:00 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: drbd_sync_handshake:
  743. Feb 26 18:33:00 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: self 15F5D43C9A0B7F26:0000000000000000:15F5D43C9A0B7F26:2603DB27E7D1C272 bits:0 flags:0
  744. Feb 26 18:33:00 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: peer 15F5D43C9A0B7F26:0000000000000000:351C9E61C6B06FB6:2603DB27E7D1C272 bits:0 flags:120
  745. Feb 26 18:33:00 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: uuid_compare()=0 by rule 38
  746. Feb 26 18:33:00 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-01: pdsk( DUnknown -> UpToDate ) repl( Off -> Established )
  747. Feb 26 18:33:00 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( UpToDate -> Outdated )
  748.  
  749.  
  750. Network disable!!!!!!!!!!!!!!!!!!!!!
  751.  
  752.  
  753.  
  754. Feb 26 18:33:06 drbd9-03 kernel: vmxnet3 0000:04:00.0 eth1: NIC Link is Down
  755. Feb 26 18:33:11 drbd9-03 kernel: drbd r0 drbd9-02: PingAck did not arrive in time.
  756. Feb 26 18:33:11 drbd9-03 kernel: drbd r0 drbd9-02: conn( Connected -> NetworkFailure ) peer( Secondary -> Unknown )
  757. Feb 26 18:33:11 drbd9-03 kernel: drbd r0/0 drbd1 drbd9-02: repl( Established -> Off )
  758. Feb 26 18:33:11 drbd9-03 kernel: drbd r0 drbd9-02: ack_receiver terminated
  759. Feb 26 18:33:11 drbd9-03 kernel: drbd r0 drbd9-02: Terminating ack_recv thread
  760. Feb 26 18:33:11 drbd9-03 kernel: drbd r0 drbd9-02: Connection closed
  761. Feb 26 18:33:11 drbd9-03 kernel: drbd r0 drbd9-02: conn( NetworkFailure -> Unconnected )
  762. Feb 26 18:33:11 drbd9-03 kernel: drbd r0 drbd9-02: Restarting receiver thread
  763. Feb 26 18:33:11 drbd9-03 kernel: drbd r0 drbd9-02: conn( Unconnected -> Connecting )
  764.  
  765.  
  766. Feb 26 18:33:16 drbd9-03 kernel: vmxnet3 0000:04:00.0 eth1: NIC Link is Up 10000 Mbps
  767.  
  768.  
  769. Feb 26 18:33:33 drbd9-03 kernel: drbd r0 drbd9-02: sock_recvmsg returned -11
  770. Feb 26 18:33:33 drbd9-03 kernel: drbd r0 drbd9-02: conn( Connecting -> BrokenPipe )
  771. Feb 26 18:33:33 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4127932492 (2->0 496/16)
  772. Feb 26 18:33:33 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 549ms
  773. Feb 26 18:33:33 drbd9-03 kernel: drbd r0 drbd9-02: Connection closed
  774. Feb 26 18:33:33 drbd9-03 kernel: drbd r0 drbd9-02: conn( BrokenPipe -> Unconnected )
  775. Feb 26 18:33:33 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 736373838 (2->0 496/16)
  776. Feb 26 18:33:33 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 654ms
  777. Feb 26 18:33:34 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2905864915 (2->0 496/16)
  778. Feb 26 18:33:34 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 499ms
  779. Feb 26 18:33:34 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1264025253 (2->0 496/16)
  780. Feb 26 18:33:34 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 2391ms
  781. Feb 26 18:33:37 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1696483929 (2->0 496/16)
  782. Feb 26 18:33:37 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 5358ms
  783. Feb 26 18:33:42 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  784. Feb 26 18:33:42 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3028844404 (2->0 496/16)
  785. Feb 26 18:33:42 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 3828ms
  786. Feb 26 18:33:45 drbd9-03 kernel: drbd r0 drbd9-01: Preparing remote state change 202060381 (primary_nodes=0, weak_nodes=0)
  787. Feb 26 18:33:45 drbd9-03 kernel: drbd r0 drbd9-01: Committing remote state change 202060381
  788. Feb 26 18:33:46 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3560067254 (2->0 496/16)
  789. Feb 26 18:33:46 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 3318ms
  790. Feb 26 18:33:49 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3499444799 (2->0 496/16)
  791. Feb 26 18:33:49 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 8731ms
  792. Feb 26 18:33:51 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  793. Feb 26 18:33:58 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1499230488 (2->0 496/16)
  794. Feb 26 18:33:58 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 7358ms
  795. Feb 26 18:34:03 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  796. Feb 26 18:34:05 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1988657507 (2->0 496/16)
  797. Feb 26 18:34:05 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 6150ms
  798. Feb 26 18:34:12 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1830613693 (2->0 496/16)
  799. Feb 26 18:34:12 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 4078ms
  800. Feb 26 18:34:15 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  801. Feb 26 18:34:16 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3472824171 (2->0 496/16)
  802. Feb 26 18:34:16 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 7319ms
  803. Feb 26 18:34:23 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2003518388 (2->0 496/16)
  804. Feb 26 18:34:23 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 6411ms
  805. Feb 26 18:34:24 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  806. Feb 26 18:34:29 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2872586560 (2->0 496/16)
  807. Feb 26 18:34:29 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 6771ms
  808. Feb 26 18:34:36 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 192783146 (2->0 496/16)
  809. Feb 26 18:34:36 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 6452ms
  810. Feb 26 18:34:36 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  811. Feb 26 18:34:43 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3017968589 (2->0 496/16)
  812. Feb 26 18:34:43 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 5479ms
  813. Feb 26 18:34:45 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  814. Feb 26 18:34:48 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2103455658 (2->0 496/16)
  815. Feb 26 18:34:48 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 8112ms
  816. Feb 26 18:34:56 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2170927023 (2->0 496/16)
  817. Feb 26 18:34:56 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 299ms
  818. Feb 26 18:34:57 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1173042560 (2->0 496/16)
  819. Feb 26 18:34:57 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 7373ms
  820. Feb 26 18:34:57 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  821. Feb 26 18:35:04 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3923345721 (2->0 496/16)
  822. Feb 26 18:35:04 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 3145ms
  823. Feb 26 18:35:07 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2962791380 (2->0 496/16)
  824. Feb 26 18:35:07 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 4526ms
  825. Feb 26 18:35:09 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  826. Feb 26 18:35:12 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2059240641 (2->0 496/16)
  827. Feb 26 18:35:12 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 8356ms
  828. Feb 26 18:35:18 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  829. Feb 26 18:35:20 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 953230976 (2->0 496/16)
  830. Feb 26 18:35:20 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 554ms
  831. Feb 26 18:35:20 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 516175354 (2->0 496/16)
  832. Feb 26 18:35:20 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 7507ms
  833. Feb 26 18:35:28 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3830064125 (2->0 496/16)
  834. Feb 26 18:35:28 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 3563ms
  835. Feb 26 18:35:28 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  836. Feb 26 18:35:28 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  837. Feb 26 18:35:32 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4278418275 (2->0 496/16)
  838. Feb 26 18:35:32 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 4328ms
  839. Feb 26 18:35:36 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2587216505 (2->0 496/16)
  840. Feb 26 18:35:36 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 6343ms
  841. Feb 26 18:35:39 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  842. Feb 26 18:35:42 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2421676245 (2->0 496/16)
  843. Feb 26 18:35:42 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 1320ms
  844. Feb 26 18:35:44 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1155148395 (2->0 496/16)
  845. Feb 26 18:35:44 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 1891ms
  846. Feb 26 18:35:45 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4236434574 (2->0 496/16)
  847. Feb 26 18:35:45 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 1726ms
  848. Feb 26 18:35:47 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1783213420 (2->0 496/16)
  849. Feb 26 18:35:47 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 472ms
  850. Feb 26 18:35:48 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2325110552 (2->0 496/16)
  851. Feb 26 18:35:48 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 3857ms
  852. Feb 26 18:35:51 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  853. Feb 26 18:35:52 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2216963373 (2->0 496/16)
  854. Feb 26 18:35:52 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 1159ms
  855. Feb 26 18:35:53 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1072294531 (2->0 496/16)
  856. Feb 26 18:35:53 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 6311ms
  857. Feb 26 18:35:59 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4237505305 (2->0 496/16)
  858. Feb 26 18:35:59 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 3502ms
  859. Feb 26 18:36:00 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  860. Feb 26 18:36:02 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 271816173 (2->0 496/16)
  861. Feb 26 18:36:02 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 5686ms
  862. Feb 26 18:36:08 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2533839464 (2->0 496/16)
  863. Feb 26 18:36:08 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 6072ms
  864. Feb 26 18:36:09 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  865. Feb 26 18:36:14 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3606934239 (2->0 496/16)
  866. Feb 26 18:36:14 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 8167ms
  867. Feb 26 18:36:20 drbd9-03 kernel: drbd r0 tcp:drbd9-04: Closing unexpected connection from 100.100.10.21
  868. Feb 26 18:36:22 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 983721643 (2->0 496/16)
  869. Feb 26 18:36:22 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 1320ms
  870. Feb 26 18:36:24 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 661765498 (2->0 496/16)
  871. Feb 26 18:36:24 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 1732ms
  872. Feb 26 18:36:25 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1657392586 (2->0 496/16)
  873. Feb 26 18:36:25 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 5846ms
  874.  
  875. !!!!!!!!!!!!!!!!!!!!!!!!!!
  876.  
  877. Feb 26 18:36:28 drbd9-03 kernel: INFO: task drbd_r_r0:3114 blocked for more than 120 seconds.
  878. Feb 26 18:36:28 drbd9-03 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  879. Feb 26 18:36:28 drbd9-03 kernel: drbd_r_r0 D ffff88003d613680 0 3114 2 0x00000080
  880. Feb 26 18:36:28 drbd9-03 kernel: ffff8800369a3b70 0000000000000046 ffff88001c8ab8e0 ffff8800369a3fd8
  881. Feb 26 18:36:28 drbd9-03 kernel: ffff8800369a3fd8 ffff8800369a3fd8 ffff88001c8ab8e0 ffff880039f5e960
  882. Feb 26 18:36:28 drbd9-03 kernel: 7fffffffffffffff ffff88001c8ab8e0 0000000000000002 0000000000000000
  883. Feb 26 18:36:28 drbd9-03 kernel: Call Trace:
  884. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
  885. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffff816075f9>] schedule_timeout+0x209/0x2d0
  886. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffff810a9512>] ? default_wake_function+0x12/0x20
  887. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffff8109825b>] ? autoremove_wake_function+0x2b/0x40
  888. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffff810b1c7c>] ? update_curr+0xcc/0x150
  889. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffff81608b72>] __down_common+0xd2/0x14a
  890. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffff810b2a6e>] ? dequeue_task_fair+0x40e/0x620
  891. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffff81608c07>] __down+0x1d/0x1f
  892. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffff8109d201>] down+0x41/0x50
  893. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffffa04b23c4>] state_change_lock+0x54/0x60 [drbd]
  894. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffffa04b7700>] ? __change_peer_disk_states+0x70/0x70 [drbd]
  895. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffffa04b5cc6>] change_cluster_wide_state+0x46/0xe90 [drbd]
  896. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffff8108007e>] ? try_to_del_timer_sync+0x5e/0x90
  897. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffff81080102>] ? del_timer_sync+0x52/0x60
  898. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffff8160756d>] ? schedule_timeout+0x17d/0x2d0
  899. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffff8107ed90>] ? __internal_add_timer+0x130/0x130
  900. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffffa04b7336>] change_cstate+0x86/0xc0 [drbd]
  901. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffffa0486de1>] drbd_receiver+0x151/0x630 [drbd]
  902. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffffa0492521>] drbd_thread_setup+0x61/0x120 [drbd]
  903. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffffa04924c0>] ? w_complete+0x20/0x20 [drbd]
  904. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffff8109726f>] kthread+0xcf/0xe0
  905. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
  906. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffff81614158>] ret_from_fork+0x58/0x90
  907. Feb 26 18:36:28 drbd9-03 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
  908. Feb 26 18:36:31 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2357617602 (2->0 496/16)
  909. Feb 26 18:36:31 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 1790ms
  910. Feb 26 18:36:32 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  911. Feb 26 18:36:33 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 905347632 (2->0 496/16)
  912. Feb 26 18:36:33 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 5193ms
  913. Feb 26 18:36:38 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1539036085 (2->0 496/16)
  914. Feb 26 18:36:38 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 2832ms
  915. Feb 26 18:36:41 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  916. Feb 26 18:36:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3407485329 (2->0 496/16)
  917. Feb 26 18:36:41 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 5385ms
  918. Feb 26 18:36:47 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 621469889 (2->0 496/16)
  919. Feb 26 18:36:47 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 6198ms
  920. Feb 26 18:36:50 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  921. Feb 26 18:36:53 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1544542665 (2->0 496/16)
  922. Feb 26 18:36:53 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 8156ms
  923. Feb 26 18:36:58 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  924. Feb 26 18:37:01 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1806218545 (2->0 496/16)
  925. Feb 26 18:37:01 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 5610ms
  926. Feb 26 18:37:06 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2318514728 (2->0 496/16)
  927. Feb 26 18:37:06 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 5352ms
  928. Feb 26 18:37:08 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  929. Feb 26 18:37:12 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3234375198 (2->0 496/16)
  930. Feb 26 18:37:12 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 4635ms
  931. Feb 26 18:37:16 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 975113547 (2->0 496/16)
  932. Feb 26 18:37:16 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 212ms
  933. Feb 26 18:37:17 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3094546250 (2->0 496/16)
  934. Feb 26 18:37:17 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 4356ms
  935. Feb 26 18:37:19 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  936. Feb 26 18:37:21 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3250393572 (2->0 496/16)
  937. Feb 26 18:37:21 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 715ms
  938. Feb 26 18:37:22 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3661432340 (2->0 496/16)
  939. Feb 26 18:37:22 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 6403ms
  940. Feb 26 18:37:28 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2282588060 (2->0 496/16)
  941. Feb 26 18:37:28 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 3349ms
  942. Feb 26 18:37:31 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  943. Feb 26 18:37:32 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4014327130 (2->0 496/16)
  944. Feb 26 18:37:32 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 7264ms
  945. Feb 26 18:37:39 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 571867440 (2->0 496/16)
  946. Feb 26 18:37:39 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 4409ms
  947. Feb 26 18:37:40 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  948. Feb 26 18:37:43 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3516582956 (2->0 496/16)
  949. Feb 26 18:37:43 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 5627ms
  950. Feb 26 18:37:49 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4292968949 (2->0 496/16)
  951. Feb 26 18:37:49 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 5396ms
  952. Feb 26 18:37:51 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  953. Feb 26 18:37:54 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3935421702 (2->0 496/16)
  954. Feb 26 18:37:54 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 2690ms
  955. Feb 26 18:37:57 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1326207274 (2->0 496/16)
  956. Feb 26 18:37:57 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 3673ms
  957. Feb 26 18:38:01 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3661155097 (2->0 496/16)
  958. Feb 26 18:38:01 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 4022ms
  959. Feb 26 18:38:03 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  960. Feb 26 18:38:05 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 104161368 (2->0 496/16)
  961. Feb 26 18:38:05 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 2647ms
  962. Feb 26 18:38:07 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 422285090 (2->0 496/16)
  963. Feb 26 18:38:07 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 9274ms
  964. Feb 26 18:38:15 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  965. Feb 26 18:38:17 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2706459626 (2->0 496/16)
  966. Feb 26 18:38:17 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 5120ms
  967. Feb 26 18:38:22 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 519992869 (2->0 496/16)
  968. Feb 26 18:38:22 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 4769ms
  969. Feb 26 18:38:23 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  970. Feb 26 18:38:26 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 413766697 (2->0 496/16)
  971. Feb 26 18:38:26 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 4420ms
  972.  
  973. !!!!!!!!!!!!!!!!!!!!!!!!
  974.  
  975. Feb 26 18:38:28 drbd9-03 kernel: INFO: task drbd_r_r0:3114 blocked for more than 120 seconds.
  976. Feb 26 18:38:28 drbd9-03 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  977. Feb 26 18:38:28 drbd9-03 kernel: drbd_r_r0 D ffff88003d613680 0 3114 2 0x00000080
  978. Feb 26 18:38:28 drbd9-03 kernel: ffff8800369a3b70 0000000000000046 ffff88001c8ab8e0 ffff8800369a3fd8
  979. Feb 26 18:38:28 drbd9-03 kernel: ffff8800369a3fd8 ffff8800369a3fd8 ffff88001c8ab8e0 ffff880039f5e960
  980. Feb 26 18:38:28 drbd9-03 kernel: 7fffffffffffffff ffff88001c8ab8e0 0000000000000002 0000000000000000
  981. Feb 26 18:38:28 drbd9-03 kernel: Call Trace:
  982. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
  983. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffff816075f9>] schedule_timeout+0x209/0x2d0
  984. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffff810a9512>] ? default_wake_function+0x12/0x20
  985. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffff8109825b>] ? autoremove_wake_function+0x2b/0x40
  986. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffff810b1c7c>] ? update_curr+0xcc/0x150
  987. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffff81608b72>] __down_common+0xd2/0x14a
  988. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffff810b2a6e>] ? dequeue_task_fair+0x40e/0x620
  989. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffff81608c07>] __down+0x1d/0x1f
  990. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffff8109d201>] down+0x41/0x50
  991. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffffa04b23c4>] state_change_lock+0x54/0x60 [drbd]
  992. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffffa04b7700>] ? __change_peer_disk_states+0x70/0x70 [drbd]
  993. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffffa04b5cc6>] change_cluster_wide_state+0x46/0xe90 [drbd]
  994. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffff8108007e>] ? try_to_del_timer_sync+0x5e/0x90
  995. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffff81080102>] ? del_timer_sync+0x52/0x60
  996. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffff8160756d>] ? schedule_timeout+0x17d/0x2d0
  997. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffff8107ed90>] ? __internal_add_timer+0x130/0x130
  998. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffffa04b7336>] change_cstate+0x86/0xc0 [drbd]
  999. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffffa0486de1>] drbd_receiver+0x151/0x630 [drbd]
  1000. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffffa0492521>] drbd_thread_setup+0x61/0x120 [drbd]
  1001. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffffa04924c0>] ? w_complete+0x20/0x20 [drbd]
  1002. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffff8109726f>] kthread+0xcf/0xe0
  1003. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
  1004. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffff81614158>] ret_from_fork+0x58/0x90
  1005. Feb 26 18:38:28 drbd9-03 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
  1006. Feb 26 18:38:31 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3430510247 (2->0 496/16)
  1007. Feb 26 18:38:31 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 1401ms
  1008. Feb 26 18:38:32 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2266865952 (2->0 496/16)
  1009. Feb 26 18:38:32 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 6928ms
  1010. Feb 26 18:38:36 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  1011. Feb 26 18:38:39 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2377129242 (2->0 496/16)
  1012. Feb 26 18:38:39 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 3217ms
  1013. Feb 26 18:38:42 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2160023316 (2->0 496/16)
  1014. Feb 26 18:38:42 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 6209ms
  1015. Feb 26 18:38:44 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  1016. Feb 26 18:38:49 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 571211202 (2->0 496/16)
  1017. Feb 26 18:38:49 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 7038ms
  1018. Feb 26 18:38:53 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  1019. Feb 26 18:38:56 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1127143557 (2->0 496/16)
  1020. Feb 26 18:38:56 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 6538ms
  1021. Feb 26 18:39:02 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  1022. Feb 26 18:39:02 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1240761710 (2->0 496/16)
  1023. Feb 26 18:39:02 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 5798ms
  1024. Feb 26 18:39:08 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1675500083 (2->0 496/16)
  1025. Feb 26 18:39:08 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 9298ms
  1026. Feb 26 18:39:11 drbd9-03 kernel: drbd r0 tcp:drbd9-04: Closing unexpected connection from 100.100.10.21
  1027. Feb 26 18:39:17 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2718483551 (2->0 496/16)
  1028. Feb 26 18:39:17 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 7233ms
  1029. Feb 26 18:39:22 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  1030. Feb 26 18:39:25 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3872798115 (2->0 496/16)
  1031. Feb 26 18:39:25 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 7974ms
  1032. Feb 26 18:39:31 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  1033. Feb 26 18:39:32 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4016494118 (2->0 496/16)
  1034. Feb 26 18:39:32 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 695ms
  1035. Feb 26 18:39:33 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2999253673 (2->0 496/16)
  1036. Feb 26 18:39:33 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 2487ms
  1037. Feb 26 18:39:36 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 684281626 (2->0 496/16)
  1038. Feb 26 18:39:36 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 5331ms
  1039. Feb 26 18:39:40 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  1040. Feb 26 18:39:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3108402569 (2->0 496/16)
  1041. Feb 26 18:39:41 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 3161ms
  1042. Feb 26 18:39:44 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2560220061 (2->0 496/16)
  1043. Feb 26 18:39:44 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 9585ms
  1044. Feb 26 18:39:51 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  1045. Feb 26 18:39:54 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 611248467 (2->0 496/16)
  1046. Feb 26 18:39:54 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 3468ms
  1047. Feb 26 18:39:57 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1532026530 (2->0 496/16)
  1048. Feb 26 18:39:57 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 7608ms
  1049. Feb 26 18:39:57 drbd9-03 gnome-session: (gnome-settings-daemon:2096): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
  1050. Feb 26 18:39:57 drbd9-03 gnome-session: (gnome-settings-daemon:2096): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
  1051. Feb 26 18:39:57 drbd9-03 gnome-session: (gnome-settings-daemon:2096): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
  1052. Feb 26 18:39:57 drbd9-03 gnome-session: (gnome-settings-daemon:2096): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
  1053. Feb 26 18:40:01 drbd9-03 systemd: Starting Session 4 of user root.
  1054. Feb 26 18:40:01 drbd9-03 systemd: Started Session 4 of user root.
  1055. Feb 26 18:40:03 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  1056. Feb 26 18:40:05 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3086691970 (2->0 496/16)
  1057. Feb 26 18:40:05 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 6861ms
  1058. Feb 26 18:40:09 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  1059. Feb 26 18:40:12 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 2499805130 (2->0 496/16)
  1060. Feb 26 18:40:12 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 4731ms
  1061. Feb 26 18:40:16 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1116660183 (2->0 496/16)
  1062. Feb 26 18:40:16 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 5761ms
  1063. Feb 26 18:40:21 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  1064. Feb 26 18:40:22 drbd9-03 gnome-session: (gnome-settings-daemon:2096): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
  1065. Feb 26 18:40:22 drbd9-03 gnome-session: (gnome-settings-daemon:2096): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
  1066. Feb 26 18:40:22 drbd9-03 gnome-session: (gnome-settings-daemon:2096): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
  1067. Feb 26 18:40:22 drbd9-03 gnome-session: (gnome-settings-daemon:2096): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
  1068. Feb 26 18:40:22 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 256981273 (2->0 496/16)
  1069. Feb 26 18:40:22 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 5790ms
  1070. Feb 26 18:40:28 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 4278699290 (2->0 496/16)
  1071. Feb 26 18:40:28 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 7574ms
  1072. Feb 26 18:40:28 drbd9-03 kernel: INFO: task drbd_r_r0:3114 blocked for more than 120 seconds.
  1073. Feb 26 18:40:28 drbd9-03 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  1074. Feb 26 18:40:28 drbd9-03 kernel: drbd_r_r0 D ffff88003d613680 0 3114 2 0x00000080
  1075. Feb 26 18:40:28 drbd9-03 kernel: ffff8800369a3b70 0000000000000046 ffff88001c8ab8e0 ffff8800369a3fd8
  1076. Feb 26 18:40:28 drbd9-03 kernel: ffff8800369a3fd8 ffff8800369a3fd8 ffff88001c8ab8e0 ffff880039f5e960
  1077. Feb 26 18:40:28 drbd9-03 kernel: 7fffffffffffffff ffff88001c8ab8e0 0000000000000002 0000000000000000
  1078. Feb 26 18:40:28 drbd9-03 kernel: Call Trace:
  1079. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
  1080. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffff816075f9>] schedule_timeout+0x209/0x2d0
  1081. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffff810a9512>] ? default_wake_function+0x12/0x20
  1082. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffff8109825b>] ? autoremove_wake_function+0x2b/0x40
  1083. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffff810b1c7c>] ? update_curr+0xcc/0x150
  1084. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffff81608b72>] __down_common+0xd2/0x14a
  1085. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffff810b2a6e>] ? dequeue_task_fair+0x40e/0x620
  1086. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffff81608c07>] __down+0x1d/0x1f
  1087. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffff8109d201>] down+0x41/0x50
  1088. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffffa04b23c4>] state_change_lock+0x54/0x60 [drbd]
  1089. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffffa04b7700>] ? __change_peer_disk_states+0x70/0x70 [drbd]
  1090. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffffa04b5cc6>] change_cluster_wide_state+0x46/0xe90 [drbd]
  1091. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffff8108007e>] ? try_to_del_timer_sync+0x5e/0x90
  1092. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffff81080102>] ? del_timer_sync+0x52/0x60
  1093. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffff8160756d>] ? schedule_timeout+0x17d/0x2d0
  1094. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffff8107ed90>] ? __internal_add_timer+0x130/0x130
  1095. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffffa04b7336>] change_cstate+0x86/0xc0 [drbd]
  1096. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffffa0486de1>] drbd_receiver+0x151/0x630 [drbd]
  1097. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffffa0492521>] drbd_thread_setup+0x61/0x120 [drbd]
  1098. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffffa04924c0>] ? w_complete+0x20/0x20 [drbd]
  1099. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffff8109726f>] kthread+0xcf/0xe0
  1100. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
  1101. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffff81614158>] ret_from_fork+0x58/0x90
  1102. Feb 26 18:40:28 drbd9-03 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
  1103. Feb 26 18:40:32 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  1104. Feb 26 18:40:36 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 181018252 (2->0 496/16)
  1105. Feb 26 18:40:36 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 1952ms
  1106. Feb 26 18:40:38 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 3960951220 (2->0 496/16)
  1107. Feb 26 18:40:38 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 1671ms
  1108. Feb 26 18:40:39 drbd9-03 kernel: drbd r0 tcp:drbd9-05: Closing unexpected connection from 100.100.10.21
  1109. Feb 26 18:40:39 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 356215126 (2->0 496/16)
  1110. Feb 26 18:40:39 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 1729ms
  1111. Feb 26 18:40:41 drbd9-03 kernel: drbd r0: Preparing cluster-wide state change 1214575198 (2->0 496/16)
  1112. Feb 26 18:40:41 drbd9-03 kernel: drbd r0: Retrying cluster-wide state change after 3533ms
RAW Paste Data