Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- Mar 3 14:30:13 drbd9-01 kernel: drbd r0/0 drbd1: peer does not support WRITE_SAME
- Mar 3 14:30:13 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: unexpected
- Mar 3 14:30:13 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: In UUIDs from node 1 found equal UUID (B35BFF8B3A09C3D4) for nodes 2 3
- Mar 3 14:30:13 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: I have 4784B438C8271CE8 for node_id=2
- Mar 3 14:30:13 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: I have 4784B438C8271CE8 for node_id=3
- Mar 3 14:30:13 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: drbd_sync_handshake:
- Mar 3 14:30:13 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: self 8E3E5FF304474C76:0000000000000000:B35BFF8B3A09C3D4:A85E76C8215C798A bits:165416 flags:0
- Mar 3 14:30:13 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: peer 8E3E5FF304474C76:FFFFFFFFFFFFFFFF:B35BFF8B3A09C3D4:0000000000000000 bits:22194 flags:4
- Mar 3 14:30:13 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: uuid_compare()=0 by rule 38
- Mar 3 14:30:13 drbd9-01 kernel: drbd r0/0 drbd1: Becoming sync source due to disk states.
- Mar 3 14:30:13 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( Off -> WFBitMapS )
- Mar 3 14:30:13 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 336(1), total 336; compression: 99.8%
- Mar 3 14:30:13 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 336(1), total 336; compression: 99.8%
- Mar 3 14:30:13 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source
- Mar 3 14:30:13 drbd9-01 drbdadm[6016]: Don't know which config file belongs to resource r0, trying default ones...
- Mar 3 14:30:13 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
- Mar 3 14:30:13 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( WFBitMapS -> SyncSource )
- Mar 3 14:30:13 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: Began resync as SyncSource (will sync 661664 KB [165416 bits set]).
- Mar 3 14:31:30 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: unexpected
- Mar 3 14:31:30 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: In UUIDs from node 1 found equal UUID (8E3E5FF304474C76) for nodes 2 3
- Mar 3 14:31:30 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: I have 4784B438C8271CE8 for node_id=2
- Mar 3 14:31:30 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: I have 4784B438C8271CE8 for node_id=3
- Mar 3 14:31:30 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: Resync done (total 77 sec; paused 0 sec; 8592 K/sec)
- Mar 3 14:31:30 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( Inconsistent -> UpToDate ) repl( SyncSource -> Established )
- Mar 3 14:37:23 drbd9-01 dbus-daemon: dbus[938]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service'
- Mar 3 14:37:23 drbd9-01 dbus[938]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service'
- Mar 3 14:37:24 drbd9-01 systemd: Starting Hostname Service...
- Mar 3 14:37:24 drbd9-01 dbus-daemon: dbus[938]: [system] Successfully activated service 'org.freedesktop.hostname1'
- Mar 3 14:37:24 drbd9-01 dbus[938]: [system] Successfully activated service 'org.freedesktop.hostname1'
- Mar 3 14:37:24 drbd9-01 systemd: Started Hostname Service.
- Mar 3 14:37:44 drbd9-01 kernel: EXT4-fs (drbd1): mounting ext3 file system using the ext4 subsystem
- Mar 3 14:37:44 drbd9-01 kernel: EXT4-fs (drbd1): mounted filesystem with ordered data mode. Opts: (null)
- Mar 3 14:38:06 drbd9-01 dbus-daemon: dbus[938]: [system] Activating service name='org.freedesktop.PackageKit' (using servicehelper)
- Mar 3 14:38:06 drbd9-01 dbus[938]: [system] Activating service name='org.freedesktop.PackageKit' (using servicehelper)
- Mar 3 14:38:07 drbd9-01 dbus-daemon: dbus[938]: [system] Successfully activated service 'org.freedesktop.PackageKit'
- Mar 3 14:38:07 drbd9-01 dbus[938]: [system] Successfully activated service 'org.freedesktop.PackageKit'
- Mar 3 14:38:26 drbd9-01 kernel: drbd r0/0 drbd1: Congestion-fill threshold reached
- Mar 3 14:38:26 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( UpToDate -> Consistent ) repl( Established -> Ahead )
- Mar 3 14:38:27 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( Consistent -> Outdated )
- Mar 3 14:38:28 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source
- Mar 3 14:38:28 drbd9-01 drbdadm[6340]: Don't know which config file belongs to resource r0, trying default ones...
- Mar 3 14:38:28 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
- Mar 3 14:38:28 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( Outdated -> Inconsistent ) repl( Ahead -> SyncSource )
- Mar 3 14:38:28 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: Began resync as SyncSource (will sync 49788 KB [12447 bits set]).
- Mar 3 14:38:28 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: unexpected
- Mar 3 14:38:28 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: In UUIDs from node 1 found equal UUID (8E3E5FF304474C76) for nodes 2 3
- Mar 3 14:38:28 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: I have 4784B438C8271CE8 for node_id=2
- Mar 3 14:38:28 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: I have 4784B438C8271CE8 for node_id=3
- Mar 3 14:38:28 drbd9-01 kernel: drbd r0/0 drbd1: Congestion-fill threshold reached
- Mar 3 14:38:28 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( SyncSource -> Ahead )
- Mar 3 14:38:28 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( Inconsistent -> Outdated )
- Mar 3 14:38:29 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source
- Mar 3 14:38:29 drbd9-01 drbdadm[6343]: Don't know which config file belongs to resource r0, trying default ones...
- Mar 3 14:38:29 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
- Mar 3 14:38:29 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( Outdated -> Inconsistent ) repl( Ahead -> SyncSource )
- Mar 3 14:38:29 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: Began resync as SyncSource (will sync 77348 KB [19337 bits set]).
- Mar 3 14:38:29 drbd9-01 kernel: drbd r0/0 drbd1: Congestion-fill threshold reached
- Mar 3 14:38:29 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( SyncSource -> Ahead )
- Mar 3 14:38:30 drbd9-01 drbdadm[6345]: Don't know which config file belongs to resource r0, trying default ones...
- Mar 3 14:38:30 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source
- Mar 3 14:38:30 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
- Mar 3 14:38:30 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( Ahead -> SyncSource )
- Mar 3 14:38:30 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: Began resync as SyncSource (will sync 91548 KB [22887 bits set]).
- Mar 3 14:38:31 drbd9-01 kernel: drbd r0/0 drbd1: Congestion-fill threshold reached
- Mar 3 14:38:31 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( SyncSource -> Ahead )
- Mar 3 14:38:32 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source
- Mar 3 14:38:32 drbd9-01 drbdadm[6347]: Don't know which config file belongs to resource r0, trying default ones...
- Mar 3 14:38:32 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
- Mar 3 14:38:32 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( Ahead -> SyncSource )
- Mar 3 14:38:32 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: Began resync as SyncSource (will sync 174080 KB [43520 bits set]).
- Mar 3 14:38:33 drbd9-01 kernel: drbd r0/0 drbd1: Congestion-fill threshold reached
- Mar 3 14:38:33 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( SyncSource -> Ahead )
- Mar 3 14:38:34 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source
- Mar 3 14:38:34 drbd9-01 drbdadm[6349]: Don't know which config file belongs to resource r0, trying default ones...
- Mar 3 14:38:34 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
- Mar 3 14:38:34 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( Ahead -> SyncSource )
- Mar 3 14:38:34 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: Began resync as SyncSource (will sync 184568 KB [46142 bits set]).
- Mar 3 14:38:41 drbd9-01 kernel: EXT4-fs (drbd1): mounting ext3 file system using the ext4 subsystem
- Mar 3 14:38:41 drbd9-01 kernel: EXT4-fs (drbd1): mounted filesystem with ordered data mode. Opts: (null)
- Mar 3 14:39:09 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: unexpected
- Mar 3 14:39:09 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: In UUIDs from node 1 found equal UUID (8E3E5FF304474C76) for nodes 2 3
- Mar 3 14:39:09 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: I have 4784B438C8271CE8 for node_id=2
- Mar 3 14:39:09 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: I have 4784B438C8271CE8 for node_id=3
- Mar 3 14:40:01 drbd9-01 systemd: Starting Session 17 of user root.
- Mar 3 14:40:01 drbd9-01 systemd: Started Session 17 of user root.
- Mar 3 14:42:46 drbd9-01 kernel: end_request: I/O error, dev fd0, sector 0
- Mar 3 14:42:46 drbd9-01 kernel: end_request: I/O error, dev fd0, sector 0
- Mar 3 14:43:03 drbd9-01 kernel: end_request: I/O error, dev fd0, sector 0
- Mar 3 14:43:04 drbd9-01 kernel: end_request: I/O error, dev fd0, sector 0
- Mar 3 14:44:20 drbd9-01 kernel: EXT4-fs (drbd1): mounting ext3 file system using the ext4 subsystem
- Mar 3 14:44:20 drbd9-01 kernel: EXT4-fs (drbd1): mounted filesystem with ordered data mode. Opts: (null)
- Mar 3 14:44:31 drbd9-01 kernel: EXT4-fs (drbd1): mounting ext3 file system using the ext4 subsystem
- Mar 3 14:44:31 drbd9-01 kernel: EXT4-fs (drbd1): mounted filesystem with ordered data mode. Opts: (null)
- 1) start big file copy
- Mar 3 14:48:18 drbd9-01 kernel: drbd r0/0 drbd1: Congestion-fill threshold reached
- Mar 3 14:48:18 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( SyncSource -> Ahead )
- Mar 3 14:48:18 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( Inconsistent -> Outdated )
- Mar 3 14:48:19 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source
- Mar 3 14:48:19 drbd9-01 drbdadm[6734]: Don't know which config file belongs to resource r0, trying default ones...
- Mar 3 14:48:19 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
- Mar 3 14:48:19 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( Outdated -> Inconsistent ) repl( Ahead -> SyncSource )
- Mar 3 14:48:19 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: Began resync as SyncSource (will sync 6144 KB [1536 bits set]).
- Mar 3 14:48:19 drbd9-01 kernel: drbd r0/0 drbd1: Congestion-fill threshold reached
- Mar 3 14:48:19 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( SyncSource -> Ahead )
- Mar 3 14:48:22 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( Inconsistent -> Consistent )
- Mar 3 14:48:22 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: unexpected
- Mar 3 14:48:22 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: In UUIDs from node 1 found equal UUID (8E3E5FF304474C76) for nodes 2 3
- Mar 3 14:48:22 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: I have 4784B438C8271CE8 for node_id=2
- Mar 3 14:48:22 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: I have 4784B438C8271CE8 for node_id=3
- Mar 3 14:48:22 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( Consistent -> Outdated )
- Mar 3 14:48:23 drbd9-01 kernel: drbd r0/0 drbd1: new current UUID: 9AC70FCEB869EB6F weak: FFFFFFFFFFFFFFFE
- Mar 3 14:48:23 drbd9-01 drbdadm[6736]: Don't know which config file belongs to resource r0, trying default ones...
- Mar 3 14:48:23 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source
- Mar 3 14:48:23 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
- Mar 3 14:48:23 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: pdsk( Outdated -> Inconsistent ) repl( Ahead -> SyncSource )
- Mar 3 14:48:23 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: Began resync as SyncSource (will sync 70412 KB [17603 bits set]).
- Mar 3 14:48:23 drbd9-01 kernel: drbd r0/0 drbd1: Congestion-fill threshold reached
- Mar 3 14:48:23 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( SyncSource -> Ahead )
- Mar 3 14:48:25 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source
- Mar 3 14:48:25 drbd9-01 drbdadm[6738]: Don't know which config file belongs to resource r0, trying default ones...
- Mar 3 14:48:25 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
- Mar 3 14:48:25 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( Ahead -> SyncSource )
- Mar 3 14:48:25 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: Began resync as SyncSource (will sync 78576 KB [19644 bits set]).
- Mar 3 14:48:26 drbd9-01 kernel: drbd r0/0 drbd1: Congestion-fill threshold reached
- Mar 3 14:48:26 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( SyncSource -> Ahead )
- Mar 3 14:48:29 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source
- Mar 3 14:48:29 drbd9-01 drbdadm[6740]: Don't know which config file belongs to resource r0, trying default ones...
- Mar 3 14:48:29 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
- Mar 3 14:48:29 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( Ahead -> SyncSource )
- Mar 3 14:48:29 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: Began resync as SyncSource (will sync 140008 KB [35002 bits set]).
- Mar 3 14:48:29 drbd9-01 kernel: drbd r0/0 drbd1: Congestion-fill threshold reached
- Mar 3 14:48:29 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( SyncSource -> Ahead )
- Mar 3 14:48:31 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source
- Mar 3 14:48:31 drbd9-01 drbdadm[6743]: Don't know which config file belongs to resource r0, trying default ones...
- Mar 3 14:48:31 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
- Mar 3 14:48:31 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( Ahead -> SyncSource )
- Mar 3 14:48:31 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: Began resync as SyncSource (will sync 271052 KB [67763 bits set]).
- Mar 3 14:48:31 drbd9-01 kernel: drbd r0/0 drbd1: Congestion-fill threshold reached
- Mar 3 14:48:31 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( SyncSource -> Ahead )
- Mar 3 14:48:32 drbd9-01 drbdadm[6745]: Don't know which config file belongs to resource r0, trying default ones...
- Mar 3 14:48:32 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source
- Mar 3 14:48:32 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
- Mar 3 14:48:32 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( Ahead -> SyncSource )
- Mar 3 14:48:32 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: Began resync as SyncSource (will sync 385712 KB [96428 bits set]).
- Mar 3 14:48:32 drbd9-01 kernel: drbd r0/0 drbd1: Congestion-fill threshold reached
- Mar 3 14:48:32 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( SyncSource -> Ahead )
- Mar 3 14:48:34 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source
- Mar 3 14:48:34 drbd9-01 drbdadm[6747]: Don't know which config file belongs to resource r0, trying default ones...
- Mar 3 14:48:34 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
- Mar 3 14:48:34 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( Ahead -> SyncSource )
- Mar 3 14:48:34 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: Began resync as SyncSource (will sync 517324 KB [129331 bits set]).
- Mar 3 14:48:34 drbd9-01 kernel: drbd r0/0 drbd1: Congestion-fill threshold reached
- Mar 3 14:48:34 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( SyncSource -> Ahead )
- 2) Disable netwok
- Mar 3 14:48:35 drbd9-01 kernel: vmxnet3 0000:04:00.0 eth1: NIC Link is Down
- Mar 3 14:48:35 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source
- Mar 3 14:48:35 drbd9-01 drbdadm[6749]: Don't know which config file belongs to resource r0, trying default ones...
- Mar 3 14:48:35 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
- Mar 3 14:48:35 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( Ahead -> SyncSource )
- Mar 3 14:48:35 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: Began resync as SyncSource (will sync 522416 KB [130604 bits set]).
- Mar 3 14:48:35 drbd9-01 kernel: drbd r0/0 drbd1: Congestion-fill threshold reached
- Mar 3 14:48:35 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( SyncSource -> Ahead )
- 3) ping timeout
- Mar 3 14:48:45 drbd9-01 kernel: drbd r0 drbd9-02: PingAck did not arrive in time.
- Mar 3 14:48:45 drbd9-01 kernel: drbd r0 drbd9-02: conn( Connected -> NetworkFailure ) peer( Secondary -> Unknown )
- Mar 3 14:48:45 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( Ahead -> Off )
- Mar 3 14:48:45 drbd9-01 kernel: drbd r0 drbd9-02: ack_receiver terminated
- Mar 3 14:48:45 drbd9-01 kernel: drbd r0 drbd9-02: Terminating ack_recv thread
- Mar 3 14:48:45 drbd9-01 kernel: drbd r0 drbd9-02: Connection closed
- Mar 3 14:48:45 drbd9-01 kernel: drbd r0 drbd9-02: conn( NetworkFailure -> Unconnected )
- Mar 3 14:48:45 drbd9-01 kernel: drbd r0 drbd9-02: Restarting receiver thread
- Mar 3 14:48:45 drbd9-01 kernel: drbd r0 drbd9-02: conn( Unconnected -> Connecting )
- 4) Re-connect network
- Mar 3 14:49:07 drbd9-01 kernel: vmxnet3 0000:04:00.0 eth1: NIC Link is Up 10000 Mbps
- Mar 3 14:49:10 drbd9-01 kernel: drbd r0 tcp:drbd9-02: initial packet S crossed
- Mar 3 14:49:11 drbd9-01 kernel: drbd r0 drbd9-02: Handshake successful: Agreed network protocol version 111
- Mar 3 14:49:11 drbd9-01 kernel: drbd r0 drbd9-02: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
- Mar 3 14:49:11 drbd9-01 kernel: drbd r0 drbd9-02: Starting ack_recv thread (from drbd_r_r0 [5607])
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0: Preparing cluster-wide state change 3446365049 (0->1 499/145)
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0: State change 3446365049: primary_nodes=1, weak_nodes=FFFFFFFFFFFFFFFC
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0: Committing cluster-wide state change 3446365049 (0ms)
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0 drbd9-02: conn( Connecting -> Connected ) peer( Unknown -> Secondary )
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0/0 drbd1: peer does not support WRITE_SAME
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: updated UUIDs 9AC70FCEB869EB6F:0000000000000000:8E3E5FF304474C76:B35BFF8B3A09C3D4
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: unexpected
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: In UUIDs from node 1 found equal UUID (8E3E5FF304474C76) for nodes 2 3
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: I have 4784B438C8271CE8 for node_id=2
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: I have 4784B438C8271CE8 for node_id=3
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0/0 drbd1: bitmap page idx 85 changed during IO!
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0/0 drbd1: bitmap WRITE of 64 pages took 37 ms
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: drbd_sync_handshake:
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: self 9AC70FCEB869EB6F:0000000000000000:8E3E5FF304474C76:B35BFF8B3A09C3D4 bits:593419 flags:0
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: peer 9AC70FCEB869EB6E:0000000000000000:8E3E5FF304474C76:0000000000000000 bits:129324 flags:104
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: uuid_compare()=0 by rule 38
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0/0 drbd1: Becoming sync source due to disk states.
- Mar 3 14:49:12 drbd9-01 kernel: drbd r0/0 drbd1 drbd9-02: repl( Off -> WFBitMapS )
- 5) pending!! there is no resync start signal, I think.
- Mar 3 14:50:01 drbd9-01 systemd: Starting Session 18 of user root.
- Mar 3 14:50:01 drbd9-01 systemd: Started Session 18 of user root.
- Mar 3 14:51:47 drbd9-01 gnome-session: (gnome-settings-daemon:2380): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
- Mar 3 14:51:47 drbd9-01 gnome-session: (gnome-settings-daemon:2380): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
- Mar 3 14:51:47 drbd9-01 gnome-session: (gnome-settings-daemon:2380): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
- 6) kernel hung_task_timeout_secs!!!
- Mar 3 14:52:04 drbd9-01 kernel: INFO: task pool:6755 blocked for more than 120 seconds.
- Mar 3 14:52:04 drbd9-01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Mar 3 14:52:04 drbd9-01 kernel: pool D ffff88003d613680 0 6755 2232 0x00000080
- Mar 3 14:52:04 drbd9-01 kernel: ffff880018867bd8 0000000000000086 ffff88001f3d4440 ffff880018867fd8
- Mar 3 14:52:04 drbd9-01 kernel: ffff880018867fd8 ffff880018867fd8 ffff88001f3d4440 ffff880039d9a000
- Mar 3 14:52:04 drbd9-01 kernel: ffff880039d9a078 000000000000000c ffff88000a144100 ffff88000a11b030
- Mar 3 14:52:04 drbd9-01 kernel: Call Trace:
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffffa0507085>] wait_transaction_locked+0x85/0xd0 [jbd2]
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffffa0507400>] start_this_handle+0x2b0/0x5d0 [jbd2]
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff811ab72a>] ? kmem_cache_alloc+0x1ba/0x1d0
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffffa0507933>] jbd2__journal_start+0xf3/0x1e0 [jbd2]
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffffa0530e1a>] ? ext4_dirty_inode+0x2a/0x60 [ext4]
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffffa055b299>] __ext4_journal_start_sb+0x69/0xe0 [ext4]
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffffa0530e1a>] ext4_dirty_inode+0x2a/0x60 [ext4]
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff811f069a>] __mark_inode_dirty+0xca/0x290
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff811e1721>] update_time+0x81/0xd0
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff811e1970>] file_update_time+0xa0/0xf0
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff811f40d8>] generic_file_splice_write+0xc8/0x180
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff811f4010>] ? pipe_to_file+0x170/0x170
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff811f4603>] do_splice_from+0xc3/0x140
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff811f619e>] SyS_splice+0x27e/0x690
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff81614209>] system_call_fastpath+0x16/0x1b
- Mar 3 14:52:04 drbd9-01 kernel: INFO: task kworker/u256:1:6338 blocked for more than 120 seconds.
- Mar 3 14:52:04 drbd9-01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Mar 3 14:52:04 drbd9-01 kernel: kworker/u256:1 D ffff88003d613680 0 6338 2 0x00000080
- Mar 3 14:52:04 drbd9-01 kernel: Workqueue: writeback bdi_writeback_workfn (flush-147:1)
- Mar 3 14:52:04 drbd9-01 kernel: ffff880009a0b7d8 0000000000000046 ffff880039142220 ffff880009a0bfd8
- Mar 3 14:52:04 drbd9-01 kernel: ffff880009a0bfd8 ffff880009a0bfd8 ffff880039142220 ffff88000a465400
- Mar 3 14:52:04 drbd9-01 kernel: ffff88000a465550 ffff88002ba9e300 00000001007e7682 ffff88000a465574
- Mar 3 14:52:04 drbd9-01 kernel: Call Trace:
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
- 7) drbd pengind point #1
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffffa04a03da>] drbd_make_request+0x14a/0x360 [drbd]
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff812aa56f>] ? generic_make_request_checks+0x24f/0x380
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff812aa782>] generic_make_request+0xe2/0x130
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff812aa841>] submit_bio+0x71/0x150
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffffa0531c55>] ext4_io_submit+0x25/0x50 [ext4]
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffffa0531e09>] ext4_bio_write_page+0x159/0x2e0 [ext4]
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffffa0528f6d>] mpage_submit_page+0x5d/0x80 [ext4]
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffffa05290a0>] mpage_process_page_bufs+0x110/0x130 [ext4]
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffffa0529ef5>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffffa052e043>] ext4_writepages+0x463/0xd60 [ext4]
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff81162bfe>] do_writepages+0x1e/0x40
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff811f0000>] __writeback_single_inode+0x40/0x220
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff811f0cfe>] writeback_sb_inodes+0x25e/0x420
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff811f0f5f>] __writeback_inodes_wb+0x9f/0xd0
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff811f17a3>] wb_writeback+0x263/0x2f0
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff811f2cdc>] bdi_writeback_workfn+0x1cc/0x460
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff8108f0bb>] process_one_work+0x17b/0x470
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff8108fe8b>] worker_thread+0x11b/0x400
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff8108fd70>] ? rescuer_thread+0x400/0x400
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff8109726f>] kthread+0xcf/0xe0
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff81614158>] ret_from_fork+0x58/0x90
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Mar 3 14:52:04 drbd9-01 kernel: INFO: task jbd2/drbd1-8:6590 blocked for more than 120 seconds.
- Mar 3 14:52:04 drbd9-01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Mar 3 14:52:04 drbd9-01 kernel: jbd2/drbd1-8 D ffff88003d613680 0 6590 2 0x00000080
- Mar 3 14:52:04 drbd9-01 kernel: ffff880009adfc90 0000000000000046 ffff880036bb2d80 ffff880009adffd8
- Mar 3 14:52:04 drbd9-01 kernel: ffff880009adffd8 ffff880009adffd8 ffff880036bb2d80 ffff880009adfda0
- Mar 3 14:52:04 drbd9-01 kernel: ffff880039d9a0c0 ffff880036bb2d80 ffff880009adfd88 ffff88000a144100
- Mar 3 14:52:04 drbd9-01 kernel: Call Trace:
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffffa050a258>] jbd2_journal_commit_transaction+0x248/0x19a0 [jbd2]
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff810aec1e>] ? account_entity_dequeue+0xae/0xd0
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff810b2126>] ? dequeue_entity+0x106/0x510
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff810ad756>] ? __dequeue_entity+0x26/0x40
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff810125c6>] ? __switch_to+0x136/0x4a0
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff8108007e>] ? try_to_del_timer_sync+0x5e/0x90
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffffa050fda9>] kjournald2+0xc9/0x260 [jbd2]
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffffa050fce0>] ? commit_timeout+0x10/0x10 [jbd2]
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff8109726f>] kthread+0xcf/0xe0
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff81614158>] ret_from_fork+0x58/0x90
- Mar 3 14:52:04 drbd9-01 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Mar 3 14:52:05 drbd9-01 gnome-session: (gnome-settings-daemon:2380): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
- Mar 3 14:52:05 drbd9-01 gnome-session: (gnome-settings-daemon:2380): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
- Mar 3 14:52:05 drbd9-01 gnome-session: (gnome-settings-daemon:2380): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
- Mar 3 14:52:05 drbd9-01 gnome-session: (gnome-settings-daemon:2380): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
- Mar 3 14:52:05 drbd9-01 gnome-session: (gnome-settings-daemon:2380): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
- Mar 3 14:52:05 drbd9-01 gnome-session: (gnome-settings-daemon:2380): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
- Mar 3 14:52:05 drbd9-01 gnome-session: (gnome-settings-daemon:2380): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
- 7) drbd pengind point #2 after 2 min
- Mar 3 14:54:04 drbd9-01 kernel: INFO: task kswapd0:279 blocked for more than 120 seconds.
- Mar 3 14:54:04 drbd9-01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Mar 3 14:54:04 drbd9-01 kernel: kswapd0 D ffff88003d613680 0 279 2 0x00000000
- Mar 3 14:54:04 drbd9-01 kernel: ffff880036807848 0000000000000046 ffff88003c1fe660 ffff880036807fd8
- Mar 3 14:54:04 drbd9-01 kernel: ffff880036807fd8 ffff880036807fd8 ffff88003c1fe660 ffff88000a465400
- Mar 3 14:54:04 drbd9-01 kernel: ffff88000a465550 ffff880038d66800 000000010080f8a8 ffff88000a465574
- Mar 3 14:54:04 drbd9-01 kernel: Call Trace:
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa04a03da>] drbd_make_request+0x14a/0x360 [drbd]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff812aa56f>] ? generic_make_request_checks+0x24f/0x380
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff812aa782>] generic_make_request+0xe2/0x130
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff812aa841>] submit_bio+0x71/0x150
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff811569ab>] ? unlock_page+0x2b/0x30
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa0531c55>] ext4_io_submit+0x25/0x50 [ext4]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa052cea2>] ext4_writepage+0x1f2/0x470 [ext4]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff81169f6d>] shrink_page_list+0x80d/0xad0
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff8116a8ea>] shrink_inactive_list+0x1ea/0x560
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff8116b3ad>] shrink_lruvec+0x36d/0x730
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff8116b7e6>] shrink_zone+0x76/0x1a0
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff8116ca9c>] balance_pgdat+0x48c/0x5e0
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff8116cd63>] kswapd+0x173/0x450
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff8116cbf0>] ? balance_pgdat+0x5e0/0x5e0
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff8109726f>] kthread+0xcf/0xe0
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff81614158>] ret_from_fork+0x58/0x90
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Mar 3 14:54:04 drbd9-01 kernel: INFO: task pool:6755 blocked for more than 120 seconds.
- Mar 3 14:54:04 drbd9-01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Mar 3 14:54:04 drbd9-01 kernel: pool D ffff88003d613680 0 6755 2232 0x00000080
- Mar 3 14:54:04 drbd9-01 kernel: ffff880018867bd8 0000000000000086 ffff88001f3d4440 ffff880018867fd8
- Mar 3 14:54:04 drbd9-01 kernel: ffff880018867fd8 ffff880018867fd8 ffff88001f3d4440 ffff880039d9a000
- Mar 3 14:54:04 drbd9-01 kernel: ffff880039d9a078 000000000000000c ffff88000a144100 ffff88000a11b030
- Mar 3 14:54:04 drbd9-01 kernel: Call Trace:
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa0507085>] wait_transaction_locked+0x85/0xd0 [jbd2]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa0507400>] start_this_handle+0x2b0/0x5d0 [jbd2]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff811ab72a>] ? kmem_cache_alloc+0x1ba/0x1d0
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa0507933>] jbd2__journal_start+0xf3/0x1e0 [jbd2]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa0530e1a>] ? ext4_dirty_inode+0x2a/0x60 [ext4]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa055b299>] __ext4_journal_start_sb+0x69/0xe0 [ext4]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa0530e1a>] ext4_dirty_inode+0x2a/0x60 [ext4]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff811f069a>] __mark_inode_dirty+0xca/0x290
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff811e1721>] update_time+0x81/0xd0
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff811e1970>] file_update_time+0xa0/0xf0
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff811f40d8>] generic_file_splice_write+0xc8/0x180
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff811f4010>] ? pipe_to_file+0x170/0x170
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff811f4603>] do_splice_from+0xc3/0x140
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff811f619e>] SyS_splice+0x27e/0x690
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff81614209>] system_call_fastpath+0x16/0x1b
- Mar 3 14:54:04 drbd9-01 kernel: INFO: task kworker/u256:1:6338 blocked for more than 120 seconds.
- Mar 3 14:54:04 drbd9-01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Mar 3 14:54:04 drbd9-01 kernel: kworker/u256:1 D ffff88003d613680 0 6338 2 0x00000080
- Mar 3 14:54:04 drbd9-01 kernel: Workqueue: writeback bdi_writeback_workfn (flush-147:1)
- Mar 3 14:54:04 drbd9-01 kernel: ffff880009a0b7d8 0000000000000046 ffff880039142220 ffff880009a0bfd8
- Mar 3 14:54:04 drbd9-01 kernel: ffff880009a0bfd8 ffff880009a0bfd8 ffff880039142220 ffff88000a465400
- Mar 3 14:54:04 drbd9-01 kernel: ffff88000a465550 ffff88002ba9e300 00000001007e7682 ffff88000a465574
- Mar 3 14:54:04 drbd9-01 kernel: Call Trace:
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa04a03da>] drbd_make_request+0x14a/0x360 [drbd]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff812aa56f>] ? generic_make_request_checks+0x24f/0x380
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff812aa782>] generic_make_request+0xe2/0x130
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff812aa841>] submit_bio+0x71/0x150
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa0531c55>] ext4_io_submit+0x25/0x50 [ext4]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa0531e09>] ext4_bio_write_page+0x159/0x2e0 [ext4]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa0528f6d>] mpage_submit_page+0x5d/0x80 [ext4]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa05290a0>] mpage_process_page_bufs+0x110/0x130 [ext4]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa0529ef5>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa052e043>] ext4_writepages+0x463/0xd60 [ext4]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff81162bfe>] do_writepages+0x1e/0x40
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff811f0000>] __writeback_single_inode+0x40/0x220
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff811f0cfe>] writeback_sb_inodes+0x25e/0x420
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff811f0f5f>] __writeback_inodes_wb+0x9f/0xd0
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff811f17a3>] wb_writeback+0x263/0x2f0
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff811f2cdc>] bdi_writeback_workfn+0x1cc/0x460
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff8108f0bb>] process_one_work+0x17b/0x470
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff8108fe8b>] worker_thread+0x11b/0x400
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff8108fd70>] ? rescuer_thread+0x400/0x400
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff8109726f>] kthread+0xcf/0xe0
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff81614158>] ret_from_fork+0x58/0x90
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Mar 3 14:54:04 drbd9-01 kernel: INFO: task jbd2/drbd1-8:6590 blocked for more than 120 seconds.
- Mar 3 14:54:04 drbd9-01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Mar 3 14:54:04 drbd9-01 kernel: jbd2/drbd1-8 D ffff88003d613680 0 6590 2 0x00000080
- Mar 3 14:54:04 drbd9-01 kernel: ffff880009adfc90 0000000000000046 ffff880036bb2d80 ffff880009adffd8
- Mar 3 14:54:04 drbd9-01 kernel: ffff880009adffd8 ffff880009adffd8 ffff880036bb2d80 ffff880009adfda0
- Mar 3 14:54:04 drbd9-01 kernel: ffff880039d9a0c0 ffff880036bb2d80 ffff880009adfd88 ffff88000a144100
- Mar 3 14:54:04 drbd9-01 kernel: Call Trace:
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa050a258>] jbd2_journal_commit_transaction+0x248/0x19a0 [jbd2]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff810aec1e>] ? account_entity_dequeue+0xae/0xd0
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff810b2126>] ? dequeue_entity+0x106/0x510
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff810ad756>] ? __dequeue_entity+0x26/0x40
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff810125c6>] ? __switch_to+0x136/0x4a0
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff8108007e>] ? try_to_del_timer_sync+0x5e/0x90
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa050fda9>] kjournald2+0xc9/0x260 [jbd2]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffffa050fce0>] ? commit_timeout+0x10/0x10 [jbd2]
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff8109726f>] kthread+0xcf/0xe0
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff81614158>] ret_from_fork+0x58/0x90
- Mar 3 14:54:04 drbd9-01 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Mar 3 14:56:04 drbd9-01 kernel: INFO: task kswapd0:279 blocked for more than 120 seconds.
- Mar 3 14:56:04 drbd9-01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Mar 3 14:56:04 drbd9-01 kernel: kswapd0 D ffff88003d613680 0 279 2 0x00000000
- Mar 3 14:56:04 drbd9-01 kernel: ffff880036807848 0000000000000046 ffff88003c1fe660 ffff880036807fd8
- Mar 3 14:56:04 drbd9-01 kernel: ffff880036807fd8 ffff880036807fd8 ffff88003c1fe660 ffff88000a465400
- Mar 3 14:56:04 drbd9-01 kernel: ffff88000a465550 ffff880038d66800 000000010080f8a8 ffff88000a465574
- Mar 3 14:56:04 drbd9-01 kernel: Call Trace:
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffffa04a03da>] drbd_make_request+0x14a/0x360 [drbd]
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff812aa56f>] ? generic_make_request_checks+0x24f/0x380
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff812aa782>] generic_make_request+0xe2/0x130
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff812aa841>] submit_bio+0x71/0x150
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff811569ab>] ? unlock_page+0x2b/0x30
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffffa0531c55>] ext4_io_submit+0x25/0x50 [ext4]
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffffa052cea2>] ext4_writepage+0x1f2/0x470 [ext4]
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff81169f6d>] shrink_page_list+0x80d/0xad0
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff8116a8ea>] shrink_inactive_list+0x1ea/0x560
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff8116b3ad>] shrink_lruvec+0x36d/0x730
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff8116b7e6>] shrink_zone+0x76/0x1a0
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff8116ca9c>] balance_pgdat+0x48c/0x5e0
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff8116cd63>] kswapd+0x173/0x450
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff8116cbf0>] ? balance_pgdat+0x5e0/0x5e0
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff8109726f>] kthread+0xcf/0xe0
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff81614158>] ret_from_fork+0x58/0x90
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Mar 3 14:56:04 drbd9-01 kernel: INFO: task pool:6755 blocked for more than 120 seconds.
- Mar 3 14:56:04 drbd9-01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Mar 3 14:56:04 drbd9-01 kernel: pool D ffff88003d613680 0 6755 2232 0x00000080
- Mar 3 14:56:04 drbd9-01 kernel: ffff880018867bd8 0000000000000086 ffff88001f3d4440 ffff880018867fd8
- Mar 3 14:56:04 drbd9-01 kernel: ffff880018867fd8 ffff880018867fd8 ffff88001f3d4440 ffff880039d9a000
- Mar 3 14:56:04 drbd9-01 kernel: ffff880039d9a078 000000000000000c ffff88000a144100 ffff88000a11b030
- Mar 3 14:56:04 drbd9-01 kernel: Call Trace:
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffffa0507085>] wait_transaction_locked+0x85/0xd0 [jbd2]
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffffa0507400>] start_this_handle+0x2b0/0x5d0 [jbd2]
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff811ab72a>] ? kmem_cache_alloc+0x1ba/0x1d0
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffffa0507933>] jbd2__journal_start+0xf3/0x1e0 [jbd2]
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffffa0530e1a>] ? ext4_dirty_inode+0x2a/0x60 [ext4]
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffffa055b299>] __ext4_journal_start_sb+0x69/0xe0 [ext4]
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffffa0530e1a>] ext4_dirty_inode+0x2a/0x60 [ext4]
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff811f069a>] __mark_inode_dirty+0xca/0x290
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff811e1721>] update_time+0x81/0xd0
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff811e1970>] file_update_time+0xa0/0xf0
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff811f40d8>] generic_file_splice_write+0xc8/0x180
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff811f4010>] ? pipe_to_file+0x170/0x170
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff811f4603>] do_splice_from+0xc3/0x140
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff811f619e>] SyS_splice+0x27e/0x690
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff81614209>] system_call_fastpath+0x16/0x1b
- Mar 3 14:56:04 drbd9-01 kernel: INFO: task kworker/u256:1:6338 blocked for more than 120 seconds.
- Mar 3 14:56:04 drbd9-01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Mar 3 14:56:04 drbd9-01 kernel: kworker/u256:1 D ffff88003d613680 0 6338 2 0x00000080
- Mar 3 14:56:04 drbd9-01 kernel: Workqueue: writeback bdi_writeback_workfn (flush-147:1)
- Mar 3 14:56:04 drbd9-01 kernel: ffff880009a0b7d8 0000000000000046 ffff880039142220 ffff880009a0bfd8
- Mar 3 14:56:04 drbd9-01 kernel: ffff880009a0bfd8 ffff880009a0bfd8 ffff880039142220 ffff88000a465400
- Mar 3 14:56:04 drbd9-01 kernel: ffff88000a465550 ffff88002ba9e300 00000001007e7682 ffff88000a465574
- Mar 3 14:56:04 drbd9-01 kernel: Call Trace:
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffffa04a03da>] drbd_make_request+0x14a/0x360 [drbd]
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff812aa56f>] ? generic_make_request_checks+0x24f/0x380
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff812aa782>] generic_make_request+0xe2/0x130
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff812aa841>] submit_bio+0x71/0x150
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffffa0531c55>] ext4_io_submit+0x25/0x50 [ext4]
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffffa0531e09>] ext4_bio_write_page+0x159/0x2e0 [ext4]
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffffa0528f6d>] mpage_submit_page+0x5d/0x80 [ext4]
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffffa05290a0>] mpage_process_page_bufs+0x110/0x130 [ext4]
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffffa0529ef5>] mpage_prepare_extent_to_map+0x235/0x2e0 [ext4]
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffffa052e043>] ext4_writepages+0x463/0xd60 [ext4]
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff81162bfe>] do_writepages+0x1e/0x40
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff811f0000>] __writeback_single_inode+0x40/0x220
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff811f0cfe>] writeback_sb_inodes+0x25e/0x420
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff811f0f5f>] __writeback_inodes_wb+0x9f/0xd0
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff811f17a3>] wb_writeback+0x263/0x2f0
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff811f2cdc>] bdi_writeback_workfn+0x1cc/0x460
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff8108f0bb>] process_one_work+0x17b/0x470
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff8108fe8b>] worker_thread+0x11b/0x400
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff8108fd70>] ? rescuer_thread+0x400/0x400
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff8109726f>] kthread+0xcf/0xe0
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff81614158>] ret_from_fork+0x58/0x90
- Mar 3 14:56:04 drbd9-01 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Mar 3 14:56:12 drbd9-01 gnome-session: (gnome-settings-daemon:2380): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
- Mar 3 14:56:12 drbd9-01 gnome-session: (gnome-settings-daemon:2380): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
- Mar 3 14:56:12 drbd9-01 gnome-session: (gnome-settings-daemon:2380): color-plugin-WARNING **: unable to get EDID for xrandr-Virtual1: unable to get EDID for output
- Mar 3 14:59:07 drbd9-01 vmsvc[901]: [ warning] [powerops] Unable to send the status RPC.
- Mar 3 14:59:07 drbd9-01 systemd: Stopping Session 1 of user root.
- Mar 3 14:59:07 drbd9-01 systemd: Stopped Session 1 of user root.
- Mar 3 14:59:07 drbd9-01 systemd: Stopping user-0.slice.
- Mar 3 14:59:07 drbd9-01 systemd: Removed slice user-0.slice.
- Mar 3 14:59:07 drbd9-01 systemd: Stopping user-42.slice.
- Mar 3 14:59:07 drbd9-01 systemd: Removed slice user-42.slice.
- Mar 3 14:59:07 drbd9-01 systemd: Stopping Sound Card.
- Mar 3 14:59:07 drbd9-01 systemd: Stopped target Sound Card.
- Mar 3 14:59:07 drbd9-01 systemd: Stopping LVM2 PV scan on device 8:2...
- Mar 3 14:59:07 drbd9-01 systemd: Stopping Manage Sound Card State (restore and store)...
- Mar 3 14:59:07 drbd9-01 systemd: Stopping system-getty.slice.
- Mar 3 14:59:07 drbd9-01 gnome-keyring-daemon[2230]: g_dbus_connection_real_closed: Remote peer vanished with error: Underlying GIOStream returned 0 bytes on an async read (g-io-error-quark, 0). Exiting.
- Mar 3 14:59:07 drbd9-01 colord: device removed: xrandr-Virtual1
- Mar 3 14:59:07 drbd9-01 systemd: Removed slice system-getty.slice.
- Mar 3 14:59:07 drbd9-01 systemd: Stopping Dump dmesg to /var/log/dmesg...
- Mar 3 14:59:07 drbd9-01 systemd: Stopped Dump dmesg to /var/log/dmesg.
- Mar 3 14:59:07 drbd9-01 systemd: Stopping CUPS Printing Service...
- Mar 3 14:59:07 drbd9-01 systemd: Stopping Graphical Interface.
- Mar 3 14:59:07 drbd9-01 systemd: Stopped target Graphical Interface.
- Mar 3 14:59:07 drbd9-01 systemd: Stopping Multi-User System.
- Mar 3 14:59:07 drbd9-01 systemd: Stopped target Multi-User System.
- Mar 3 14:59:07 drbd9-01 systemd: Stopping SYSV: Manages the services needed to run VMware software...
- Mar 3 14:59:07 drbd9-01 rsyslogd: [origin software="rsyslogd" swVersion="7.4.7" x-pid="908" x-info="http://www.rsyslog.com"] exiting on signal 15.
- Mar 3 14:59:22 drbd9-01 rsyslogd: [origin software="rsyslogd" swVersion="7.4.7" x-pid="910" x-info="http://www.rsyslog.com"] start
- Mar 3 14:59:19 drbd9-01 journal: Runtime journal is using 6.1M (max 48.9M, leaving 73.4M of free 483.6M, current limit 48.9M).
- Mar 3 14:59:19 drbd9-01 kernel: Initializing cgroup subsys cpuset
- Mar 3 14:59:19 drbd9-01 kernel: Initializing cgroup subsys cpu
- Mar 3 14:59:19 drbd9-01 kernel: Initializing cgroup subsys cpuacct
- Mar 3 14:59:19 drbd9-01 kernel: Linux version 3.10.0-229.7.2.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) ) #1 SMP Tue Jun 23 22:06:11 UTC 2015
- Mar 3 14:59:19 drbd9-01 kernel: Command line: BOOT_IMAGE=/vmlinuz-3.10.0-229.7.2.el7.x86_64 root=/dev/mapper/centos-root ro rd.lvm.lv=centos/root rd.lvm.lv=centos/swap net.ifnames=0 rhgb quiet systemd.debug LANG=ko_KR.UTF-8
- Mar 3 14:59:19 drbd9-01 kernel: Disabled fast string operations
- Mar 3 14:59:19 drbd9-01 kernel: e820: BIOS-provided physical RAM map:
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement