Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- Apr 11 10:13:46 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: repl( Ahead -> SyncSource )
- Apr 11 10:13:46 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: Began resync as SyncSource (will sync 345288 KB [86322 bits set
- Apr 11 10:13:46 drbd9-02 kernel: drbd r1/0 drbd1: Congestion-fill threshold reached
- Apr 11 10:13:46 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: repl( SyncSource -> Ahead )
- Apr 11 10:13:47 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: helper command: /sbin/drbdadm before-resync-source
- Apr 11 10:13:47 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
- Apr 11 10:13:47 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: repl( Ahead -> SyncSource )
- Apr 11 10:13:47 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: Began resync as SyncSource (will sync 358536 KB [89634 bits set]).
- Apr 11 10:13:47 drbd9-02 kernel: drbd r1/0 drbd1: Congestion-fill threshold reached
- Apr 11 10:13:47 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: repl( SyncSource -> Ahead )
- Apr 11 10:13:48 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: helper command: /sbin/drbdadm before-resync-source
- Apr 11 10:13:48 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
- Apr 11 10:13:48 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: repl( Ahead -> SyncSource )
- Apr 11 10:13:48 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: Began resync as SyncSource (will sync 602836 KB [150709 bits set]).
- Apr 11 10:13:48 drbd9-02 kernel: drbd r1/0 drbd1: Congestion-fill threshold reached
- Apr 11 10:13:48 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: repl( SyncSource -> Ahead )
- Apr 11 10:13:49 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: helper command: /sbin/drbdadm before-resync-source
- Apr 11 10:13:49 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
- Apr 11 10:13:49 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: repl( Ahead -> SyncSource )
- Apr 11 10:13:49 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: Began resync as SyncSource (will sync 715444 KB [178861 bits set]).
- Apr 11 10:13:50 drbd9-02 kernel: drbd r1/0 drbd1: Congestion-fill threshold reached
- Apr 11 10:13:50 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: repl( SyncSource -> Ahead ) //-----------------> VM pause
- Apr 11 10:14:45 drbd9-02 kernel: drbd r1 drbd9-03: meta connection shut down by peer.
- Apr 11 10:14:45 drbd9-02 kernel: drbd r1 drbd9-03: conn( Connected -> NetworkFailure ) peer( Secondary -> Unknown )
- Apr 11 10:14:45 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: repl( Ahead -> Off )
- Apr 11 10:14:45 drbd9-02 kernel: drbd r1 drbd9-03: ack_receiver terminated
- Apr 11 10:14:45 drbd9-02 kernel: drbd r1 drbd9-03: Terminating ack_recv thread
- Apr 11 10:14:45 drbd9-02 kernel: drbd r1 drbd9-03: sock was shut down by peer
- Apr 11 10:14:45 drbd9-02 kernel: drbd r1 drbd9-03: Connection closed
- Apr 11 10:14:45 drbd9-02 kernel: drbd r1 drbd9-03: conn( NetworkFailure -> Unconnected )
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1 drbd9-03: Restarting receiver thread
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1 drbd9-03: conn( Unconnected -> Connecting )
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1 drbd9-03: Handshake successful: Agreed network protocol version 111
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1 drbd9-03: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1 drbd9-03: Starting ack_recv thread (from drbd_r_r1 [1704])
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1: Preparing cluster-wide state change 3076172425 (1->2 499/145)
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1: State change 3076172425: primary_nodes=2, weak_nodes=FFFFFFFFFFFFFFF9
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1: Committing cluster-wide state change 3076172425 (0ms)
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1 drbd9-03: conn( Connecting -> Connected ) peer( Unknown -> Secondary )
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1/0 drbd1: peer does not support WRITE_SAME
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: updated UUIDs 9D2D64E77C03610B:0000000000000000:72BF2B4A67324B6C:15D43B83F5FF2550
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1/0 drbd1: bitmap WRITE of 5 pages took 7 ms
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: drbd_sync_handshake:
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: self 9D2D64E77C03610B:0000000000000000:72BF2B4A67324B6C:15D43B83F5FF2550 bits:235985 flags:0
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: peer 9D2D64E77C03610A:0000000000000000:72BF2B4A67324B6C:0000000000000000 bits:190914 flags:104
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: uuid_compare()=0 by rule 38
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1/0 drbd1: Becoming sync source due to disk states.
- Apr 11 10:14:46 drbd9-02 kernel: drbd r1/0 drbd1 drbd9-03: repl( Off -> WFBitMapS ) //-------------------------> pending!
- Apr 11 10:16:57 drbd9-02 kernel: INFO: task pool:2916 blocked for more than 120 seconds.
- Apr 11 10:16:57 drbd9-02 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Apr 11 10:16:57 drbd9-02 kernel: pool D ffff88003d613680 0 2916 1 0x00000080
- Apr 11 10:16:57 drbd9-02 kernel: ffff880027637b28 0000000000000086 ffff880014ef5b00 ffff880027637fd8
- Apr 11 10:16:57 drbd9-02 kernel: ffff880027637fd8 ffff880027637fd8 ffff880014ef5b00 ffff88003d613f48
- Apr 11 10:16:57 drbd9-02 kernel: ffff88003ffb8f50 0000000000000002 ffffffffa04e0720 ffff880027637ba0
- Apr 11 10:16:57 drbd9-02 kernel: Call Trace:
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffffa04e0720>] ? start_this_handle+0x5d0/0x5d0 [jbd2]
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffff816099ad>] io_schedule+0x9d/0x130
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffffa04e072e>] sleep_on_shadow_bh+0xe/0x20 [jbd2]
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffff81607780>] __wait_on_bit+0x60/0x90
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffff81012609>] ? __switch_to+0x179/0x4a0
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffffa04e0720>] ? start_this_handle+0x5d0/0x5d0 [jbd2]
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffff81607837>] out_of_line_wait_on_bit+0x87/0xb0
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffff81098270>] ? autoremove_wake_function+0x40/0x40
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffffa04e1c0d>] do_get_write_access+0x2ed/0x500 [jbd2]
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffff811f9eed>] ? __getblk+0x2d/0x2e0
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffffa04e1e47>] jbd2_journal_get_write_access+0x27/0x40 [jbd2]
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffffa05434cb>] __ext4_journal_get_write_access+0x3b/0x80 [ext4]
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffffa05159a0>] ext4_reserve_inode_write+0x70/0xa0 [ext4]
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffffa0518e30>] ? ext4_dirty_inode+0x40/0x60 [ext4]
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffffa0515a23>] ext4_mark_inode_dirty+0x53/0x210 [ext4]
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffffa0518e30>] ext4_dirty_inode+0x40/0x60 [ext4]
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffff811f069a>] __mark_inode_dirty+0xca/0x290
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffff811e1721>] update_time+0x81/0xd0
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffff811e1970>] file_update_time+0xa0/0xf0
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffff811f40d8>] generic_file_splice_write+0xc8/0x180
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffff811f4010>] ? pipe_to_file+0x170/0x170
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffff811f4603>] do_splice_from+0xc3/0x140
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffff811f619e>] SyS_splice+0x27e/0x690
- Apr 11 10:16:57 drbd9-02 kernel: [<ffffffff81614209>] system_call_fastpath+0x16/0x1b
- Apr 11 10:18:57 drbd9-02 kernel: INFO: task kworker/u256:0:6 blocked for more than 120 seconds.
- Apr 11 10:18:57 drbd9-02 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Apr 11 10:18:57 drbd9-02 kernel: kworker/u256:0 D ffff88003d613680 0 6 2 0x00000000
- Apr 11 10:18:57 drbd9-02 kernel: Workqueue: writeback bdi_writeback_workfn (flush-147:1)
- Apr 11 10:18:57 drbd9-02 kernel: ffff88003cfff948 0000000000000046 ffff88003cee38e0 ffff88003cffffd8
- Apr 11 10:18:57 drbd9-02 kernel: ffff88003cffffd8 ffff88003cffffd8 ffff88003cee38e0 ffff88002a743c00
- Apr 11 10:18:57 drbd9-02 kernel: ffff88002a743d50 ffff88002d595000 000000010000c0e1 ffff88002a743d74
- Apr 11 10:18:57 drbd9-02 kernel: Call Trace:
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffffa040759a>] drbd_make_request+0x14a/0x360 [drbd]
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff812aa56f>] ? generic_make_request_checks+0x24f/0x380
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff812aa782>] generic_make_request+0xe2/0x130
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff812aa841>] submit_bio+0x71/0x150
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffffa0519c55>] ext4_io_submit+0x25/0x50 [ext4]
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffffa05164b3>] ext4_writepages+0x8d3/0xd60 [ext4]
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff81162bfe>] do_writepages+0x1e/0x40
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff811f0000>] __writeback_single_inode+0x40/0x220
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff811f0cfe>] writeback_sb_inodes+0x25e/0x420
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff811f0f5f>] __writeback_inodes_wb+0x9f/0xd0
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff811f17a3>] wb_writeback+0x263/0x2f0
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff811f2cdc>] bdi_writeback_workfn+0x1cc/0x460
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff8108f0bb>] process_one_work+0x17b/0x470
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff8108fe8b>] worker_thread+0x11b/0x400
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff8108fd70>] ? rescuer_thread+0x400/0x400
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff8109726f>] kthread+0xcf/0xe0
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff81614158>] ret_from_fork+0x58/0x90
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Apr 11 10:18:57 drbd9-02 kernel: INFO: task pool:2916 blocked for more than 120 seconds.
- Apr 11 10:18:57 drbd9-02 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Apr 11 10:18:57 drbd9-02 kernel: pool D ffff88003d613680 0 2916 1 0x00000080
- Apr 11 10:18:57 drbd9-02 kernel: ffff880027637b28 0000000000000086 ffff880014ef5b00 ffff880027637fd8
- Apr 11 10:18:57 drbd9-02 kernel: ffff880027637fd8 ffff880027637fd8 ffff880014ef5b00 ffff88003d613f48
- Apr 11 10:18:57 drbd9-02 kernel: ffff88003ffb8f50 0000000000000002 ffffffffa04e0720 ffff880027637ba0
- Apr 11 10:18:57 drbd9-02 kernel: Call Trace:
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffffa04e0720>] ? start_this_handle+0x5d0/0x5d0 [jbd2]
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff816099ad>] io_schedule+0x9d/0x130
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffffa04e072e>] sleep_on_shadow_bh+0xe/0x20 [jbd2]
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff81607780>] __wait_on_bit+0x60/0x90
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff81012609>] ? __switch_to+0x179/0x4a0
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffffa04e0720>] ? start_this_handle+0x5d0/0x5d0 [jbd2]
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff81607837>] out_of_line_wait_on_bit+0x87/0xb0
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff81098270>] ? autoremove_wake_function+0x40/0x40
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffffa04e1c0d>] do_get_write_access+0x2ed/0x500 [jbd2]
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff811f9eed>] ? __getblk+0x2d/0x2e0
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffffa04e1e47>] jbd2_journal_get_write_access+0x27/0x40 [jbd2]
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffffa05434cb>] __ext4_journal_get_write_access+0x3b/0x80 [ext4]
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffffa05159a0>] ext4_reserve_inode_write+0x70/0xa0 [ext4]
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffffa0518e30>] ? ext4_dirty_inode+0x40/0x60 [ext4]
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffffa0515a23>] ext4_mark_inode_dirty+0x53/0x210 [ext4]
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffffa0518e30>] ext4_dirty_inode+0x40/0x60 [ext4]
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff811f069a>] __mark_inode_dirty+0xca/0x290
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff811e1721>] update_time+0x81/0xd0
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff811e1970>] file_update_time+0xa0/0xf0
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff811f40d8>] generic_file_splice_write+0xc8/0x180
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff811f4010>] ? pipe_to_file+0x170/0x170
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff811f4603>] do_splice_from+0xc3/0x140
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff811f619e>] SyS_splice+0x27e/0x690
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff81614209>] system_call_fastpath+0x16/0x1b
- Apr 11 10:18:57 drbd9-02 kernel: INFO: task jbd2/drbd1-8:2900 blocked for more than 120 seconds.
- Apr 11 10:18:57 drbd9-02 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Apr 11 10:18:57 drbd9-02 kernel: jbd2/drbd1-8 D ffff88003d613680 0 2900 2 0x00000080
- Apr 11 10:18:57 drbd9-02 kernel: ffff8800189c7b48 0000000000000046 ffff880014cda220 ffff8800189c7fd8
- Apr 11 10:18:57 drbd9-02 kernel: ffff8800189c7fd8 ffff8800189c7fd8 ffff880014cda220 ffff88002a743c00
- Apr 11 10:18:57 drbd9-02 kernel: ffff88002a743d50 ffff880014f86400 000000010000d287 ffff88002a743d74
- Apr 11 10:18:57 drbd9-02 kernel: Call Trace:
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffffa040759a>] drbd_make_request+0x14a/0x360 [drbd]
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff812aa56f>] ? generic_make_request_checks+0x24f/0x380
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff812aa782>] generic_make_request+0xe2/0x130
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff812aa841>] submit_bio+0x71/0x150
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff811fe98e>] ? bio_alloc_bioset+0x1be/0x2e0
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff811f99b3>] _submit_bh+0x143/0x210
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff811f9a90>] submit_bh+0x10/0x20
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffffa04e36d0>] jbd2_journal_commit_transaction+0x6c0/0x19a0 [jbd2]
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff810ad756>] ? __dequeue_entity+0x26/0x40
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff810125c6>] ? __switch_to+0x136/0x4a0
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffffa04e8da9>] kjournald2+0xc9/0x260 [jbd2]
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffffa04e8ce0>] ? commit_timeout+0x10/0x10 [jbd2]
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff8109726f>] kthread+0xcf/0xe0
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff81614158>] ret_from_fork+0x58/0x90
- Apr 11 10:18:57 drbd9-02 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Apr 11 10:19:05 drbd9-02 chronyd[913]: Selected source 114.70.64.57
- Apr 11 10:20:02 drbd9-02 systemd: Starting Session 3 of user root.
- Apr 11 10:20:02 drbd9-02 systemd: Started Session 3 of user root.
- Apr 11 10:20:57 drbd9-02 kernel: INFO: task kworker/u256:0:6 blocked for more than 120 seconds.
- Apr 11 10:20:57 drbd9-02 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Apr 11 10:20:57 drbd9-02 kernel: kworker/u256:0 D ffff88003d613680 0 6 2 0x00000000
- Apr 11 10:20:57 drbd9-02 kernel: Workqueue: writeback bdi_writeback_workfn (flush-147:1)
- Apr 11 10:20:57 drbd9-02 kernel: ffff88003cfff948 0000000000000046 ffff88003cee38e0 ffff88003cffffd8
- Apr 11 10:20:57 drbd9-02 kernel: ffff88003cffffd8 ffff88003cffffd8 ffff88003cee38e0 ffff88002a743c00
- Apr 11 10:20:57 drbd9-02 kernel: ffff88002a743d50 ffff88002d595000 000000010000c0e1 ffff88002a743d74
- Apr 11 10:20:57 drbd9-02 kernel: Call Trace:
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffffa040759a>] drbd_make_request+0x14a/0x360 [drbd]
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff812aa56f>] ? generic_make_request_checks+0x24f/0x380
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff812aa782>] generic_make_request+0xe2/0x130
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff812aa841>] submit_bio+0x71/0x150
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffffa0519c55>] ext4_io_submit+0x25/0x50 [ext4]
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffffa05164b3>] ext4_writepages+0x8d3/0xd60 [ext4]
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff81162bfe>] do_writepages+0x1e/0x40
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff811f0000>] __writeback_single_inode+0x40/0x220
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff811f0cfe>] writeback_sb_inodes+0x25e/0x420
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff811f0f5f>] __writeback_inodes_wb+0x9f/0xd0
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff811f17a3>] wb_writeback+0x263/0x2f0
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff811f2cdc>] bdi_writeback_workfn+0x1cc/0x460
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff8108f0bb>] process_one_work+0x17b/0x470
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff8108fe8b>] worker_thread+0x11b/0x400
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff8108fd70>] ? rescuer_thread+0x400/0x400
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff8109726f>] kthread+0xcf/0xe0
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff81614158>] ret_from_fork+0x58/0x90
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Apr 11 10:20:57 drbd9-02 kernel: INFO: task pool:2916 blocked for more than 120 seconds.
- Apr 11 10:20:57 drbd9-02 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Apr 11 10:20:57 drbd9-02 kernel: pool D ffff88003d613680 0 2916 1 0x00000080
- Apr 11 10:20:57 drbd9-02 kernel: ffff880027637b28 0000000000000086 ffff880014ef5b00 ffff880027637fd8
- Apr 11 10:20:57 drbd9-02 kernel: ffff880027637fd8 ffff880027637fd8 ffff880014ef5b00 ffff88003d613f48
- Apr 11 10:20:57 drbd9-02 kernel: ffff88003ffb8f50 0000000000000002 ffffffffa04e0720 ffff880027637ba0
- Apr 11 10:20:57 drbd9-02 kernel: Call Trace:
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffffa04e0720>] ? start_this_handle+0x5d0/0x5d0 [jbd2]
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff816099ad>] io_schedule+0x9d/0x130
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffffa04e072e>] sleep_on_shadow_bh+0xe/0x20 [jbd2]
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff81607780>] __wait_on_bit+0x60/0x90
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff81012609>] ? __switch_to+0x179/0x4a0
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffffa04e0720>] ? start_this_handle+0x5d0/0x5d0 [jbd2]
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff81607837>] out_of_line_wait_on_bit+0x87/0xb0
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff81098270>] ? autoremove_wake_function+0x40/0x40
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffffa04e1c0d>] do_get_write_access+0x2ed/0x500 [jbd2]
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff811f9eed>] ? __getblk+0x2d/0x2e0
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffffa04e1e47>] jbd2_journal_get_write_access+0x27/0x40 [jbd2]
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffffa05434cb>] __ext4_journal_get_write_access+0x3b/0x80 [ext4]
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffffa05159a0>] ext4_reserve_inode_write+0x70/0xa0 [ext4]
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffffa0518e30>] ? ext4_dirty_inode+0x40/0x60 [ext4]
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffffa0515a23>] ext4_mark_inode_dirty+0x53/0x210 [ext4]
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffffa0518e30>] ext4_dirty_inode+0x40/0x60 [ext4]
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff811f069a>] __mark_inode_dirty+0xca/0x290
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff811e1721>] update_time+0x81/0xd0
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff811e1970>] file_update_time+0xa0/0xf0
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff811f40d8>] generic_file_splice_write+0xc8/0x180
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff811f4010>] ? pipe_to_file+0x170/0x170
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff811f4603>] do_splice_from+0xc3/0x140
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff811f619e>] SyS_splice+0x27e/0x690
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff81614209>] system_call_fastpath+0x16/0x1b
- Apr 11 10:20:57 drbd9-02 kernel: INFO: task jbd2/drbd1-8:2900 blocked for more than 120 seconds.
- Apr 11 10:20:57 drbd9-02 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Apr 11 10:20:57 drbd9-02 kernel: jbd2/drbd1-8 D ffff88003d613680 0 2900 2 0x00000080
- Apr 11 10:20:57 drbd9-02 kernel: ffff8800189c7b48 0000000000000046 ffff880014cda220 ffff8800189c7fd8
- Apr 11 10:20:57 drbd9-02 kernel: ffff8800189c7fd8 ffff8800189c7fd8 ffff880014cda220 ffff88002a743c00
- Apr 11 10:20:57 drbd9-02 kernel: ffff88002a743d50 ffff880014f86400 000000010000d287 ffff88002a743d74
- Apr 11 10:20:57 drbd9-02 kernel: Call Trace:
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffffa040759a>] drbd_make_request+0x14a/0x360 [drbd]
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff812aa56f>] ? generic_make_request_checks+0x24f/0x380
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff812aa782>] generic_make_request+0xe2/0x130
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff812aa841>] submit_bio+0x71/0x150
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff811fe98e>] ? bio_alloc_bioset+0x1be/0x2e0
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff811f99b3>] _submit_bh+0x143/0x210
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff811f9a90>] submit_bh+0x10/0x20
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffffa04e36d0>] jbd2_journal_commit_transaction+0x6c0/0x19a0 [jbd2]
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff810ad756>] ? __dequeue_entity+0x26/0x40
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff810125c6>] ? __switch_to+0x136/0x4a0
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffffa04e8da9>] kjournald2+0xc9/0x260 [jbd2]
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffffa04e8ce0>] ? commit_timeout+0x10/0x10 [jbd2]
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff8109726f>] kthread+0xcf/0xe0
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff81614158>] ret_from_fork+0x58/0x90
- Apr 11 10:20:57 drbd9-02 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Apr 11 10:22:57 drbd9-02 kernel: INFO: task kworker/u256:0:6 blocked for more than 120 seconds.
- Apr 11 10:22:57 drbd9-02 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Apr 11 10:22:57 drbd9-02 kernel: kworker/u256:0 D ffff88003d613680 0 6 2 0x00000000
- Apr 11 10:22:57 drbd9-02 kernel: Workqueue: writeback bdi_writeback_workfn (flush-147:1)
- Apr 11 10:22:57 drbd9-02 kernel: ffff88003cfff948 0000000000000046 ffff88003cee38e0 ffff88003cffffd8
- Apr 11 10:22:57 drbd9-02 kernel: ffff88003cffffd8 ffff88003cffffd8 ffff88003cee38e0 ffff88002a743c00
- Apr 11 10:22:57 drbd9-02 kernel: ffff88002a743d50 ffff88002d595000 000000010000c0e1 ffff88002a743d74
- Apr 11 10:22:57 drbd9-02 kernel: Call Trace:
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffffa040759a>] drbd_make_request+0x14a/0x360 [drbd]
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff812aa56f>] ? generic_make_request_checks+0x24f/0x380
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff812aa782>] generic_make_request+0xe2/0x130
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff812aa841>] submit_bio+0x71/0x150
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffffa0519c55>] ext4_io_submit+0x25/0x50 [ext4]
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffffa05164b3>] ext4_writepages+0x8d3/0xd60 [ext4]
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff81162bfe>] do_writepages+0x1e/0x40
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff811f0000>] __writeback_single_inode+0x40/0x220
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff811f0cfe>] writeback_sb_inodes+0x25e/0x420
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff811f0f5f>] __writeback_inodes_wb+0x9f/0xd0
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff811f17a3>] wb_writeback+0x263/0x2f0
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff811f2cdc>] bdi_writeback_workfn+0x1cc/0x460
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff8108f0bb>] process_one_work+0x17b/0x470
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff8108fe8b>] worker_thread+0x11b/0x400
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff8108fd70>] ? rescuer_thread+0x400/0x400
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff8109726f>] kthread+0xcf/0xe0
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff81614158>] ret_from_fork+0x58/0x90
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Apr 11 10:22:57 drbd9-02 kernel: INFO: task pool:2916 blocked for more than 120 seconds.
- Apr 11 10:22:57 drbd9-02 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Apr 11 10:22:57 drbd9-02 kernel: pool D ffff88003d613680 0 2916 1 0x00000080
- Apr 11 10:22:57 drbd9-02 kernel: ffff880027637b28 0000000000000086 ffff880014ef5b00 ffff880027637fd8
- Apr 11 10:22:57 drbd9-02 kernel: ffff880027637fd8 ffff880027637fd8 ffff880014ef5b00 ffff88003d613f48
- Apr 11 10:22:57 drbd9-02 kernel: ffff88003ffb8f50 0000000000000002 ffffffffa04e0720 ffff880027637ba0
- Apr 11 10:22:57 drbd9-02 kernel: Call Trace:
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffffa04e0720>] ? start_this_handle+0x5d0/0x5d0 [jbd2]
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff816099ad>] io_schedule+0x9d/0x130
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffffa04e072e>] sleep_on_shadow_bh+0xe/0x20 [jbd2]
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff81607780>] __wait_on_bit+0x60/0x90
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff81012609>] ? __switch_to+0x179/0x4a0
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffffa04e0720>] ? start_this_handle+0x5d0/0x5d0 [jbd2]
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff81607837>] out_of_line_wait_on_bit+0x87/0xb0
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff81098270>] ? autoremove_wake_function+0x40/0x40
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffffa04e1c0d>] do_get_write_access+0x2ed/0x500 [jbd2]
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff811f9eed>] ? __getblk+0x2d/0x2e0
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffffa04e1e47>] jbd2_journal_get_write_access+0x27/0x40 [jbd2]
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffffa05434cb>] __ext4_journal_get_write_access+0x3b/0x80 [ext4]
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffffa05159a0>] ext4_reserve_inode_write+0x70/0xa0 [ext4]
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffffa0518e30>] ? ext4_dirty_inode+0x40/0x60 [ext4]
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffffa0515a23>] ext4_mark_inode_dirty+0x53/0x210 [ext4]
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffffa0518e30>] ext4_dirty_inode+0x40/0x60 [ext4]
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff811f069a>] __mark_inode_dirty+0xca/0x290
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff811e1721>] update_time+0x81/0xd0
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff811e1970>] file_update_time+0xa0/0xf0
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff811f40d8>] generic_file_splice_write+0xc8/0x180
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff811f4010>] ? pipe_to_file+0x170/0x170
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff811f4603>] do_splice_from+0xc3/0x140
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff811f619e>] SyS_splice+0x27e/0x690
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff81614209>] system_call_fastpath+0x16/0x1b
- Apr 11 10:22:57 drbd9-02 kernel: INFO: task jbd2/drbd1-8:2900 blocked for more than 120 seconds.
- Apr 11 10:22:57 drbd9-02 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
- Apr 11 10:22:57 drbd9-02 kernel: jbd2/drbd1-8 D ffff88003d613680 0 2900 2 0x00000080
- Apr 11 10:22:57 drbd9-02 kernel: ffff8800189c7b48 0000000000000046 ffff880014cda220 ffff8800189c7fd8
- Apr 11 10:22:57 drbd9-02 kernel: ffff8800189c7fd8 ffff8800189c7fd8 ffff880014cda220 ffff88002a743c00
- Apr 11 10:22:57 drbd9-02 kernel: ffff88002a743d50 ffff880014f86400 000000010000d287 ffff88002a743d74
- Apr 11 10:22:57 drbd9-02 kernel: Call Trace:
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff816096a9>] schedule+0x29/0x70
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffffa040759a>] drbd_make_request+0x14a/0x360 [drbd]
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff812aa56f>] ? generic_make_request_checks+0x24f/0x380
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff812aa782>] generic_make_request+0xe2/0x130
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff812aa841>] submit_bio+0x71/0x150
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff811fe98e>] ? bio_alloc_bioset+0x1be/0x2e0
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff811f99b3>] _submit_bh+0x143/0x210
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff811f9a90>] submit_bh+0x10/0x20
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffffa04e36d0>] jbd2_journal_commit_transaction+0x6c0/0x19a0 [jbd2]
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff810ad756>] ? __dequeue_entity+0x26/0x40
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff810125c6>] ? __switch_to+0x136/0x4a0
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffffa04e8da9>] kjournald2+0xc9/0x260 [jbd2]
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff81098230>] ? wake_up_bit+0x30/0x30
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffffa04e8ce0>] ? commit_timeout+0x10/0x10 [jbd2]
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff8109726f>] kthread+0xcf/0xe0
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff81614158>] ret_from_fork+0x58/0x90
- Apr 11 10:22:57 drbd9-02 kernel: [<ffffffff810971a0>] ? kthread_create_on_node+0x140/0x140
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement