[ 622.580106] [ 622.581632] ========================================================= [ 622.588063] [ INFO: possible irq lock inversion dependency detected ] [ 622.594498] 3.19.0-rc1-ARCH+ #1 Not tainted [ 622.598681] --------------------------------------------------------- [ 622.605112] kworker/0:1/162 just changed the state of lock: [ 622.610674] (&(&blkcg->lock)->rlock){+.+...}, at: [] blkcg_css_offline+0x48/0x60 [ 622.619135] but this lock was taken by another, SOFTIRQ-safe lock in the past: [ 622.626345] (&(&q->__queue_lock)->rlock){+.-...} and interrupts could create inverse lock ordering between them. [ 622.637078] [ 622.637078] other info that might help us debug this: [ 622.643601] Possible interrupt unsafe locking scenario: [ 622.643601] [ 622.650381] CPU0 CPU1 [ 622.654909] ---- ---- [ 622.659435] lock(&(&blkcg->lock)->rlock); [ 622.663663] local_irq_disable(); [ 622.669570] lock(&(&q->__queue_lock)->rlock); [ 622.676639] lock(&(&blkcg->lock)->rlock); [ 622.683369] [ 622.685994] lock(&(&q->__queue_lock)->rlock); [ 622.690739] [ 622.690739] *** DEADLOCK *** [ 622.690739] [ 622.696664] 5 locks held by kworker/0:1/162: [ 622.700935] #0: ("cgroup_destroy"){.+.+..}, at: [] process_one_work+0x198/0x55c [ 622.709377] #1: ((&css->destroy_work)#3){+.+...}, at: [] process_one_work+0x198/0x55c [ 622.718345] #2: (cgroup_mutex){+.+.+.}, at: [] css_killed_work_fn+0x18/0x1b4 [ 622.726519] #3: (&(&blkcg->lock)->rlock){+.+...}, at: [] blkcg_css_offline+0x48/0x60 [ 622.735392] #4: (&(&q->__queue_lock)->rlock){+.-...}, at: [] blkcg_css_offline+0x1c/0x60 [ 622.744621] [ 622.744621] the shortest dependencies between 2nd lock and 1st lock: [ 622.752482] -> (&(&q->__queue_lock)->rlock){+.-...} ops: 16731 { [ 622.758669] HARDIRQ-ON-W at: [ 622.761917] [] cfqg_stats_merge+0x3c/0x2a0 [ 622.768821] [] cfq_pd_offline+0xa4/0xe8 [ 622.775465] [] blkg_destroy+0x108/0x238 [ 622.782109] [] blkcg_css_offline+0x2c/0x60 [ 622.789012] [] css_killed_work_fn+0x5c/0x1b4 [ 622.796080] [] process_one_work+0x2a4/0x55c [ 622.803069] [] worker_thread+0x2a8/0x42c [ 622.809791] [] kthread+0xc0/0xd4 [ 622.815822] [] ret_from_fork+0x14/0x34 [ 622.822373] IN-SOFTIRQ-W at: [ 622.825620] [] _raw_spin_lock_irqsave+0x44/0x58 [ 622.832946] [] scsi_end_request+0x114/0x15c [ 622.839927] [] scsi_io_completion+0x1c0/0x50c [ 622.847073] [] blk_done_softirq+0x78/0x8c [ 622.853890] [] __do_softirq+0x1a8/0x448 [ 622.860533] [] run_ksoftirqd+0x2c/0x60 [ 622.867074] [] smpboot_thread_fn+0x284/0x298 [ 622.874150] [] kthread+0xc0/0xd4 [ 622.880172] [] ret_from_fork+0x14/0x34 [ 622.886715] INITIAL USE at: [ 622.889875] [] _raw_spin_lock_irq+0x48/0x58 [ 622.896769] [] blk_queue_bypass_start+0x10/0xa8 [ 622.904027] [] blkcg_activate_policy+0x4c/0x2f4 [ 622.911267] [] blk_throtl_init+0xa8/0xec [ 622.917894] [] blk_alloc_queue_node+0x1ac/0x220 [ 622.925135] [] loop_add+0x84/0x1ec [ 622.931261] [] loop_init+0xcc/0x124 [ 622.937473] [] do_one_initcall+0xf8/0x1d0 [ 622.944186] [] kernel_init_freeable+0x1d4/0x2a4 [ 622.951436] [] kernel_init+0x8/0xe4 [ 622.957656] [] ret_from_fork+0x14/0x34 [ 622.964119] } [ 622.965880] ... key at: [] __key.36420+0x0/0x8 [ 622.971827] ... acquired at: [ 622.974882] [] blk_queue_bypass_start+0x10/0xa8 [ 622.980578] [] blkcg_activate_policy+0x4c/0x2f4 [ 622.986262] [] blk_throtl_init+0xa8/0xec [ 622.991334] [] blk_alloc_queue_node+0x1ac/0x220 [ 622.997019] [] loop_add+0x84/0x1ec [ 623.001572] [] loop_init+0xcc/0x124 [ 623.006212] [] do_one_initcall+0xf8/0x1d0 [ 623.011370] [] kernel_init_freeable+0x1d4/0x2a4 [ 623.017047] [] kernel_init+0x8/0xe4 [ 623.021704] [] ret_from_fork+0x14/0x34 [ 623.026602] [ 623.028106] -> (&(&blkcg->lock)->rlock){+.+...} ops: 42 { [ 623.033594] HARDIRQ-ON-W at: [ 623.036755] [] cfqg_stats_merge+0x3c/0x2a0 [ 623.043470] [] cfq_pd_offline+0xa4/0xe8 [ 623.049932] [] blkg_destroy+0x108/0x238 [ 623.056403] [] blkcg_css_offline+0x2c/0x60 [ 623.063133] [] css_killed_work_fn+0x5c/0x1b4 [ 623.070036] [] process_one_work+0x2a4/0x55c [ 623.076844] [] worker_thread+0x2a8/0x42c [ 623.083386] [] kthread+0xc0/0xd4 [ 623.089242] [] ret_from_fork+0x14/0x34 [ 623.095612] SOFTIRQ-ON-W at: [ 623.098773] [] cfqg_stats_merge+0x3c/0x2a0 [ 623.105487] [] cfq_pd_offline+0xa4/0xe8 [ 623.111940] [] blkg_destroy+0x108/0x238 [ 623.118403] [] blkcg_css_offline+0x2c/0x60 [ 623.125134] [] css_killed_work_fn+0x5c/0x1b4 [ 623.132028] [] process_one_work+0x2a4/0x55c [ 623.138837] [] worker_thread+0x2a8/0x42c [ 623.145386] [] kthread+0xc0/0xd4 [ 623.151235] [] ret_from_fork+0x14/0x34 [ 623.157604] INITIAL USE at: [ 623.160679] [] _raw_spin_lock+0x38/0x48 [ 623.167055] [] blkg_create+0x274/0x518 [ 623.173345] [] blkcg_activate_policy+0xbc/0x2f4 [ 623.180421] [] blk_throtl_init+0xa8/0xec [ 623.186875] [] blk_alloc_queue_node+0x1ac/0x220 [ 623.193952] [] loop_add+0x84/0x1ec [ 623.199905] [] loop_init+0xcc/0x124 [ 623.205943] [] do_one_initcall+0xf8/0x1d0 [ 623.212484] [] kernel_init_freeable+0x1d4/0x2a4 [ 623.219543] [] kernel_init+0x8/0xe4 [ 623.225583] [] ret_from_fork+0x14/0x34 [ 623.231873] } [ 623.233548] ... key at: [] __key.30063+0x0/0x8 [ 623.239407] ... acquired at: [ 623.242376] [] mark_held_locks+0x64/0x84 [ 623.247467] [] trace_hardirqs_on_caller+0x14c/0x1e8 [ 623.253498] [] cfqg_stats_merge+0x3c/0x2a0 [ 623.258742] [] cfq_pd_offline+0xa4/0xe8 [ 623.263727] [] blkg_destroy+0x108/0x238 [ 623.268721] [] blkcg_css_offline+0x2c/0x60 [ 623.273974] [] css_killed_work_fn+0x5c/0x1b4 [ 623.279400] [] process_one_work+0x2a4/0x55c [ 623.284740] [] worker_thread+0x2a8/0x42c [ 623.289811] [] kthread+0xc0/0xd4 [ 623.294192] [] ret_from_fork+0x14/0x34 [ 623.299091] [ 623.300594] [ 623.300594] stack backtrace: [ 623.304970] CPU: 0 PID: 162 Comm: kworker/0:1 Not tainted 3.19.0-rc1-ARCH+ #1 [ 623.312097] Hardware name: Marvell Orion5x (Flattened Device Tree) [ 623.318281] Workqueue: cgroup_destroy css_killed_work_fn [ 623.323645] [] (unwind_backtrace) from [] (show_stack+0x10/0x14) [ 623.331413] [] (show_stack) from [] (print_irq_inversion_bug.part.24+0x170/0x1a8) [ 623.340654] [] (print_irq_inversion_bug.part.24) from [] (check_usage_backwards+0x10c/0x148) [ 623.350837] [] (check_usage_backwards) from [] (mark_lock+0x600/0x660) [ 623.359120] [] (mark_lock) from [] (mark_held_locks+0x64/0x84) [ 623.366709] [] (mark_held_locks) from [] (trace_hardirqs_on_caller+0x14c/0x1e8) [ 623.375771] [] (trace_hardirqs_on_caller) from [] (cfqg_stats_merge+0x3c/0x2a0) [ 623.384820] [] (cfqg_stats_merge) from [] (cfq_pd_offline+0xa4/0xe8) [ 623.392928] [] (cfq_pd_offline) from [] (blkg_destroy+0x108/0x238) [ 623.400863] [] (blkg_destroy) from [] (blkcg_css_offline+0x2c/0x60) [ 623.408891] [] (blkcg_css_offline) from [] (css_killed_work_fn+0x5c/0x1b4) [ 623.417516] [] (css_killed_work_fn) from [] (process_one_work+0x2a4/0x55c) [ 623.426140] [] (process_one_work) from [] (worker_thread+0x2a8/0x42c) [ 623.434327] [] (worker_thread) from [] (kthread+0xc0/0xd4) [ 623.441564] [] (kthread) from [] (ret_from_fork+0x14/0x34)