INFO: task xfsaild/sda3:598 blocked for more than 120 seconds. Not tainted 4.1.14+63-ph #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. xfsaild/sda3 D ffff88042a8d7d18 0 598 2 0x00000000 ffff88042a8d7d18 ffff880102c93aa0 ffff88042c248000 0000000000000282 ffff88042a8d8000 ffff88042c248000 0000000000000000 ffff88042b1f9800 ffff88042bba4100 ffff88042a8d7d38 ffffffff9363c167 ffff88042b1f9928 Call Trace: [] schedule+0x37/0x90 [] _xfs_log_force+0x189/0x280 [] ? try_to_wake_up+0x330/0x330 [] xfs_log_force+0x2a/0x90 [] ? xfs_trans_ail_cursor_first+0x90/0x90 [] xfsaild+0x143/0x5f0 [] ? __schedule+0x4f1/0x990 [] ? xfs_trans_ail_cursor_first+0x90/0x90 [] kthread+0xc9/0xe0 [] ? kthread_create_on_node+0x1a0/0x1a0 [] ret_from_fork+0x42/0x70 [] ? kthread_create_on_node+0x1a0/0x1a0 INFO: task xfsaild/sdb1:2068 blocked for more than 120 seconds. Not tainted 4.1.14+63-ph #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. xfsaild/sdb1 D ffff88042529fad8 0 2068 2 0x00000000 ffff88042529fad8 ffff88082d948000 ffff8800b94ad7f0 ffff88082a8401d0 ffff8804252a0000 ffff88042529fc40 7fffffffffffffff ffff8800b94ad7f0 ffffffffffffffff ffff88042529faf8 ffffffff9363c167 0000000000000000 Call Trace: [] schedule+0x37/0x90 [] schedule_timeout+0x1fc/0x260 [] ? part_round_stats+0x5c/0x70 [] ? kernfs_path_locked+0x49/0xa0 [] ? elv_rb_add+0x62/0x70 [] wait_for_completion+0x9f/0x170 [] ? try_to_wake_up+0x330/0x330 [] flush_work+0xff/0x1d0 [] ? worker_detach_from_pool+0xd0/0xd0 [] xlog_cil_force_lsn+0x7e/0x200 [] ? try_to_del_timer_sync+0x4f/0x70 [] _xfs_log_force+0x6b/0x280 [] ? xfs_buf_delwri_submit_nowait+0x20/0x30 [] xfs_log_force+0x2a/0x90 [] ? xfs_trans_ail_cursor_first+0x90/0x90 [] xfsaild+0x143/0x5f0 [] ? __schedule+0x4f1/0x990 [] ? xfs_trans_ail_cursor_first+0x90/0x90 [] kthread+0xc9/0xe0 [] ? kthread_create_on_node+0x1a0/0x1a0 [] ret_from_fork+0x42/0x70 [] ? kthread_create_on_node+0x1a0/0x1a0 INFO: task mysqld:7538 blocked for more than 120 seconds. Not tainted 4.1.14+63-ph #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. mysqld D ffff8803e708bd98 0 7538 3771 0x00000000 ffff8803e708bd98 ffff88082d941d50 ffff8803f1ebd7f0 ffff88042bba4c00 ffff8803e708c000 ffff88042b1f9928 ffff88042b1f9800 0000000000000000 ffff8803f1ebd7f0 ffff8803e708bdb8 ffffffff9363c167 ffff8803f1ebd7f0 Call Trace: [] schedule+0x37/0x90 [] _xfs_log_force_lsn+0x26b/0x2c0 [] ? try_to_wake_up+0x330/0x330 [] xfs_file_fsync+0x18f/0x1e0 [] vfs_fsync_range+0x46/0xc0 [] vfs_fsync+0x1c/0x20 [] do_fsync+0x38/0x70 [] ? context_tracking_user_enter+0x13/0x20 [] ? syscall_trace_leave+0x95/0xf0 [] SyS_fsync+0x10/0x20 [] system_call_fastpath+0x12/0x71 INFO: task pop3-login:7921 blocked for more than 120 seconds. Not tainted 4.1.14+63-ph #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. pop3-login D ffff88082c5234f8 0 7921 4498 0x00000000 ffff88082c5234f8 ffff88042dbdbaa0 ffff880827443aa0 ffff88082c523568 ffff88082c524000 ffff88082bb63b6c ffff880827443aa0 00000000ffffffff ffff88082bb63b70 ffff88082c523518 ffffffff9363c167 0000000000000000 Call Trace: [] schedule+0x37/0x90 [] schedule_preempt_disabled+0xe/0x10 [] __mutex_lock_slowpath+0x95/0x110 [] mutex_lock+0x1b/0x2f [] xfs_reclaim_inodes_ag+0x307/0x3d0 [] ? spin_unlock+0x10/0x10 [] ? __call_rcu+0x3f/0x360 [] ? inode_lru_isolate+0x89/0x1c0 [] ? __list_lru_walk_one.isra.6+0x158/0x170 [] ? insert_inode_locked+0x180/0x180 [] ? insert_inode_locked+0x180/0x180 [] xfs_reclaim_inodes_nr+0x33/0x40 [] xfs_fs_free_cached_objects+0x19/0x20 [] super_cache_scan+0x1a3/0x1b0 [] shrink_slab+0x216/0x3d0 [] shrink_zone+0x2b9/0x2d0 [] do_try_to_free_pages+0x16a/0x400 [] try_to_free_pages+0xda/0x160 [] __alloc_pages_slowpath+0x3ff/0x697 [] __alloc_pages_nodemask+0x230/0x240 [] alloc_pages_current+0x9c/0x110 [] new_slab+0x32b/0x450 [] __slab_alloc+0x2de/0x433 [] ? sk_prot_alloc.isra.38+0x41/0x170 [] kmem_cache_alloc+0x193/0x1f0 [] ? tcp_write_xmit+0x2b1/0xda0 [] sk_prot_alloc.isra.38+0x41/0x170 [] sk_alloc+0x33/0xb0 [] unix_create1+0x51/0x1a0 [] unix_create+0x73/0x90 [] __sock_create+0xf0/0x1f0 [] ? rcu_eqs_enter+0x68/0x90 [] sock_create+0x30/0x40 [] SyS_socketpair+0x6c/0x1d0 [] ? syscall_trace_leave+0x95/0xf0 [] system_call_fastpath+0x12/0x71 INFO: task imap:26593 blocked for more than 120 seconds. Not tainted 4.1.14+63-ph #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. imap D ffff880675a5fdc8 0 26593 4498 0x00000000 ffff880675a5fdc8 ffff88082d943aa0 ffff8807abab0000 ffff8806b0702f00 ffff880675a60000 ffff88042b1f9928 ffff88042b1f9800 ffff88042bba4c18 ffff8807abab0000 ffff880675a5fde8 ffffffff9363c167 0000fddf00010800 Call Trace: [] schedule+0x37/0x90 [] _xfs_log_force_lsn+0x153/0x2c0 [] ? try_to_wake_up+0x330/0x330 [] xfs_dir_fsync+0x86/0xf0 [] vfs_fsync_range+0x46/0xc0 [] vfs_fsync+0x1c/0x20 [] do_fsync+0x38/0x70 [] ? context_tracking_user_enter+0x13/0x20 [] ? syscall_trace_leave+0x95/0xf0 [] SyS_fdatasync+0x13/0x20 [] system_call_fastpath+0x12/0x71 INFO: task kworker/7:2:22195 blocked for more than 120 seconds. Not tainted 4.1.14+63-ph #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kworker/7:2 D ffff88009ac93108 0 22195 2 0x00000000 ffff88009ac93108 ffff88082d941d50 ffff88044840d7f0 0000000950d64c61 ffff88009ac94000 ffff88009ac93270 7fffffffffffffff ffff88044840d7f0 ffffffffffffffff ffff88009ac93128 ffffffff9363c167 74612e656e6f6568 Call Trace: [] schedule+0x37/0x90 [] schedule_timeout+0x1fc/0x260 [] wait_for_completion+0x9f/0x170 [] ? try_to_wake_up+0x330/0x330 [] flush_work+0xff/0x1d0 [] ? worker_detach_from_pool+0xd0/0xd0 [] xlog_cil_force_lsn+0x7e/0x200 [] _xfs_log_force_lsn+0x58/0x2c0 [] ? xfs_iext_bno_to_ext+0x97/0x170 [] xfs_log_force_lsn+0x3b/0xa0 [] ? xfs_reclaim_inode+0xd0/0x330 [] xfs_iunpin_wait+0xae/0x140 [] ? wake_atomic_t_function+0x40/0x40 [] xfs_reclaim_inode+0xd0/0x330 [] xfs_reclaim_inodes_ag+0x277/0x3d0 [] xfs_reclaim_inodes_nr+0x33/0x40 [] xfs_fs_free_cached_objects+0x19/0x20 [] super_cache_scan+0x1a3/0x1b0 [] shrink_slab+0x216/0x3d0 [] shrink_zone+0x2b9/0x2d0 [] do_try_to_free_pages+0x16a/0x400 [] try_to_free_pages+0xda/0x160 [] __alloc_pages_slowpath+0x3ff/0x697 [] __alloc_pages_nodemask+0x230/0x240 [] alloc_pages_current+0x9c/0x110 [] new_slab+0x32b/0x450 [] __slab_alloc+0x2de/0x433 [] ? ida_pre_get+0x69/0x100 [] kmem_cache_alloc+0x193/0x1f0 [] ? native_sched_clock+0x35/0x80 [] ida_pre_get+0x69/0x100 [] ida_simple_get+0x4e/0xf0 [] ? finish_task_switch+0x11e/0x1a0 [] create_worker+0x4b/0x1f0 [] ? mod_timer+0x14a/0x1c0 [] worker_thread+0x3ef/0x510 [] ? __schedule+0x4f1/0x990 [] ? process_scheduled_works+0x40/0x40 [] kthread+0xc9/0xe0 [] ? kthread_create_on_node+0x1a0/0x1a0 [] ret_from_fork+0x42/0x70 [] ? kthread_create_on_node+0x1a0/0x1a0 INFO: task kworker/7:0H:7845 blocked for more than 120 seconds. Not tainted 4.1.14+63-ph #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kworker/7:0H D ffff8806ec7ff108 0 7845 2 0x00000000 ffff8806ec7ff108 ffff88044840d7f0 ffff88082762baa0 ffff88044840d7f0 ffff8806ec800000 ffff8806ec7ff270 7fffffffffffffff ffff88082762baa0 ffffffffffffffff ffff8806ec7ff128 ffffffff9363c167 ffff8806ec7ff148 Call Trace: [] schedule+0x37/0x90 [] schedule_timeout+0x1fc/0x260 [] ? ttwu_do_activate.constprop.91+0x66/0x70 [] ? try_to_wake_up+0x1f3/0x330 [] ? cfq_set_request+0x2ff/0x4c0 [] wait_for_completion+0x9f/0x170 [] ? try_to_wake_up+0x330/0x330 [] flush_work+0xff/0x1d0 [] ? worker_detach_from_pool+0xd0/0xd0 [] xlog_cil_force_lsn+0x7e/0x200 [] ? __cfq_set_active_queue+0xf8/0x230 [] _xfs_log_force_lsn+0x58/0x2c0 [] xfs_log_force_lsn+0x3b/0xa0 [] ? xfs_reclaim_inode+0xd0/0x330 [] xfs_iunpin_wait+0xae/0x140 [] ? wake_atomic_t_function+0x40/0x40 [] xfs_reclaim_inode+0xd0/0x330 [] xfs_reclaim_inodes_ag+0x277/0x3d0 [] xfs_reclaim_inodes_nr+0x33/0x40 [] xfs_fs_free_cached_objects+0x19/0x20 [] super_cache_scan+0x1a3/0x1b0 [] shrink_slab+0x216/0x3d0 [] shrink_zone+0x2b9/0x2d0 [] do_try_to_free_pages+0x16a/0x400 [] try_to_free_pages+0xda/0x160 [] __alloc_pages_slowpath+0x3ff/0x697 [] __alloc_pages_nodemask+0x230/0x240 [] alloc_pages_current+0x9c/0x110 [] new_slab+0x32b/0x450 [] ? try_to_wake_up+0x1f3/0x330 [] __slab_alloc+0x2de/0x433 [] ? ida_pre_get+0x69/0x100 [] ? __wake_up+0x53/0x70 [] kmem_cache_alloc+0x193/0x1f0 [] ? pick_next_entity+0x90/0x170 [] ida_pre_get+0x69/0x100 [] ida_simple_get+0x4e/0xf0 [] ? finish_task_switch+0x11e/0x1a0 [] create_worker+0x4b/0x1f0 [] ? mod_timer+0x14a/0x1c0 [] worker_thread+0x3ef/0x510 [] ? __schedule+0x4f1/0x990 [] ? process_scheduled_works+0x40/0x40 [] kthread+0xc9/0xe0 [] ? kthread_create_on_node+0x1a0/0x1a0 [] ret_from_fork+0x42/0x70 [] ? kthread_create_on_node+0x1a0/0x1a0 INFO: task httpd:22880 blocked for more than 120 seconds. Not tainted 4.1.14+63-ph #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. httpd D ffff88005557b108 0 22880 22644 0x00000000 ffff88005557b108 ffff88042dbdbaa0 ffff8807e65a57f0 ffff88005557b1b8 ffff88005557c000 ffff88005557b270 7fffffffffffffff ffff8807e65a57f0 ffffffffffffffff ffff88005557b128 ffffffff9363c167 0000000000000292 Call Trace: [] schedule+0x37/0x90 [] schedule_timeout+0x1fc/0x260 [] ? cfq_set_request+0xa5/0x4c0 [] wait_for_completion+0x9f/0x170 [] ? try_to_wake_up+0x330/0x330 [] flush_work+0xff/0x1d0 [] ? worker_detach_from_pool+0xd0/0xd0 [] xlog_cil_force_lsn+0x7e/0x200 [] ? aac_fib_send+0x2c4/0x4f0 [aacraid] [] _xfs_log_force_lsn+0x58/0x2c0 [] ? xfs_iext_bno_to_ext+0x97/0x170 [] xfs_log_force_lsn+0x3b/0xa0 [] ? xfs_reclaim_inode+0xd0/0x330 [] xfs_iunpin_wait+0xae/0x140 [] ? wake_atomic_t_function+0x40/0x40 [] xfs_reclaim_inode+0xd0/0x330 [] xfs_reclaim_inodes_ag+0x277/0x3d0 [] xfs_reclaim_inodes_nr+0x33/0x40 [] xfs_fs_free_cached_objects+0x19/0x20 [] super_cache_scan+0x1a3/0x1b0 [] shrink_slab+0x216/0x3d0 [] shrink_zone+0x2b9/0x2d0 [] do_try_to_free_pages+0x16a/0x400 [] try_to_free_pages+0xda/0x160 [] __alloc_pages_slowpath+0x3ff/0x697 [] __alloc_pages_nodemask+0x230/0x240 [] alloc_pages_current+0x9c/0x110 [] new_slab+0x32b/0x450 [] ? zone_statistics+0x98/0xc0 [] __slab_alloc+0x2de/0x433 [] ? alloc_fdmem+0x24/0x40 [] __kmalloc+0x1c7/0x240 [] alloc_fdmem+0x24/0x40 [] alloc_fdtable+0x77/0x100 [] dup_fd+0x262/0x330 [] copy_process.part.37+0x76a/0x1a80 [] ? acct_account_cputime+0x1c/0x20 [] do_fork+0xbc/0x350 [] ? syscall_trace_enter_phase1+0xf8/0x160 [] ? context_tracking_user_enter+0x13/0x20 [] SyS_clone+0x16/0x20 [] system_call_fastpath+0x12/0x71 INFO: task kworker/7:0:23773 blocked for more than 120 seconds. Not tainted 4.1.14+63-ph #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kworker/7:0 D ffff8805243bbb38 0 23773 2 0x00000000 Workqueue: xfs-cil/sdb1 xlog_cil_push_work ffff8805243bbb38 ffff88082d941d50 ffff880790560000 0000000000000000 ffff8805243bc000 ffff88010af7c0b8 ffff88042c067d28 ffff880790560000 ffff88042c067d00 ffff8805243bbb58 ffffffff9363c167 ffff88042c067c00 Call Trace: [] schedule+0x37/0x90 [] xlog_state_get_iclog_space+0xdb/0x2e0 [] ? xlog_bdstrat+0x34/0x70 [] ? try_to_wake_up+0x330/0x330 [] xlog_write+0x18e/0x700 [] xlog_cil_push+0x22f/0x3f0 [] xlog_cil_push_work+0x15/0x20 [] process_one_work+0x14f/0x410 [] process_scheduled_works+0x2c/0x40 [] worker_thread+0x28a/0x510 [] ? __schedule+0x4f1/0x990 [] ? process_scheduled_works+0x40/0x40 [] kthread+0xc9/0xe0 [] ? kthread_create_on_node+0x1a0/0x1a0 [] ret_from_fork+0x42/0x70 [] ? kthread_create_on_node+0x1a0/0x1a0 INFO: task qpsmtpd p:25 :::12850 blocked for more than 120 seconds. Not tainted 4.1.14+63-ph #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. qpsmtpd p:25 :: D ffff88071f807c48 0 12850 6124 0x00000000 ffff88071f807c48 ffff8805aabf0000 ffff8806c5780000 ffff88083ffd7000 ffff88071f808000 ffffffff93ce6444 ffff8806c5780000 00000000ffffffff ffffffff93ce6448 ffff88071f807c68 ffffffff9363c167 ffff88042b9f0000 Call Trace: [] schedule+0x37/0x90 [] schedule_preempt_disabled+0xe/0x10 [] __mutex_lock_slowpath+0x95/0x110 [] mutex_lock+0x1b/0x2f [] rtnl_lock+0x15/0x20 [] rtnetlink_rcv+0x16/0x40 [] netlink_unicast+0x13a/0x1d0 [] netlink_sendmsg+0x20a/0x600 [] SYSC_sendto+0x111/0x170 [] ? context_tracking_user_exit+0x13/0x20 [] ? syscall_trace_enter_phase1+0xf8/0x160 [] SyS_sendto+0xe/0x10 [] system_call_fastpath+0x12/0x71