API tools faq
paste
Advertisement
Guest User

timing out command, waited 360s

a guest
Jan 24th, 2013
249
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 29.13 KB | None | 0 0
raw download clone embed print report
  1. $ dmesg
  2.  
  3. [ 5641.572292] INFO: task ls:4830 blocked for more than 120 seconds.
  4. [ 5641.572333] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  5. [ 5641.572385] ls D ffff88007f4139c0 0 4830 3950 0x00000000
  6. [ 5641.572390] ffff880078f65bc8 0000000000000082 ffff88007bfc5c00 ffff880078f65fd8
  7. [ 5641.572396] ffff880078f65fd8 ffff880078f65fd8 ffff88006c0cae00 ffff88007bfc5c00
  8. [ 5641.572401] ffff880078f65ba8 ffff88007bfc5c00 ffff88007f414258 0000000000000002
  9. [ 5641.572406] Call Trace:
  10. [ 5641.572412] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  11. [ 5641.572416] [<ffffffff81680b89>] schedule+0x29/0x70
  12. [ 5641.572420] [<ffffffff81680c5f>] io_schedule+0x8f/0xd0
  13. [ 5641.572424] [<ffffffff811b1f0e>] sleep_on_buffer+0xe/0x20
  14. [ 5641.572428] [<ffffffff8167f2f0>] __wait_on_bit+0x60/0x90
  15. [ 5641.572432] [<ffffffff811b747a>] ? bio_alloc_bioset+0x5a/0xf0
  16. [ 5641.572436] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  17. [ 5641.572440] [<ffffffff8167f39c>] out_of_line_wait_on_bit+0x7c/0x90
  18. [ 5641.572445] [<ffffffff81076810>] ? autoremove_wake_function+0x40/0x40
  19. [ 5641.572449] [<ffffffff811b248e>] __wait_on_buffer+0x2e/0x30
  20. [ 5641.572454] [<ffffffff8121ccc4>] ext4_bread+0x64/0x80
  21. [ 5641.572459] [<ffffffff81227c98>] htree_dirblock_to_tree+0x38/0x1c0
  22. [ 5641.572463] [<ffffffff81153b1d>] ? page_add_new_anon_rmap+0xbd/0xe0
  23. [ 5641.572468] [<ffffffff81227ea3>] ext4_htree_fill_tree+0x83/0x1f0
  24. [ 5641.572473] [<ffffffff8116da8f>] ? kmem_cache_alloc_trace+0x11f/0x130
  25. [ 5641.572477] [<ffffffff81217022>] ? ext4_readdir+0x662/0x750
  26. [ 5641.572481] [<ffffffff81217022>] ? ext4_readdir+0x662/0x750
  27. [ 5641.572484] [<ffffffff81216f36>] ext4_readdir+0x576/0x750
  28. [ 5641.572489] [<ffffffff81194150>] ? fillonedir+0xe0/0xe0
  29. [ 5641.572493] [<ffffffff81194150>] ? fillonedir+0xe0/0xe0
  30. [ 5641.572496] [<ffffffff81194150>] ? fillonedir+0xe0/0xe0
  31. [ 5641.572500] [<ffffffff81194048>] vfs_readdir+0xb8/0xe0
  32. [ 5641.572505] [<ffffffff8118c8f3>] ? putname+0x33/0x50
  33. [ 5641.572509] [<ffffffff81194340>] sys_getdents+0x80/0x100
  34. [ 5641.572514] [<ffffffff81689d29>] system_call_fastpath+0x16/0x1b
  35. [ 5641.812969] sd 4:0:0:0: >timing out command, waited 360s
  36. [ 5761.572056] INFO: task jbd2/sdb1-8:2903 blocked for more than 120 seconds.
  37. [ 5761.572113] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  38. [ 5761.572166] jbd2/sdb1-8 D ffff88007f4139c0 0 2903 2 0x00000000
  39. [ 5761.572173] ffff880078f41bc0 0000000000000046 ffff880078f2ae00 ffff880078f41fd8
  40. [ 5761.572179] ffff880078f41fd8 ffff880078f41fd8 ffff88006c0d8000 ffff880078f2ae00
  41. [ 5761.572185] ffff880078f41ba0 ffff880078f2ae00 ffff88007f414258 0000000000000002
  42. [ 5761.572191] Call Trace:
  43. [ 5761.572203] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  44. [ 5761.572211] [<ffffffff81680b89>] schedule+0x29/0x70
  45. [ 5761.572216] [<ffffffff81680c5f>] io_schedule+0x8f/0xd0
  46. [ 5761.572221] [<ffffffff811b1f0e>] sleep_on_buffer+0xe/0x20
  47. [ 5761.572226] [<ffffffff8167f2f0>] __wait_on_bit+0x60/0x90
  48. [ 5761.572231] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  49. [ 5761.572235] [<ffffffff8167f39c>] out_of_line_wait_on_bit+0x7c/0x90
  50. [ 5761.572242] [<ffffffff81076810>] ? autoremove_wake_function+0x40/0x40
  51. [ 5761.572247] [<ffffffff811b248e>] __wait_on_buffer+0x2e/0x30
  52. [ 5761.572254] [<ffffffff8126ab9f>] jbd2_journal_commit_transaction+0x181f/0x19a0
  53. [ 5761.572260] [<ffffffff8126f79b>] kjournald2+0xbb/0x240
  54. [ 5761.572265] [<ffffffff810767d0>] ? finish_wait+0x80/0x80
  55. [ 5761.572270] [<ffffffff8126f6e0>] ? jbd2_journal_release_jbd_inode+0x130/0x130
  56. [ 5761.572275] [<ffffffff81075e33>] kthread+0x93/0xa0
  57. [ 5761.572280] [<ffffffff8168b024>] kernel_thread_helper+0x4/0x10
  58. [ 5761.572286] [<ffffffff81075da0>] ? kthread_freezable_should_stop+0x70/0x70
  59. [ 5761.572290] [<ffffffff8168b020>] ? gs_change+0x13/0x13
  60. [ 6001.817038] sd 4:0:0:0: >timing out command, waited 360s
  61.  
  62.  
  63.  
  64. $ kern.log
  65.  
  66. Dec 16 20:40:59 ubplex kernel: [ 5641.572059] INFO: task jbd2/sdb1-8:2903 blocked for more than 120 seconds.
  67. Dec 16 20:40:59 ubplex kernel: [ 5641.572116] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  68. Dec 16 20:40:59 ubplex kernel: [ 5641.572169] jbd2/sdb1-8 D ffff88007f4139c0 0 2903 2 0x00000000
  69. Dec 16 20:40:59 ubplex kernel: [ 5641.572176] ffff880078f41bc0 0000000000000046 ffff880078f2ae00 ffff880078f41fd8
  70. Dec 16 20:40:59 ubplex kernel: [ 5641.572183] ffff880078f41fd8 ffff880078f41fd8 ffff88006c0d8000 ffff880078f2ae00
  71. Dec 16 20:40:59 ubplex kernel: [ 5641.572188] ffff880078f41ba0 ffff880078f2ae00 ffff88007f414258 0000000000000002
  72. Dec 16 20:40:59 ubplex kernel: [ 5641.572194] Call Trace:
  73. Dec 16 20:40:59 ubplex kernel: [ 5641.572207] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  74. Dec 16 20:40:59 ubplex kernel: [ 5641.572216] [<ffffffff81680b89>] schedule+0x29/0x70
  75. Dec 16 20:40:59 ubplex kernel: [ 5641.572221] [<ffffffff81680c5f>] io_schedule+0x8f/0xd0
  76. Dec 16 20:40:59 ubplex kernel: [ 5641.572225] [<ffffffff811b1f0e>] sleep_on_buffer+0xe/0x20
  77. Dec 16 20:40:59 ubplex kernel: [ 5641.572230] [<ffffffff8167f2f0>] __wait_on_bit+0x60/0x90
  78. Dec 16 20:40:59 ubplex kernel: [ 5641.572234] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  79. Dec 16 20:40:59 ubplex kernel: [ 5641.572238] [<ffffffff8167f39c>] out_of_line_wait_on_bit+0x7c/0x90
  80. Dec 16 20:40:59 ubplex kernel: [ 5641.572244] [<ffffffff81076810>] ? autoremove_wake_function+0x40/0x40
  81. Dec 16 20:40:59 ubplex kernel: [ 5641.572249] [<ffffffff811b248e>] __wait_on_buffer+0x2e/0x30
  82. Dec 16 20:40:59 ubplex kernel: [ 5641.572255] [<ffffffff8126ab9f>] jbd2_journal_commit_transaction+0x181f/0x19a0
  83. Dec 16 20:40:59 ubplex kernel: [ 5641.572261] [<ffffffff8126f79b>] kjournald2+0xbb/0x240
  84. Dec 16 20:40:59 ubplex kernel: [ 5641.572265] [<ffffffff810767d0>] ? finish_wait+0x80/0x80
  85. Dec 16 20:40:59 ubplex kernel: [ 5641.572270] [<ffffffff8126f6e0>] ? jbd2_journal_release_jbd_inode+0x130/0x130
  86. Dec 16 20:40:59 ubplex kernel: [ 5641.572274] [<ffffffff81075e33>] kthread+0x93/0xa0
  87. Dec 16 20:40:59 ubplex kernel: [ 5641.572279] [<ffffffff8168b024>] kernel_thread_helper+0x4/0x10
  88. Dec 16 20:40:59 ubplex kernel: [ 5641.572283] [<ffffffff81075da0>] ? kthread_freezable_should_stop+0x70/0x70
  89. Dec 16 20:40:59 ubplex kernel: [ 5641.572287] [<ffffffff8168b020>] ? gs_change+0x13/0x13
  90. Dec 16 20:40:59 ubplex kernel: [ 5641.572292] INFO: task ls:4830 blocked for more than 120 seconds.
  91. Dec 16 20:40:59 ubplex kernel: [ 5641.572333] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  92. Dec 16 20:40:59 ubplex kernel: [ 5641.572385] ls D ffff88007f4139c0 0 4830 3950 0x00000000
  93. Dec 16 20:40:59 ubplex kernel: [ 5641.572390] ffff880078f65bc8 0000000000000082 ffff88007bfc5c00 ffff880078f65fd8
  94. Dec 16 20:40:59 ubplex kernel: [ 5641.572396] ffff880078f65fd8 ffff880078f65fd8 ffff88006c0cae00 ffff88007bfc5c00
  95. Dec 16 20:40:59 ubplex kernel: [ 5641.572401] ffff880078f65ba8 ffff88007bfc5c00 ffff88007f414258 0000000000000002
  96. Dec 16 20:40:59 ubplex kernel: [ 5641.572406] Call Trace:
  97. Dec 16 20:40:59 ubplex kernel: [ 5641.572412] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  98. Dec 16 20:40:59 ubplex kernel: [ 5641.572416] [<ffffffff81680b89>] schedule+0x29/0x70
  99. Dec 16 20:40:59 ubplex kernel: [ 5641.572420] [<ffffffff81680c5f>] io_schedule+0x8f/0xd0
  100. Dec 16 20:40:59 ubplex kernel: [ 5641.572424] [<ffffffff811b1f0e>] sleep_on_buffer+0xe/0x20
  101. Dec 16 20:40:59 ubplex kernel: [ 5641.572428] [<ffffffff8167f2f0>] __wait_on_bit+0x60/0x90
  102. Dec 16 20:40:59 ubplex kernel: [ 5641.572432] [<ffffffff811b747a>] ? bio_alloc_bioset+0x5a/0xf0
  103. Dec 16 20:40:59 ubplex kernel: [ 5641.572436] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  104. Dec 16 20:40:59 ubplex kernel: [ 5641.572440] [<ffffffff8167f39c>] out_of_line_wait_on_bit+0x7c/0x90
  105. Dec 16 20:40:59 ubplex kernel: [ 5641.572445] [<ffffffff81076810>] ? autoremove_wake_function+0x40/0x40
  106. Dec 16 20:40:59 ubplex kernel: [ 5641.572449] [<ffffffff811b248e>] __wait_on_buffer+0x2e/0x30
  107. Dec 16 20:40:59 ubplex kernel: [ 5641.572454] [<ffffffff8121ccc4>] ext4_bread+0x64/0x80
  108. Dec 16 20:40:59 ubplex kernel: [ 5641.572459] [<ffffffff81227c98>] htree_dirblock_to_tree+0x38/0x1c0
  109. Dec 16 20:40:59 ubplex kernel: [ 5641.572463] [<ffffffff81153b1d>] ? page_add_new_anon_rmap+0xbd/0xe0
  110. Dec 16 20:40:59 ubplex kernel: [ 5641.572468] [<ffffffff81227ea3>] ext4_htree_fill_tree+0x83/0x1f0
  111. Dec 16 20:40:59 ubplex kernel: [ 5641.572473] [<ffffffff8116da8f>] ? kmem_cache_alloc_trace+0x11f/0x130
  112. Dec 16 20:40:59 ubplex kernel: [ 5641.572477] [<ffffffff81217022>] ? ext4_readdir+0x662/0x750
  113. Dec 16 20:40:59 ubplex kernel: [ 5641.572481] [<ffffffff81217022>] ? ext4_readdir+0x662/0x750
  114. Dec 16 20:40:59 ubplex kernel: [ 5641.572484] [<ffffffff81216f36>] ext4_readdir+0x576/0x750
  115. Dec 16 20:40:59 ubplex kernel: [ 5641.572489] [<ffffffff81194150>] ? fillonedir+0xe0/0xe0
  116. Dec 16 20:40:59 ubplex kernel: [ 5641.572493] [<ffffffff81194150>] ? fillonedir+0xe0/0xe0
  117. Dec 16 20:40:59 ubplex kernel: [ 5641.572496] [<ffffffff81194150>] ? fillonedir+0xe0/0xe0
  118. Dec 16 20:40:59 ubplex kernel: [ 5641.572500] [<ffffffff81194048>] vfs_readdir+0xb8/0xe0
  119. Dec 16 20:40:59 ubplex kernel: [ 5641.572505] [<ffffffff8118c8f3>] ? putname+0x33/0x50
  120. Dec 16 20:40:59 ubplex kernel: [ 5641.572509] [<ffffffff81194340>] sys_getdents+0x80/0x100
  121. Dec 16 20:40:59 ubplex kernel: [ 5641.572514] [<ffffffff81689d29>] system_call_fastpath+0x16/0x1b
  122. Dec 16 20:40:59 ubplex kernel: [ 5641.812969] sd 4:0:0:0: >timing out command, waited 360s
  123. Dec 16 20:42:59 ubplex kernel: [ 5761.572056] INFO: task jbd2/sdb1-8:2903 blocked for more than 120 seconds.
  124. Dec 16 20:42:59 ubplex kernel: [ 5761.572113] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  125. Dec 16 20:42:59 ubplex kernel: [ 5761.572166] jbd2/sdb1-8 D ffff88007f4139c0 0 2903 2 0x00000000
  126. Dec 16 20:42:59 ubplex kernel: [ 5761.572173] ffff880078f41bc0 0000000000000046 ffff880078f2ae00 ffff880078f41fd8
  127. Dec 16 20:42:59 ubplex kernel: [ 5761.572179] ffff880078f41fd8 ffff880078f41fd8 ffff88006c0d8000 ffff880078f2ae00
  128. Dec 16 20:42:59 ubplex kernel: [ 5761.572185] ffff880078f41ba0 ffff880078f2ae00 ffff88007f414258 0000000000000002
  129. Dec 16 20:42:59 ubplex kernel: [ 5761.572191] Call Trace:
  130. Dec 16 20:42:59 ubplex kernel: [ 5761.572203] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  131. Dec 16 20:42:59 ubplex kernel: [ 5761.572211] [<ffffffff81680b89>] schedule+0x29/0x70
  132. Dec 16 20:42:59 ubplex kernel: [ 5761.572216] [<ffffffff81680c5f>] io_schedule+0x8f/0xd0
  133. Dec 16 20:42:59 ubplex kernel: [ 5761.572221] [<ffffffff811b1f0e>] sleep_on_buffer+0xe/0x20
  134. Dec 16 20:42:59 ubplex kernel: [ 5761.572226] [<ffffffff8167f2f0>] __wait_on_bit+0x60/0x90
  135. Dec 16 20:42:59 ubplex kernel: [ 5761.572231] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  136. Dec 16 20:42:59 ubplex kernel: [ 5761.572235] [<ffffffff8167f39c>] out_of_line_wait_on_bit+0x7c/0x90
  137. Dec 16 20:42:59 ubplex kernel: [ 5761.572242] [<ffffffff81076810>] ? autoremove_wake_function+0x40/0x40
  138. Dec 16 20:42:59 ubplex kernel: [ 5761.572247] [<ffffffff811b248e>] __wait_on_buffer+0x2e/0x30
  139. Dec 16 20:42:59 ubplex kernel: [ 5761.572254] [<ffffffff8126ab9f>] jbd2_journal_commit_transaction+0x181f/0x19a0
  140. Dec 16 20:42:59 ubplex kernel: [ 5761.572260] [<ffffffff8126f79b>] kjournald2+0xbb/0x240
  141. Dec 16 20:42:59 ubplex kernel: [ 5761.572265] [<ffffffff810767d0>] ? finish_wait+0x80/0x80
  142. Dec 16 20:42:59 ubplex kernel: [ 5761.572270] [<ffffffff8126f6e0>] ? jbd2_journal_release_jbd_inode+0x130/0x130
  143. Dec 16 20:42:59 ubplex kernel: [ 5761.572275] [<ffffffff81075e33>] kthread+0x93/0xa0
  144. Dec 16 20:42:59 ubplex kernel: [ 5761.572280] [<ffffffff8168b024>] kernel_thread_helper+0x4/0x10
  145. Dec 16 20:42:59 ubplex kernel: [ 5761.572286] [<ffffffff81075da0>] ? kthread_freezable_should_stop+0x70/0x70
  146. Dec 16 20:42:59 ubplex kernel: [ 5761.572290] [<ffffffff8168b020>] ? gs_change+0x13/0x13
  147. Dec 16 20:46:59 ubplex kernel: [ 6001.817038] sd 4:0:0:0: >timing out command, waited 360s
  148. Dec 16 20:52:59 ubplex kernel: [ 6361.836981] sd 4:0:0:0: >timing out command, waited 360s
  149. Dec 16 20:58:59 ubplex kernel: [ 6721.841050] sd 4:0:0:0: >timing out command, waited 360s
  150. Dec 16 20:58:59 ubplex kernel: [ 6721.842671] sd 4:0:0:0: >[sdb] Unhandled sense code
  151. Dec 16 20:58:59 ubplex kernel: [ 6721.842677] sd 4:0:0:0: >[sdb]
  152. Dec 16 20:58:59 ubplex kernel: [ 6721.842680] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
  153. Dec 16 20:58:59 ubplex kernel: [ 6721.842683] sd 4:0:0:0: >[sdb]
  154. Dec 16 20:58:59 ubplex kernel: [ 6721.842685] Sense Key : Medium Error [current]
  155. Dec 16 20:58:59 ubplex kernel: [ 6721.842691] sd 4:0:0:0: >[sdb]
  156. Dec 16 20:58:59 ubplex kernel: [ 6721.842694] Add. Sense: Unrecovered read error
  157. Dec 16 20:58:59 ubplex kernel: [ 6721.842698] sd 4:0:0:0: >[sdb] CDB:
  158. Dec 16 20:58:59 ubplex kernel: [ 6721.842699] Read(10): 28 00 34 81 09 10 00 00 08 00
  159. Dec 16 20:58:59 ubplex kernel: [ 6721.842712] end_request: critical target error, dev sdb, sector 880871696
  160. Dec 16 21:04:59 ubplex kernel: [ 7081.848993] sd 4:0:0:0: >timing out command, waited 360s
  161. Dec 16 21:10:59 ubplex kernel: [ 7441.853061] sd 4:0:0:0: >timing out command, waited 360s
  162. Dec 16 21:16:59 ubplex kernel: [ 7801.921005] sd 4:0:0:0: >timing out command, waited 360s
  163. Dec 16 21:22:59 ubplex kernel: [ 8161.925073] sd 4:0:0:0: >timing out command, waited 360s
  164.  
  165.  
  166.  
  167. $ syslog
  168.  
  169. Dec 16 20:42:59 ubplex kernel: [ 5761.572056] INFO: task jbd2/sdb1-8:2903 blocked for more than 120 seconds.
  170. Dec 16 20:42:59 ubplex kernel: [ 5761.572113] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  171. Dec 16 20:42:59 ubplex kernel: [ 5761.572166] jbd2/sdb1-8 D ffff88007f4139c0 0 2903 2 0x00000000
  172. Dec 16 20:42:59 ubplex kernel: [ 5761.572173] ffff880078f41bc0 0000000000000046 ffff880078f2ae00 ffff880078f41fd8
  173. Dec 16 20:42:59 ubplex kernel: [ 5761.572179] ffff880078f41fd8 ffff880078f41fd8 ffff88006c0d8000 ffff880078f2ae00
  174. Dec 16 20:42:59 ubplex kernel: [ 5761.572185] ffff880078f41ba0 ffff880078f2ae00 ffff88007f414258 0000000000000002
  175. Dec 16 20:42:59 ubplex kernel: [ 5761.572191] Call Trace:
  176. Dec 16 20:42:59 ubplex kernel: [ 5761.572203] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  177. Dec 16 20:42:59 ubplex kernel: [ 5761.572211] [<ffffffff81680b89>] schedule+0x29/0x70
  178. Dec 16 20:42:59 ubplex kernel: [ 5761.572216] [<ffffffff81680c5f>] io_schedule+0x8f/0xd0
  179. Dec 16 20:42:59 ubplex kernel: [ 5761.572221] [<ffffffff811b1f0e>] sleep_on_buffer+0xe/0x20
  180. Dec 16 20:42:59 ubplex kernel: [ 5761.572226] [<ffffffff8167f2f0>] __wait_on_bit+0x60/0x90
  181. Dec 16 20:42:59 ubplex kernel: [ 5761.572231] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  182. Dec 16 20:42:59 ubplex kernel: [ 5761.572235] [<ffffffff8167f39c>] out_of_line_wait_on_bit+0x7c/0x90
  183. Dec 16 20:42:59 ubplex kernel: [ 5761.572242] [<ffffffff81076810>] ? autoremove_wake_function+0x40/0x40
  184. Dec 16 20:42:59 ubplex kernel: [ 5761.572247] [<ffffffff811b248e>] __wait_on_buffer+0x2e/0x30
  185. Dec 16 20:42:59 ubplex kernel: [ 5761.572254] [<ffffffff8126ab9f>] jbd2_journal_commit_transaction+0x181f/0x19a0
  186. Dec 16 20:42:59 ubplex kernel: [ 5761.572260] [<ffffffff8126f79b>] kjournald2+0xbb/0x240
  187. Dec 16 20:42:59 ubplex kernel: [ 5761.572265] [<ffffffff810767d0>] ? finish_wait+0x80/0x80
  188. Dec 16 20:42:59 ubplex kernel: [ 5761.572270] [<ffffffff8126f6e0>] ? jbd2_journal_release_jbd_inode+0x130/0x130
  189. Dec 16 20:42:59 ubplex kernel: [ 5761.572275] [<ffffffff81075e33>] kthread+0x93/0xa0
  190. Dec 16 20:42:59 ubplex kernel: [ 5761.572280] [<ffffffff8168b024>] kernel_thread_helper+0x4/0x10
  191. Dec 16 20:42:59 ubplex kernel: [ 5761.572286] [<ffffffff81075da0>] ? kthread_freezable_should_stop+0x70/0x70
  192. Dec 16 20:42:59 ubplex kernel: [ 5761.572290] [<ffffffff8168b020>] ? gs_change+0x13/0x13
  193. Dec 16 20:46:59 ubplex kernel: [ 6001.817038] sd 4:0:0:0: >timing out command, waited 360s
  194. Dec 16 20:52:59 ubplex kernel: [ 6361.836981] sd 4:0:0:0: >timing out command, waited 360s
  195.  
  196.  
  197. $ top
  198.  
  199. top - 20:51:07 up 1:44, 2 users, load average: 1.99, 1.82, 1.24
  200. Tasks: 107 total, 1 running, 106 sleeping, 0 stopped, 0 zombie
  201. %Cpu(s): 0.0 us, 0.0 sy, 0.0 ni, 50.3 id, 49.7 wa, 0.0 hi, 0.0 si, 0.0 st
  202. KiB Mem: 2041456 total, 457496 used, 1583960 free, 33316 buffers
  203. KiB Swap: 2084860 total, 0 used, 2084860 free, 209972 cached
  204.  
  205. PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
  206. 2683 root 20 0 0 0 0 S 0.7 0.0 0:11.36 usb-storage
  207. 1283 plex 35 15 1254m 52m 5748 S 0.3 2.6 0:11.56 python
  208. 5647 dom 20 0 20624 1528 1056 R 0.3 0.1 0:00.01 top
  209. 1 root 20 0 24464 2300 1284 S 0.0 0.1 0:01.65 init
  210. 2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
  211.  
  212.  
  213.  
  214. dom@ubplex:~$ sudo lsof /media/seagate/Photos/Canada\ 2010/
  215. COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
  216. ls 5537 dom 3r DIR 8,17 4096 27525123 /media/seagate/Photos/Canada 2010
  217. dom@ubplex:~$ sudo lsof -p 5537
  218. COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
  219. ls 5537 dom cwd DIR 8,17 4096 27525121 /media/seagate/Photos
  220. ls 5537 dom rtd DIR 252,0 4096 2 /
  221. ls 5537 dom txt REG 252,0 109944 534517 /bin/ls
  222. ls 5537 dom mem REG 252,0 2915696 1840230 /usr/lib/locale/locale-archive
  223. ls 5537 dom mem REG 252,0 18560 1310955 /lib/x86_64-linux-gnu/libattr.so.1.1.0
  224. ls 5537 dom mem REG 252,0 135398 1311032 /lib/x86_64-linux-gnu/libpthread-2.15.so
  225. ls 5537 dom mem REG 252,0 14792 1310975 /lib/x86_64-linux-gnu/libdl-2.15.so
  226. ls 5537 dom mem REG 252,0 1811160 1310963 /lib/x86_64-linux-gnu/libc-2.15.so
  227. ls 5537 dom mem REG 252,0 31168 1310951 /lib/x86_64-linux-gnu/libacl.so.1.1.0
  228. ls 5537 dom mem REG 252,0 31784 1311038 /lib/x86_64-linux-gnu/librt-2.15.so
  229. ls 5537 dom mem REG 252,0 121824 1311040 /lib/x86_64-linux-gnu/libselinux.so.1
  230. ls 5537 dom mem REG 252,0 149312 1310945 /lib/x86_64-linux-gnu/ld-2.15.so
  231. ls 5537 dom 0u CHR 136,1 0t0 4 /dev/pts/1
  232. ls 5537 dom 1u CHR 136,1 0t0 4 /dev/pts/1
  233. ls 5537 dom 2u CHR 136,1 0t0 4 /dev/pts/1
  234. ls 5537 dom 3r DIR 8,17 4096 27525123 /media/seagate/Photos/Canada 2010
  235.  
  236.  
  237.  
  238. dom@ubplex:/media/seagate/Photos$ ls Canada\ 2010/
  239. ls: reading directory Canada 2010/: Input/output error
  240.  
  241.  
  242.  
  243. dom@ubplex:/media/seagate/Photos$ sudo smartctl -d sat -l selftest /dev/sdb1
  244. [sudo] password for dom:
  245. smartctl 5.43 2012-06-30 r3573 [x86_64-linux-3.5.0-17-generic] (local build)
  246. Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net
  247.  
  248. > several minute pause <
  249.  
  250. === START OF READ SMART DATA SECTION ===
  251. SMART Self-test log structure revision number 1
  252. Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
  253. # 1 Short offline Completed without error 00% 278 -
  254.  
  255.  
  256.  
  257.  
  258. $ dmesg
  259.  
  260. [ 6001.817038] sd 4:0:0:0: >timing out command, waited 360s
  261. [ 6361.836981] sd 4:0:0:0: >timing out command, waited 360s
  262. [ 6721.841050] sd 4:0:0:0: >timing out command, waited 360s
  263. [ 6721.842671] sd 4:0:0:0: >[sdb] Unhandled sense code
  264. [ 6721.842677] sd 4:0:0:0: >[sdb]
  265. [ 6721.842680] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
  266. [ 6721.842683] sd 4:0:0:0: >[sdb]
  267. [ 6721.842685] Sense Key : Medium Error [current]
  268. [ 6721.842691] sd 4:0:0:0: >[sdb]
  269. [ 6721.842694] Add. Sense: Unrecovered read error
  270. [ 6721.842698] sd 4:0:0:0: >[sdb] CDB:
  271. [ 6721.842699] Read(10): 28 00 34 81 09 10 00 00 08 00
  272. [ 6721.842712] end_request: critical target error, dev sdb, sector 880871696
  273. [ 7081.848993] sd 4:0:0:0: >timing out command, waited 360s
  274. [ 7441.853061] sd 4:0:0:0: >timing out command, waited 360s
  275. [ 7801.921005] sd 4:0:0:0: >timing out command, waited 360s
  276. [ 8161.925073] sd 4:0:0:0: >timing out command, waited 360s
  277.  
  278.  
  279.  
  280. dom@ubplex:/media/seagate/Photos$ sudo smartctl -d sat -l selftest /dev/sdb1
  281. [sudo] password for dom:
  282. smartctl 5.43 2012-06-30 r3573 [x86_64-linux-3.5.0-17-generic] (local build)
  283. Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net
  284.  
  285. === START OF READ SMART DATA SECTION ===
  286. SMART Self-test log structure revision number 1
  287. Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
  288. # 1 Extended offline Completed without error 00% 283 -
  289. # 2 Short offline Completed without error 00% 278 -
  290.  
  291.  
  292.  
  293.  
  294.  
  295.  
  296. few days later...
  297. $ dmesg
  298.  
  299. [ 5641.572059] INFO: task jbd2/sdb1-8:2903 blocked for more than 120 seconds.
  300. [ 5641.572116] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  301. [ 5641.572169] jbd2/sdb1-8 D ffff88007f4139c0 0 2903 2 0x00000000
  302. [ 5641.572176] ffff880078f41bc0 0000000000000046 ffff880078f2ae00 ffff880078f41fd8
  303. [ 5641.572183] ffff880078f41fd8 ffff880078f41fd8 ffff88006c0d8000 ffff880078f2ae00
  304. [ 5641.572188] ffff880078f41ba0 ffff880078f2ae00 ffff88007f414258 0000000000000002
  305. [ 5641.572194] Call Trace:
  306. [ 5641.572207] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  307. [ 5641.572216] [<ffffffff81680b89>] schedule+0x29/0x70
  308. [ 5641.572221] [<ffffffff81680c5f>] io_schedule+0x8f/0xd0
  309. [ 5641.572225] [<ffffffff811b1f0e>] sleep_on_buffer+0xe/0x20
  310. [ 5641.572230] [<ffffffff8167f2f0>] __wait_on_bit+0x60/0x90
  311. [ 5641.572234] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  312. [ 5641.572238] [<ffffffff8167f39c>] out_of_line_wait_on_bit+0x7c/0x90
  313. [ 5641.572244] [<ffffffff81076810>] ? autoremove_wake_function+0x40/0x40
  314. [ 5641.572249] [<ffffffff811b248e>] __wait_on_buffer+0x2e/0x30
  315. [ 5641.572255] [<ffffffff8126ab9f>] jbd2_journal_commit_transaction+0x181f/0x19a0
  316. [ 5641.572261] [<ffffffff8126f79b>] kjournald2+0xbb/0x240
  317. [ 5641.572265] [<ffffffff810767d0>] ? finish_wait+0x80/0x80
  318. [ 5641.572270] [<ffffffff8126f6e0>] ? jbd2_journal_release_jbd_inode+0x130/0x130
  319. [ 5641.572274] [<ffffffff81075e33>] kthread+0x93/0xa0
  320. [ 5641.572279] [<ffffffff8168b024>] kernel_thread_helper+0x4/0x10
  321. [ 5641.572283] [<ffffffff81075da0>] ? kthread_freezable_should_stop+0x70/0x70
  322. [ 5641.572287] [<ffffffff8168b020>] ? gs_change+0x13/0x13
  323. [ 5641.572292] INFO: task ls:4830 blocked for more than 120 seconds.
  324. [ 5641.572333] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  325. [ 5641.572385] ls D ffff88007f4139c0 0 4830 3950 0x00000000
  326. [ 5641.572390] ffff880078f65bc8 0000000000000082 ffff88007bfc5c00 ffff880078f65fd8
  327. [ 5641.572396] ffff880078f65fd8 ffff880078f65fd8 ffff88006c0cae00 ffff88007bfc5c00
  328. [ 5641.572401] ffff880078f65ba8 ffff88007bfc5c00 ffff88007f414258 0000000000000002
  329. [ 5641.572406] Call Trace:
  330. [ 5641.572412] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  331. [ 5641.572416] [<ffffffff81680b89>] schedule+0x29/0x70
  332. [ 5641.572420] [<ffffffff81680c5f>] io_schedule+0x8f/0xd0
  333. [ 5641.572424] [<ffffffff811b1f0e>] sleep_on_buffer+0xe/0x20
  334. [ 5641.572428] [<ffffffff8167f2f0>] __wait_on_bit+0x60/0x90
  335. [ 5641.572432] [<ffffffff811b747a>] ? bio_alloc_bioset+0x5a/0xf0
  336. [ 5641.572436] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  337. [ 5641.572440] [<ffffffff8167f39c>] out_of_line_wait_on_bit+0x7c/0x90
  338. [ 5641.572445] [<ffffffff81076810>] ? autoremove_wake_function+0x40/0x40
  339. [ 5641.572449] [<ffffffff811b248e>] __wait_on_buffer+0x2e/0x30
  340. [ 5641.572454] [<ffffffff8121ccc4>] ext4_bread+0x64/0x80
  341. [ 5641.572459] [<ffffffff81227c98>] htree_dirblock_to_tree+0x38/0x1c0
  342. [ 5641.572463] [<ffffffff81153b1d>] ? page_add_new_anon_rmap+0xbd/0xe0
  343. [ 5641.572468] [<ffffffff81227ea3>] ext4_htree_fill_tree+0x83/0x1f0
  344. [ 5641.572473] [<ffffffff8116da8f>] ? kmem_cache_alloc_trace+0x11f/0x130
  345. [ 5641.572477] [<ffffffff81217022>] ? ext4_readdir+0x662/0x750
  346. [ 5641.572481] [<ffffffff81217022>] ? ext4_readdir+0x662/0x750
  347. [ 5641.572484] [<ffffffff81216f36>] ext4_readdir+0x576/0x750
  348. [ 5641.572489] [<ffffffff81194150>] ? fillonedir+0xe0/0xe0
  349. [ 5641.572493] [<ffffffff81194150>] ? fillonedir+0xe0/0xe0
  350. [ 5641.572496] [<ffffffff81194150>] ? fillonedir+0xe0/0xe0
  351. [ 5641.572500] [<ffffffff81194048>] vfs_readdir+0xb8/0xe0
  352. [ 5641.572505] [<ffffffff8118c8f3>] ? putname+0x33/0x50
  353. [ 5641.572509] [<ffffffff81194340>] sys_getdents+0x80/0x100
  354. [ 5641.572514] [<ffffffff81689d29>] system_call_fastpath+0x16/0x1b
  355. [ 5641.812969] sd 4:0:0:0: >timing out command, waited 360s
  356. [ 5761.572056] INFO: task jbd2/sdb1-8:2903 blocked for more than 120 seconds.
  357. [ 5761.572113] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
  358. [ 5761.572166] jbd2/sdb1-8 D ffff88007f4139c0 0 2903 2 0x00000000
  359. [ 5761.572173] ffff880078f41bc0 0000000000000046 ffff880078f2ae00 ffff880078f41fd8
  360. [ 5761.572179] ffff880078f41fd8 ffff880078f41fd8 ffff88006c0d8000 ffff880078f2ae00
  361. [ 5761.572185] ffff880078f41ba0 ffff880078f2ae00 ffff88007f414258 0000000000000002
  362. [ 5761.572191] Call Trace:
  363. [ 5761.572203] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  364. [ 5761.572211] [<ffffffff81680b89>] schedule+0x29/0x70
  365. [ 5761.572216] [<ffffffff81680c5f>] io_schedule+0x8f/0xd0
  366. [ 5761.572221] [<ffffffff811b1f0e>] sleep_on_buffer+0xe/0x20
  367. [ 5761.572226] [<ffffffff8167f2f0>] __wait_on_bit+0x60/0x90
  368. [ 5761.572231] [<ffffffff811b1f00>] ? do_thaw_one+0x90/0x90
  369. [ 5761.572235] [<ffffffff8167f39c>] out_of_line_wait_on_bit+0x7c/0x90
  370. [ 5761.572242] [<ffffffff81076810>] ? autoremove_wake_function+0x40/0x40
  371. [ 5761.572247] [<ffffffff811b248e>] __wait_on_buffer+0x2e/0x30
  372. [ 5761.572254] [<ffffffff8126ab9f>] jbd2_journal_commit_transaction+0x181f/0x19a0
  373. [ 5761.572260] [<ffffffff8126f79b>] kjournald2+0xbb/0x240
  374. [ 5761.572265] [<ffffffff810767d0>] ? finish_wait+0x80/0x80
  375. [ 5761.572270] [<ffffffff8126f6e0>] ? jbd2_journal_release_jbd_inode+0x130/0x130
  376. [ 5761.572275] [<ffffffff81075e33>] kthread+0x93/0xa0
  377. [ 5761.572280] [<ffffffff8168b024>] kernel_thread_helper+0x4/0x10
  378. [ 5761.572286] [<ffffffff81075da0>] ? kthread_freezable_should_stop+0x70/0x70
  379. [ 5761.572290] [<ffffffff8168b020>] ? gs_change+0x13/0x13
  380. [ 6001.817038] sd 4:0:0:0: >timing out command, waited 360s
  381. [ 6361.836981] sd 4:0:0:0: >timing out command, waited 360s
  382. [ 6721.841050] sd 4:0:0:0: >timing out command, waited 360s
  383. [ 6721.842671] sd 4:0:0:0: >[sdb] Unhandled sense code
  384. [ 6721.842677] sd 4:0:0:0: >[sdb]
  385. [ 6721.842680] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
  386. [ 6721.842683] sd 4:0:0:0: >[sdb]
  387. [ 6721.842685] Sense Key : Medium Error [current]
  388. [ 6721.842691] sd 4:0:0:0: >[sdb]
  389. [ 6721.842694] Add. Sense: Unrecovered read error
  390. [ 6721.842698] sd 4:0:0:0: >[sdb] CDB:
  391. [ 6721.842699] Read(10): 28 00 34 81 09 10 00 00 08 00
  392. [ 6721.842712] end_request: critical target error, dev sdb, sector 880871696
  393. [ 7081.848993] sd 4:0:0:0: >timing out command, waited 360s
  394. [ 7441.853061] sd 4:0:0:0: >timing out command, waited 360s
  395. [ 7801.921005] sd 4:0:0:0: >timing out command, waited 360s
  396. [ 8161.925073] sd 4:0:0:0: >timing out command, waited 360s
  397. [41797.813081] sd 4:0:0:0: >timing out command, waited 360s
  398. [42157.817025] sd 4:0:0:0: >timing out command, waited 360s
  399. [128989.829004] sd 4:0:0:0: >timing out command, waited 360s
  400. [129349.837074] sd 4:0:0:0: >timing out command, waited 360s
  401. [184524.821045] sd 4:0:0:0: >timing out command, waited 360s
  402. [184884.824988] sd 4:0:0:0: >timing out command, waited 360s
  403. [185244.869056] sd 4:0:0:0: >timing out command, waited 360s
  404. [185604.873000] sd 4:0:0:0: >timing out command, waited 360s
  405. [185964.901068] sd 4:0:0:0: >timing out command, waited 360s
  406. [186324.909012] sd 4:0:0:0: >timing out command, waited 360s
  407. [186684.997079] sd 4:0:0:0: >timing out command, waited 360s
  408. [187045.005023] sd 4:0:0:0: >timing out command, waited 360s
  409. [187405.097091] sd 4:0:0:0: >timing out command, waited 360s
  410. [187765.101034] sd 4:0:0:0: >timing out command, waited 360s
  411. [188125.164978] sd 4:0:0:0: >timing out command, waited 360s
  412. [188485.169047] sd 4:0:0:0: >timing out command, waited 360s
  413. [188845.172990] sd 4:0:0:0: >timing out command, waited 360s
  414. [189205.177065] sd 4:0:0:0: >timing out command, waited 360s
  415. [301220.837068] sd 4:0:0:0: >timing out command, waited 360s
  416. [301580.845011] sd 4:0:0:0: >timing out command, waited 360s
  417. [388012.821039] sd 4:0:0:0: >timing out command, waited 360s
  418. [388372.828983] sd 4:0:0:0: >timing out command, waited 360s
  419. [561237.833042] sd 4:0:0:0: >timing out command, waited 360s
  420. [561597.840985] sd 4:0:0:0: >timing out command, waited 360s
  421. [733982.821077] sd 4:0:0:0: >timing out command, waited 360s
  422. [734342.825021] sd 4:0:0:0: >timing out command, waited 360s
  423. [906581.833086] sd 4:0:0:0: >timing out command, waited 360s
  424. [906941.837027] sd 4:0:0:0: >timing out command, waited 360s
  425. [1078292.825049] sd 4:0:0:0: >timing out command, waited 360s
  426. [1078652.832993] sd 4:0:0:0: >timing out command, waited 360s
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!