API tools faq
paste
Guest User

kernel BUG at drivers/scsi/scsi_lib.c:1153!

a guest
Jun 28th, 2012
216
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 108.59 KB | None | 0 0
raw download clone embed print report
  1. Jun 29 05:06:42 borg03b kernel: [231632.877579] sd 8:0:5:0: [sdj] Unhandled sense code
  2. Jun 29 05:06:42 borg03b kernel: [231632.877583] sd 8:0:5:0: [sdj] Result: hostbyte=invalid driverbyte=DRIVER_SENSE
  3. Jun 29 05:06:42 borg03b kernel: [231632.877586] sd 8:0:5:0: [sdj] Sense Key : Medium Error [current]
  4. Jun 29 05:06:42 borg03b kernel: [231632.877590] Info fld=0x904ff8b8
  5. Jun 29 05:06:42 borg03b kernel: [231632.877591] sd 8:0:5:0: [sdj] Add. Sense: Unrecovered read error
  6. Jun 29 05:06:42 borg03b kernel: [231632.877595] sd 8:0:5:0: [sdj] CDB: Read(10): 28 00 90 4f f8 3f 00 00 f8 00
  7. Jun 29 05:06:42 borg03b kernel: [231632.877602] end_request: critical target error, dev sdj, sector 2421159999
  8. Jun 29 05:06:42 borg03b kernel: [231632.881963] md/raid10:md4: sdj1: rescheduling sector 6052895744
  9. Jun 29 05:06:46 borg03b kernel: [231636.380147] sd 8:0:5:0: [sdj] Unhandled sense code
  10. Jun 29 05:06:46 borg03b kernel: [231636.380150] sd 8:0:5:0: [sdj] Result: hostbyte=invalid driverbyte=DRIVER_SENSE
  11. Jun 29 05:06:46 borg03b kernel: [231636.380153] sd 8:0:5:0: [sdj] Sense Key : Medium Error [current]
  12. Jun 29 05:06:46 borg03b kernel: [231636.380157] Info fld=0x904ff8b8
  13. Jun 29 05:06:46 borg03b kernel: [231636.380159] sd 8:0:5:0: [sdj] Add. Sense: Unrecovered read error
  14. Jun 29 05:06:46 borg03b kernel: [231636.380162] sd 8:0:5:0: [sdj] CDB: Read(10): 28 00 90 4f f8 b7 00 00 08 00
  15. Jun 29 05:06:46 borg03b kernel: [231636.380168] end_request: critical target error, dev sdj, sector 2421160119
  16. Jun 29 05:06:46 borg03b kernel: [231636.401781] ------------[ cut here ]------------
  17. Jun 29 05:06:46 borg03b kernel: [231636.405694] kernel BUG at drivers/scsi/scsi_lib.c:1153!
  18. Jun 29 05:06:46 borg03b kernel: [231636.405694] invalid opcode: 0000 [#1] SMP
  19. Jun 29 05:06:46 borg03b kernel: [231636.405694] CPU 7
  20. Jun 29 05:06:46 borg03b kernel: [231636.405694] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  21. Jun 29 05:06:46 borg03b kernel: [231636.405694]
  22. Jun 29 05:06:46 borg03b kernel: [231636.405694] Pid: 14506, comm: kworker/7:0 Tainted: G O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  23. Jun 29 05:06:46 borg03b kernel: [231636.405694] RIP: 0010:[<ffffffffa0014da1>] [<ffffffffa0014da1>] scsi_setup_fs_cmnd+0x4c/0xbf [scsi_mod]
  24. Jun 29 05:06:46 borg03b kernel: [231636.405694] RSP: 0018:ffff880388a05d20 EFLAGS: 00010046
  25. Jun 29 05:06:46 borg03b kernel: [231636.405694] RAX: 0000000000000000 RBX: ffff880415b3d330 RCX: ffff88022a47eb60
  26. Jun 29 05:06:46 borg03b kernel: [231636.405694] RDX: ffff88022ab1d800 RSI: ffff880415b3d330 RDI: ffff880229f1f800
  27. Jun 29 05:06:46 borg03b kernel: [231636.405694] RBP: ffff880229f1f800 R08: 0000000000000082 R09: 0000000000000001
  28. Jun 29 05:06:46 borg03b kernel: [231636.405694] R10: ffffffff811bfbd5 R11: ffff880437d91cc0 R12: 00000000904ff878
  29. Jun 29 05:06:46 borg03b kernel: [231636.405694] R13: ffff88022a47eb60 R14: ffff88022a6fdc00 R15: ffff880229f1f800
  30. Jun 29 05:06:46 borg03b kernel: [231636.405694] FS: 00007f2948786700(0000) GS:ffff880437d80000(0000) knlGS:00000000ad27fbb0
  31. Jun 29 05:06:46 borg03b kernel: [231636.405694] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  32. Jun 29 05:06:46 borg03b kernel: [231636.405694] CR2: 00007fd97a65fd70 CR3: 000000042aacb000 CR4: 00000000000006e0
  33. Jun 29 05:06:46 borg03b kernel: [231636.405694] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  34. Jun 29 05:06:46 borg03b kernel: [231636.405694] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  35. Jun 29 05:06:46 borg03b kernel: [231636.405694] Process kworker/7:0 (pid: 14506, threadinfo ffff880388a04000, task ffff880413cecea0)
  36. Jun 29 05:06:46 borg03b kernel: [231636.405694] Stack:
  37. Jun 29 05:06:46 borg03b kernel: [231636.405694] ffff8804294fde10 0000000000000000 ffff880415b3d330 ffffffffa0061835
  38. Jun 29 05:06:46 borg03b kernel: [231636.405694] ffff880437d91cc0 0000000000011c00 ffff880229f1f088 ffff880229f1f108
  39. Jun 29 05:06:46 borg03b kernel: [231636.405694] ffff88022a47e788 ffff880229c037c8 ffff880229c037c8 ffff880415b3d330
  40. Jun 29 05:06:46 borg03b kernel: [231636.405694] Call Trace:
  41. Jun 29 05:06:46 borg03b kernel: [231636.405694] [<ffffffffa0061835>] ? sd_prep_fn+0x29c/0xb23 [sd_mod]
  42. Jun 29 05:06:46 borg03b kernel: [231636.405694] [<ffffffff811b2817>] ? blk_peek_request+0xd9/0x1b9
  43. Jun 29 05:06:46 borg03b kernel: [231636.405694] [<ffffffffa00144a2>] ? scsi_request_fn+0x75/0x479 [scsi_mod]
  44. Jun 29 05:06:46 borg03b kernel: [231636.405694] [<ffffffff811b0c96>] ? freed_request+0x5a/0x5a
  45. Jun 29 05:06:46 borg03b kernel: [231636.405694] [<ffffffff811b0cb2>] ? blk_delay_work+0x1c/0x28
  46. Jun 29 05:06:46 borg03b kernel: [231636.405694] [<ffffffff8105028b>] ? process_one_work+0x1c7/0x2e5
  47. Jun 29 05:06:46 borg03b kernel: [231636.405694] [<ffffffff810504cc>] ? worker_thread+0x123/0x234
  48. Jun 29 05:06:46 borg03b kernel: [231636.405694] [<ffffffff810503a9>] ? process_one_work+0x2e5/0x2e5
  49. Jun 29 05:06:46 borg03b kernel: [231636.405694] [<ffffffff810503a9>] ? process_one_work+0x2e5/0x2e5
  50. Jun 29 05:06:46 borg03b kernel: [231636.405694] [<ffffffff81053d3e>] ? kthread+0x7a/0x82
  51. Jun 29 05:06:46 borg03b kernel: [231636.405694] [<ffffffff8134beb4>] ? kernel_thread_helper+0x4/0x10
  52. Jun 29 05:06:46 borg03b kernel: [231636.405694] [<ffffffff81053cc4>] ? kthread_worker_fn+0x139/0x139
  53. Jun 29 05:06:46 borg03b kernel: [231636.405694] [<ffffffff8134beb0>] ? gs_change+0xb/0xb
  54. Jun 29 05:06:46 borg03b kernel: [231636.405694] Code: 85 c0 74 1d 48 8b 00 48 85 c0 74 15 48 8b 40 50 48 85 c0 74 0c 48 89 de 48 89 ef ff d0 85 c0 75 79 66 83 bb e0 00 00 00 00 75 04 <0f> 0b eb fe 48 8b 93 e8 00 00 00 48 85 d2 75 21 be 20 00 00 00
  55. Jun 29 05:06:46 borg03b kernel: [231636.405694] RIP [<ffffffffa0014da1>] scsi_setup_fs_cmnd+0x4c/0xbf [scsi_mod]
  56. Jun 29 05:06:46 borg03b kernel: [231636.405694] RSP <ffff880388a05d20>
  57. Jun 29 05:06:46 borg03b kernel: [231636.405694] ---[ end trace d1ccaa1c8031f614 ]---
  58. Jun 29 05:06:46 borg03b kernel: [231636.619825] BUG: unable to handle kernel paging request at fffffffffffffff8
  59. Jun 29 05:07:47 borg03b kernel: [231636.623767] IP: [<ffffffff810539c7>] kthread_data+0x7/0xc
  60. Jun 29 05:07:47 borg03b kernel: [231636.623767] PGD 1607067 PUD 1608067 PMD 0
  61. Jun 29 05:07:47 borg03b kernel: [231636.623767] Oops: 0000 [#2] SMP
  62. Jun 29 05:07:47 borg03b kernel: [231636.623767] CPU 7
  63. Jun 29 05:07:47 borg03b kernel: [231636.623767] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  64. Jun 29 05:07:47 borg03b kernel: [231636.623767]
  65. Jun 29 05:07:47 borg03b kernel: [231636.623767] Pid: 14506, comm: kworker/7:0 Tainted: G D O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  66. Jun 29 05:07:47 borg03b kernel: [231636.623767] RIP: 0010:[<ffffffff810539c7>] [<ffffffff810539c7>] kthread_data+0x7/0xc
  67. Jun 29 05:07:47 borg03b kernel: [231636.623767] RSP: 0018:ffff880388a05a80 EFLAGS: 00010002
  68. Jun 29 05:07:47 borg03b kernel: [231636.623767] RAX: 0000000000000000 RBX: 0000000000000007 RCX: 0000000000000007
  69. Jun 29 05:07:47 borg03b kernel: [231636.623767] RDX: 0000000000000007 RSI: 0000000000000007 RDI: ffff880413cecea0
  70. Jun 29 05:07:47 borg03b kernel: [231636.623767] RBP: ffff880413cecea0 R08: 0000000000000800 R09: ffff880413cecea0
  71. Jun 29 05:07:47 borg03b kernel: [231636.623767] R10: 0000000000000000 R11: dead000000200200 R12: ffff880388a05b58
  72. Jun 29 05:07:47 borg03b kernel: [231636.623767] R13: 0000000000000007 R14: ffff880413ced158 R15: ffff880413ced060
  73. Jun 29 05:07:47 borg03b kernel: [231636.623767] FS: 00007f2948786700(0000) GS:ffff880437d80000(0000) knlGS:00000000ad27fbb0
  74. Jun 29 05:07:47 borg03b kernel: [231636.623767] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  75. Jun 29 05:07:47 borg03b kernel: [231636.623767] CR2: fffffffffffffff8 CR3: 000000042aacb000 CR4: 00000000000006e0
  76. Jun 29 05:07:47 borg03b kernel: [231636.623767] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  77. Jun 29 05:07:47 borg03b kernel: [231636.623767] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  78. Jun 29 05:07:47 borg03b kernel: [231636.623767] Process kworker/7:0 (pid: 14506, threadinfo ffff880388a04000, task ffff880413cecea0)
  79. Jun 29 05:07:47 borg03b kernel: [231636.623767] Stack:
  80. Jun 29 05:07:47 borg03b kernel: [231636.623767] ffffffff8104f65f ffff880437d91cc0 ffffffff813441fd ffff88022a095280
  81. Jun 29 05:07:47 borg03b kernel: [231636.623767] ffff88022a095280 0000000000000001 0000000000000296 0000000000011cc0
  82. Jun 29 05:07:47 borg03b kernel: [231636.623767] ffff880388a05fd8 ffff880388a05fd8 0000000000011cc0 ffff880413cecea0
  83. Jun 29 05:07:47 borg03b kernel: [231636.623767] Call Trace:
  84. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8104f65f>] ? wq_worker_sleeping+0x8/0x7f
  85. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff813441fd>] ? __schedule+0x13c/0x57e
  86. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8103dddc>] ? do_exit+0x74f/0x76d
  87. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff81346599>] ? oops_end+0xa8/0xad
  88. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff81002d19>] ? do_invalid_op+0x85/0x8f
  89. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffffa0014da1>] ? scsi_setup_fs_cmnd+0x4c/0xbf [scsi_mod]
  90. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8134bd35>] ? invalid_op+0x15/0x20
  91. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff811bfbd5>] ? cfq_init_queue+0x3bd/0x3bd
  92. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffffa0014da1>] ? scsi_setup_fs_cmnd+0x4c/0xbf [scsi_mod]
  93. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffffa0061835>] ? sd_prep_fn+0x29c/0xb23 [sd_mod]
  94. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff811b2817>] ? blk_peek_request+0xd9/0x1b9
  95. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffffa00144a2>] ? scsi_request_fn+0x75/0x479 [scsi_mod]
  96. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff811b0c96>] ? freed_request+0x5a/0x5a
  97. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff811b0cb2>] ? blk_delay_work+0x1c/0x28
  98. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8105028b>] ? process_one_work+0x1c7/0x2e5
  99. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff810504cc>] ? worker_thread+0x123/0x234
  100. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff810503a9>] ? process_one_work+0x2e5/0x2e5
  101. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff810503a9>] ? process_one_work+0x2e5/0x2e5
  102. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff81053d3e>] ? kthread+0x7a/0x82
  103. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8134beb4>] ? kernel_thread_helper+0x4/0x10
  104. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff81053cc4>] ? kthread_worker_fn+0x139/0x139
  105. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8134beb0>] ? gs_change+0xb/0xb
  106. Jun 29 05:07:47 borg03b kernel: [231636.623767] Code: 41 5e 41 5f c3 41 bf ea ff ff ff eb 9c 90 90 90 65 48 8b 04 25 00 b6 00 00 48 8b 80 60 02 00 00 8b 40 f0 c3 48 8b 87 60 02 00 00 <48> 8b 40 f8 c3 48 3b 3d fd d3 70 00 75 08 0f bf 87 4a 06 00 00
  107. Jun 29 05:07:47 borg03b kernel: [231636.623767] RIP [<ffffffff810539c7>] kthread_data+0x7/0xc
  108. Jun 29 05:07:47 borg03b kernel: [231636.623767] RSP <ffff880388a05a80>
  109. Jun 29 05:07:47 borg03b kernel: [231636.623767] CR2: fffffffffffffff8
  110. Jun 29 05:07:47 borg03b kernel: [231636.623767] ---[ end trace d1ccaa1c8031f615 ]---
  111. Jun 29 05:07:47 borg03b kernel: [231636.623767] Fixing recursive fault but reboot is needed!
  112. Jun 29 05:07:47 borg03b kernel: [231636.623767] ------------[ cut here ]------------
  113. Jun 29 05:07:47 borg03b kernel: [231636.623767] WARNING: at kernel/watchdog.c:241 watchdog_overflow_callback+0x96/0xa1()
  114. Jun 29 05:07:47 borg03b kernel: [231636.623767] Hardware name: H8DM3-2
  115. Jun 29 05:07:47 borg03b kernel: [231636.623767] Watchdog detected hard LOCKUP on cpu 7
  116. Jun 29 05:07:47 borg03b kernel: [231636.623767] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  117. Jun 29 05:07:47 borg03b kernel: [231636.623767] Pid: 14506, comm: kworker/7:0 Tainted: G D O 3.2.20-vs2.3.2.10 #1
  118. Jun 29 05:07:47 borg03b kernel: [231636.623767] Call Trace:
  119. Jun 29 05:07:47 borg03b kernel: [231636.623767] <NMI> [<ffffffff8103a606>] ? warn_slowpath_common+0x78/0x8c
  120. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8103a6b8>] ? warn_slowpath_fmt+0x45/0x4a
  121. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff81343cf5>] ? printk+0x40/0x4b
  122. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8108887a>] ? watchdog_overflow_callback+0x96/0xa1
  123. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff810aa532>] ? __perf_event_overflow+0xf9/0x176
  124. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff810587e3>] ? sched_clock_local+0x13/0x76
  125. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8100e4d4>] ? x86_pmu_handle_irq+0xc3/0xf8
  126. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff812235f6>] ? ghes_read_estatus+0x7d/0x136
  127. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff81223b41>] ? ghes_notify_nmi+0x55/0x1ce
  128. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff813466c9>] ? do_nmi+0x9e/0x2ca
  129. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff81345f8a>] ? nmi+0x1a/0x20
  130. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8134583a>] ? _raw_spin_lock_irq+0x17/0x1c
  131. Jun 29 05:07:47 borg03b kernel: [231636.623767] <<EOE>> [<ffffffff81344188>] ? __schedule+0xc7/0x57e
  132. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8103d78c>] ? do_exit+0xff/0x76d
  133. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff81343cf5>] ? printk+0x40/0x4b
  134. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8103aa6c>] ? kmsg_dump+0x41/0xdd
  135. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff81346599>] ? oops_end+0xa8/0xad
  136. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff81020927>] ? no_context+0x1ff/0x20c
  137. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8134850a>] ? do_page_fault+0x190/0x328
  138. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff811c9ac4>] ? number+0x13d/0x243
  139. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff81054000>] ? kthread_stop+0x3b/0xa7
  140. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff811cb3cf>] ? vsnprintf+0x3ef/0x428
  141. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff81345d2f>] ? page_fault+0x1f/0x30
  142. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff810539c7>] ? kthread_data+0x7/0xc
  143. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8104f65f>] ? wq_worker_sleeping+0x8/0x7f
  144. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff813441fd>] ? __schedule+0x13c/0x57e
  145. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8103dddc>] ? do_exit+0x74f/0x76d
  146. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff81346599>] ? oops_end+0xa8/0xad
  147. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff81002d19>] ? do_invalid_op+0x85/0x8f
  148. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffffa0014da1>] ? scsi_setup_fs_cmnd+0x4c/0xbf [scsi_mod]
  149. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8134bd35>] ? invalid_op+0x15/0x20
  150. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff811bfbd5>] ? cfq_init_queue+0x3bd/0x3bd
  151. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffffa0014da1>] ? scsi_setup_fs_cmnd+0x4c/0xbf [scsi_mod]
  152. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffffa0061835>] ? sd_prep_fn+0x29c/0xb23 [sd_mod]
  153. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff811b2817>] ? blk_peek_request+0xd9/0x1b9
  154. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffffa00144a2>] ? scsi_request_fn+0x75/0x479 [scsi_mod]
  155. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff811b0c96>] ? freed_request+0x5a/0x5a
  156. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff811b0cb2>] ? blk_delay_work+0x1c/0x28
  157. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8105028b>] ? process_one_work+0x1c7/0x2e5
  158. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff810504cc>] ? worker_thread+0x123/0x234
  159. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff810503a9>] ? process_one_work+0x2e5/0x2e5
  160. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff810503a9>] ? process_one_work+0x2e5/0x2e5
  161. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff81053d3e>] ? kthread+0x7a/0x82
  162. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8134beb4>] ? kernel_thread_helper+0x4/0x10
  163. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff81053cc4>] ? kthread_worker_fn+0x139/0x139
  164. Jun 29 05:07:47 borg03b kernel: [231636.623767] [<ffffffff8134beb0>] ? gs_change+0xb/0xb
  165. Jun 29 05:07:47 borg03b kernel: [231636.623767] ---[ end trace d1ccaa1c8031f616 ]---
  166. Jun 29 05:07:47 borg03b kernel: [231644.579585] ------------[ cut here ]------------
  167. Jun 29 05:07:47 borg03b kernel: [231644.579585] WARNING: at kernel/watchdog.c:241 watchdog_overflow_callback+0x96/0xa1()
  168. Jun 29 05:07:47 borg03b kernel: [231644.579585] Hardware name: H8DM3-2
  169. Jun 29 05:07:47 borg03b kernel: [231644.579585] Watchdog detected hard LOCKUP on cpu 1
  170. Jun 29 05:07:47 borg03b kernel: [231644.579585] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  171. Jun 29 05:07:47 borg03b kernel: [231644.579585] Pid: 0, comm: swapper/1 Tainted: G D W O 3.2.20-vs2.3.2.10 #1
  172. Jun 29 05:07:47 borg03b kernel: [231644.579585] Call Trace:
  173. Jun 29 05:07:47 borg03b kernel: [231644.579585] <NMI> [<ffffffff8103a606>] ? warn_slowpath_common+0x78/0x8c
  174. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8103a6b8>] ? warn_slowpath_fmt+0x45/0x4a
  175. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81343cf5>] ? printk+0x40/0x4b
  176. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8108887a>] ? watchdog_overflow_callback+0x96/0xa1
  177. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff810aa532>] ? __perf_event_overflow+0xf9/0x176
  178. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff810587e3>] ? sched_clock_local+0x13/0x76
  179. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8100e4d4>] ? x86_pmu_handle_irq+0xc3/0xf8
  180. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff812235f6>] ? ghes_read_estatus+0x7d/0x136
  181. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81223b41>] ? ghes_notify_nmi+0x55/0x1ce
  182. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff813466c9>] ? do_nmi+0x9e/0x2ca
  183. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81345f8a>] ? nmi+0x1a/0x20
  184. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff813459f1>] ? _raw_spin_lock_irqsave+0x19/0x1e
  185. Jun 29 05:07:47 borg03b kernel: [231644.579585] <<EOE>> <IRQ> [<ffffffff811b6082>] ? blk_rq_timed_out_timer+0x2a/0x100
  186. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff810aa3f6>] ? perf_event_task_tick+0x1f0/0x233
  187. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81046874>] ? run_timer_softirq+0x1b8/0x27b
  188. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff811b6058>] ? blk_abort_queue+0xf6/0xf6
  189. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8103fb92>] ? __do_softirq+0xbe/0x194
  190. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81017d47>] ? lapic_next_event+0x18/0x1d
  191. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8134bfac>] ? call_softirq+0x1c/0x30
  192. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8100393d>] ? do_softirq+0x31/0x63
  193. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8103f96d>] ? irq_exit+0x44/0xb5
  194. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff810188bd>] ? smp_apic_timer_interrupt+0x85/0x93
  195. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8134b54b>] ? apic_timer_interrupt+0x6b/0x70
  196. Jun 29 05:07:47 borg03b kernel: [231644.579585] <EOI> [<ffffffff81008d63>] ? default_idle+0x48/0x79
  197. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81008d43>] ? default_idle+0x28/0x79
  198. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  199. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  200. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81345843>] ? _raw_spin_unlock_irqrestore+0x4/0x5
  201. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  202. Jun 29 05:07:47 borg03b kernel: [231644.579585] ---[ end trace d1ccaa1c8031f617 ]---
  203. Jun 29 05:07:47 borg03b kernel: [231696.404015] INFO: rcu_sched detected stalls on CPUs/tasks: { 7} (detected by 3, t=15002 jiffies)
  204. Jun 29 05:07:47 borg03b kernel: [231696.404018] sending NMI to all CPUs:
  205. Jun 29 05:07:47 borg03b kernel: [231644.579585] NMI backtrace for cpu 1
  206. Jun 29 05:07:47 borg03b kernel: [231644.579585] CPU 1
  207. Jun 29 05:07:47 borg03b kernel: [231644.579585] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  208. Jun 29 05:07:47 borg03b kernel: [231644.579585]
  209. Jun 29 05:07:47 borg03b kernel: [231644.579585] Pid: 0, comm: swapper/1 Tainted: G D W O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  210. Jun 29 05:07:47 borg03b kernel: [231644.579585] RIP: 0010:[<ffffffff813459f1>] [<ffffffff813459f1>] _raw_spin_lock_irqsave+0x19/0x1e
  211. Jun 29 05:07:47 borg03b kernel: [231644.579585] RSP: 0000:ffff880237c83e18 EFLAGS: 00000097
  212. Jun 29 05:07:47 borg03b kernel: [231644.579585] RAX: 0000000000000282 RBX: ffff88022a47e788 RCX: 0000000000000050
  213. Jun 29 05:07:47 borg03b kernel: [231644.579585] RDX: 0000000000000051 RSI: 0000000000000001 RDI: ffff88022a47eb6c
  214. Jun 29 05:07:47 borg03b kernel: [231644.579585] RBP: ffff88022a47ec08 R08: 0000d2acde6ea100 R09: dead000000200200
  215. Jun 29 05:07:47 borg03b kernel: [231644.579585] R10: ffff880237c8b870 R11: ffffffff81404595 R12: 0000000000000000
  216. Jun 29 05:07:47 borg03b kernel: [231644.579585] R13: 0000000000000000 R14: ffff88022b4c5fd8 R15: ffff88022a47ec58
  217. Jun 29 05:07:47 borg03b kernel: [231644.579585] FS: 00007ff3fb7f37c0(0000) GS:ffff880237c80000(0000) knlGS:00000000e4fffbb0
  218. Jun 29 05:07:47 borg03b kernel: [231644.579585] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  219. Jun 29 05:07:47 borg03b kernel: [231644.579585] CR2: 00007f8dbc87f338 CR3: 000000021d8e7000 CR4: 00000000000006e0
  220. Jun 29 05:07:47 borg03b kernel: [231644.579585] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  221. Jun 29 05:07:47 borg03b kernel: [231644.579585] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  222. Jun 29 05:07:47 borg03b kernel: [231644.579585] Process swapper/1 (pid: 0, threadinfo ffff88022b4c4000, task ffff88022b4b60c0)
  223. Jun 29 05:07:47 borg03b kernel: [231644.579585] Stack:
  224. Jun 29 05:07:47 borg03b kernel: [231644.579585] ffffffff811b6082 ffffffff810aa3f6 ffff88022b4bc000 ffff88022a47ec08
  225. Jun 29 05:07:47 borg03b kernel: [231644.579585] 0000000000000100 ffff880237c83ea0 ffff88022b4c5fd8 ffff88022b4c5fd8
  226. Jun 29 05:07:47 borg03b kernel: [231644.579585] ffffffff81046874 ffff880237c83e58 ffffffff811b6058 ffff88022b4bd020
  227. Jun 29 05:07:47 borg03b kernel: [231644.579585] Call Trace:
  228. Jun 29 05:07:47 borg03b kernel: [231644.579585] <IRQ>
  229. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff811b6082>] ? blk_rq_timed_out_timer+0x2a/0x100
  230. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff810aa3f6>] ? perf_event_task_tick+0x1f0/0x233
  231. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81046874>] ? run_timer_softirq+0x1b8/0x27b
  232. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff811b6058>] ? blk_abort_queue+0xf6/0xf6
  233. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8103fb92>] ? __do_softirq+0xbe/0x194
  234. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81017d47>] ? lapic_next_event+0x18/0x1d
  235. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8134bfac>] ? call_softirq+0x1c/0x30
  236. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8100393d>] ? do_softirq+0x31/0x63
  237. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8103f96d>] ? irq_exit+0x44/0xb5
  238. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff810188bd>] ? smp_apic_timer_interrupt+0x85/0x93
  239. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8134b54b>] ? apic_timer_interrupt+0x6b/0x70
  240. Jun 29 05:07:47 borg03b kernel: [231644.579585] <EOI>
  241. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81008d63>] ? default_idle+0x48/0x79
  242. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81008d43>] ? default_idle+0x28/0x79
  243. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  244. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  245. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81345843>] ? _raw_spin_unlock_irqrestore+0x4/0x5
  246. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  247. Jun 29 05:07:47 borg03b kernel: [231644.579585] Code: 07 88 c2 66 c1 e8 08 38 c2 74 06 f3 90 8a 17 eb f6 c3 9c 58 fa ba 00 01 00 00 f0 66 0f c1 17 88 d1 66 c1 ea 08 38 d1 74 06 f3 90 <8a> 0f eb f6 c3 48 83 ec 08 48 c7 c7 90 34 64 81 58 e9 3a f2 ff
  248. Jun 29 05:07:47 borg03b kernel: [231644.579585] Call Trace:
  249. Jun 29 05:07:47 borg03b kernel: [231644.579585] <IRQ> [<ffffffff811b6082>] ? blk_rq_timed_out_timer+0x2a/0x100
  250. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff810aa3f6>] ? perf_event_task_tick+0x1f0/0x233
  251. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81046874>] ? run_timer_softirq+0x1b8/0x27b
  252. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff811b6058>] ? blk_abort_queue+0xf6/0xf6
  253. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8103fb92>] ? __do_softirq+0xbe/0x194
  254. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81017d47>] ? lapic_next_event+0x18/0x1d
  255. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8134bfac>] ? call_softirq+0x1c/0x30
  256. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8100393d>] ? do_softirq+0x31/0x63
  257. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8103f96d>] ? irq_exit+0x44/0xb5
  258. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff810188bd>] ? smp_apic_timer_interrupt+0x85/0x93
  259. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8134b54b>] ? apic_timer_interrupt+0x6b/0x70
  260. Jun 29 05:07:47 borg03b kernel: [231644.579585] <EOI> [<ffffffff81008d63>] ? default_idle+0x48/0x79
  261. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81008d43>] ? default_idle+0x28/0x79
  262. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  263. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  264. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff81345843>] ? _raw_spin_unlock_irqrestore+0x4/0x5
  265. Jun 29 05:07:47 borg03b kernel: [231644.579585] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  266. Jun 29 05:07:47 borg03b kernel: [231670.098823] NMI backtrace for cpu 7
  267. Jun 29 05:07:47 borg03b kernel: [231670.098823] CPU 7
  268. Jun 29 05:07:47 borg03b kernel: [231670.098823] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  269. Jun 29 05:07:47 borg03b kernel: [231670.098823]
  270. Jun 29 05:07:47 borg03b kernel: [231670.098823] Pid: 14506, comm: kworker/7:0 Tainted: G D W O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  271. Jun 29 05:07:47 borg03b kernel: [231670.098823] RIP: 0010:[<ffffffff8134583c>] [<ffffffff8134583c>] _raw_spin_lock_irq+0x19/0x1c
  272. Jun 29 05:07:47 borg03b kernel: [231670.098823] RSP: 0018:ffff880388a05740 EFLAGS: 00000097
  273. Jun 29 05:07:47 borg03b kernel: [231670.098823] RAX: 0000000000000044 RBX: ffff880437d91cc0 RCX: 000000000000006c
  274. Jun 29 05:07:47 borg03b kernel: [231670.098823] RDX: 000000000073b543 RSI: ffff880413cecea0 RDI: ffff880437d91cc0
  275. Jun 29 05:07:47 borg03b kernel: [231670.098823] RBP: ffff880413cecea0 R08: 0000000000000000 R09: 0000000000000000
  276. Jun 29 05:07:47 borg03b kernel: [231670.098823] R10: ffff88022a839290 R11: ffffffff81433e60 R12: 0000000000000009
  277. Jun 29 05:07:47 borg03b kernel: [231670.098823] R13: 0000000000000007 R14: ffff880413ced158 R15: 0000000000000000
  278. Jun 29 05:07:47 borg03b kernel: [231670.098823] FS: 00007f2948786700(0000) GS:ffff880437d80000(0000) knlGS:00000000ad27fbb0
  279. Jun 29 05:07:47 borg03b kernel: [231670.098823] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  280. Jun 29 05:07:47 borg03b kernel: [231670.098823] CR2: fffffffffffffff8 CR3: 000000042aacb000 CR4: 00000000000006e0
  281. Jun 29 05:07:47 borg03b kernel: [231670.098823] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  282. Jun 29 05:07:47 borg03b kernel: [231670.098823] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  283. Jun 29 05:07:47 borg03b kernel: [231670.098823] Process kworker/7:0 (pid: 14506, threadinfo ffff880388a04000, task ffff880413cecea0)
  284. Jun 29 05:07:47 borg03b kernel: [231670.098823] Stack:
  285. Jun 29 05:07:47 borg03b kernel: [231670.098823] ffffffff81344188 ffffffff00000010 ffff880413cecea0 0000000000000009
  286. Jun 29 05:07:47 borg03b kernel: [231670.098823] 0000000000000009 0000000000011cc0 ffff880388a05fd8 ffff880388a05fd8
  287. Jun 29 05:07:47 borg03b kernel: [231670.098823] 0000000000011cc0 ffff880413cecea0 ffff880388a04010 ffff880388a057a8
  288. Jun 29 05:07:47 borg03b kernel: [231670.098823] Call Trace:
  289. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81344188>] ? __schedule+0xc7/0x57e
  290. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8103d78c>] ? do_exit+0xff/0x76d
  291. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81343cf5>] ? printk+0x40/0x4b
  292. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8103aa6c>] ? kmsg_dump+0x41/0xdd
  293. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81346599>] ? oops_end+0xa8/0xad
  294. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81020927>] ? no_context+0x1ff/0x20c
  295. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8134850a>] ? do_page_fault+0x190/0x328
  296. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff811c9ac4>] ? number+0x13d/0x243
  297. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81054000>] ? kthread_stop+0x3b/0xa7
  298. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff811cb3cf>] ? vsnprintf+0x3ef/0x428
  299. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81345d2f>] ? page_fault+0x1f/0x30
  300. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff810539c7>] ? kthread_data+0x7/0xc
  301. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8104f65f>] ? wq_worker_sleeping+0x8/0x7f
  302. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff813441fd>] ? __schedule+0x13c/0x57e
  303. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8103dddc>] ? do_exit+0x74f/0x76d
  304. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81346599>] ? oops_end+0xa8/0xad
  305. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81002d19>] ? do_invalid_op+0x85/0x8f
  306. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffffa0014da1>] ? scsi_setup_fs_cmnd+0x4c/0xbf [scsi_mod]
  307. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8134bd35>] ? invalid_op+0x15/0x20
  308. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff811bfbd5>] ? cfq_init_queue+0x3bd/0x3bd
  309. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffffa0014da1>] ? scsi_setup_fs_cmnd+0x4c/0xbf [scsi_mod]
  310. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffffa0061835>] ? sd_prep_fn+0x29c/0xb23 [sd_mod]
  311. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff811b2817>] ? blk_peek_request+0xd9/0x1b9
  312. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffffa00144a2>] ? scsi_request_fn+0x75/0x479 [scsi_mod]
  313. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff811b0c96>] ? freed_request+0x5a/0x5a
  314. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff811b0cb2>] ? blk_delay_work+0x1c/0x28
  315. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8105028b>] ? process_one_work+0x1c7/0x2e5
  316. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff810504cc>] ? worker_thread+0x123/0x234
  317. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff810503a9>] ? process_one_work+0x2e5/0x2e5
  318. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff810503a9>] ? process_one_work+0x2e5/0x2e5
  319. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81053d3e>] ? kthread+0x7a/0x82
  320. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8134beb4>] ? kernel_thread_helper+0x4/0x10
  321. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81053cc4>] ? kthread_worker_fn+0x139/0x139
  322. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8134beb0>] ? gs_change+0xb/0xb
  323. Jun 29 05:07:47 borg03b kernel: [231670.098823] Code: 00 89 d0 f0 66 0f b1 0f 66 39 d0 0f 94 c0 0f b6 c0 c3 fa b8 00 01 00 00 f0 66 0f c1 07 88 c2 66 c1 e8 08 38 c2 74 06 f3 90 8a 17 <eb> f6 c3 fe 07 56 9d c3 83 ca ff f0 0f c1 17 ff ca b8 01 00 00
  324. Jun 29 05:07:47 borg03b kernel: [231670.098823] Call Trace:
  325. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81344188>] ? __schedule+0xc7/0x57e
  326. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8103d78c>] ? do_exit+0xff/0x76d
  327. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81343cf5>] ? printk+0x40/0x4b
  328. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8103aa6c>] ? kmsg_dump+0x41/0xdd
  329. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81346599>] ? oops_end+0xa8/0xad
  330. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81020927>] ? no_context+0x1ff/0x20c
  331. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8134850a>] ? do_page_fault+0x190/0x328
  332. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff811c9ac4>] ? number+0x13d/0x243
  333. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81054000>] ? kthread_stop+0x3b/0xa7
  334. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff811cb3cf>] ? vsnprintf+0x3ef/0x428
  335. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81345d2f>] ? page_fault+0x1f/0x30
  336. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff810539c7>] ? kthread_data+0x7/0xc
  337. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8104f65f>] ? wq_worker_sleeping+0x8/0x7f
  338. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff813441fd>] ? __schedule+0x13c/0x57e
  339. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8103dddc>] ? do_exit+0x74f/0x76d
  340. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81346599>] ? oops_end+0xa8/0xad
  341. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81002d19>] ? do_invalid_op+0x85/0x8f
  342. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffffa0014da1>] ? scsi_setup_fs_cmnd+0x4c/0xbf [scsi_mod]
  343. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8134bd35>] ? invalid_op+0x15/0x20
  344. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff811bfbd5>] ? cfq_init_queue+0x3bd/0x3bd
  345. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffffa0014da1>] ? scsi_setup_fs_cmnd+0x4c/0xbf [scsi_mod]
  346. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffffa0061835>] ? sd_prep_fn+0x29c/0xb23 [sd_mod]
  347. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff811b2817>] ? blk_peek_request+0xd9/0x1b9
  348. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffffa00144a2>] ? scsi_request_fn+0x75/0x479 [scsi_mod]
  349. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff811b0c96>] ? freed_request+0x5a/0x5a
  350. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff811b0cb2>] ? blk_delay_work+0x1c/0x28
  351. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8105028b>] ? process_one_work+0x1c7/0x2e5
  352. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff810504cc>] ? worker_thread+0x123/0x234
  353. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff810503a9>] ? process_one_work+0x2e5/0x2e5
  354. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff810503a9>] ? process_one_work+0x2e5/0x2e5
  355. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81053d3e>] ? kthread+0x7a/0x82
  356. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8134beb4>] ? kernel_thread_helper+0x4/0x10
  357. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff81053cc4>] ? kthread_worker_fn+0x139/0x139
  358. Jun 29 05:07:47 borg03b kernel: [231670.098823] [<ffffffff8134beb0>] ? gs_change+0xb/0xb
  359. Jun 29 05:07:47 borg03b kernel: [231696.414666] NMI backtrace for cpu 0
  360. Jun 29 05:07:47 borg03b kernel: [231696.414666] CPU 0
  361. Jun 29 05:07:47 borg03b kernel: [231696.414666] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  362. Jun 29 05:07:47 borg03b kernel: [231696.414666]
  363. Jun 29 05:07:47 borg03b kernel: [231696.414666] Pid: 0, comm: swapper/0 Tainted: G D W O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  364. Jun 29 05:07:47 borg03b kernel: [231696.414666] RIP: 0010:[<ffffffff81008d63>] [<ffffffff81008d63>] default_idle+0x48/0x79
  365. Jun 29 05:07:47 borg03b kernel: [231696.414666] RSP: 0018:ffffffff81601f38 EFLAGS: 00000246
  366. Jun 29 05:07:47 borg03b kernel: [231696.414666] RAX: 0000000000000000 RBX: ffffffff81600010 RCX: 00000000c0010055
  367. Jun 29 05:07:47 borg03b kernel: [231696.414666] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000001
  368. Jun 29 05:07:47 borg03b kernel: [231696.414666] RBP: ffffffff816715c8 R08: 0000000000000000 R09: 0000000000000000
  369. Jun 29 05:07:47 borg03b kernel: [231696.414666] R10: 00000000ffffffff R11: 0000000000000000 R12: 0000000000000000
  370. Jun 29 05:07:47 borg03b kernel: [231696.414666] R13: ffffffffffffffff R14: 0000000000000000 R15: 0000000000000000
  371. Jun 29 05:07:47 borg03b kernel: [231696.414666] FS: 00007fabdb6eb6f0(0000) GS:ffff880237c00000(0000) knlGS:00000000e55ffbb0
  372. Jun 29 05:07:47 borg03b kernel: [231696.414666] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  373. Jun 29 05:07:47 borg03b kernel: [231696.414666] CR2: 0000000000f349fc CR3: 000000041436b000 CR4: 00000000000006f0
  374. Jun 29 05:07:47 borg03b kernel: [231696.414666] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  375. Jun 29 05:07:47 borg03b kernel: [231696.414666] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  376. Jun 29 05:07:47 borg03b kernel: [231696.414666] Process swapper/0 (pid: 0, threadinfo ffffffff81600000, task ffffffff8160d020)
  377. Jun 29 05:07:47 borg03b kernel: [231696.414666] Stack:
  378. Jun 29 05:07:47 borg03b kernel: [231696.414666] ffffffff81600010 ffffffff81008e77 0000000000000000 ffffffffffffffff
  379. Jun 29 05:07:47 borg03b kernel: [231696.414666] ffffffff81600010 ffffffff81001dc3 ffffffffffffffff 10dc3b883dbbe2b8
  380. Jun 29 05:07:47 borg03b kernel: [231696.414666] 0000000000000000 0000000000000000 6db6db6db6db6db7 ffffffff8168bbcc
  381. Jun 29 05:07:47 borg03b kernel: [231696.414666] Call Trace:
  382. Jun 29 05:07:47 borg03b kernel: [231696.414666] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  383. Jun 29 05:07:47 borg03b kernel: [231696.414666] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  384. Jun 29 05:07:47 borg03b kernel: [231696.414666] [<ffffffff8168bbcc>] ? start_kernel+0x3a1/0x3ac
  385. Jun 29 05:07:47 borg03b kernel: [231696.414666] [<ffffffff8168b38f>] ? x86_64_start_kernel+0xee/0xf2
  386. Jun 29 05:07:47 borg03b kernel: [231696.414666] Code: 34 25 f0 ca 00 00 e8 53 f8 ff ff 65 48 8b 04 25 08 b6 00 00 83 a0 3c e0 ff ff fb 0f ae f0 48 8b 80 38 e0 ff ff a8 08 75 04 fb f4 <eb> 01 fb 65 48 8b 04 25 08 b6 00 00 83 88 3c e0 ff ff 04 65 8b
  387. Jun 29 05:07:47 borg03b kernel: [231696.414666] Call Trace:
  388. Jun 29 05:07:47 borg03b kernel: [231696.414666] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  389. Jun 29 05:07:47 borg03b kernel: [231696.414666] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  390. Jun 29 05:07:47 borg03b kernel: [231696.414666] [<ffffffff8168bbcc>] ? start_kernel+0x3a1/0x3ac
  391. Jun 29 05:07:47 borg03b kernel: [231696.414666] [<ffffffff8168b38f>] ? x86_64_start_kernel+0xee/0xf2
  392. Jun 29 05:07:47 borg03b kernel: [231696.404018] NMI backtrace for cpu 3
  393. Jun 29 05:07:47 borg03b kernel: [231696.404018] CPU 3
  394. Jun 29 05:07:47 borg03b kernel: [231696.404018] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  395. Jun 29 05:07:47 borg03b kernel: [231696.404018]
  396. Jun 29 05:07:47 borg03b kernel: [231696.404018] Pid: 9852, comm: insserv Tainted: G D W O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  397. Jun 29 05:07:47 borg03b kernel: [231696.404018] RIP: 0010:[<ffffffff811cc154>] [<ffffffff811cc154>] delay_tsc+0x8/0x54
  398. Jun 29 05:07:47 borg03b kernel: [231696.404018] RSP: 0000:ffff880237d83e20 EFLAGS: 00000897
  399. Jun 29 05:07:47 borg03b kernel: [231696.404018] RAX: ffffffff811cc14c RBX: 0000000000000001 RCX: 0000000000000001
  400. Jun 29 05:07:47 borg03b kernel: [231696.404018] RDX: 0000000000248429 RSI: 0000000000000003 RDI: 000000000024842a
  401. Jun 29 05:07:47 borg03b kernel: [231696.404018] RBP: ffffffff8161e580 R08: 0000000000000000 R09: ffff880237d83de8
  402. Jun 29 05:07:47 borg03b kernel: [231696.404018] R10: 0000000000000000 R11: ffffffff81433e60 R12: ffffffff8161e580
  403. Jun 29 05:07:47 borg03b kernel: [231696.404018] R13: 0000000000000001 R14: ffffffff8161e680 R15: 0000000000000008
  404. Jun 29 05:07:47 borg03b kernel: [231696.404018] FS: 00007fac02fb2700(0000) GS:ffff880237d80000(0000) knlGS:00000000e55ffbb0
  405. Jun 29 05:07:47 borg03b kernel: [231696.404018] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
  406. Jun 29 05:07:47 borg03b kernel: [231696.404018] CR2: 00007fac02fb9000 CR3: 00000001e5077000 CR4: 00000000000006e0
  407. Jun 29 05:07:47 borg03b kernel: [231696.404018] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  408. Jun 29 05:07:47 borg03b kernel: [231696.404018] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  409. Jun 29 05:07:47 borg03b kernel: [231696.404018] Process insserv (pid: 9852, threadinfo ffff88001c6e0000, task ffff8800091820c0)
  410. Jun 29 05:07:47 borg03b kernel: [231696.404018] Stack:
  411. Jun 29 05:07:47 borg03b kernel: [231696.404018] ffffffff81018c35 ffff880237d8d550 ffffffff8108df92 0000000037d91d30
  412. Jun 29 05:07:47 borg03b kernel: [231696.404018] ffffffff8161e680 ffffffff5d0fc600 0000000000000003 0000000000000001
  413. Jun 29 05:07:47 borg03b kernel: [231696.404018] 0000000000000003 ffff880237d83f48 0000000000000001 ffffffff81068684
  414. Jun 29 05:07:47 borg03b kernel: [231696.404018] Call Trace:
  415. Jun 29 05:07:47 borg03b kernel: [231696.404018] <IRQ>
  416. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff81018c35>] ? arch_trigger_all_cpu_backtrace+0x58/0x6b
  417. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff8108df92>] ? __rcu_pending+0x21a/0x336
  418. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff81068684>] ? tick_nohz_handler+0xc6/0xc6
  419. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff8108e1cc>] ? rcu_check_callbacks+0xc8/0x108
  420. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff81068684>] ? tick_nohz_handler+0xc6/0xc6
  421. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff81045fc2>] ? update_process_times+0x31/0x63
  422. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff810686ee>] ? tick_sched_timer+0x6a/0x86
  423. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff810572c5>] ? __run_hrtimer+0xab/0x136
  424. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff8103fc0b>] ? __do_softirq+0x137/0x194
  425. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff81057588>] ? hrtimer_interrupt+0xd8/0x1a5
  426. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff810188b8>] ? smp_apic_timer_interrupt+0x80/0x93
  427. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff8134b54b>] ? apic_timer_interrupt+0x6b/0x70
  428. Jun 29 05:07:47 borg03b kernel: [231696.404018] <EOI>
  429. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff8134aabb>] ? system_call_fastpath+0x16/0x1b
  430. Jun 29 05:07:47 borg03b kernel: [231696.404018] Code: 59 ff e0 48 83 ec 08 48 69 ff c7 10 00 00 5e e9 c6 ff ff ff 48 83 ec 08 48 8d 3c bf 41 58 e9 b7 ff ff ff 65 8b 34 25 f0 ca 00 00 <0f> ae f0 66 66 90 0f 31 89 c1 0f ae f0 66 66 90 0f 31 89 c2 29
  431. Jun 29 05:07:47 borg03b kernel: [231696.404018] Call Trace:
  432. Jun 29 05:07:47 borg03b kernel: [231696.404018] <IRQ> [<ffffffff81018c35>] ? arch_trigger_all_cpu_backtrace+0x58/0x6b
  433. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff8108df92>] ? __rcu_pending+0x21a/0x336
  434. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff81068684>] ? tick_nohz_handler+0xc6/0xc6
  435. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff8108e1cc>] ? rcu_check_callbacks+0xc8/0x108
  436. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff81068684>] ? tick_nohz_handler+0xc6/0xc6
  437. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff81045fc2>] ? update_process_times+0x31/0x63
  438. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff810686ee>] ? tick_sched_timer+0x6a/0x86
  439. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff810572c5>] ? __run_hrtimer+0xab/0x136
  440. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff8103fc0b>] ? __do_softirq+0x137/0x194
  441. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff81057588>] ? hrtimer_interrupt+0xd8/0x1a5
  442. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff810188b8>] ? smp_apic_timer_interrupt+0x80/0x93
  443. Jun 29 05:07:47 borg03b kernel: [231696.404018] [<ffffffff8134b54b>] ? apic_timer_interrupt+0x6b/0x70
  444. Jun 29 05:07:47 borg03b kernel: [231696.404018] <EOI> [<ffffffff8134aabb>] ? system_call_fastpath+0x16/0x1b
  445. Jun 29 05:07:47 borg03b kernel: [231697.077082] NMI backtrace for cpu 2
  446. Jun 29 05:07:47 borg03b kernel: [231697.077082] CPU 2
  447. Jun 29 05:07:47 borg03b kernel: [231697.077082] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  448. Jun 29 05:07:47 borg03b kernel: [231697.077082]
  449. Jun 29 05:07:47 borg03b kernel: [231697.077082] Pid: 0, comm: swapper/2 Tainted: G D W O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  450. Jun 29 05:07:47 borg03b kernel: [231697.077082] RIP: 0010:[<ffffffff81008d63>] [<ffffffff81008d63>] default_idle+0x48/0x79
  451. Jun 29 05:07:47 borg03b kernel: [231697.077082] RSP: 0018:ffff88022b4fbed8 EFLAGS: 00000246
  452. Jun 29 05:07:47 borg03b kernel: [231697.077082] RAX: 0000000000000000 RBX: ffff88022b4fa010 RCX: 00000000c0010055
  453. Jun 29 05:07:47 borg03b kernel: [231697.077082] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000001
  454. Jun 29 05:07:47 borg03b kernel: [231697.077082] RBP: ffffffff816715c8 R08: 0000000000000000 R09: ffff88022b4fa010
  455. Jun 29 05:07:47 borg03b kernel: [231697.077082] R10: 00000000ffffffff R11: 0000000000000246 R12: 0000000000000000
  456. Jun 29 05:07:47 borg03b kernel: [231697.077082] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
  457. Jun 29 05:07:47 borg03b kernel: [231697.077082] FS: 00007fa346512720(0000) GS:ffff880237d00000(0000) knlGS:00000000558c06c0
  458. Jun 29 05:07:47 borg03b kernel: [231697.077082] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  459. Jun 29 05:07:47 borg03b kernel: [231697.077082] CR2: 00007f5a599e8cd0 CR3: 000000041a6fb000 CR4: 00000000000006e0
  460. Jun 29 05:07:47 borg03b kernel: [231697.077082] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  461. Jun 29 05:07:47 borg03b kernel: [231697.077082] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  462. Jun 29 05:07:47 borg03b kernel: [231697.077082] Process swapper/2 (pid: 0, threadinfo ffff88022b4fa000, task ffff88022b4f95b0)
  463. Jun 29 05:07:47 borg03b kernel: [231697.077082] Stack:
  464. Jun 29 05:07:47 borg03b kernel: [231697.077082] ffff88022b4fa010 ffffffff81008e77 0000000000000000 0000000000000000
  465. Jun 29 05:07:47 borg03b kernel: [231697.077082] ffff88022b4fa010 ffffffff81001dc3 0000000000000000 2f6d759c3bc5c62c
  466. Jun 29 05:07:47 borg03b kernel: [231697.077082] 0000000000000000 0000000000000002 0000000000000000 ffffffff8134063c
  467. Jun 29 05:07:47 borg03b kernel: [231697.077082] Call Trace:
  468. Jun 29 05:07:47 borg03b kernel: [231697.077082] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  469. Jun 29 05:07:47 borg03b kernel: [231697.077082] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  470. Jun 29 05:07:47 borg03b kernel: [231697.077082] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  471. Jun 29 05:07:47 borg03b kernel: [231697.077082] Code: 34 25 f0 ca 00 00 e8 53 f8 ff ff 65 48 8b 04 25 08 b6 00 00 83 a0 3c e0 ff ff fb 0f ae f0 48 8b 80 38 e0 ff ff a8 08 75 04 fb f4 <eb> 01 fb 65 48 8b 04 25 08 b6 00 00 83 88 3c e0 ff ff 04 65 8b
  472. Jun 29 05:07:47 borg03b kernel: [231697.077082] Call Trace:
  473. Jun 29 05:07:47 borg03b kernel: [231697.077082] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  474. Jun 29 05:07:47 borg03b kernel: [231697.077082] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  475. Jun 29 05:07:47 borg03b kernel: [231697.077082] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  476. Jun 29 05:07:47 borg03b kernel: [231697.077078] NMI backtrace for cpu 4
  477. Jun 29 05:07:47 borg03b kernel: [231697.077078] CPU 4
  478. Jun 29 05:07:47 borg03b kernel: [231697.077078] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  479. Jun 29 05:07:47 borg03b kernel: [231697.077078]
  480. Jun 29 05:07:47 borg03b kernel: [231697.077078] Pid: 0, comm: swapper/4 Tainted: G D W O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  481. Jun 29 05:07:47 borg03b kernel: [231697.077078] RIP: 0010:[<ffffffff81008d63>] [<ffffffff81008d63>] default_idle+0x48/0x79
  482. Jun 29 05:07:47 borg03b kernel: [231697.077078] RSP: 0018:ffff88022b519ed8 EFLAGS: 00000246
  483. Jun 29 05:07:47 borg03b kernel: [231697.077078] RAX: 0000000000000000 RBX: ffff88022b518010 RCX: 00000000c0010055
  484. Jun 29 05:07:47 borg03b kernel: [231697.077078] RDX: 0000000000000000 RSI: 0000000000000004 RDI: 0000000000000001
  485. Jun 29 05:07:47 borg03b kernel: [231697.077078] RBP: ffffffff816715c8 R08: 0000000000000000 R09: ffff88022b518010
  486. Jun 29 05:07:47 borg03b kernel: [231697.077078] R10: 00000000ffffffff R11: 0000000000000246 R12: 0000000000000000
  487. Jun 29 05:07:47 borg03b kernel: [231697.077078] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
  488. Jun 29 05:07:47 borg03b kernel: [231697.077078] FS: 00007f84265bc700(0000) GS:ffff880437c00000(0000) knlGS:00000000e1d65bb0
  489. Jun 29 05:07:47 borg03b kernel: [231697.077078] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  490. Jun 29 05:07:47 borg03b kernel: [231697.077078] CR2: 0000000001402000 CR3: 0000000221ef8000 CR4: 00000000000006e0
  491. Jun 29 05:07:47 borg03b kernel: [231697.077078] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  492. Jun 29 05:07:47 borg03b kernel: [231697.077078] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  493. Jun 29 05:07:47 borg03b kernel: [231697.077078] Process swapper/4 (pid: 0, threadinfo ffff88022b518000, task ffff88022b506180)
  494. Jun 29 05:07:47 borg03b kernel: [231697.077078] Stack:
  495. Jun 29 05:07:47 borg03b kernel: [231697.077078] ffff88022b518010 ffffffff81008e77 0000000000000000 0000000000000000
  496. Jun 29 05:07:47 borg03b kernel: [231697.077078] ffff88022b518010 ffffffff81001dc3 0000000000000000 3877dae599b93bbb
  497. Jun 29 05:07:47 borg03b kernel: [231697.077078] 0000000000000000 0000000000000004 0000000000000000 ffffffff8134063c
  498. Jun 29 05:07:47 borg03b kernel: [231697.077078] Call Trace:
  499. Jun 29 05:07:47 borg03b kernel: [231697.077078] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  500. Jun 29 05:07:47 borg03b kernel: [231697.077078] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  501. Jun 29 05:07:47 borg03b kernel: [231697.077078] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  502. Jun 29 05:07:47 borg03b kernel: [231697.077078] Code: 34 25 f0 ca 00 00 e8 53 f8 ff ff 65 48 8b 04 25 08 b6 00 00 83 a0 3c e0 ff ff fb 0f ae f0 48 8b 80 38 e0 ff ff a8 08 75 04 fb f4 <eb> 01 fb 65 48 8b 04 25 08 b6 00 00 83 88 3c e0 ff ff 04 65 8b
  503. Jun 29 05:07:47 borg03b kernel: [231697.077078] Call Trace:
  504. Jun 29 05:07:47 borg03b kernel: [231697.077078] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  505. Jun 29 05:07:47 borg03b kernel: [231697.077078] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  506. Jun 29 05:07:47 borg03b kernel: [231697.077078] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  507. Jun 29 05:07:47 borg03b kernel: [231697.216975] NMI backtrace for cpu 5
  508. Jun 29 05:07:47 borg03b kernel: [231697.593749] CPU 5
  509. Jun 29 05:07:47 borg03b kernel: [231697.593749] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  510. Jun 29 05:07:47 borg03b kernel: [231697.593749]
  511. Jun 29 05:07:47 borg03b kernel: [231697.593749] Pid: 0, comm: swapper/5 Tainted: G D W O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  512. Jun 29 05:07:47 borg03b kernel: [231697.593749] RIP: 0010:[<ffffffff81008d63>] [<ffffffff81008d63>] default_idle+0x48/0x79
  513. Jun 29 05:07:47 borg03b kernel: [231697.593749] RSP: 0018:ffff88022b521ed8 EFLAGS: 00000246
  514. Jun 29 05:07:47 borg03b kernel: [231697.593749] RAX: 0000000000000000 RBX: ffff88022b520010 RCX: 00000000c0010055
  515. Jun 29 05:07:47 borg03b kernel: [231697.593749] RDX: 0000000000000000 RSI: 0000000000000005 RDI: 0000000000000001
  516. Jun 29 05:07:47 borg03b kernel: [231697.593749] RBP: ffffffff816715c8 R08: 0000000000000000 R09: ffff88022b520010
  517. Jun 29 05:07:47 borg03b kernel: [231697.593749] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
  518. Jun 29 05:07:47 borg03b kernel: [231697.593749] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
  519. Jun 29 05:07:47 borg03b kernel: [231697.593749] FS: 00007f4a45433700(0000) GS:ffff880437c80000(0000) knlGS:000000005582aa80
  520. Jun 29 05:07:47 borg03b kernel: [231697.593749] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  521. Jun 29 05:07:47 borg03b kernel: [231697.593749] CR2: 00007f4a44ed2842 CR3: 0000000001605000 CR4: 00000000000006e0
  522. Jun 29 05:07:47 borg03b kernel: [231697.593749] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  523. Jun 29 05:07:47 borg03b kernel: [231697.593749] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  524. Jun 29 05:07:47 borg03b kernel: [231697.593749] Process swapper/5 (pid: 0, threadinfo ffff88022b520000, task ffff88022b51f630)
  525. Jun 29 05:07:47 borg03b kernel: [231697.593749] Stack:
  526. Jun 29 05:07:47 borg03b kernel: [231697.593749] ffff88022b520010 ffffffff81008e77 0000000000000000 0000000000000000
  527. Jun 29 05:07:47 borg03b kernel: [231697.593749] ffff88022b520010 ffffffff81001dc3 0000000000000000 5b9e23006e04fcc1
  528. Jun 29 05:07:47 borg03b kernel: [231697.593749] 0000000000000000 0000000000000005 0000000000000000 ffffffff8134063c
  529. Jun 29 05:07:47 borg03b kernel: [231697.593749] Call Trace:
  530. Jun 29 05:07:47 borg03b kernel: [231697.593749] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  531. Jun 29 05:07:47 borg03b kernel: [231697.593749] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  532. Jun 29 05:07:47 borg03b kernel: [231697.593749] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  533. Jun 29 05:07:47 borg03b kernel: [231697.593749] Code: 34 25 f0 ca 00 00 e8 53 f8 ff ff 65 48 8b 04 25 08 b6 00 00 83 a0 3c e0 ff ff fb 0f ae f0 48 8b 80 38 e0 ff ff a8 08 75 04 fb f4 <eb> 01 fb 65 48 8b 04 25 08 b6 00 00 83 88 3c e0 ff ff 04 65 8b
  534. Jun 29 05:07:47 borg03b kernel: [231697.593749] Call Trace:
  535. Jun 29 05:07:47 borg03b kernel: [231697.593749] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  536. Jun 29 05:07:47 borg03b kernel: [231697.593749] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  537. Jun 29 05:07:47 borg03b kernel: [231697.593749] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  538. Jun 29 05:07:47 borg03b kernel: [231697.593740] NMI backtrace for cpu 6
  539. Jun 29 05:07:47 borg03b kernel: [231697.593740] CPU 6
  540. Jun 29 05:07:47 borg03b kernel: [231697.593740] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  541. Jun 29 05:07:47 borg03b kernel: [231697.593740]
  542. Jun 29 05:07:47 borg03b kernel: [231697.593740] Pid: 0, comm: swapper/6 Tainted: G D W O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  543. Jun 29 05:07:47 borg03b kernel: [231697.593740] RIP: 0010:[<ffffffff81008d63>] [<ffffffff81008d63>] default_idle+0x48/0x79
  544. Jun 29 05:07:47 borg03b kernel: [231697.593740] RSP: 0018:ffff88022b529ed8 EFLAGS: 00000246
  545. Jun 29 05:07:47 borg03b kernel: [231697.593740] RAX: 0000000000000000 RBX: ffff88022b528010 RCX: 00000000c0010055
  546. Jun 29 05:07:47 borg03b kernel: [231697.593740] RDX: 0000000000000000 RSI: 0000000000000006 RDI: 0000000000000001
  547. Jun 29 05:07:47 borg03b kernel: [231697.593740] RBP: ffffffff816715c8 R08: 0000000000000000 R09: ffff88022b528010
  548. Jun 29 05:07:47 borg03b kernel: [231697.593740] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
  549. Jun 29 05:07:47 borg03b kernel: [231697.593740] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
  550. Jun 29 05:07:47 borg03b kernel: [231697.593740] FS: 00007f5a5a8ea700(0000) GS:ffff880437d00000(0000) knlGS:00000000558c06c0
  551. Jun 29 05:07:47 borg03b kernel: [231697.593740] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  552. Jun 29 05:07:47 borg03b kernel: [231697.593740] CR2: 00007f5a5a6d57f0 CR3: 000000021e6b5000 CR4: 00000000000006e0
  553. Jun 29 05:07:47 borg03b kernel: [231697.593740] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  554. Jun 29 05:07:47 borg03b kernel: [231697.593740] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  555. Jun 29 05:07:47 borg03b kernel: [231697.593740] Process swapper/6 (pid: 0, threadinfo ffff88022b528000, task ffff88022b51ef60)
  556. Jun 29 05:07:47 borg03b kernel: [231697.593740] Stack:
  557. Jun 29 05:07:47 borg03b kernel: [231697.593740] ffff88022b528010 ffffffff81008e77 0000000000000000 0000000000000000
  558. Jun 29 05:07:47 borg03b kernel: [231697.593740] ffff88022b528010 ffffffff81001dc3 0000000000000000 133676006c70cf56
  559. Jun 29 05:07:47 borg03b kernel: [231697.593740] 0000000000000000 0000000000000006 0000000000000000 ffffffff8134063c
  560. Jun 29 05:07:47 borg03b kernel: [231697.593740] Call Trace:
  561. Jun 29 05:07:47 borg03b kernel: [231697.593740] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  562. Jun 29 05:07:47 borg03b kernel: [231697.593740] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  563. Jun 29 05:07:47 borg03b kernel: [231697.593740] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  564. Jun 29 05:07:47 borg03b kernel: [231697.593740] Code: 34 25 f0 ca 00 00 e8 53 f8 ff ff 65 48 8b 04 25 08 b6 00 00 83 a0 3c e0 ff ff fb 0f ae f0 48 8b 80 38 e0 ff ff a8 08 75 04 fb f4 <eb> 01 fb 65 48 8b 04 25 08 b6 00 00 83 88 3c e0 ff ff 04 65 8b
  565. Jun 29 05:07:47 borg03b kernel: [231697.593740] Call Trace:
  566. Jun 29 05:07:47 borg03b kernel: [231697.593740] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  567. Jun 29 05:07:47 borg03b kernel: [231697.593740] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  568. Jun 29 05:07:47 borg03b kernel: [231697.593740] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  569. Jun 29 05:08:08 borg03b kernel: [231717.708014] INFO: rcu_bh detected stalls on CPUs/tasks: { 1 7} (detected by 3, t=15002 jiffies)
  570. Jun 29 05:08:08 borg03b kernel: [231717.712000] sending NMI to all CPUs:
  571. Jun 29 05:08:08 borg03b kernel: [231644.579585] NMI backtrace for cpu 1
  572. Jun 29 05:08:08 borg03b kernel: [231644.579585] CPU 1
  573. Jun 29 05:08:08 borg03b kernel: [231644.579585] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  574. Jun 29 05:08:08 borg03b kernel: [231644.579585]
  575. Jun 29 05:08:08 borg03b kernel: [231644.579585] Pid: 0, comm: swapper/1 Tainted: G D W O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  576. Jun 29 05:08:08 borg03b kernel: [231644.579585] RIP: 0010:[<ffffffff813459f3>] [<ffffffff813459f3>] _raw_spin_lock_irqsave+0x1b/0x1e
  577. Jun 29 05:08:08 borg03b kernel: [231644.579585] RSP: 0000:ffff880237c83e18 EFLAGS: 00000097
  578. Jun 29 05:08:08 borg03b kernel: [231644.579585] RAX: 0000000000000282 RBX: ffff88022a47e788 RCX: 0000000000000050
  579. Jun 29 05:08:08 borg03b kernel: [231644.579585] RDX: 0000000000000051 RSI: 0000000000000001 RDI: ffff88022a47eb6c
  580. Jun 29 05:08:08 borg03b kernel: [231644.579585] RBP: ffff88022a47ec08 R08: 0000d2acde6ea100 R09: dead000000200200
  581. Jun 29 05:08:08 borg03b kernel: [231644.579585] R10: ffff880237c8b870 R11: ffffffff81404595 R12: 0000000000000000
  582. Jun 29 05:08:08 borg03b kernel: [231644.579585] R13: 0000000000000000 R14: ffff88022b4c5fd8 R15: ffff88022a47ec58
  583. Jun 29 05:08:08 borg03b kernel: [231644.579585] FS: 00007ff3fb7f37c0(0000) GS:ffff880237c80000(0000) knlGS:00000000e4fffbb0
  584. Jun 29 05:08:08 borg03b kernel: [231644.579585] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  585. Jun 29 05:08:08 borg03b kernel: [231644.579585] CR2: 00007f8dbc87f338 CR3: 000000021d8e7000 CR4: 00000000000006e0
  586. Jun 29 05:08:08 borg03b kernel: [231644.579585] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  587. Jun 29 05:08:08 borg03b kernel: [231644.579585] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  588. Jun 29 05:08:08 borg03b kernel: [231644.579585] Process swapper/1 (pid: 0, threadinfo ffff88022b4c4000, task ffff88022b4b60c0)
  589. Jun 29 05:08:08 borg03b kernel: [231644.579585] Stack:
  590. Jun 29 05:08:08 borg03b kernel: [231644.579585] ffffffff811b6082 ffffffff810aa3f6 ffff88022b4bc000 ffff88022a47ec08
  591. Jun 29 05:08:08 borg03b kernel: [231644.579585] 0000000000000100 ffff880237c83ea0 ffff88022b4c5fd8 ffff88022b4c5fd8
  592. Jun 29 05:08:08 borg03b kernel: [231644.579585] ffffffff81046874 ffff880237c83e58 ffffffff811b6058 ffff88022b4bd020
  593. Jun 29 05:08:08 borg03b kernel: [231644.579585] Call Trace:
  594. Jun 29 05:08:08 borg03b kernel: [231644.579585] <IRQ>
  595. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff811b6082>] ? blk_rq_timed_out_timer+0x2a/0x100
  596. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff810aa3f6>] ? perf_event_task_tick+0x1f0/0x233
  597. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff81046874>] ? run_timer_softirq+0x1b8/0x27b
  598. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff811b6058>] ? blk_abort_queue+0xf6/0xf6
  599. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff8103fb92>] ? __do_softirq+0xbe/0x194
  600. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff81017d47>] ? lapic_next_event+0x18/0x1d
  601. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff8134bfac>] ? call_softirq+0x1c/0x30
  602. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff8100393d>] ? do_softirq+0x31/0x63
  603. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff8103f96d>] ? irq_exit+0x44/0xb5
  604. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff810188bd>] ? smp_apic_timer_interrupt+0x85/0x93
  605. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff8134b54b>] ? apic_timer_interrupt+0x6b/0x70
  606. Jun 29 05:08:08 borg03b kernel: [231644.579585] <EOI>
  607. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff81008d63>] ? default_idle+0x48/0x79
  608. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff81008d43>] ? default_idle+0x28/0x79
  609. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  610. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  611. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff81345843>] ? _raw_spin_unlock_irqrestore+0x4/0x5
  612. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  613. Jun 29 05:08:08 borg03b kernel: [231644.579585] Code: c2 66 c1 e8 08 38 c2 74 06 f3 90 8a 17 eb f6 c3 9c 58 fa ba 00 01 00 00 f0 66 0f c1 17 88 d1 66 c1 ea 08 38 d1 74 06 f3 90 8a 0f <eb> f6 c3 48 83 ec 08 48 c7 c7 90 34 64 81 58 e9 3a f2 ff ff 48
  614. Jun 29 05:08:08 borg03b kernel: [231644.579585] Call Trace:
  615. Jun 29 05:08:08 borg03b kernel: [231644.579585] <IRQ> [<ffffffff811b6082>] ? blk_rq_timed_out_timer+0x2a/0x100
  616. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff810aa3f6>] ? perf_event_task_tick+0x1f0/0x233
  617. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff81046874>] ? run_timer_softirq+0x1b8/0x27b
  618. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff811b6058>] ? blk_abort_queue+0xf6/0xf6
  619. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff8103fb92>] ? __do_softirq+0xbe/0x194
  620. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff81017d47>] ? lapic_next_event+0x18/0x1d
  621. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff8134bfac>] ? call_softirq+0x1c/0x30
  622. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff8100393d>] ? do_softirq+0x31/0x63
  623. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff8103f96d>] ? irq_exit+0x44/0xb5
  624. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff810188bd>] ? smp_apic_timer_interrupt+0x85/0x93
  625. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff8134b54b>] ? apic_timer_interrupt+0x6b/0x70
  626. Jun 29 05:08:08 borg03b kernel: [231644.579585] <EOI> [<ffffffff81008d63>] ? default_idle+0x48/0x79
  627. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff81008d43>] ? default_idle+0x28/0x79
  628. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  629. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  630. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff81345843>] ? _raw_spin_unlock_irqrestore+0x4/0x5
  631. Jun 29 05:08:08 borg03b kernel: [231644.579585] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  632. Jun 29 05:08:08 borg03b kernel: [231670.098823] NMI backtrace for cpu 7
  633. Jun 29 05:08:08 borg03b kernel: [231670.098823] CPU 7
  634. Jun 29 05:08:08 borg03b kernel: [231670.098823] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  635. Jun 29 05:08:08 borg03b kernel: [231670.098823]
  636. Jun 29 05:08:08 borg03b kernel: [231670.098823] Pid: 14506, comm: kworker/7:0 Tainted: G D W O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  637. Jun 29 05:08:08 borg03b kernel: [231670.098823] RIP: 0010:[<ffffffff8134583a>] [<ffffffff8134583a>] _raw_spin_lock_irq+0x17/0x1c
  638. Jun 29 05:08:08 borg03b kernel: [231670.098823] RSP: 0018:ffff880388a05740 EFLAGS: 00000097
  639. Jun 29 05:08:08 borg03b kernel: [231670.098823] RAX: 0000000000000044 RBX: ffff880437d91cc0 RCX: 000000000000006c
  640. Jun 29 05:08:08 borg03b kernel: [231670.098823] RDX: 000000000073b543 RSI: ffff880413cecea0 RDI: ffff880437d91cc0
  641. Jun 29 05:08:08 borg03b kernel: [231670.098823] RBP: ffff880413cecea0 R08: 0000000000000000 R09: 0000000000000000
  642. Jun 29 05:08:08 borg03b kernel: [231670.098823] R10: ffff88022a839290 R11: ffffffff81433e60 R12: 0000000000000009
  643. Jun 29 05:08:08 borg03b kernel: [231670.098823] R13: 0000000000000007 R14: ffff880413ced158 R15: 0000000000000000
  644. Jun 29 05:08:08 borg03b kernel: [231670.098823] FS: 00007f2948786700(0000) GS:ffff880437d80000(0000) knlGS:00000000ad27fbb0
  645. Jun 29 05:08:08 borg03b kernel: [231670.098823] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  646. Jun 29 05:08:08 borg03b kernel: [231670.098823] CR2: fffffffffffffff8 CR3: 000000042aacb000 CR4: 00000000000006e0
  647. Jun 29 05:08:08 borg03b kernel: [231670.098823] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  648. Jun 29 05:08:08 borg03b kernel: [231670.098823] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  649. Jun 29 05:08:08 borg03b kernel: [231670.098823] Process kworker/7:0 (pid: 14506, threadinfo ffff880388a04000, task ffff880413cecea0)
  650. Jun 29 05:08:08 borg03b kernel: [231670.098823] Stack:
  651. Jun 29 05:08:08 borg03b kernel: [231670.098823] ffffffff81344188 ffffffff00000010 ffff880413cecea0 0000000000000009
  652. Jun 29 05:08:08 borg03b kernel: [231670.098823] 0000000000000009 0000000000011cc0 ffff880388a05fd8 ffff880388a05fd8
  653. Jun 29 05:08:08 borg03b kernel: [231670.098823] 0000000000011cc0 ffff880413cecea0 ffff880388a04010 ffff880388a057a8
  654. Jun 29 05:08:08 borg03b kernel: [231670.098823] Call Trace:
  655. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81344188>] ? __schedule+0xc7/0x57e
  656. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8103d78c>] ? do_exit+0xff/0x76d
  657. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81343cf5>] ? printk+0x40/0x4b
  658. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8103aa6c>] ? kmsg_dump+0x41/0xdd
  659. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81346599>] ? oops_end+0xa8/0xad
  660. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81020927>] ? no_context+0x1ff/0x20c
  661. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8134850a>] ? do_page_fault+0x190/0x328
  662. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff811c9ac4>] ? number+0x13d/0x243
  663. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81054000>] ? kthread_stop+0x3b/0xa7
  664. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff811cb3cf>] ? vsnprintf+0x3ef/0x428
  665. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81345d2f>] ? page_fault+0x1f/0x30
  666. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff810539c7>] ? kthread_data+0x7/0xc
  667. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8104f65f>] ? wq_worker_sleeping+0x8/0x7f
  668. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff813441fd>] ? __schedule+0x13c/0x57e
  669. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8103dddc>] ? do_exit+0x74f/0x76d
  670. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81346599>] ? oops_end+0xa8/0xad
  671. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81002d19>] ? do_invalid_op+0x85/0x8f
  672. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffffa0014da1>] ? scsi_setup_fs_cmnd+0x4c/0xbf [scsi_mod]
  673. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8134bd35>] ? invalid_op+0x15/0x20
  674. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff811bfbd5>] ? cfq_init_queue+0x3bd/0x3bd
  675. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffffa0014da1>] ? scsi_setup_fs_cmnd+0x4c/0xbf [scsi_mod]
  676. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffffa0061835>] ? sd_prep_fn+0x29c/0xb23 [sd_mod]
  677. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff811b2817>] ? blk_peek_request+0xd9/0x1b9
  678. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffffa00144a2>] ? scsi_request_fn+0x75/0x479 [scsi_mod]
  679. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff811b0c96>] ? freed_request+0x5a/0x5a
  680. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff811b0cb2>] ? blk_delay_work+0x1c/0x28
  681. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8105028b>] ? process_one_work+0x1c7/0x2e5
  682. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff810504cc>] ? worker_thread+0x123/0x234
  683. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff810503a9>] ? process_one_work+0x2e5/0x2e5
  684. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff810503a9>] ? process_one_work+0x2e5/0x2e5
  685. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81053d3e>] ? kthread+0x7a/0x82
  686. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8134beb4>] ? kernel_thread_helper+0x4/0x10
  687. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81053cc4>] ? kthread_worker_fn+0x139/0x139
  688. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8134beb0>] ? gs_change+0xb/0xb
  689. Jun 29 05:08:08 borg03b kernel: [231670.098823] Code: 01 00 00 89 d0 f0 66 0f b1 0f 66 39 d0 0f 94 c0 0f b6 c0 c3 fa b8 00 01 00 00 f0 66 0f c1 07 88 c2 66 c1 e8 08 38 c2 74 06 f3 90 <8a> 17 eb f6 c3 fe 07 56 9d c3 83 ca ff f0 0f c1 17 ff ca b8 01
  690. Jun 29 05:08:08 borg03b kernel: [231670.098823] Call Trace:
  691. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81344188>] ? __schedule+0xc7/0x57e
  692. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8103d78c>] ? do_exit+0xff/0x76d
  693. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81343cf5>] ? printk+0x40/0x4b
  694. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8103aa6c>] ? kmsg_dump+0x41/0xdd
  695. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81346599>] ? oops_end+0xa8/0xad
  696. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81020927>] ? no_context+0x1ff/0x20c
  697. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8134850a>] ? do_page_fault+0x190/0x328
  698. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff811c9ac4>] ? number+0x13d/0x243
  699. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81054000>] ? kthread_stop+0x3b/0xa7
  700. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff811cb3cf>] ? vsnprintf+0x3ef/0x428
  701. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81345d2f>] ? page_fault+0x1f/0x30
  702. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff810539c7>] ? kthread_data+0x7/0xc
  703. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8104f65f>] ? wq_worker_sleeping+0x8/0x7f
  704. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff813441fd>] ? __schedule+0x13c/0x57e
  705. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8103dddc>] ? do_exit+0x74f/0x76d
  706. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81346599>] ? oops_end+0xa8/0xad
  707. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81002d19>] ? do_invalid_op+0x85/0x8f
  708. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffffa0014da1>] ? scsi_setup_fs_cmnd+0x4c/0xbf [scsi_mod]
  709. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8134bd35>] ? invalid_op+0x15/0x20
  710. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff811bfbd5>] ? cfq_init_queue+0x3bd/0x3bd
  711. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffffa0014da1>] ? scsi_setup_fs_cmnd+0x4c/0xbf [scsi_mod]
  712. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffffa0061835>] ? sd_prep_fn+0x29c/0xb23 [sd_mod]
  713. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff811b2817>] ? blk_peek_request+0xd9/0x1b9
  714. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffffa00144a2>] ? scsi_request_fn+0x75/0x479 [scsi_mod]
  715. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff811b0c96>] ? freed_request+0x5a/0x5a
  716. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff811b0cb2>] ? blk_delay_work+0x1c/0x28
  717. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8105028b>] ? process_one_work+0x1c7/0x2e5
  718. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff810504cc>] ? worker_thread+0x123/0x234
  719. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff810503a9>] ? process_one_work+0x2e5/0x2e5
  720. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff810503a9>] ? process_one_work+0x2e5/0x2e5
  721. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81053d3e>] ? kthread+0x7a/0x82
  722. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8134beb4>] ? kernel_thread_helper+0x4/0x10
  723. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff81053cc4>] ? kthread_worker_fn+0x139/0x139
  724. Jun 29 05:08:08 borg03b kernel: [231670.098823] [<ffffffff8134beb0>] ? gs_change+0xb/0xb
  725. Jun 29 05:08:08 borg03b kernel: [231701.987537] NMI backtrace for cpu 5
  726. Jun 29 05:08:08 borg03b kernel: [231701.987537] CPU 5
  727. Jun 29 05:08:08 borg03b kernel: [231701.987537] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  728. Jun 29 05:08:08 borg03b kernel: [231701.987537]
  729. Jun 29 05:08:08 borg03b kernel: [231701.987537] Pid: 8409, comm: smtp Tainted: G D W O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  730. Jun 29 05:08:08 borg03b kernel: [231701.987537] RIP: 0010:[<ffffffff813459f3>] [<ffffffff813459f3>] _raw_spin_lock_irqsave+0x1b/0x1e
  731. Jun 29 05:08:08 borg03b kernel: [231701.987537] RSP: 0018:ffff88022a265e50 EFLAGS: 00000093
  732. Jun 29 05:08:08 borg03b kernel: [231701.987537] RAX: 0000000000000286 RBX: ffff880229f1f000 RCX: 0000000000000050
  733. Jun 29 05:08:08 borg03b kernel: [231701.987537] RDX: 0000000000000052 RSI: ffff8804145df528 RDI: ffff88022a47eb6c
  734. Jun 29 05:08:08 borg03b kernel: [231701.987537] RBP: ffff88022a47e788 R08: ffff8801536a0f20 R09: 0000000000000000
  735. Jun 29 05:08:08 borg03b kernel: [231701.987537] R10: 0000000000000000 R11: 00000000000003ff R12: ffff88022a265ee8
  736. Jun 29 05:08:08 borg03b kernel: [231701.987537] R13: ffff88022b4794b0 R14: 0000000000000001 R15: ffff8801536a10e0
  737. Jun 29 05:08:08 borg03b kernel: [231701.987537] FS: 00002b57f1f7d000(0000) GS:ffff880437c80000(0000) knlGS:000000005582aa80
  738. Jun 29 05:08:08 borg03b kernel: [231701.987537] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  739. Jun 29 05:08:08 borg03b kernel: [231701.987537] CR2: 00002b57f0ea0098 CR3: 0000000001605000 CR4: 00000000000006e0
  740. Jun 29 05:08:08 borg03b kernel: [231701.987537] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  741. Jun 29 05:08:08 borg03b kernel: [231701.987537] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  742. Jun 29 05:08:08 borg03b kernel: [231701.987537] Process smtp (pid: 8409, threadinfo ffff88022a264000, task ffff8801536a0f20)
  743. Jun 29 05:08:08 borg03b kernel: [231701.987537] Stack:
  744. Jun 29 05:08:08 borg03b kernel: [231701.987537] ffffffff811bf65f 0000000000000001 ffff8804145df528 ffff8804145df578
  745. Jun 29 05:08:08 borg03b kernel: [231701.987537] ffff880418648820 ffff88022a265ee8 ffffffff811bf6af ffff8801536a0f20
  746. Jun 29 05:08:08 borg03b kernel: [231701.987537] ffff880418648820 ffff8801536a0f20 ffffffff811b4c0b ffff88022a265ee8
  747. Jun 29 05:08:08 borg03b kernel: [231701.987537] Call Trace:
  748. Jun 29 05:08:08 borg03b kernel: [231701.987537] [<ffffffff811bf65f>] ? cfq_exit_single_io_context+0x29/0x5b
  749. Jun 29 05:08:08 borg03b kernel: [231701.987537] [<ffffffff811bf6af>] ? cfq_exit_io_context+0x1e/0x28
  750. Jun 29 05:08:08 borg03b kernel: [231701.987537] [<ffffffff811b4c0b>] ? exit_io_context+0x47/0x54
  751. Jun 29 05:08:08 borg03b kernel: [231701.987537] [<ffffffff8103dd97>] ? do_exit+0x70a/0x76d
  752. Jun 29 05:08:08 borg03b kernel: [231701.987537] [<ffffffff8103de6d>] ? do_group_exit+0x73/0x9d
  753. Jun 29 05:08:08 borg03b kernel: [231701.987537] [<ffffffff8103dea9>] ? sys_exit_group+0x12/0x17
  754. Jun 29 05:08:08 borg03b kernel: [231701.987537] [<ffffffff8134aabb>] ? system_call_fastpath+0x16/0x1b
  755. Jun 29 05:08:08 borg03b kernel: [231701.987537] Code: c2 66 c1 e8 08 38 c2 74 06 f3 90 8a 17 eb f6 c3 9c 58 fa ba 00 01 00 00 f0 66 0f c1 17 88 d1 66 c1 ea 08 38 d1 74 06 f3 90 8a 0f <eb> f6 c3 48 83 ec 08 48 c7 c7 90 34 64 81 58 e9 3a f2 ff ff 48
  756. Jun 29 05:08:08 borg03b kernel: [231701.987537] Call Trace:
  757. Jun 29 05:08:08 borg03b kernel: [231701.987537] [<ffffffff811bf65f>] ? cfq_exit_single_io_context+0x29/0x5b
  758. Jun 29 05:08:08 borg03b kernel: [231701.987537] [<ffffffff811bf6af>] ? cfq_exit_io_context+0x1e/0x28
  759. Jun 29 05:08:08 borg03b kernel: [231701.987537] [<ffffffff811b4c0b>] ? exit_io_context+0x47/0x54
  760. Jun 29 05:08:08 borg03b kernel: [231701.987537] [<ffffffff8103dd97>] ? do_exit+0x70a/0x76d
  761. Jun 29 05:08:08 borg03b kernel: [231701.987537] [<ffffffff8103de6d>] ? do_group_exit+0x73/0x9d
  762. Jun 29 05:08:08 borg03b kernel: [231701.987537] [<ffffffff8103dea9>] ? sys_exit_group+0x12/0x17
  763. Jun 29 05:08:08 borg03b kernel: [231701.987537] [<ffffffff8134aabb>] ? system_call_fastpath+0x16/0x1b
  764. Jun 29 05:08:08 borg03b kernel: [231717.720008] NMI backtrace for cpu 0
  765. Jun 29 05:08:08 borg03b kernel: [231717.720008] CPU 0
  766. Jun 29 05:08:08 borg03b kernel: [231717.720008] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  767. Jun 29 05:08:08 borg03b kernel: [231717.720008]
  768. Jun 29 05:08:08 borg03b kernel: [231717.720008] Pid: 0, comm: swapper/0 Tainted: G D W O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  769. Jun 29 05:08:08 borg03b kernel: [231717.720008] RIP: 0010:[<ffffffff81008d63>] [<ffffffff81008d63>] default_idle+0x48/0x79
  770. Jun 29 05:08:08 borg03b kernel: [231717.720008] RSP: 0018:ffffffff81601f38 EFLAGS: 00000246
  771. Jun 29 05:08:08 borg03b kernel: [231717.720008] RAX: 0000000000000000 RBX: ffffffff81600010 RCX: 00000000c0010055
  772. Jun 29 05:08:08 borg03b kernel: [231717.720008] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000001
  773. Jun 29 05:08:08 borg03b kernel: [231717.720008] RBP: ffffffff816715c8 R08: 0000000000000000 R09: ffffffff81600010
  774. Jun 29 05:08:08 borg03b kernel: [231717.720008] R10: 00000000ffffffff R11: 0000000000000246 R12: 0000000000000000
  775. Jun 29 05:08:08 borg03b kernel: [231717.720008] R13: ffffffffffffffff R14: 0000000000000000 R15: 0000000000000000
  776. Jun 29 05:08:08 borg03b kernel: [231717.720008] FS: 00002ade72dbc500(0000) GS:ffff880237c00000(0000) knlGS:00000000558c06c0
  777. Jun 29 05:08:08 borg03b kernel: [231717.720008] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  778. Jun 29 05:08:08 borg03b kernel: [231717.720008] CR2: 00000000028d6b28 CR3: 0000000416388000 CR4: 00000000000006f0
  779. Jun 29 05:08:08 borg03b kernel: [231717.720008] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  780. Jun 29 05:08:08 borg03b kernel: [231717.720008] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  781. Jun 29 05:08:08 borg03b kernel: [231717.720008] Process swapper/0 (pid: 0, threadinfo ffffffff81600000, task ffffffff8160d020)
  782. Jun 29 05:08:08 borg03b kernel: [231717.720008] Stack:
  783. Jun 29 05:08:08 borg03b kernel: [231717.720008] ffffffff81600010 ffffffff81008e77 0000000000000000 ffffffffffffffff
  784. Jun 29 05:08:08 borg03b kernel: [231717.720008] ffffffff81600010 ffffffff81001dc3 ffffffffffffffff 10dc3b883dbbe2b8
  785. Jun 29 05:08:08 borg03b kernel: [231717.720008] 0000000000000000 0000000000000000 6db6db6db6db6db7 ffffffff8168bbcc
  786. Jun 29 05:08:08 borg03b kernel: [231717.720008] Call Trace:
  787. Jun 29 05:08:08 borg03b kernel: [231717.720008] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  788. Jun 29 05:08:08 borg03b kernel: [231717.720008] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  789. Jun 29 05:08:08 borg03b kernel: [231717.720008] [<ffffffff8168bbcc>] ? start_kernel+0x3a1/0x3ac
  790. Jun 29 05:08:08 borg03b kernel: [231717.720008] [<ffffffff8168b38f>] ? x86_64_start_kernel+0xee/0xf2
  791. Jun 29 05:08:08 borg03b kernel: [231717.720008] Code: 34 25 f0 ca 00 00 e8 53 f8 ff ff 65 48 8b 04 25 08 b6 00 00 83 a0 3c e0 ff ff fb 0f ae f0 48 8b 80 38 e0 ff ff a8 08 75 04 fb f4 <eb> 01 fb 65 48 8b 04 25 08 b6 00 00 83 88 3c e0 ff ff 04 65 8b
  792. Jun 29 05:08:08 borg03b kernel: [231717.720008] Call Trace:
  793. Jun 29 05:08:08 borg03b kernel: [231717.720008] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  794. Jun 29 05:08:08 borg03b kernel: [231717.720008] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  795. Jun 29 05:08:08 borg03b kernel: [231717.720008] [<ffffffff8168bbcc>] ? start_kernel+0x3a1/0x3ac
  796. Jun 29 05:08:08 borg03b kernel: [231717.720008] [<ffffffff8168b38f>] ? x86_64_start_kernel+0xee/0xf2
  797. Jun 29 05:08:08 borg03b kernel: [231717.720015] NMI backtrace for cpu 2
  798. Jun 29 05:08:08 borg03b kernel: [231718.579789] CPU 2
  799. Jun 29 05:08:08 borg03b kernel: [231718.579789] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  800. Jun 29 05:08:08 borg03b kernel: [231718.584939]
  801. Jun 29 05:08:08 borg03b kernel: [231718.584939] Pid: 0, comm: swapper/2 Tainted: G D W O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  802. Jun 29 05:08:08 borg03b kernel: [231718.584939] RIP: 0010:[<ffffffff81008d63>] [<ffffffff81008d63>] default_idle+0x48/0x79
  803. Jun 29 05:08:08 borg03b kernel: [231718.584939] RSP: 0018:ffff88022b4fbed8 EFLAGS: 00000246
  804. Jun 29 05:08:08 borg03b kernel: [231718.584939] RAX: 0000000000000000 RBX: ffff88022b4fa010 RCX: 00000000c0010055
  805. Jun 29 05:08:08 borg03b kernel: [231718.584939] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000001
  806. Jun 29 05:08:08 borg03b kernel: [231718.584939] RBP: ffffffff816715c8 R08: 0000000000000000 R09: ffff88022b4fa010
  807. Jun 29 05:08:08 borg03b kernel: [231718.584939] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
  808. Jun 29 05:08:08 borg03b kernel: [231718.584939] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
  809. Jun 29 05:08:08 borg03b kernel: [231718.584939] FS: 00007fd89455d740(0000) GS:ffff880237d00000(0000) knlGS:00000000558c06c0
  810. Jun 29 05:08:08 borg03b kernel: [231718.584939] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  811. Jun 29 05:08:08 borg03b kernel: [231718.584939] CR2: 00007f8adbb0fed0 CR3: 00000004171bd000 CR4: 00000000000006e0
  812. Jun 29 05:08:08 borg03b kernel: [231718.584939] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  813. Jun 29 05:08:08 borg03b kernel: [231718.584939] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  814. Jun 29 05:08:08 borg03b kernel: [231718.584939] Process swapper/2 (pid: 0, threadinfo ffff88022b4fa000, task ffff88022b4f95b0)
  815. Jun 29 05:08:08 borg03b kernel: [231718.584939] Stack:
  816. Jun 29 05:08:08 borg03b kernel: [231718.584939] ffff88022b4fa010 ffffffff81008e77 0000000000000000 0000000000000000
  817. Jun 29 05:08:08 borg03b kernel: [231718.584939] ffff88022b4fa010 ffffffff81001dc3 0000000000000000 2f6d759c3bc5c62c
  818. Jun 29 05:08:08 borg03b kernel: [231718.584939] 0000000000000000 0000000000000002 0000000000000000 ffffffff8134063c
  819. Jun 29 05:08:08 borg03b kernel: [231718.584939] Call Trace:
  820. Jun 29 05:08:08 borg03b kernel: [231718.584939] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  821. Jun 29 05:08:08 borg03b kernel: [231718.584939] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  822. Jun 29 05:08:08 borg03b kernel: [231718.584939] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  823. Jun 29 05:08:08 borg03b kernel: [231718.584939] Code: 34 25 f0 ca 00 00 e8 53 f8 ff ff 65 48 8b 04 25 08 b6 00 00 83 a0 3c e0 ff ff fb 0f ae f0 48 8b 80 38 e0 ff ff a8 08 75 04 fb f4 <eb> 01 fb 65 48 8b 04 25 08 b6 00 00 83 88 3c e0 ff ff 04 65 8b
  824. Jun 29 05:08:08 borg03b kernel: [231718.584939] Call Trace:
  825. Jun 29 05:08:08 borg03b kernel: [231718.584939] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  826. Jun 29 05:08:08 borg03b kernel: [231718.584939] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  827. Jun 29 05:08:08 borg03b kernel: [231718.584939] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  828. Jun 29 05:08:08 borg03b kernel: [231718.579791] NMI backtrace for cpu 3
  829. Jun 29 05:08:09 borg03b kernel: [231718.579791] CPU 3
  830. Jun 29 05:08:09 borg03b kernel: [231718.579791] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  831. Jun 29 05:08:09 borg03b kernel: [231718.579791]
  832. Jun 29 05:08:09 borg03b kernel: [231718.579791] Pid: 0, comm: swapper/3 Tainted: G D W O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  833. Jun 29 05:08:09 borg03b kernel: [231718.579791] RIP: 0010:[<ffffffff811cc107>] [<ffffffff811cc107>] __const_udelay+0x4/0x29
  834. Jun 29 05:08:09 borg03b kernel: [231718.579791] RSP: 0018:ffff880237d83e18 EFLAGS: 00000096
  835. Jun 29 05:08:09 borg03b kernel: [231718.579791] RAX: 0000000000000c00 RBX: 0000000000000001 RCX: 0000000000000000
  836. Jun 29 05:08:09 borg03b kernel: [231718.579791] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000418958
  837. Jun 29 05:08:09 borg03b kernel: [231718.579791] RBP: ffffffff8161e780 R08: 0000000000000000 R09: ffff880237d83de8
  838. Jun 29 05:08:09 borg03b kernel: [231718.579791] R10: 0000000000000000 R11: ffffffff81433e60 R12: ffffffff8161e780
  839. Jun 29 05:08:09 borg03b kernel: [231718.579791] R13: 0000000000000002 R14: ffffffff8161e880 R15: 0000000000000008
  840. Jun 29 05:08:09 borg03b kernel: [231718.579791] FS: 00007fabdb6eb6f0(0000) GS:ffff880237d80000(0000) knlGS:00000000558c06c0
  841. Jun 29 05:08:09 borg03b kernel: [231718.579791] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  842. Jun 29 05:08:09 borg03b kernel: [231718.579791] CR2: 00007f13c8480338 CR3: 000000041436b000 CR4: 00000000000006e0
  843. Jun 29 05:08:09 borg03b kernel: [231718.579791] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  844. Jun 29 05:08:09 borg03b kernel: [231718.579791] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  845. Jun 29 05:08:09 borg03b kernel: [231718.579791] Process swapper/3 (pid: 0, threadinfo ffff88022b50e000, task ffff88022b506f20)
  846. Jun 29 05:08:09 borg03b kernel: [231718.579791] Stack:
  847. Jun 29 05:08:09 borg03b kernel: [231718.579791] 0000000000000002 ffffffff81018c35 ffff880237d8d660 ffffffff8108df92
  848. Jun 29 05:08:09 borg03b kernel: [231718.579791] ffff880415cd71a0 ffffffff8161e880 ffffffff64d8f800 0000000000000003
  849. Jun 29 05:08:09 borg03b kernel: [231718.579791] 0000000000000000 0000000000000003 ffff880237d83f48 0000000000000001
  850. Jun 29 05:08:09 borg03b kernel: [231718.579791] Call Trace:
  851. Jun 29 05:08:09 borg03b kernel: [231718.579791] <IRQ>
  852. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff81018c35>] ? arch_trigger_all_cpu_backtrace+0x58/0x6b
  853. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff8108df92>] ? __rcu_pending+0x21a/0x336
  854. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff81068684>] ? tick_nohz_handler+0xc6/0xc6
  855. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff8108e1eb>] ? rcu_check_callbacks+0xe7/0x108
  856. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff81068684>] ? tick_nohz_handler+0xc6/0xc6
  857. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff81045fc2>] ? update_process_times+0x31/0x63
  858. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff810686ee>] ? tick_sched_timer+0x6a/0x86
  859. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff810572c5>] ? __run_hrtimer+0xab/0x136
  860. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff81057588>] ? hrtimer_interrupt+0xd8/0x1a5
  861. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff810589c5>] ? sched_clock_idle_wakeup_event+0x12/0x1a
  862. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff810188b8>] ? smp_apic_timer_interrupt+0x80/0x93
  863. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff8134b54b>] ? apic_timer_interrupt+0x6b/0x70
  864. Jun 29 05:08:09 borg03b kernel: [231718.579791] <EOI>
  865. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff81008d63>] ? default_idle+0x48/0x79
  866. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff81008d43>] ? default_idle+0x28/0x79
  867. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  868. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  869. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  870. Jun 29 05:08:09 borg03b kernel: [231718.579791] Code: 00 00 00 00 48 ff c8 75 fb 48 ff c8 c3 48 c7 05 cc 16 47 00 4c c1 1c 81 c3 48 83 ec 08 48 8b 05 c0 16 47 00 5a ff e0 48 83 ec 08 <48> 8d 04 bd 00 00 00 00 65 48 8b 14 25 58 1c 01 00 48 6b d2 3e
  871. Jun 29 05:08:09 borg03b kernel: [231718.579791] Call Trace:
  872. Jun 29 05:08:09 borg03b kernel: [231718.579791] <IRQ> [<ffffffff81018c35>] ? arch_trigger_all_cpu_backtrace+0x58/0x6b
  873. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff8108df92>] ? __rcu_pending+0x21a/0x336
  874. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff81068684>] ? tick_nohz_handler+0xc6/0xc6
  875. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff8108e1eb>] ? rcu_check_callbacks+0xe7/0x108
  876. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff81068684>] ? tick_nohz_handler+0xc6/0xc6
  877. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff81045fc2>] ? update_process_times+0x31/0x63
  878. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff810686ee>] ? tick_sched_timer+0x6a/0x86
  879. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff810572c5>] ? __run_hrtimer+0xab/0x136
  880. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff81057588>] ? hrtimer_interrupt+0xd8/0x1a5
  881. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff810589c5>] ? sched_clock_idle_wakeup_event+0x12/0x1a
  882. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff810188b8>] ? smp_apic_timer_interrupt+0x80/0x93
  883. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff8134b54b>] ? apic_timer_interrupt+0x6b/0x70
  884. Jun 29 05:08:09 borg03b kernel: [231718.579791] <EOI> [<ffffffff81008d63>] ? default_idle+0x48/0x79
  885. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff81008d43>] ? default_idle+0x28/0x79
  886. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  887. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  888. Jun 29 05:08:09 borg03b kernel: [231718.579791] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  889. Jun 29 05:08:09 borg03b kernel: [231718.656862] NMI backtrace for cpu 4
  890. Jun 29 05:08:09 borg03b kernel: [231718.656862] CPU 4
  891. Jun 29 05:08:09 borg03b kernel: [231718.656862] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  892. Jun 29 05:08:09 borg03b kernel: [231718.993546]
  893. Jun 29 05:08:09 borg03b kernel: [231718.993546] Pid: 0, comm: swapper/4 Tainted: G D W O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  894. Jun 29 05:08:09 borg03b kernel: [231718.993546] RIP: 0010:[<ffffffff81008d63>] [<ffffffff81008d63>] default_idle+0x48/0x79
  895. Jun 29 05:08:09 borg03b kernel: [231718.993546] RSP: 0018:ffff88022b519ed8 EFLAGS: 00000246
  896. Jun 29 05:08:09 borg03b kernel: [231718.993546] RAX: 0000000000000000 RBX: ffff88022b518010 RCX: 00000000c0010055
  897. Jun 29 05:08:09 borg03b kernel: [231718.993546] RDX: 0000000000000000 RSI: 0000000000000004 RDI: 0000000000000001
  898. Jun 29 05:08:09 borg03b kernel: [231718.993546] RBP: ffffffff816715c8 R08: 0000000000000000 R09: ffff88022b518010
  899. Jun 29 05:08:09 borg03b kernel: [231718.993546] R10: 00000000ffffffff R11: 0000000000000246 R12: 0000000000000000
  900. Jun 29 05:08:09 borg03b kernel: [231718.993546] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
  901. Jun 29 05:08:09 borg03b kernel: [231718.993546] FS: 00007f728f467700(0000) GS:ffff880437c00000(0000) knlGS:00000000558c06c0
  902. Jun 29 05:08:09 borg03b kernel: [231718.993546] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  903. Jun 29 05:08:09 borg03b kernel: [231718.993546] CR2: 0000000002625818 CR3: 000000042a1ac000 CR4: 00000000000006e0
  904. Jun 29 05:08:09 borg03b kernel: [231718.993546] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  905. Jun 29 05:08:09 borg03b kernel: [231718.993546] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  906. Jun 29 05:08:09 borg03b kernel: [231718.993546] Process swapper/4 (pid: 0, threadinfo ffff88022b518000, task ffff88022b506180)
  907. Jun 29 05:08:09 borg03b kernel: [231718.993546] Stack:
  908. Jun 29 05:08:09 borg03b kernel: [231718.993546] ffff88022b518010 ffffffff81008e77 0000000000000000 0000000000000000
  909. Jun 29 05:08:09 borg03b kernel: [231718.993546] ffff88022b518010 ffffffff81001dc3 0000000000000000 3877dae599b93bbb
  910. Jun 29 05:08:09 borg03b kernel: [231718.993546] 0000000000000000 0000000000000004 0000000000000000 ffffffff8134063c
  911. Jun 29 05:08:09 borg03b kernel: [231718.993546] Call Trace:
  912. Jun 29 05:08:09 borg03b kernel: [231718.993546] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  913. Jun 29 05:08:09 borg03b kernel: [231718.993546] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  914. Jun 29 05:08:09 borg03b kernel: [231718.993546] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  915. Jun 29 05:08:09 borg03b kernel: [231718.993546] Code: 34 25 f0 ca 00 00 e8 53 f8 ff ff 65 48 8b 04 25 08 b6 00 00 83 a0 3c e0 ff ff fb 0f ae f0 48 8b 80 38 e0 ff ff a8 08 75 04 fb f4 <eb> 01 fb 65 48 8b 04 25 08 b6 00 00 83 88 3c e0 ff ff 04 65 8b
  916. Jun 29 05:08:09 borg03b kernel: [231718.993546] Call Trace:
  917. Jun 29 05:08:09 borg03b kernel: [231718.993546] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  918. Jun 29 05:08:09 borg03b kernel: [231718.993546] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  919. Jun 29 05:08:09 borg03b kernel: [231718.993546] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  920. Jun 29 05:08:09 borg03b kernel: [231718.584924] NMI backtrace for cpu 6
  921. Jun 29 05:08:09 borg03b kernel: [231719.114757] CPU 6
  922. Jun 29 05:08:09 borg03b kernel: [231719.114757] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  923. Jun 29 05:08:09 borg03b kernel: [231719.114757]
  924. Jun 29 05:08:09 borg03b kernel: [231719.114757] Pid: 0, comm: swapper/6 Tainted: G D W O 3.2.20-vs2.3.2.10 #1 Supermicro H8DM3-2/H8DM3-2
  925. Jun 29 05:08:09 borg03b kernel: [231719.114757] RIP: 0010:[<ffffffff81008d63>] [<ffffffff81008d63>] default_idle+0x48/0x79
  926. Jun 29 05:08:09 borg03b kernel: [231719.114757] RSP: 0018:ffff88022b529ed8 EFLAGS: 00000246
  927. Jun 29 05:08:09 borg03b kernel: [231719.114757] RAX: 0000000000000000 RBX: ffff88022b528010 RCX: 00000000c0010055
  928. Jun 29 05:08:09 borg03b kernel: [231719.114757] RDX: 0000000000000000 RSI: 0000000000000006 RDI: 0000000000000001
  929. Jun 29 05:08:09 borg03b kernel: [231719.114757] RBP: ffffffff816715c8 R08: 0000000000000000 R09: ffff88022b528010
  930. Jun 29 05:08:09 borg03b kernel: [231719.114757] R10: 00000000ffffffff R11: 0000000000000246 R12: 0000000000000000
  931. Jun 29 05:08:09 borg03b kernel: [231719.114757] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
  932. Jun 29 05:08:09 borg03b kernel: [231719.114757] FS: 00002aec74e0d700(0000) GS:ffff880437d00000(0000) knlGS:00000000558c06c0
  933. Jun 29 05:08:09 borg03b kernel: [231719.114757] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  934. Jun 29 05:08:09 borg03b kernel: [231719.114757] CR2: 000000000061b798 CR3: 0000000416ef8000 CR4: 00000000000006e0
  935. Jun 29 05:08:09 borg03b kernel: [231719.114757] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  936. Jun 29 05:08:09 borg03b kernel: [231719.114757] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  937. Jun 29 05:08:09 borg03b kernel: [231719.114757] Process swapper/6 (pid: 0, threadinfo ffff88022b528000, task ffff88022b51ef60)
  938. Jun 29 05:08:09 borg03b kernel: [231719.114757] Stack:
  939. Jun 29 05:08:09 borg03b kernel: [231719.114757] ffff88022b528010 ffffffff81008e77 0000000000000000 0000000000000000
  940. Jun 29 05:08:09 borg03b kernel: [231719.114757] ffff88022b528010 ffffffff81001dc3 0000000000000000 133676006c70cf56
  941. Jun 29 05:08:09 borg03b kernel: [231719.114757] 0000000000000000 0000000000000006 0000000000000000 ffffffff8134063c
  942. Jun 29 05:08:09 borg03b kernel: [231719.114757] Call Trace:
  943. Jun 29 05:08:09 borg03b kernel: [231719.114757] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  944. Jun 29 05:08:09 borg03b kernel: [231719.114757] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  945. Jun 29 05:08:09 borg03b kernel: [231719.114757] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  946. Jun 29 05:08:09 borg03b kernel: [231719.114757] Code: 34 25 f0 ca 00 00 e8 53 f8 ff ff 65 48 8b 04 25 08 b6 00 00 83 a0 3c e0 ff ff fb 0f ae f0 48 8b 80 38 e0 ff ff a8 08 75 04 fb f4 <eb> 01 fb 65 48 8b 04 25 08 b6 00 00 83 88 3c e0 ff ff 04 65 8b
  947. Jun 29 05:08:09 borg03b kernel: [231719.114757] Call Trace:
  948. Jun 29 05:08:09 borg03b kernel: [231719.114757] [<ffffffff81008e77>] ? amd_e400_idle+0xe3/0xe8
  949. Jun 29 05:08:09 borg03b kernel: [231719.114757] [<ffffffff81001dc3>] ? cpu_idle+0xb3/0xe9
  950. Jun 29 05:08:09 borg03b kernel: [231719.114757] [<ffffffff8134063c>] ? start_secondary+0x1c9/0x1cf
  951. Jun 29 05:10:47 borg03b kernel: [231732.397302] ------------[ cut here ]------------
  952. Jun 29 05:10:47 borg03b kernel: [231732.397302] WARNING: at kernel/watchdog.c:241 watchdog_overflow_callback+0x96/0xa1()
  953. Jun 29 05:10:47 borg03b kernel: [231732.397302] Hardware name: H8DM3-2
  954. Jun 29 05:10:47 borg03b kernel: [231732.397302] Watchdog detected hard LOCKUP on cpu 5
  955. Jun 29 05:10:47 borg03b kernel: [231732.397302] Modules linked in: ocfs2 quota_tree drbd(O) xt_multiport iptable_filter ip_tables x_tables ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs bonding 8021q w83793 w83627hf hwmon_vid loop nv_tco radeon ttm drm_kms_helper drm i2c_nforce2 amd64_edac_mod serio_raw edac_core processor button thermal_sys i2c_algo_bit i2c_core evdev pcspkr k10temp power_supply edac_mce_amd raid10 raid1 md_mod usbhid hid sd_mod ata_generic ohci_hcd pata_amd igb mptsas dca mptscsih mptbase scsi_transport_sas floppy sata_nv ehci_hcd libata usbcore scsi_mod usb_common forcedeth [last unloaded: scsi_wait_scan]
  956. Jun 29 05:10:47 borg03b kernel: [231732.397302] Pid: 8409, comm: smtp Tainted: G D W O 3.2.20-vs2.3.2.10 #1
  957. Jun 29 05:10:47 borg03b kernel: [231732.397302] Call Trace:
  958. Jun 29 05:10:47 borg03b kernel: [231732.397302] <NMI> [<ffffffff8103a606>] ? warn_slowpath_common+0x78/0x8c
  959. Jun 29 05:10:47 borg03b kernel: [231732.397302] [<ffffffff8103a6b8>] ? warn_slowpath_fmt+0x45/0x4a
  960. Jun 29 05:10:47 borg03b kernel: [231732.397302] [<ffffffff81343cf5>] ? printk+0x40/0x4b
  961. Jun 29 05:10:47 borg03b kernel: [231732.397302] [<ffffffff8108887a>] ? watchdog_overflow_callback+0x96/0xa1
  962. Jun 29 05:10:47 borg03b kernel: [231732.397302] [<ffffffff810aa532>] ? __perf_event_overflow+0xf9/0x176
  963. Jun 29 05:10:47 borg03b kernel: [231732.397302] [<ffffffff810587e3>] ? sched_clock_local+0x13/0x76
  964. Jun 29 05:10:47 borg03b kernel: [231732.397302] [<ffffffff8100e4d4>] ? x86_pmu_han
  965. [yes, it ends like this, in tears and disrupted]
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!