11 00:21:25 node-10-157-128-23 kernel: [6471022.698435] INFO: rcu_sched detected stalls on CPUs/tasks: { 1} (detected by 0, t=15002 jiffies, g=70535696, c=70535695, q=1427)
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847112] sending NMI to all CPUs:
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847119] NMI backtrace for cpu 0
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847120] CPU 0
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847133] Pid: 1548, comm: kvm Tainted: GF       W    3.8.0-32-generic #47~precise1-Ubuntu Supermicro X10SLL-F/X10SLL-F
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847134] RIP: 0010:[<ffffffff81362580>]  [<ffffffff81362580>] __bitmap_andnot+0x20/0x50
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847141] RSP: 0018:ffff88081fc03cc8  EFLAGS: 00000046
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847142] RAX: 0000000000000000 RBX: ffff88081fc0dd00 RCX: 0000000000000000
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847143] RDX: ffff88081fc0dd20 RSI: ffff88081fc0dd00 RDI: ffff88081fc0dd00
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847144] RBP: ffff88081fc03cc8 R08: 0000000000000000 R09: 0000000000000004
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847146] R10: 0000000000003d56 R11: 0000000000003d55 R12: ffff88081fc0dd20
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847147] R13: 000000000000dd40 R14: 0000000000080000 R15: 00000000000000ff
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847149] FS:  00007f465e958700(0000) GS:ffff88081fc00000(0000) knlGS:ffff880002200000
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847150] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847151] CR2: 00000000006d3ca0 CR3: 000000037a40e000 CR4: 00000000001427e0
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847153] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847154] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847156] Process kvm (pid: 1548, threadinfo ffff8805005d6000, task ffff8807ce645d00)
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847156] Stack:
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847158]  ffff88081fc03d38 ffffffff8103ea76 ffffffff81ce7040 0000000000000096
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847161]  000000001fc03d48 000000000000dd20 0000000000000002 0000000000000000
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847163]  ffff88081fc03d58 0000000000002710 0000000000000001 0000000000000593
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847166] Call Trace:
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847167]  <IRQ>
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847168]  [<ffffffff8103ea76>] __x2apic_send_IPI_mask+0x156/0x180
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847173]  [<ffffffff8103eabc>] x2apic_send_IPI_all+0x1c/0x20
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847178]  [<ffffffff8103ab01>] arch_trigger_all_cpu_backtrace+0x61/0xa0
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847181]  [<ffffffff810f5bab>] print_other_cpu_stall+0x21b/0x280
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847184]  [<ffffffff810f6830>] __rcu_pending+0x1a0/0x210
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847186]  [<ffffffff810f68fd>] rcu_pending+0x5d/0x80
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847189]  [<ffffffff810f86e3>] rcu_check_callbacks+0xa3/0x160
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847194]  [<ffffffff8106c038>] update_process_times+0x48/0x90
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847197]  [<ffffffff810b6bc3>] tick_sched_handle.isra.8+0x33/0x70
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847199]  [<ffffffff810b6cec>] tick_sched_timer+0x4c/0x80
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847202]  [<ffffffff810839d8>] __run_hrtimer+0x78/0x1f0
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847204]  [<ffffffff810b6ca0>] ? tick_nohz_handler+0xa0/0xa0
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847207]  [<ffffffff810b004c>] ? ktime_get_update_offsets+0x4c/0xd0
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847209]  [<ffffffff81084267>] hrtimer_interrupt+0xf7/0x240
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847213]  [<ffffffff816fec59>] smp_apic_timer_interrupt+0x69/0x99
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847224]  [<ffffffffa041b3f0>] ? vmx_invpcid_supported+0x20/0x20 [kvm_intel]
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847227]  [<ffffffff816fdb9d>] apic_timer_interrupt+0x6d/0x80
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847228]  <EOI>
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847229]  [<ffffffffa038958b>] ? vcpu_enter_guest+0x34b/0x690 [kvm]
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847249]  [<ffffffffa0389576>] ? vcpu_enter_guest+0x336/0x690 [kvm]
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847253]  [<ffffffff8107fdd0>] ? add_wait_queue+0x60/0x60
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847260]  [<ffffffffa03899e8>] __vcpu_run+0x118/0x2f0 [kvm]
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847267]  [<ffffffffa0389c5f>] kvm_arch_vcpu_ioctl_run+0x9f/0x170 [kvm]
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847273]  [<ffffffffa0370516>] kvm_vcpu_ioctl+0x556/0x680 [kvm]
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847276]  [<ffffffff811dcf92>] ? fsnotify+0x1d2/0x2b0
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847279]  [<ffffffff81088ac8>] ? __wake_up_locked_key+0x18/0x20
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847282]  [<ffffffff811e5be6>] ? eventfd_write+0x1b6/0x220
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847284]  [<ffffffff811ad69a>] do_vfs_ioctl+0x8a/0x340
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847286]  [<ffffffff811ad9e1>] sys_ioctl+0x91/0xb0
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847289]  [<ffffffff8119bceb>] ? sys_write+0x6b/0xa0
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847292]  [<ffffffff816fce5d>] system_call_fastpath+0x1a/0x1f
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847293] Code: 5d c3 66 0f 1f 84 00 00 00 00 00 4c 63 c9 55 31 c0 49 83 c1 3f 49 c1 e9 06 48 89 e5 45 85 c9 7e 2d 31 c9 45 31 c0 0f 1f 44 00 00 <48> 8b 04 ca 48 f7 d0 48 23 04 ce 48 89 04 cf 48 83 c1 01 49 09
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847317] NMI backtrace for cpu 4
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847318] CPU 4
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847320] Pid: 26477, comm: kvm Tainted: GF       W    3.8.0-32-generic #47~precise1-Ubuntu Supermicro X10SLL-F/X10SLL-F
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847322] RIP: 0010:[<ffffffffa041d322>]  [<ffffffffa041d322>] vmx_complete_atomic_exit+0x72/0xb0 [kvm_intel]
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847326] RSP: 0018:ffff88056b407cc8  EFLAGS: 00000046
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847327] RAX: 0000000000000001 RBX: ffff88007fb33e20 RCX: ffff88007fb33e20
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847328] RDX: 0000000000004404 RSI: 0000000000325ab2 RDI: ffff88007fb33e20
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847329] RBP: ffff88056b407cd8 R08: 0000000000000000 R09: 0000000000000000
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847330] R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000200
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847332] R13: 0000000000000000 R14: 0000000000325acc R15: 0000000000000006
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847333] FS:  00007f42b920e700(0000) GS:ffff88081fd00000(0000) knlGS:0000000000000000
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847334] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847336] CR2: 000037baaa09d000 CR3: 00000006cc1f0000 CR4: 00000000001427e0
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847337] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847338] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847339] Process kvm (pid: 26477, threadinfo ffff88056b406000, task ffff8807cdac1740)
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847340] Stack:
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847341]  0000000000000000 0000000034699a09 ffff88056b407d38 ffffffffa0420207
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847344]  0000000000000000 ffff88007fb33e20 ffff88056b407fd8 000000002e00e880
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847346]  ffff88007fb33e20 ffff88007fb33e20 0000000000000000 0000000000000000
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847349] Call Trace:
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847353]  [<ffffffffa0420207>] vmx_vcpu_run+0x287/0x440 [kvm_intel]
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847362]  [<ffffffffa0389549>] vcpu_enter_guest+0x309/0x690 [kvm]
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847365]  [<ffffffff8108f8aa>] ? __cond_resched+0x2a/0x40
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847375]  [<ffffffffa03899e8>] __vcpu_run+0x118/0x2f0 [kvm]
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847385]  [<ffffffffa0389c5f>] kvm_arch_vcpu_ioctl_run+0x9f/0x170 [kvm]
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847393]  [<ffffffffa0370516>] kvm_vcpu_ioctl+0x556/0x680 [kvm]
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847395]  [<ffffffff810921c0>] ? wake_up_state+0x10/0x20
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847398]  [<ffffffff810b8276>] ? wake_futex+0x76/0xa0
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847400]  [<ffffffff810b95e3>] ? futex_wake+0x113/0x130
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847404]  [<ffffffff810135ed>] ? __switch_to+0x16d/0x4a0
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847406]  [<ffffffff811ad69a>] do_vfs_ioctl+0x8a/0x340
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847415]  [<ffffffffa037c904>] ? kvm_on_user_return+0x74/0x80 [kvm]
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847417]  [<ffffffff811ad9e1>] sys_ioctl+0x91/0xb0
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847420]  [<ffffffff81014dd7>] ? do_notify_resume+0x67/0xc0
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847422]  [<ffffffff816fce5d>] system_call_fastpath+0x1a/0x1f
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847423] Code: 75 0f 48 8b 5d f0 4c 8b 65 f8 c9 c3 0f 1f 44 00 00 41 81 e4 00 07 00 00 41 81 fc 00 02 00 00 75 e1 48 89 df e8 60 bb f5 ff cd 02 <48> 89 df e8 76 bb f5 ff 48 8b 5d f0 4c 8b 65 f8 c9 c3 0f 1f 40
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847448] NMI backtrace for cpu 5
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847450] CPU 5
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847452] Pid: 0, comm: swapper/5 Tainted: GF       W    3.8.0-32-generic #47~precise1-Ubuntu Supermicro X10SLL-F/X10SLL-F
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847454] RIP: 0010:[<ffffffff813c6687>]  [<ffffffff813c6687>] intel_idle+0xa7/0x100
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847458] RSP: 0018:ffff8807f3d93df8  EFLAGS: 00000046
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847460] RAX: 0000000000000020 RBX: 0000000000000008 RCX: 0000000000000001
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847461] RDX: 0000000000000000 RSI: ffff8807f3d93fd8 RDI: 0000000001c0d000
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847462] RBP: ffff8807f3d93e28 R08: 0000000000000020 R09: 0000000000005dd9
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847463] R10: 0000000000000e60 R11: 0000000000000000 R12: 0000000000000003
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847464] R13: 0000000000000020 R14: 0000000000000003 R15: 0000000000000003
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847466] FS:  0000000000000000(0000) GS:ffff88081fd40000(0000) knlGS:0000000000000000
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847467] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847468] CR2: 00000000006dbff8 CR3: 0000000001c0d000 CR4: 00000000001427e0
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847470] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847471] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847472] Process swapper/5 (pid: 0, threadinfo ffff8807f3d92000, task ffff8807f3d98000)
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847473] Stack:
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847474]  ffff8807f3d93e08 000000058101bad9 ffff88081fd59f70 001703087280b82c
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847477]  ffffffff8158b290 ffffffff81c7a380 ffff8807f3d93e38 ffffffff8158b2a9
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847480]  ffff8807f3d93e98 ffffffff8158bd21 ffff8807f3d93ea8 ffffffff8158d16e
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847482] Call Trace:
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847488]  [<ffffffff8158b290>] ? centrino_target+0x360/0x360
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847490]  [<ffffffff8158b2a9>] cpuidle_enter+0x19/0x20
Nov 11 00:21:25 node-10-157-128-23 kernel: [6471022.847492]  [<ffffffff8158bd21>] cpuidle_wrap_enter+0x41/0xa0