API tools faq
paste
Advertisement
Guest User

rcu stall on x86

a guest
Mar 24th, 2022
109
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 10.45 KB | None | 0 0
raw download clone embed print report
  1. [ 907.892075][ C1] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
  2. [ 907.899101][ C1] rcu: 0-...!: (1 GPs behind) idle=a59/1/0x4000000000000000 softirq=5468/5472 fqs=5
  3. [ 907.909178][ C1] (detected by 1, t=10502 jiffies, g=4825, q=140)
  4. [ 907.915713][ C1] Sending NMI from CPU 1 to CPUs 0:
  5. [ 907.920925][ C0] NMI backtrace for cpu 0
  6. [ 907.920934][ C0] CPU: 0 PID: 3644 Comm: kworker/0:1 Not tainted 5.17.0-rc6-next-20220303-syzkaller #0
  7. [ 907.920954][ C0] Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
  8. [ 907.920995][ C0] Workqueue: events_power_efficient gc_worker
  9. [ 907.921188][ C0] RIP: 0010:__lock_acquire+0x1460/0x56c0
  10. [ 907.921231][ C0] Code: 0f b7 db be 08 00 00 00 48 89 d8 48 c1 f8 06 48 8d 3c c5 80 59 01 90 e8 fe 54 67 00 48 0f a3 1d f6 ed a3 0e 0f 83 2c 06 00 00 <48> 8d 1c 5b 48 c1 e3 06 48 81 c3 a0 5d 01 90 48 8d 7b 40 48 b8 00
  11. [ 907.921251][ C0] RSP: 0018:ffffc90000007be0 EFLAGS: 00000047
  12. [ 907.921266][ C0] RAX: 0000000000000001 RBX: 0000000000000029 RCX: ffffffff815d6b82
  13. [ 907.921280][ C0] RDX: fffffbfff2002b31 RSI: 0000000000000008 RDI: ffffffff90015980
  14. [ 907.921294][ C0] RBP: ffff88801bc4a842 R08: 0000000000000000 R09: ffffffff90015987
  15. [ 907.921308][ C0] R10: fffffbfff2002b30 R11: 0000000000000001 R12: ffff88801bc4a820
  16. [ 907.921322][ C0] R13: ffff88801bc49d40 R14: 0000000000000001 R15: 3d0b7b89c89fff91
  17. [ 907.921336][ C0] FS: 0000000000000000(0000) GS:ffff8880b9c00000(0000) knlGS:0000000000000000
  18. [ 907.921360][ C0] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
  19. [ 907.921375][ C0] CR2: 0000000020000600 CR3: 0000000076bbe000 CR4: 00000000003506f0
  20. [ 907.921389][ C0] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  21. [ 907.921401][ C0] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
  22. [ 907.921414][ C0] Call Trace:
  23. [ 907.921419][ C0] <IRQ>
  24. [ 907.921432][ C0] ? lockdep_hardirqs_on_prepare+0x400/0x400
  25. [ 907.921466][ C0] lock_acquire+0x1ab/0x510
  26. [ 907.921491][ C0] ? __hrtimer_run_queues+0x243/0xe50
  27. [ 907.921551][ C0] ? lock_release+0x720/0x720
  28. [ 907.921581][ C0] _raw_spin_lock_irq+0x32/0x50
  29. [ 907.921651][ C0] ? __hrtimer_run_queues+0x243/0xe50
  30. [ 907.921676][ C0] __hrtimer_run_queues+0x243/0xe50
  31. [ 907.921704][ C0] ? hrtimer_sleeper_start_expires+0x80/0x80
  32. [ 907.921730][ C0] ? ktime_get_update_offsets_now+0x3eb/0x5c0
  33. [ 907.921762][ C0] hrtimer_interrupt+0x31c/0x790
  34. [ 907.921791][ C0] __sysvec_apic_timer_interrupt+0x146/0x530
  35. [ 907.921847][ C0] sysvec_apic_timer_interrupt+0x8e/0xc0
  36. [ 907.921876][ C0] </IRQ>
  37. [ 907.921881][ C0] <TASK>
  38. [ 907.921887][ C0] asm_sysvec_apic_timer_interrupt+0x12/0x20
  39. [ 907.921915][ C0] RIP: 0010:__sanitizer_cov_trace_pc+0x0/0x60
  40. [ 907.921952][ C0] Code: 48 89 ef 5d e9 51 f3 4a 00 5d be 03 00 00 00 e9 96 96 74 02 66 0f 1f 44 00 00 48 8b be b0 01 00 00 e8 b4 ff ff ff 31 c0 c3 90 <65> 8b 05 29 49 89 7e 89 c1 48 8b 34 24 81 e1 00 01 00 00 65 48 8b
  41. [ 907.921970][ C0] RSP: 0018:ffffc90003ccfc30 EFLAGS: 00000293
  42. [ 907.921987][ C0] RAX: 0000000000000000 RBX: 0000000000000200 RCX: 0000000000000000
  43. [ 907.921999][ C0] RDX: ffff88801bc49d40 RSI: ffffffff87821df7 RDI: 0000000000000003
  44. [ 907.922013][ C0] RBP: 0000000000000000 R08: 0000000000000000 R09: ffffffff90015a17
  45. [ 907.922026][ C0] R10: ffffffff87821ded R11: 0000000000000001 R12: dffffc0000000000
  46. [ 907.922040][ C0] R13: 0000000000000000 R14: 0000000000040000 R15: 0000000000040000
  47. [ 907.922055][ C0] ? gc_worker+0x23d/0x12b0
  48. [ 907.922080][ C0] ? gc_worker+0x247/0x12b0
  49. [ 907.922107][ C0] gc_worker+0x24d/0x12b0
  50. [ 907.922133][ C0] ? lock_release+0x720/0x720
  51. [ 907.922159][ C0] ? nf_ct_kill_acct+0xc0/0xc0
  52. [ 907.922188][ C0] process_one_work+0x996/0x1610
  53. [ 907.922227][ C0] ? pwq_dec_nr_in_flight+0x2a0/0x2a0
  54. [ 907.922250][ C0] ? rwlock_bug.part.0+0x90/0x90
  55. [ 907.922277][ C0] ? _raw_spin_lock_irq+0x41/0x50
  56. [ 907.922303][ C0] worker_thread+0x665/0x1080
  57. [ 907.922326][ C0] ? __kthread_parkme+0x15f/0x220
  58. [ 907.922360][ C0] ? process_one_work+0x1610/0x1610
  59. [ 907.922381][ C0] kthread+0x2e9/0x3a0
  60. [ 907.922422][ C0] ? kthread_complete_and_exit+0x40/0x40
  61. [ 907.922452][ C0] ret_from_fork+0x1f/0x30
  62. [ 907.922488][ C0] </TASK>
  63. [ 907.922495][ C0] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.570 msecs
  64. [ 907.922919][ C1] rcu: rcu_preempt kthread starved for 10492 jiffies! g4825 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
  65. [ 908.311859][ C1] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
  66. [ 908.321827][ C1] rcu: RCU grace-period kthread stack dump:
  67. [ 908.327706][ C1] task:rcu_preempt state:R running task stack:28752 pid: 16 ppid: 2 flags:0x00004000
  68. [ 908.338490][ C1] Call Trace:
  69. [ 908.341770][ C1] <TASK>
  70. [ 908.344709][ C1] __schedule+0xa94/0x4910
  71. [ 908.349152][ C1] ? lockdep_hardirqs_on+0x79/0x100
  72. [ 908.354358][ C1] ? __mod_timer+0x83c/0xe30
  73. [ 908.358966][ C1] ? io_schedule_timeout+0x140/0x140
  74. [ 908.364270][ C1] ? debug_object_free+0x350/0x350
  75. [ 908.369447][ C1] schedule+0xd2/0x1f0
  76. [ 908.373534][ C1] schedule_timeout+0x14a/0x2a0
  77. [ 908.378394][ C1] ? usleep_range_state+0x1b0/0x1b0
  78. [ 908.383606][ C1] ? destroy_timer_on_stack+0x20/0x20
  79. [ 908.388988][ C1] ? hci_se_io+0x30/0x110
  80. [ 908.393377][ C1] ? _raw_spin_unlock_irqrestore+0x3d/0x70
  81. [ 908.399195][ C1] ? prepare_to_swait_event+0xee/0x470
  82. [ 908.404698][ C1] rcu_gp_fqs_loop+0x186/0x810
  83. [ 908.409501][ C1] ? force_qs_rnp+0x800/0x800
  84. [ 908.414197][ C1] ? _raw_spin_unlock_irqrestore+0x50/0x70
  85. [ 908.420023][ C1] rcu_gp_kthread+0x1de/0x320
  86. [ 908.424715][ C1] ? rcu_gp_init+0x1420/0x1420
  87. [ 908.429499][ C1] ? __sanitizer_cov_trace_const_cmp1+0x22/0x80
  88. [ 908.435755][ C1] ? __kthread_parkme+0x15f/0x220
  89. [ 908.440804][ C1] ? rcu_gp_init+0x1420/0x1420
  90. [ 908.445589][ C1] kthread+0x2e9/0x3a0
  91. [ 908.449674][ C1] ? kthread_complete_and_exit+0x40/0x40
  92. [ 908.455329][ C1] ret_from_fork+0x1f/0x30
  93. [ 908.459775][ C1] </TASK>
  94. [ 908.462798][ C1] rcu: Stack dump where RCU GP kthread last ran:
  95. [ 908.469115][ C1] NMI backtrace for cpu 1
  96. [ 908.473437][ C1] CPU: 1 PID: 45 Comm: kworker/u4:2 Not tainted 5.17.0-rc6-next-20220303-syzkaller #0
  97. [ 908.482978][ C1] Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
  98. [ 908.493034][ C1] Workqueue: events_unbound toggle_allocation_gate
  99. [ 908.499598][ C1] Call Trace:
  100. [ 908.502877][ C1] <IRQ>
  101. [ 908.505718][ C1] dump_stack_lvl+0xcd/0x134
  102. [ 908.510358][ C1] nmi_cpu_backtrace.cold+0x47/0x144
  103. [ 908.515653][ C1] ? lapic_can_unplug_cpu+0x80/0x80
  104. [ 908.520859][ C1] nmi_trigger_cpumask_backtrace+0x1e6/0x230
  105. [ 908.526863][ C1] rcu_check_gp_kthread_starvation.cold+0x1fb/0x200
  106. [ 908.533477][ C1] ? do_raw_spin_unlock+0x171/0x230
  107. [ 908.538696][ C1] rcu_sched_clock_irq+0x21ae/0x22a0
  108. [ 908.544345][ C1] ? rcutree_dead_cpu+0x30/0x30
  109. [ 908.549196][ C1] ? timekeeping_advance+0x978/0xe30
  110. [ 908.554517][ C1] ? tick_sched_do_timer+0x470/0x470
  111. [ 908.559829][ C1] update_process_times+0x16d/0x200
  112. [ 908.565037][ C1] tick_sched_handle+0x9b/0x180
  113. [ 908.569903][ C1] tick_sched_timer+0xee/0x120
  114. [ 908.574671][ C1] __hrtimer_run_queues+0x1c0/0xe50
  115. [ 908.579887][ C1] ? hrtimer_sleeper_start_expires+0x80/0x80
  116. [ 908.585879][ C1] ? ktime_get_update_offsets_now+0x3eb/0x5c0
  117. [ 908.591971][ C1] hrtimer_interrupt+0x31c/0x790
  118. [ 908.596937][ C1] __sysvec_apic_timer_interrupt+0x146/0x530
  119. [ 908.602939][ C1] sysvec_apic_timer_interrupt+0x8e/0xc0
  120. [ 908.608581][ C1] </IRQ>
  121. [ 908.611507][ C1] <TASK>
  122. [ 908.614435][ C1] asm_sysvec_apic_timer_interrupt+0x12/0x20
  123. [ 908.620427][ C1] RIP: 0010:check_kcov_mode+0xf/0x40
  124. [ 908.625731][ C1] Code: 7c 24 08 e8 b3 9b 4b 00 e9 61 fd ff ff cc cc cc cc cc cc cc cc cc cc cc cc cc cc 65 8b 05 79 4d 89 7e 89 c2 81 e2 00 01 00 00 <a9> 00 01 ff 00 74 10 31 c0 85 d2 74 15 8b 96 ac 15 00 00 85 d2 74
  125. [ 908.645345][ C1] RSP: 0018:ffffc90000b679c8 EFLAGS: 00000246
  126. [ 908.651415][ C1] RAX: 0000000000000001 RBX: ffff8880b9c42380 RCX: 0000000000000001
  127. [ 908.659386][ C1] RDX: 0000000000000000 RSI: ffff888016df8000 RDI: 0000000000000003
  128. [ 908.667359][ C1] RBP: 0000000000000003 R08: 0000000000000000 R09: 0000000000000001
  129. [ 908.675327][ C1] R10: ffffffff816d99aa R11: 0000000000000000 R12: ffffed1017388471
  130. [ 908.683298][ C1] R13: 0000000000000000 R14: ffff8880b9c42388 R15: 0000000000000001
  131. [ 908.691272][ C1] ? smp_call_function_many_cond+0x50a/0xc90
  132. [ 908.697309][ C1] __sanitizer_cov_trace_const_cmp4+0x1c/0x70
  133. [ 908.703391][ C1] smp_call_function_many_cond+0x50a/0xc90
  134. [ 908.709216][ C1] ? optimize_nops+0x2d0/0x2d0
  135. [ 908.714028][ C1] ? optimize_nops+0x2d0/0x2d0
  136. [ 908.718804][ C1] on_each_cpu_cond_mask+0x56/0xa0
  137. [ 908.723927][ C1] ? kmem_cache_alloc+0xaf/0x3b0
  138. [ 908.728906][ C1] text_poke_bp_batch+0x3e9/0x6b0
  139. [ 908.733953][ C1] ? do_sync_core+0x20/0x20
  140. [ 908.738466][ C1] ? __jump_label_patch+0x159/0x1b0
  141. [ 908.743682][ C1] ? __sanitizer_cov_trace_const_cmp1+0x22/0x80
  142. [ 908.749937][ C1] ? __jump_label_update+0x351/0x400
  143. [ 908.755257][ C1] text_poke_finish+0x16/0x30
  144. [ 908.759944][ C1] arch_jump_label_transform_apply+0x13/0x20
  145. [ 908.765933][ C1] jump_label_update+0x32f/0x410
  146. [ 908.770882][ C1] static_key_enable_cpuslocked+0x1b1/0x260
  147. [ 908.776785][ C1] static_key_enable+0x16/0x20
  148. [ 908.781557][ C1] toggle_allocation_gate+0x100/0x390
  149. [ 908.786940][ C1] ? lock_release+0x720/0x720
  150. [ 908.791628][ C1] ? wake_up_kfence_timer+0x20/0x20
  151. [ 908.796838][ C1] ? lockdep_hardirqs_on_prepare+0x400/0x400
  152. [ 908.802841][ C1] process_one_work+0x996/0x1610
  153. [ 908.807799][ C1] ? pwq_dec_nr_in_flight+0x2a0/0x2a0
  154. [ 908.813180][ C1] ? rwlock_bug.part.0+0x90/0x90
  155. [ 908.818128][ C1] ? _raw_spin_lock_irq+0x41/0x50
  156. [ 908.823175][ C1] worker_thread+0x665/0x1080
  157. [ 908.827876][ C1] ? process_one_work+0x1610/0x1610
  158. [ 908.833081][ C1] kthread+0x2e9/0x3a0
  159. [ 908.837160][ C1] ? kthread_complete_and_exit+0x40/0x40
  160. [ 908.842810][ C1] ret_from_fork+0x1f/0x30
  161. [ 908.847256][ C1] </TASK>
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!