API tools faq
paste
Advertisement
viniciusferrao

dmesg from debug TrueNAS 12.0-U1 #1

viniciusferrao
Dec 24th, 2020
289
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 92.04 KB | None | 0 0
raw download clone embed print report
  1. root@storage:~ # dmesg
  2. ---<<BOOT>>---
  3. Copyright (c) 1992-2020 The FreeBSD Project.
  4. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
  5. The Regents of the University of California. All rights reserved.
  6. FreeBSD is a registered trademark of The FreeBSD Foundation.
  7. FreeBSD 12.2-RELEASE-p2 663e6b09467(HEAD) TRUENAS amd64
  8. FreeBSD clang version 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611aa2)
  9. WARNING: WITNESS option enabled, expect reduced performance.
  10. VT(efifb): resolution 800x600
  11. CPU: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz (2100.04-MHz K8-class CPU)
  12. Origin="GenuineIntel" Id=0x306e4 Family=0x6 Model=0x3e Stepping=4
  13. Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  14. Features2=0x7fbee3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  15. AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  16. AMD Features2=0x1<LAHF>
  17. Structured Extended Features=0x281<FSGSBASE,SMEP,ERMS>
  18. XSAVE Features=0x1<XSAVEOPT>
  19. VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
  20. TSC: P-state invariant, performance statistics
  21. real memory = 137438953472 (131072 MB)
  22. avail memory = 133614964736 (127425 MB)
  23. Event timer "LAPIC" quality 600
  24. ACPI APIC Table: <ALASKA A M I>
  25. FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs
  26. FreeBSD/SMP: 2 package(s) x 6 core(s)
  27. random: unblocking device.
  28. ioapic0 <Version 2.0> irqs 0-23 on motherboard
  29. ioapic1 <Version 2.0> irqs 24-47 on motherboard
  30. ioapic2 <Version 2.0> irqs 48-71 on motherboard
  31. Launching APs: 10 3 1 4 6 5 9 11 2 8 7
  32. Timecounter "TSC" frequency 2100041344 Hz quality 1000
  33. random: entropy device external interface
  34. kbd0 at kbdmux0
  35. random: registering fast source Intel Secure Key RNG
  36. random: fast provider: "Intel Secure Key RNG"
  37. mlx5en: Mellanox Ethernet driver 3.5.2 (September 2019)
  38. nexus0
  39. efirtc0: <EFI Realtime Clock> on motherboard
  40. efirtc0: registered as a time-of-day clock, resolution 1.000000s
  41. aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS> on motherboard
  42. padlock0: No ACE support.
  43. cryptosoft0: <software crypto> on motherboard
  44. acpi0: <ALASKA A M I> on motherboard
  45. acpi0: Power Button (fixed)
  46. cpu0: <ACPI CPU> numa-domain 0 on acpi0
  47. attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
  48. Timecounter "i8254" frequency 1193182 Hz quality 0
  49. Event timer "i8254" frequency 1193182 Hz quality 100
  50. atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
  51. atrtc0: registered as a time-of-day clock, resolution 1.000000s
  52. Event timer "RTC" frequency 32768 Hz quality 0
  53. hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
  54. Timecounter "HPET" frequency 14318180 Hz quality 950
  55. Event timer "HPET" frequency 14318180 Hz quality 350
  56. Event timer "HPET1" frequency 14318180 Hz quality 340
  57. Event timer "HPET2" frequency 14318180 Hz quality 340
  58. Event timer "HPET3" frequency 14318180 Hz quality 340
  59. Event timer "HPET4" frequency 14318180 Hz quality 340
  60. Event timer "HPET5" frequency 14318180 Hz quality 340
  61. Event timer "HPET6" frequency 14318180 Hz quality 340
  62. Event timer "HPET7" frequency 14318180 Hz quality 340
  63. Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
  64. acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
  65. pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff numa-domain 0 on acpi0
  66. pci0: <ACPI PCI bus> numa-domain 0 on pcib0
  67. pcib1: <ACPI PCI-PCI bridge> irq 26 at device 1.0 numa-domain 0 on pci0
  68. pci1: <ACPI PCI bus> numa-domain 0 on pcib1
  69. mps0: <Avago Technologies (LSI) SAS2308> port 0x8000-0x80ff mem 0xdfa40000-0xdfa4ffff,0xdfa00000-0xdfa3ffff irq 16 at device 0.0 numa-domain 0 on pci1
  70. mps0: Firmware: 20.00.04.00, Driver: 21.02.00.00-fbsd
  71. mps0: IOCCapabilities: 5a85c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,MSIXIndex,HostDisc>
  72. pcib2: <ACPI PCI-PCI bridge> irq 32 at device 2.0 numa-domain 0 on pci0
  73. pci2: <ACPI PCI bus> numa-domain 0 on pcib2
  74. pcib3: <ACPI PCI-PCI bridge> irq 32 at device 2.2 numa-domain 0 on pci0
  75. pci3: <ACPI PCI bus> numa-domain 0 on pcib3
  76. pcib4: <ACPI PCI-PCI bridge> irq 47 at device 3.0 numa-domain 0 on pci0
  77. pci4: <ACPI PCI bus> numa-domain 0 on pcib4
  78. pcib4: no PRT entry for 4.0.INTA
  79. pcib4: no PRT entry for 4.0.INTB
  80. pcib4: no PRT entry for 4.0.INTC
  81. pcib4: no PRT entry for 4.0.INTD
  82. igb0: <Intel(R) PRO/1000 PCI-Express Network Driver> port 0x7060-0x707f mem 0xdfc60000-0xdfc7ffff,0xdfc8c000-0xdfc8ffff irq 11 at device 0.0 numa-domain 0 on pci4
  83. igb0: Using 1024 TX descriptors and 1024 RX descriptors
  84. igb0: queue equality override not set, capping rx_queues at 6 and tx_queues at 6
  85. igb0: Using 6 RX queues 6 TX queues
  86. igb0: Using MSI-X interrupts with 7 vectors
  87. igb0: Ethernet address: 00:25:90:e3:96:e0
  88. igb1: <Intel(R) PRO/1000 PCI-Express Network Driver> port 0x7040-0x705f mem 0xdfc40000-0xdfc5ffff,0xdfc88000-0xdfc8bfff irq 10 at device 0.1 numa-domain 0 on pci4
  89. igb1: Using 1024 TX descriptors and 1024 RX descriptors
  90. igb1: queue equality override not set, capping rx_queues at 6 and tx_queues at 6
  91. igb1: Using 6 RX queues 6 TX queues
  92. igb1: Using MSI-X interrupts with 7 vectors
  93. igb1: Ethernet address: 00:25:90:e3:96:e1
  94. igb2: <Intel(R) PRO/1000 PCI-Express Network Driver> port 0x7020-0x703f mem 0xdfc20000-0xdfc3ffff,0xdfc84000-0xdfc87fff irq 5 at device 0.2 numa-domain 0 on pci4
  95. igb2: Using 1024 TX descriptors and 1024 RX descriptors
  96. igb2: queue equality override not set, capping rx_queues at 6 and tx_queues at 6
  97. igb2: Using 6 RX queues 6 TX queues
  98. igb2: Using MSI-X interrupts with 7 vectors
  99. igb2: Ethernet address: 00:25:90:e3:96:e2
  100. igb3: <Intel(R) PRO/1000 PCI-Express Network Driver> port 0x7000-0x701f mem 0xdfc00000-0xdfc1ffff,0xdfc80000-0xdfc83fff irq 11 at device 0.3 numa-domain 0 on pci4
  101. igb3: Using 1024 TX descriptors and 1024 RX descriptors
  102. igb3: queue equality override not set, capping rx_queues at 6 and tx_queues at 6
  103. igb3: Using 6 RX queues 6 TX queues
  104. igb3: Using MSI-X interrupts with 7 vectors
  105. igb3: Ethernet address: 00:25:90:e3:96:e3
  106. pcib5: <ACPI PCI-PCI bridge> irq 47 at device 3.2 numa-domain 0 on pci0
  107. pci5: <ACPI PCI bus> numa-domain 0 on pcib5
  108. ioat0: <IVB IOAT Ch0> mem 0xdfd1c000-0xdfd1ffff irq 31 at device 4.0 numa-domain 0 on pci0
  109. ioat0: Capabilities: 2f7<PQ,Extended_APIC_ID,Block_Fill,Move_CRC,DCA,Marker_Skipping,CRC,Page_Break>
  110. ioat1: <IVB IOAT Ch1> mem 0xdfd18000-0xdfd1bfff irq 39 at device 4.1 numa-domain 0 on pci0
  111. ioat1: Capabilities: 2f7<PQ,Extended_APIC_ID,Block_Fill,Move_CRC,DCA,Marker_Skipping,CRC,Page_Break>
  112. ioat2: <IVB IOAT Ch2> mem 0xdfd14000-0xdfd17fff irq 31 at device 4.2 numa-domain 0 on pci0
  113. ioat2: Capabilities: f7<Extended_APIC_ID,Block_Fill,Move_CRC,DCA,Marker_Skipping,CRC,Page_Break>
  114. ioat3: <IVB IOAT Ch3> mem 0xdfd10000-0xdfd13fff irq 39 at device 4.3 numa-domain 0 on pci0
  115. ioat3: Capabilities: f7<Extended_APIC_ID,Block_Fill,Move_CRC,DCA,Marker_Skipping,CRC,Page_Break>
  116. ioat4: <IVB IOAT Ch4> mem 0xdfd0c000-0xdfd0ffff irq 31 at device 4.4 numa-domain 0 on pci0
  117. ioat4: Capabilities: f7<Extended_APIC_ID,Block_Fill,Move_CRC,DCA,Marker_Skipping,CRC,Page_Break>
  118. ioat5: <IVB IOAT Ch5> mem 0xdfd08000-0xdfd0bfff irq 39 at device 4.5 numa-domain 0 on pci0
  119. ioat5: Capabilities: f7<Extended_APIC_ID,Block_Fill,Move_CRC,DCA,Marker_Skipping,CRC,Page_Break>
  120. ioat6: <IVB IOAT Ch6> mem 0xdfd04000-0xdfd07fff irq 31 at device 4.6 numa-domain 0 on pci0
  121. ioat6: Capabilities: f7<Extended_APIC_ID,Block_Fill,Move_CRC,DCA,Marker_Skipping,CRC,Page_Break>
  122. ioat7: <IVB IOAT Ch7> mem 0xdfd00000-0xdfd03fff irq 39 at device 4.7 numa-domain 0 on pci0
  123. ioat7: Capabilities: f7<Extended_APIC_ID,Block_Fill,Move_CRC,DCA,Marker_Skipping,CRC,Page_Break>
  124. pcib6: <ACPI PCI-PCI bridge> irq 16 at device 17.0 numa-domain 0 on pci0
  125. pci6: <ACPI PCI bus> numa-domain 0 on pcib6
  126. isci0: <Intel(R) C600 Series Chipset SAS Controller (SATA mode)> port 0x6000-0x60ff mem 0xde47c000-0xde47ffff,0xde000000-0xde3fffff irq 16 at device 0.0 numa-domain 0 on pci6
  127. pci0: <simple comms> at device 22.0 (no driver attached)
  128. pci0: <simple comms> at device 22.1 (no driver attached)
  129. ehci0: <Intel Patsburg USB 2.0 controller> mem 0xdfd23000-0xdfd233ff irq 16 at device 26.0 numa-domain 0 on pci0
  130. usbus0: EHCI version 1.0
  131. usbus0 numa-domain 0 on ehci0
  132. usbus0: 480Mbps High Speed USB v2.0
  133. ehci1: <Intel Patsburg USB 2.0 controller> mem 0xdfd22000-0xdfd223ff irq 23 at device 29.0 numa-domain 0 on pci0
  134. usbus1: EHCI version 1.0
  135. usbus1 numa-domain 0 on ehci1
  136. usbus1: 480Mbps High Speed USB v2.0
  137. pcib7: <ACPI PCI-PCI bridge> at device 30.0 numa-domain 0 on pci0
  138. pci7: <ACPI PCI bus> numa-domain 0 on pcib7
  139. vgapci0: <VGA-compatible display> mem 0xdd000000-0xddffffff,0xdf800000-0xdf803fff,0xdf000000-0xdf7fffff irq 16 at device 3.0 numa-domain 0 on pci7
  140. vgapci0: Boot video device
  141. isab0: <PCI-ISA bridge> at device 31.0 numa-domain 0 on pci0
  142. isa0: <ISA bus> numa-domain 0 on isab0
  143. ahci0: <Intel Patsburg AHCI SATA controller> port 0x9070-0x9077,0x9060-0x9063,0x9050-0x9057,0x9040-0x9043,0x9020-0x903f mem 0xdfd21000-0xdfd217ff irq 19 at device 31.2 numa-domain 0 on pci0
  144. ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
  145. ahcich0: <AHCI channel> at channel 0 on ahci0
  146. ahcich1: <AHCI channel> at channel 1 on ahci0
  147. ahcich2: <AHCI channel> at channel 2 on ahci0
  148. ahcich3: <AHCI channel> at channel 3 on ahci0
  149. ahcich4: <AHCI channel> at channel 4 on ahci0
  150. ahcich5: <AHCI channel> at channel 5 on ahci0
  151. ahciem0: <AHCI enclosure management bridge> on ahci0
  152. pcib8: <ACPI Host-PCI bridge> numa-domain 0 on acpi0
  153. pci8: <ACPI PCI bus> numa-domain 0 on pcib8
  154. pci8: <dasp, performance counters> at device 14.1 (no driver attached)
  155. pci8: <dasp, performance counters> at device 19.1 (no driver attached)
  156. pci8: <dasp, performance counters> at device 19.5 (no driver attached)
  157. pcib9: <ACPI Host-PCI bridge> numa-domain 1 on acpi0
  158. pci9: <ACPI PCI bus> numa-domain 1 on pcib9
  159. pcib10: <ACPI PCI-PCI bridge> irq 50 at device 1.0 numa-domain 1 on pci9
  160. pci10: <ACPI PCI bus> numa-domain 1 on pcib10
  161. pcib11: <ACPI PCI-PCI bridge> irq 56 at device 2.0 numa-domain 1 on pci9
  162. pci11: <ACPI PCI bus> numa-domain 1 on pcib11
  163. pcib12: <ACPI PCI-PCI bridge> irq 64 at device 3.0 numa-domain 1 on pci9
  164. pci12: <ACPI PCI bus> numa-domain 1 on pcib12
  165. ioat8: <IVB IOAT Ch0> mem 0xfbf1c000-0xfbf1ffff irq 55 at device 4.0 numa-domain 1 on pci9
  166. ioat8: Capabilities: 2f7<PQ,Extended_APIC_ID,Block_Fill,Move_CRC,DCA,Marker_Skipping,CRC,Page_Break>
  167. ioat9: <IVB IOAT Ch1> mem 0xfbf18000-0xfbf1bfff irq 63 at device 4.1 numa-domain 1 on pci9
  168. ioat9: Capabilities: 2f7<PQ,Extended_APIC_ID,Block_Fill,Move_CRC,DCA,Marker_Skipping,CRC,Page_Break>
  169. ioat10: <IVB IOAT Ch2> mem 0xfbf14000-0xfbf17fff irq 55 at device 4.2 numa-domain 1 on pci9
  170. ioat10: Capabilities: f7<Extended_APIC_ID,Block_Fill,Move_CRC,DCA,Marker_Skipping,CRC,Page_Break>
  171. ioat11: <IVB IOAT Ch3> mem 0xfbf10000-0xfbf13fff irq 63 at device 4.3 numa-domain 1 on pci9
  172. ioat11: Capabilities: f7<Extended_APIC_ID,Block_Fill,Move_CRC,DCA,Marker_Skipping,CRC,Page_Break>
  173. ioat12: <IVB IOAT Ch4> mem 0xfbf0c000-0xfbf0ffff irq 55 at device 4.4 numa-domain 1 on pci9
  174. ioat12: Capabilities: f7<Extended_APIC_ID,Block_Fill,Move_CRC,DCA,Marker_Skipping,CRC,Page_Break>
  175. ioat13: <IVB IOAT Ch5> mem 0xfbf08000-0xfbf0bfff irq 63 at device 4.5 numa-domain 1 on pci9
  176. ioat13: Capabilities: f7<Extended_APIC_ID,Block_Fill,Move_CRC,DCA,Marker_Skipping,CRC,Page_Break>
  177. ioat14: <IVB IOAT Ch6> mem 0xfbf04000-0xfbf07fff irq 55 at device 4.6 numa-domain 1 on pci9
  178. ioat14: Capabilities: f7<Extended_APIC_ID,Block_Fill,Move_CRC,DCA,Marker_Skipping,CRC,Page_Break>
  179. ioat15: <IVB IOAT Ch7> mem 0xfbf00000-0xfbf03fff irq 63 at device 4.7 numa-domain 1 on pci9
  180. ioat15: Capabilities: f7<Extended_APIC_ID,Block_Fill,Move_CRC,DCA,Marker_Skipping,CRC,Page_Break>
  181. pcib13: <ACPI Host-PCI bridge> numa-domain 1 on acpi0
  182. pci13: <ACPI PCI bus> numa-domain 1 on pcib13
  183. pci13: <dasp, performance counters> at device 14.1 (no driver attached)
  184. pci13: <dasp, performance counters> at device 19.1 (no driver attached)
  185. pci13: <dasp, performance counters> at device 19.5 (no driver attached)
  186. acpi_button0: <Power Button> on acpi0
  187. acpi_syscontainer0: <System Container> on acpi0
  188. acpi_syscontainer1: <System Container> on acpi0
  189. uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
  190. uart0: console (57600,n,8,1)
  191. uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
  192. ipmi0: <IPMI System Interface> port 0xca2,0xca3 on acpi0
  193. ipmi0: KCS mode found at io 0xca2 on acpi
  194. ichwd0: <Intel Patsburg watchdog timer> on isa0
  195. orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc87ff pnpid ORM0000 on isa0
  196. coretemp0: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu0
  197. est0: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu0
  198. Timecounters tick every 1.000 msec
  199. ZFS filesystem version: 5
  200. ZFS storage pool version: features support (5000)
  201. ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled
  202. ugen1.1: <Intel EHCI root HUB> at usbus1
  203. ugen0.1: <Intel EHCI root HUB> at usbus0
  204. ipmi0: IPMI device rev. 1, firmware rev. 3.39, version 2.0, device support mask 0xbf
  205. ipmi0: Number of channels 2
  206. ipmi0: Attached watchdog
  207. ipmi0: Establishing power cycle handler
  208. uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
  209. uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
  210. Trying to mount root from zfs:boot-pool/ROOT/default []...
  211. Root mount waiting for: CAM usbus0 usbus1
  212. WARNING: WITNESS option enabled, expect reduced performance.
  213. uhub0: 2 ports with 2 removable, self powered
  214. Root mount waiting for: CAM usbus0 usbus1
  215. uhub1: 2 ports with 2 removable, self powered
  216. ugen1.2: <vendor 0x8087 product 0x0024> at usbus1
  217. uhub2 numa-domain 0 on uhub0
  218. uhub2: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus1
  219. Root mount waiting for: CAM usbus0 usbus1
  220. ugen0.2: <vendor 0x8087 product 0x0024> at usbus0
  221. uhub3 numa-domain 0 on uhub1
  222. uhub3: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus0
  223. uhub2: 8 ports with 8 removable, self powered
  224. Root mount waiting for: CAM usbus0 usbus1
  225. uhub3: 6 ports with 6 removable, self powered
  226. ugen1.3: <Kingston DT 101 G2> at usbus1
  227. umass0 numa-domain 0 on uhub2
  228. umass0: <Kingston DT 101 G2, class 0/0, rev 2.00/1.00, addr 3> on usbus1
  229. umass0: SCSI over Bulk-Only; quirks = 0xc000
  230. umass0:10:0: Attached to scbus10
  231. ugen0.3: <Winbond Electronics Corp Hermon USB hidmouse Device> at usbus0
  232. ums0 numa-domain 0 on uhub3
  233. ums0: <Winbond Electronics Corp Hermon USB hidmouse Device, class 0/0, rev 1.10/0.01, addr 3> on usbus0
  234. ums0: 3 buttons and [Z] coordinates ID=0
  235. ukbd0 numa-domain 0 on uhub3
  236. ukbd0: <Winbond Electronics Corp Hermon USB hidmouse Device, class 0/0, rev 1.10/0.01, addr 3> on usbus0
  237. kbd1 at ukbd0
  238. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  239. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e096ecd0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  240. stack backtrace:
  241. #0 0xffffffff80b49d41 at witness_debugger+0x71
  242. #1 0xffffffff80b4acbd at witness_warn+0x40d
  243. #2 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  244. #3 0xffffffff80a9191a at getenv_string_buffer+0x3a
  245. #4 0xffffffff80a92047 at getenv_quad+0x17
  246. #5 0xffffffff80a92012 at getenv_int+0x12
  247. #6 0xffffffff803e6a07 at daregister+0x1e7
  248. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  249. #8 0xffffffff803e6362 at daasync+0x2c2
  250. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  251. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  252. #11 0xffffffff8038b31b at xpttargettraverse+0x6b
  253. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  254. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  255. #14 0xffffffff803e6049 at dainit+0x19
  256. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  257. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  258. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  259. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  260. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff801ee65f4d0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  261. stack backtrace:
  262. #0 0xffffffff80b49d41 at witness_debugger+0x71
  263. #1 0xffffffff80b4acbd at witness_warn+0x40d
  264. #2da0 at mps0 bus 0 scbus0 target 8 lun 0
  265. da0: <ATA ST3000NM0033-9ZM SN04> Fixed Direct Access SPC-4 SCSI device
  266. da0: Serial Number Z1Y35D6D
  267. da0: 600.000MB/s transfers
  268. da0: Command Queueing enabled
  269. da0: 2861588MB (5860533168 512 byte sectors)
  270. 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  271. #3 0xffffffff80a9191a at getenv_string_buffer+0x3a
  272. #4 0xffffffff80a92047 at getenv_quad+0x17
  273. #5 0xffffffff80a92012 at getenv_int+0x12
  274. #6 0xffffffff803e6a07 at daregister+0x1e7
  275. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  276. #8 0xffffffff803e6362 at daasync+0x2c2
  277. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  278. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  279. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  280. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  281. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  282. #14 0xffffffff803e6049 at dainit+0x19
  283. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  284. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  285. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  286. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  287. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e0971cd0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  288. stack backtrace:
  289. #0 0xffffffff80b49d41 at witness_debugger+0x71
  290. #1 0xffffffff80da1 at mps0 bus 0 scbus0 target 9 lun 0
  291. da1: <ATA ST3000NM0033-9ZM SN06> Fixed Direct Access SPC-4 SCSI device
  292. da1: Serial Number Z1Y3ZS0S
  293. da1: 600.000MB/s transfers
  294. da1: Command Queueing enabled
  295. da1: 2861588MB (5860533168 512 byte sectors)
  296. b4acbd at witness_warn+0x40d
  297. #2 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  298. #3 0xffffffff80a9191a at getenv_string_buffer+0x3a
  299. #4 0xffffffff80a92047 at getenv_quad+0x17
  300. #5 0xffffffff80a92012 at getenv_int+0x12
  301. #6 0xffffffff803e6a07 at daregister+0x1e7
  302. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  303. #8 0xffffffff803e6362 at daasync+0x2c2
  304. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  305. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  306. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  307. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  308. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  309. #14 0xffffffff803e6049 at dainit+0x19
  310. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  311. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  312. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  313. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  314. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff801eea4d4d0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  315. stack backtrace:
  316. #0 0xffffffff80b49d41 at witness_debugger+0x71
  317. #1 0xffffffff80b4acbd at witness_warn+0x40d
  318. #2 0xffffffff80e5c7d8 at uma_zallda2 at mps0 bus 0 scbus0 target 10 lun 0
  319. da2: <ATA ST3000NM0033-9ZM SN06> Fixed Direct Access SPC-4 SCSI device
  320. da2: Serial Number Z1ZAKFLS
  321. da2: 600.000MB/s transfers
  322. da2: Command Queueing enabled
  323. da2: 2861588MB (5860533168 512 byte sectors)
  324. oc_arg+0x38
  325. #3 0xffffffff80a9191a at getenv_string_buffer+0x3a
  326. #4 0xffffffff80a92047 at getenv_quad+0x17
  327. #5 0xffffffff80a92012 at getenv_int+0x12
  328. #6 0xffffffff803e6a07 at daregister+0x1e7
  329. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  330. #8 0xffffffff803e6362 at daasync+0x2c2
  331. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  332. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  333. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  334. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  335. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  336. #14 0xffffffff803e6049 at dainit+0x19
  337. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  338. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  339. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  340. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  341. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e096bcd0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  342. stack backtrace:
  343. #0 0xffffffff80b49d41 at witness_debugger+0x71
  344. #1 0xffffffff80b4acbd at witness_warn+0x40d
  345. #2 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  346. #3 0xffffffff80a919da3 at mps0 bus 0 scbus0 target 11 lun 0
  347. da3: <ATA ST3000NM0033-9ZM 0001> Fixed Direct Access SPC-4 SCSI device
  348. da3: Serial Number Z1Z0E25N
  349. da3: 600.000MB/s transfers
  350. da3: Command Queueing enabled
  351. da3: 2861588MB (5860533168 512 byte sectors)
  352. 1a at getenv_string_buffer+0x3a
  353. #4 0xffffffff80a92047 at getenv_quad+0x17
  354. #5 0xffffffff80a92012 at getenv_int+0x12
  355. #6 0xffffffff803e6a07 at daregister+0x1e7
  356. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  357. #8 0xffffffff803e6362 at daasync+0x2c2
  358. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  359. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  360. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  361. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  362. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  363. #14 0xffffffff803e6049 at dainit+0x19
  364. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  365. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  366. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  367. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  368. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e096acd0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  369. stack backtrace:
  370. #0 0xffffffff80b49d41 at witness_debugger+0x71
  371. #1 0xffffffff80b4acbd at witness_warn+0x40d
  372. #2da4 at mps0 bus 0 scbus0 target 12 lun 0
  373. da4: <ATA ST3000NM0033-9ZM SN06> Fixed Direct Access SPC-4 SCSI device
  374. da4: Serial Number Z1ZAJ256
  375. da4: 600.000MB/s transfers
  376. da4: Command Queueing enabled
  377. da4: 2861588MB (5860533168 512 byte sectors)
  378. 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  379. #3 0xffffffff80a9191a at getenv_string_buffer+0x3a
  380. #4 0xffffffff80a92047 at getenv_quad+0x17
  381. #5 0xffffffff80a92012 at getenv_int+0x12
  382. #6 0xffffffff803e6a07 at daregister+0x1e7
  383. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  384. #8 0xffffffff803e6362 at daasync+0x2c2
  385. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  386. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  387. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  388. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  389. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  390. #14 0xffffffff803e6049 at dainit+0x19
  391. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  392. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  393. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  394. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  395. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e0969cd0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  396. stack backtrace:
  397. #0 0xffffffff80b49d41 at witness_debugger+0x71
  398. #1 0xffffffff80b4acbd at witness_warn+0x40d
  399. #2 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  400. #3 0xffffffff80a919da5 at mps0 bus 0 scbus0 target 13 lun 0
  401. da5: <ATA ST3000NM0033-9ZM 0001> Fixed Direct Access SPC-4 SCSI device
  402. da5: Serial Number Z1Z0EL51
  403. da5: 600.000MB/s transfers
  404. da5: Command Queueing enabled
  405. da5: 2861588MB (5860533168 512 byte sectors)
  406. 1a at getenv_string_buffer+0x3a
  407. #4 0xffffffff80a92047 at getenv_quad+0x17
  408. #5 0xffffffff80a92012 at getenv_int+0x12
  409. #6 0xffffffff803e6a07 at daregister+0x1e7
  410. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  411. #8 0xffffffff803e6362 at daasync+0x2c2
  412. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  413. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  414. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  415. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  416. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  417. #14 0xffffffff803e6049 at dainit+0x19
  418. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  419. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  420. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  421. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  422. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e09654d0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  423. stack backtrace:
  424. #0 0xffffffff80b49d41 at witness_debugger+0x71
  425. #1 0xffffffff80b4acbd at witness_warn+0x40d
  426. #2 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  427. #3 0xffffffff80a919da6 at mps0 bus 0 scbus0 target 14 lun 0
  428. da6: <ATA ST3000NM0033-9ZM SN04> Fixed Direct Access SPC-4 SCSI device
  429. da6: Serial Number Z1Y35LAS
  430. da6: 600.000MB/s transfers
  431. da6: Command Queueing enabled
  432. da6: 2861588MB (5860533168 512 byte sectors)
  433. 1a at getenv_string_buffer+0x3a
  434. #4 0xffffffff80a92047 at getenv_quad+0x17
  435. #5 0xffffffff80a92012 at getenv_int+0x12
  436. #6 0xffffffff803e6a07 at daregister+0x1e7
  437. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  438. #8 0xffffffff803e6362 at daasync+0x2c2
  439. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  440. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  441. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  442. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  443. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  444. #14 0xffffffff803e6049 at dainit+0x19
  445. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  446. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  447. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  448. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  449. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff801eea474d0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  450. stack backtrace:
  451. #0 0xffffffff80b49d41 at witness_debugger+0x71
  452. #1 0xffffffff80b4acbd at witness_warn+0x40d
  453. #2 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  454. #3 0xffffffff80a919da7 at mps0 bus 0 scbus0 target 15 lun 0
  455. da7: <ATA ST3000NM0033-9ZM SN06> Fixed Direct Access SPC-4 SCSI device
  456. da7: Serial Number Z1ZAMV8J
  457. da7: 600.000MB/s transfers
  458. da7: Command Queueing enabled
  459. da7: 2861588MB (5860533168 512 byte sectors)
  460. 1a at getenv_string_buffer+0x3a
  461. #4 0xffffffff80a92047 at getenv_quad+0x17
  462. #5 0xffffffff80a92012 at getenv_int+0x12
  463. #6 0xffffffff803e6a07 at daregister+0x1e7
  464. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  465. #8 0xffffffff803e6362 at daasync+0x2c2
  466. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  467. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  468. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  469. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  470. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  471. #14 0xffffffff803e6049 at dainit+0x19
  472. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  473. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  474. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  475. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  476. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e0963cd0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  477. stack backtrace:
  478. #0 0xffffffff80b49d41 at witness_debugger+0x71
  479. #1 0xffffffff80b4acbd at witness_warn+0x40d
  480. #2 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  481. #3 0xffffffff80a9191a at getenv_string_buffer+0x3a
  482. #4 0xffffffff80a92047 at getenv_quad+0x17
  483. #5 0xffffffff80a92012 at getenv_int+0x12
  484. #6 0xffffffff803e6a07 at daregister+0x1e7
  485. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  486. #8 0xfffffffda8 at mps0 bus 0 scbus0 target 16 lun 0
  487. da8: <ATA ST3000NM0033-9ZM SN03> Fixed Direct Access SPC-4 SCSI device
  488. da8: Serial Number Z1Z0RNA1
  489. da8: 600.000MB/s transfers
  490. da8: Command Queueing enabled
  491. da8: 2861588MB (5860533168 512 byte sectors)
  492. f803e6362 at daasync+0x2c2
  493. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  494. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  495. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  496. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  497. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  498. #14 0xffffffff803e6049 at dainit+0x19
  499. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  500. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  501. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  502. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  503. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e0962cd0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  504. stack backtrace:
  505. #0 0xffffffff80b49d41 at witness_debugger+0x71
  506. #1 0xffffffff80b4acbd at witness_warn+0x40d
  507. #2 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  508. #3 0xffffffff80a919da9 at mps0 bus 0 scbus0 target 17 lun 0
  509. da9: <ATA ST3000NM0033-9ZM SN06> Fixed Direct Access SPC-4 SCSI device
  510. da9: Serial Number Z1Y43MTQ
  511. da9: 600.000MB/s transfers
  512. da9: Command Queueing enabled
  513. da9: 2861588MB (5860533168 512 byte sectors)
  514. 1a at getenv_string_buffer+0x3a
  515. #4 0xffffffff80a92047 at getenv_quad+0x17
  516. #5 0xffffffff80a92012 at getenv_int+0x12
  517. #6 0xffffffff803e6a07 at daregister+0x1e7
  518. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  519. #8 0xffffffff803e6362 at daasync+0x2c2
  520. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  521. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  522. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  523. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  524. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  525. #14 0xffffffff803e6049 at dainit+0x19
  526. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  527. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  528. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  529. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  530. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e0961cd0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  531. stack backtrace:
  532. #0 0xffffffff80b49d41 at witness_debugger+0x71
  533. #1 0xffffffff80b4acbd at witness_warn+0x40d
  534. #2 0xffffffff80e5c7d8 at uma_zallda10 at mps0 bus 0 scbus0 target 18 lun 0
  535. da10: <ATA ST3000NM0033-9ZM SN06> Fixed Direct Access SPC-4 SCSI device
  536. da10: Serial Number Z1ZAL9P0
  537. da10: 600.000MB/s transfers
  538. da10: Command Queueing enabled
  539. da10: 2861588MB (5860533168 512 byte sectors)
  540. oc_arg+0x38
  541. #3 0xffffffff80a9191a at getenv_string_buffer+0x3a
  542. #4 0xffffffff80a92047 at getenv_quad+0x17
  543. #5 0xffffffff80a92012 at getenv_int+0x12
  544. #6 0xffffffff803e6a07 at daregister+0x1e7
  545. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  546. #8 0xffffffff803e6362 at daasync+0x2c2
  547. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  548. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  549. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  550. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  551. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  552. #14 0xffffffff803e6049 at dainit+0x19
  553. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  554. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  555. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  556. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  557. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e095d4d0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  558. stack backtrace:
  559. #0 0xffffffff80b49d41 at witness_debugger+0x71
  560. #1 0xffffffff80b4acbd at witness_warn+0x40d
  561. #2 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  562. #3 0xffffffff80a919da11 at mps0 bus 0 scbus0 target 19 lun 0
  563. da11: <ATA ST3000NM0033-9ZM 0001> Fixed Direct Access SPC-4 SCSI device
  564. da11: Serial Number Z1Z0ETPQ
  565. da11: 600.000MB/s transfers
  566. da11: Command Queueing enabled
  567. da11: 2861588MB (5860533168 512 byte sectors)
  568. 1a at getenv_string_buffer+0x3a
  569. #4 0xffffffff80a92047 at getenv_quad+0x17
  570. #5 0xffffffff80a92012 at getenv_int+0x12
  571. #6 0xffffffff803e6a07 at daregister+0x1e7
  572. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  573. #8 0xffffffff803e6362 at daasync+0x2c2
  574. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  575. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  576. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  577. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  578. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  579. #14 0xffffffff803e6049 at dainit+0x19
  580. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  581. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  582. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  583. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  584. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff801eea414d0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  585. stack backtrace:
  586. #0 0xffffffff80b49d41 at witness_debugger+0x71
  587. #1 0xffffffff80b4acbd at witness_warn+0x40d
  588. #2 0xffffffff80e5c7d8 at uma_zallda12 at mps0 bus 0 scbus0 target 20 lun 0
  589. da12: <ATA ST3000NM0033-9ZM SN06> Fixed Direct Access SPC-4 SCSI device
  590. da12: Serial Number Z1ZAMVZC
  591. da12: 600.000MB/s transfers
  592. da12: Command Queueing enabled
  593. da12: 2861588MB (5860533168 512 byte sectors)
  594. oc_arg+0x38
  595. #3 0xffffffff80a9191a at getenv_string_buffer+0x3a
  596. #4 0xffffffff80a92047 at getenv_quad+0x17
  597. #5 0xffffffff80a92012 at getenv_int+0x12
  598. #6 0xffffffff803e6a07 at daregister+0x1e7
  599. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  600. #8 0xffffffff803e6362 at daasync+0x2c2
  601. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  602. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  603. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  604. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  605. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  606. #14 0xffffffff803e6049 at dainit+0x19
  607. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  608. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  609. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  610. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  611. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e095bcd0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  612. stack backtrace:
  613. #0 0xffffffff80b49d41 at witness_debugger+0x71
  614. #1 0xffffffff80b4acbd at witness_warn+0x40d
  615. #2 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  616. #3 0xffffffff80a9191a at getenv_string_buffer+0x3ada13 at mps0 bus 0 scbus0 target 21 lun 0
  617. da13: <ATA ST3000NM0033-9ZM SN04> Fixed Direct Access SPC-4 SCSI device
  618. da13: Serial Number Z1Y35CDJ
  619. da13: 600.000MB/s transfers
  620. da13: Command Queueing enabled
  621. da13: 2861588MB (5860533168 512 byte sectors)
  622.  
  623. #4 0xffffffff80a92047 at getenv_quad+0x17
  624. #5 0xffffffff80a92012 at getenv_int+0x12
  625. #6 0xffffffff803e6a07 at daregister+0x1e7
  626. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  627. #8 0xffffffff803e6362 at daasync+0x2c2
  628. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  629. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  630. uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port 7
  631. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  632. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  633. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  634. #14 0xffffffff803e6049 at dainit+0x19
  635. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  636. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  637. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  638. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  639. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e095acd0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  640. stack backtrace:
  641. #0 0xffffffff80b49d41 at witness_debugger+0x71
  642. #1 0xffffffff80b4acbd at witness_warn+0x40d
  643. #2 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  644. #3 0xffffffff80a919da14 at mps0 bus 0 scbus0 target 22 lun 0
  645. da14: <ATA ST3000NM0033-9ZM 0001> Fixed Direct Access SPC-4 SCSI device
  646. da14: Serial Number Z1Y01CM8
  647. da14: 600.000MB/s transfers
  648. da14: Command Queueing enabled
  649. da14: 2861588MB (5860533168 512 byte sectors)
  650. 1a at getenv_string_buffer+0x3a
  651. #4 0xffffffff80a92047 at getenv_quad+0x17
  652. #5 0xffffffff80a92012 at getenv_int+0x12
  653. #6 0xffffffff803e6a07 at daregister+0x1e7
  654. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  655. #8 0xffffffff803e6362 at daasync+0x2c2
  656. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  657. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  658. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  659. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  660. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  661. #14 0xffffffff803e6049 at dainit+0x19
  662. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  663. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  664. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  665. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  666. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e0959cd0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  667. stack backtrace:
  668. #0 0xffffffff80b49d41 at witness_debugger+0x71
  669. #1 0xffffffff80b4acbd at witness_warn+0x40d
  670. #2 0xffffffff80e5c7d8 at uma_zallda15 at mps0 bus 0 scbus0 target 23 lun 0
  671. da15: <ATA ST3000NM0033-9ZM SN03> Fixed Direct Access SPC-4 SCSI device
  672. da15: Serial Number Z1Z0QWBM
  673. da15: 600.000MB/s transfers
  674. da15: Command Queueing enabled
  675. da15: 2861588MB (5860533168 512 byte sectors)
  676. oc_arg+0x38
  677. #3 0xffffffff80a9191a at getenv_string_buffer+0x3a
  678. #4 0xffffffff80a92047 at getenv_quad+0x17
  679. #5 0xffffffff80a92012 at getenv_int+0x12
  680. #6 0xffffffff803e6a07 at daregister+0x1e7
  681. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  682. #8 0xffffffff803e6362 at daasync+0x2c2
  683. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  684. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  685. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  686. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  687. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  688. #14 0xffffffff803e6049 at dainit+0x19
  689. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  690. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  691. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  692. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  693. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e09554d0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  694. stack backtrace:
  695. #0 0xffffffff80b49d41 at witness_debugger+0x71
  696. #1 0xffffffff80b4acbd at witness_warn+0x40d
  697. #2 0xffffffff80e5c7d8 at uma_zallda16 at mps0 bus 0 scbus0 target 24 lun 0
  698. da16: <ATA ST3000NM0033-9ZM SN06> Fixed Direct Access SPC-4 SCSI device
  699. da16: Serial Number S1Z1TT92
  700. da16: 600.000MB/s transfers
  701. da16: Command Queueing enabled
  702. da16: 2861588MB (5860533168 512 byte sectors)
  703. oc_arg+0x38
  704. #3 0xffffffff80a9191a at getenv_string_buffer+0x3a
  705. #4 0xffffffff80a92047 at getenv_quad+0x17
  706. #5 0xffffffff80a92012 at getenv_int+0x12
  707. #6 0xffffffff803e6a07 at daregister+0x1e7
  708. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  709. #8 0xffffffff803e6362 at daasync+0x2c2
  710. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  711. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  712. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  713. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  714. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  715. #14 0xffffffff803e6049 at dainit+0x19
  716. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  717. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  718. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  719. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  720. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff801eea3a4d0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  721. stack backtrace:
  722. #0 0xffffffff80b49d41 at witness_debugger+0x71
  723. #1 0xffffffff80b4acbd at witness_warn+0x40d
  724. #2 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  725. #3 0xffffffff80a9191a at getenv_string_buffer+0x3a
  726. #4 0xffffffff80a92047 at getenv_quad+0x17
  727. #5 0xffffffff80a92012 at getenv_int+0x12
  728. #6 0xffffffff803e6a07 at daregister+0x1e7
  729. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  730. #8 0xffffffff803e6362 at daasync+0x2c2
  731. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  732. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  733. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  734. #12 0xffffffff8038ae5f at xptbustraverse+0x6fda17 at mps0 bus 0 scbus0 target 25 lun 0
  735. da17: <ATA ST3000NM0033-9ZM SN03> Fixed Direct Access SPC-4 SCSI device
  736. da17: Serial Number Z1Z0SRZX
  737. da17: 600.000MB/s transfers
  738. da17: Command Queueing enabled
  739. da17: 2861588MB (5860533168 512 byte sectors)
  740.  
  741. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  742. #14 0xffffffff803e6049 at dainit+0x19
  743. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  744. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  745. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  746. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  747. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e0953cd0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  748. stack backtrace:
  749. #0 0xffffffff80b49d41 at witness_debugger+0x71
  750. #1 0xffffffff80b4acbd at witness_warn+0x40d
  751. #2 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  752. #3 0xffffffff80a919da18 at mps0 bus 0 scbus0 target 26 lun 0
  753. da18: <ATA ST3000NM0033-9ZM SN03> Fixed Direct Access SPC-4 SCSI device
  754. da18: Serial Number Z1Z0T9QY
  755. da18: 600.000MB/s transfers
  756. da18: Command Queueing enabled
  757. da18: 2861588MB (5860533168 512 byte sectors)
  758. 1a at getenv_string_buffer+0x3a
  759. #4 0xffffffff80a92047 at getenv_quad+0x17
  760. #5 0xffffffff80a92012 at getenv_int+0x12
  761. #6 0xffffffff803e6a07 at daregister+0x1e7
  762. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  763. #8 0xffffffff803e6362 at daasync+0x2c2
  764. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  765. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  766. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  767. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  768. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  769. #14 0xffffffff803e6049 at dainit+0x19
  770. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  771. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  772. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  773. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  774. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e0952cd0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  775. stack backtrace:
  776. #0 0xffffffff80b49d41 at witness_debugger+0x71
  777. #1 0xffffffff80b4acbd at witness_warn+0x40d
  778. #2da19 at mps0 bus 0 scbus0 target 27 lun 0
  779. da19: <ATA ST3000NM0033-9ZM SN06> Fixed Direct Access SPC-4 SCSI device
  780. da19: Serial Number Z1ZAMTFN
  781. da19: 600.000MB/s transfers
  782. da19: Command Queueing enabled
  783. da19: 2861588MB (5860533168 512 byte sectors)
  784. 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  785. #3 0xffffffff80a9191a at getenv_string_buffer+0x3a
  786. #4 0xffffffff80a92047 at getenv_quad+0x17
  787. #5 0xffffffff80a92012 at getenv_int+0x12
  788. #6 0xffffffff803e6a07 at daregister+0x1e7
  789. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  790. #8 0xffffffff803e6362 at daasync+0x2c2
  791. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  792. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  793. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  794. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  795. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  796. #14 0xffffffff803e6049 at dainit+0x19
  797. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  798. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  799. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  800. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  801. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e0951cd0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  802. stack backtrace:
  803. #0 0xffffffff80b49d41 at witness_debugger+0x71
  804. #1 0xffffffff80b4acbd at witness_warn+0x40d
  805. #2 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  806. #3 0xffffffff80a9191a at getenv_string_buffer+0x3ada20 at mps0 bus 0 scbus0 target 28 lun 0
  807. da20: <ATA ST3000NM0033-9ZM SN04> Fixed Direct Access SPC-4 SCSI device
  808. da20: Serial Number Z1Z83LGN
  809. da20: 600.000MB/s transfers
  810. da20: Command Queueing enabled
  811. da20: 2861588MB (5860533168 512 byte sectors)
  812.  
  813. #4 0xffffffff80a92047 at getenv_quad+0x17
  814. #5 0xffffffff80a92012 at getenv_int+0x12
  815. #6 0xffffffff803e6a07 at daregister+0x1e7
  816. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  817. #8 0xffffffff803e6362 at daasync+0x2c2
  818. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  819. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  820. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  821. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  822. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  823. #14 0xffffffff803e6049 at dainit+0x19
  824. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  825. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  826. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  827. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  828. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff810e094d4d0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  829. stack backtrace:
  830. #0 0xffffffff80b49d41 at witness_debugger+0x71
  831. #1 0xffffffff80da21 at mps0 bus 0 scbus0 target 29 lun 0
  832. da21: <ATA ST3000NM0033-9ZM SN03> Fixed Direct Access SPC-4 SCSI device
  833. da21: Serial Number Z1Z0RXBT
  834. da21: 600.000MB/s transfers
  835. da21: Command Queueing enabled
  836. da21: 2861588MB (5860533168 512 byte sectors)
  837. b4acbd at witness_warn+0x40d
  838. #2 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  839. #3 0xffffffff80a9191a at getenv_string_buffer+0x3a
  840. #4 0xffffffff80a92047 at getenv_quad+0x17
  841. #5 0xffffffff80a92012 at getenv_int+0x12
  842. #6 0xffffffff803e6a07 at daregister+0x1e7
  843. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  844. #8 0xffffffff803e6362 at daasync+0x2c2
  845. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  846. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  847. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  848. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  849. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  850. #14 0xffffffff803e6049 at dainit+0x19
  851. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  852. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  853. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  854. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  855. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff801eea344d0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  856. stack backtrace:
  857. #0 0xffffffff80b49d41 at witness_debugger+0x71
  858. #1 0xffffffff80b4acbd at witness_warn+0x40d
  859. #2 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  860. #3 0xffffffff80a919da22 at mps0 bus 0 scbus0 target 30 lun 0
  861. da22: <ATA ST3000NM0033-9ZM SN06> Fixed Direct Access SPC-4 SCSI device
  862. da22: Serial Number Z1ZAN4E7
  863. da22: 600.000MB/s transfers
  864. da22: Command Queueing enabled
  865. da22: 2861588MB (5860533168 512 byte sectors)
  866. 1a at getenv_string_buffer+0x3a
  867. #4 0xffffffff80a92047 at getenv_quad+0x17
  868. #5 0xffffffff80a92012 at getenv_int+0x12
  869. #6 0xffffffff803e6a07 at daregister+0x1e7
  870. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  871. #8 0xffffffff803e6362 at daasync+0x2c2
  872. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  873. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  874. #11 0xffffffff8038b37b at xpttargettraverse+0xcb
  875. #12 0xffffffff8038ae5f at xptbustraverse+0x6f
  876. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  877. #14 0xffffffff803e6049 at dainit+0x19
  878. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  879. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  880. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  881. uma_zalloc_arg: zone "kenv" with the following non-sleepable locks held:
  882. exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff801eea324d0) locked @ /truenas-releng/freenas/_BE/os/sys/cam/cam_xpt.c:2340
  883. stack backtrace:
  884. #0 0xffffffff80b49d41 at witness_debugger+0x71
  885. #1 0xffffffff80b4acbd at witness_warn+0x40d
  886. #2da23 at mps0 bus 0 scbus0 target 31 lun 0
  887. da23: <ATA ST3000NM0033-9ZM SN04> Fixed Direct Access SPC-4 SCSI device
  888. da23: Serial Number Z1Z82VAA
  889. da23: 600.000MB/s transfers
  890. da23: Command Queueing enabled
  891. da23: 2861588MB (5860533168 512 byte sectors)
  892. 0xffffffff80e5c7d8 at uma_zalloc_arg+0x38
  893. #3 0xffffffff80a9191a at getenv_string_buffer+0x3a
  894. #4 0xffffffff80a92047 at getenv_quad+0x17
  895. #5 0xffffffff80a92012 at getenv_int+0x12
  896. #6 0xffffffff803e6a07 at daregister+0x1e7
  897. #7 0xffffffff8037f1ab at cam_periph_alloc+0x57b
  898. #8 0xffffffff803e6362 at daasync+0x2c2
  899. #9 0xffffffff8038985a at xptsetasyncfunc+0x11a
  900. #10 0xffffffff8038b595 at xptdevicetraverse+0xa5
  901. #11 0xffffffff8038b31b at xpttargettraverse+0x6b
  902. #12 0xffffffff8038aec3 at xptbustraverse+0xd3
  903. #13 0xffffffff803896ce at xpt_register_async+0x1fe
  904. #14 0xffffffff803e6049 at dainit+0x19
  905. #15 0xffffffff8037ec03 at periphdriver_init+0x43
  906. #16 0xffffffff803893f2 at xpt_finishconfig_task+0x12
  907. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  908. da24 at umass-sim0 bus 0 scbus10 target 0 lun 0
  909. da24: <Kingston DT 101 G2 PMAP> Removable Direct Access SCSI device
  910. da24: Serial Number 001CC0EC350FEC21062B000D
  911. da24: 40.000MB/s transfers
  912. da24: 14883MB (30481152 512 byte sectors)
  913. da24: quirks=0x2<NO_6_BYTE>
  914. ses0 at mps0 bus 0 scbus0 target 32 lun 0
  915. ses0: <LSI SAS2X36 0e0b> Fixed Enclosure Services SPC-3 SCSI device
  916. ses0: Serial Number
  917. ses0: 600.000MB/s transfers
  918. ses0: Command Queueing enabled
  919. ses0: SES Device
  920. ses1 at ahciem0 bus 0 scbus8 target 0 lun 0
  921. ses1: <AHCI SGPIO Enclosure 2.00 0001> SEMB S-E-S 2.00 device
  922. ses1: SEMB SES Device
  923. ada0 at ahcich1 bus 0 scbus3 target 0 lun 0
  924. ada0: <KINGSTON SEDC500R960G SCEKJ2.3> ACS-3 ATA SATA 3.x device
  925. ada0: Serial Number 50026B72822767EF
  926. ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
  927. ada0: Command Queueing enabled
  928. ada0: 915715MB (1875385008 512 byte sectors)
  929. ses1: pass25,ada0 in 'Slot 01', SATA Slot: scbus3 target 0
  930. ses0: da0,pass0 in 'Slot 01', SAS Slot: 1 phys at slot 0
  931. ses0: phy 0: SATA device
  932. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b8c
  933. ses0: da1,pass1 in 'Slot 02', SAS Slot: 1 phys at slot 1
  934. ses0: phy 0: SATA device
  935. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b8d
  936. ses0: da2,pass2 in 'Slot 03', SAS Slot: 1 phys at slot 2
  937. ses0: phy 0: SATA device
  938. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b8e
  939. ses0: da3,pass3 in 'Slot 04', SAS Slot: 1 phys at slot 3
  940. ses0: phy 0: SATA device
  941. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b8f
  942. ses0: da4,pass4 in 'Slot 05', SAS Slot: 1 phys at slot 4
  943. ses0: phy 0: SATA device
  944. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b90
  945. ses0: da5,pass5 in 'Slot 06', SAS Slot: 1 phys at slot 5
  946. ses0: phy 0: SATA device
  947. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b91
  948. ses0: da6,pass6 in 'Slot 07', SAS Slot: 1 phys at slot 6
  949. ses0: phy 0: SATA device
  950. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b92
  951. ses0: da7,pass7 in 'Slot 08', SAS Slot: 1 phys at slot 7
  952. ses0: phy 0: SATA device
  953. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b93
  954. ses0: da8,pass8 in 'Slot 09', SAS Slot: 1 phys at slot 8
  955. ses0: phy 0: SATA device
  956. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b94
  957. ses0: da9,pass9 in 'Slot 10', SAS Slot: 1 phys at slot 9
  958. ses0: phy 0: SATA device
  959. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b95
  960. ses0: da10,pass10 in 'Slot 11', SAS Slot: 1 phys at slot 10
  961. ses0: phy 0: SATA device
  962. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b96
  963. ses0: da11,pass11 in 'Slot 12', SAS Slot: 1 phys at slot 11
  964. ses0: phy 0: SATA device
  965. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b97
  966. ses0: da12,pass12 in 'Slot 13', SAS Slot: 1 phys at slot 12
  967. ses0: phy 0: SATA device
  968. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b98
  969. ses0: da13,pass13 in 'Slot 14', SAS Slot: 1 phys at slot 13
  970. ses0: phy 0: SATA device
  971. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b99
  972. ses0: da14,pass14 in 'Slot 15', SAS Slot: 1 phys at slot 14
  973. ses0: phy 0: SATA device
  974. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b9a
  975. ses0: da15,pass15 in 'Slot 16', SAS Slot: 1 phys at slot 15
  976. ses0: phy 0: SATA device
  977. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b9b
  978. ses0: da16,pass16 in 'Slot 17', SAS Slot: 1 phys at slot 16
  979. ses0: phy 0: SATA device
  980. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b9c
  981. ses0: da17,pass17 in 'Slot 18', SAS Slot: 1 phys at slot 17
  982. ses0: phy 0: SATA device
  983. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b9d
  984. ses0: da18,pass18 in 'Slot 19', SAS Slot: 1 phys at slot 18
  985. ses0: phy 0: SATA device
  986. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b9e
  987. ses0: da19,pass19 in 'Slot 20', SAS Slot: 1 phys at slot 19
  988. ses0: phy 0: SATA device
  989. ses0: phy 0: parent 5003048000459bbf addr 5003048000459b9f
  990. ses0: da20,pass20 in 'Slot 21', SAS Slot: 1 phys at slot 20
  991. ses0: phy 0: SATA device
  992. ses0: phy 0: parent 5003048000459bbf addr 5003048000459ba0
  993. ses0: da21,pass21 in 'Slot 22', SAS Slot: 1 phys at slot 21
  994. ses0: phy 0: SATA device
  995. ses0: phy 0: parent 5003048000459bbf addr 5003048000459ba1
  996. ses0: da22,pass22 in 'Slot 23', SAS Slot: 1 phys at slot 22
  997. ses0: phy 0: SATA device
  998. ses0: phy 0: parent 5003048000459bbf addr 5003048000459ba2
  999. ses0: da23,pass23 in 'Slot 24', SAS Slot: 1 phys at slot 23
  1000. ses0: phy 0: SATA device
  1001. ses0: phy 0: parent 5003048000459bbf addr 5003048000459ba3
  1002. GEOM: da21: the primary GPT table is corrupt or invalid.
  1003. GEOM: da21: using the secondary instead -- recovery strongly advised.
  1004. lock order reversal:
  1005. 1st 0xfffff801ee9f9228 db->db_rwlock (db->db_rwlock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:1256
  1006. 2nd 0xfffff810e090e938 dn->dn_mtx (dn->dn_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dnode.c:2303
  1007. stack backtrace:
  1008. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1009. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1010. #2 0xffffffff82974608 at dnode_block_freed+0xb8
  1011. #3 0xffffffff8294331a at dbuf_read_impl+0x3ca
  1012. #4 0xffffffff82939862 at dbuf_read_+0x622
  1013. #5 0xffffffff82974fa8 at dnode_hold_impl+0x2e8
  1014. #6 0xffffffff82950c78 at dmu_buf_hold_noread+0x28
  1015. #7 0xffffffff82950e7c at dmu_buf_hold+0x1c
  1016. #8 0xffffffff82a8e8ce at zap_lockdir+0x2e
  1017. #9 0xffffffff82a9006c at zap_lookup_norm+0x3c
  1018. #10 0xffffffff82a90021 at zap_lookup+0x11
  1019. #11 0xffffffff829e699a at spa_ld_trusted_config+0x4a
  1020. #12 0xffffffff829e5393 at spa_ld_mos_with_trusted_config+0x33
  1021. #13 0xffffffff829e355c at spa_load_impl+0x7c
  1022. #14 0xffffffff829dbbb3 at spa_load+0x53
  1023. #15 0xffffffff829db315 at spa_load_best+0x65
  1024. #16 0xffffffff829d7521 at spa_open_common+0x131
  1025. #17 0xffffffff829a1fc1 at dsl_pool_hold+0x21
  1026. lock order reversal:
  1027. 1st 0xfffff810e0c0d098 db->db_mtx (db->db_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:2819
  1028. 2nd 0xfffff801eeac5118 dn->dn_mtx (dn->dn_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:2847
  1029. stack backtrace:
  1030. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1031. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1032. #2 0xffffffff8294506f at dbuf_dirty_compute_state+0x11f
  1033. #3 0xffffffff8293b34f at dbuf_dirty+0xdf
  1034. #4 0xffffffff82981126 at dsl_dataset_sync+0x56
  1035. #5 0xffffffff829a069a at dsl_pool_sync+0xca
  1036. #6 0xffffffff829e0ea4 at spa_sync+0x1014
  1037. #7 0xffffffff829f91e4 at txg_sync_thread+0x484
  1038. #8 0xffffffff80aa08d0 at fork_exit+0x80
  1039. #9 0xffffffff80fcffce at fork_trampoline+0xe
  1040. lock order reversal:
  1041. lock order reversal:
  1042. lock order reversal:
  1043. lock order reversal:
  1044. 1st 0xfffff810e0927900 mls->mls_lock (mls->mls_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/multilist.c:288
  1045. 1st 0xfffff810e0927a00 mls->mls_lock (mls->mls_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs 2nd 0xfffff81080896938 dn->dn_mtx (dn->dn_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dmu_o/multilist.c:288
  1046. bjset.c:1517
  1047. 1st 0xfffff810e0927a40 mls->mls_lock (mls->mls_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs 1st 0xfffff810e0927980 mls->mls_lock (mls->mls_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/multilist.c:288
  1048. 2nd 0xfffff81080675210 buf->b_evict_lock (buf->b_evict_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/arc.c:6682
  1049. stack backtrace:
  1050. #0 0xffffffff80b49d41 at witnesstack backtrace:
  1051. /multilist.c:288
  1052. s_debugger+0x71
  1053. 2nd 0xfffff80111e24000 dn->dn_struct_rwlock (dn->dn_struct_rwlock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8#1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1054. 2nd 0xfffff810e0c0d250 db->db_mtx (db->db_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.#0 0xffffffff80b49d41 at witnes#2 0xffffffff8292897a at arc_released+0x2a
  1055. s_debugger+0x71
  1056. b9c/module/zfs/dnode.c:275
  1057. stack backtrace:
  1058. #3 0xffffffff82978b84 at dnode_sync+0x94
  1059. #0 0xffffffff80b49d41 at witnes#4 0xffffffff82c:5538
  1060. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1061. s_debugger+0x71
  1062. stack backtrace:
  1063. #0 0xffffffff80b49d41 at witnes#2 0xffffffff82959d0d at sync_dnodes_task+0x12d
  1064. s_debugger+0x71
  1065. 959c3f at sync_dnodes_task+0x5f#3 0xffffffff828d7b7f at taskq_run+0x1f
  1066. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1067.  
  1068. #4 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1069. #5 0xffffffff828d7b7f at taskq_run+0x1f
  1070. #1 0xffffffff80#6 0xffffffff80b3c868aeaec7 at _sx_slock_int+0x67
  1071. at taskqueue_run_locked+0x168
  1072. #5 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1073. #2 0xffffffff829420fc at dbuf_sync_leaf+0x16c
  1074. #7 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1075. #6 0xffffffff80aa08d0 at fork_e#3 0xffffffff82941b5b at dbuf_sync_list+0xbb
  1076. #8 0xfffffxit+0x80
  1077. #2 0xffffffff829723b8 at dnode_verify+0x98
  1078. #7 0xffffffff80fcffce at fork_trampoline+0xe
  1079. #3 0xffffffff82978b6e at dnode_sync+0x7e
  1080. #4 0xffffffff82979a07 at dnode_sync+0xf17
  1081. #4 0xfffffffff80aa08d0 at fork_exit+0x80
  1082. #5 0xffffffff82959c3f at sync_dnodes_task+0x5f
  1083. #9 0xffffffff80fcffce at fork_trampoline+0xe
  1084. #6 0xffffffff828d7b7f at taskq_run+0x1f
  1085. ff82959c3f at sync_dnodes_task+#7 0xffffffff80b3c868 at taskqueue0x5f
  1086. _run_locked+0x168
  1087. #5 0xffffffff828d7b7f at taskq_run+0x1f
  1088. #8 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1089. #6 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1090. #9 0xffffffff80aa08d0 at fork_exit+0x80
  1091. #7 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1092. #10 0xffffffff80fcffce at fork_trampoline+0xe
  1093. #8 0xffffffff80aa08d0 at fork_exit+0x80
  1094. #9 0xffffffff80fcffce at fork_trampoline+0xe
  1095. lock order reversal:
  1096. 1st 0xfffff810e0927a40 mls->mls_lock (mls->mls_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/multilist.c:288
  1097. 2nd 0xfffff801eeb36000 ms->ms_lock (ms->ms_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/metaslab.c:5992
  1098. stack backtrace:
  1099. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1100. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1101. #2 0xffffffff829bfac7 at metaslab_check_free_impl+0x107
  1102. #3 0xffffffff829c1bee at metaslab_check_free+0x5e
  1103. #4 0xffffffff82aba907 at zio_free+0x47
  1104. #5 0xffffffff8297c4b5 at dsl_dataset_block_kill+0x725
  1105. #6 0xffffffff8297a072 at free_blocks+0xd2
  1106. #7 0xffffffff8297a444 at dnode_sync_free_range+0xf4
  1107. #8 0xffffffff829cc252 at range_tree_walk+0x62
  1108. #9 0xffffffff829790ca at dnode_sync+0x5da
  1109. #10 0xffffffff82959c3f at sync_dnodes_task+0x5f
  1110. #11 0xffffffff828d7b7f at taskq_run+0x1f
  1111. #12 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1112. #13 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1113. #14 0xffffffff80aa08d0 at fork_exit+0x80
  1114. #15 0xffffffff80fcffce at fork_trampoline+0xe
  1115. lock order reversal:
  1116. 1st 0xfffff810e0927a40 mls->mls_lock (mls->mls_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/multilist.c:288
  1117. 2nd 0xffffffff82cb9b88 buf_hash_table.ht_locks[i].ht_lock (buf_hash_table.ht_locks[i].ht_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/arc.c:1012
  1118. stack backtrace:
  1119. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1120. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1121. #2 0xffffffff82923d1a at buf_hash_find+0xba
  1122. #3 0xffffffff829269af at arc_freed+0x2f
  1123. #4 0xffffffff82abaa2c at zio_free_sync+0x6c
  1124. #5 0xffffffff82aba95b at zio_free+0x9b
  1125. #6 0xffffffff8297c4b5 at dsl_dataset_block_kill+0x725
  1126. #7 0xffffffff8297a072 at free_blocks+0xd2
  1127. #8 0xffffffff8297a444 at dnode_sync_free_range+0xf4
  1128. #9 0xffffffff829cc252 at range_tree_walk+0x62
  1129. #10 0xffffffff829790ca at dnode_sync+0x5da
  1130. #11 0xffffffff82959c3f at sync_dnodes_task+0x5f
  1131. #12 0xffffffff828d7b7f at taskq_run+0x1f
  1132. #13 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1133. #14 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1134. #15 0xffffffff80aa08d0 at fork_exit+0x80
  1135. #16 0xffffffff80fcffce at fork_trampoline+0xe
  1136. lock order reversal:
  1137. 1st 0xfffff810e0927a40 mls->mls_lock (mls->mls_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/multilist.c:288
  1138. 2nd 0xfffff810e0c2e728 dn->dn_dbufs_mtx (dn->dn_dbufs_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dnode_sync.c:463
  1139. stack backtrace:
  1140. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1141. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1142. #2 0xffffffff829788c1 at dnode_evict_dbufs+0x41
  1143. #3 0xffffffff829791c0 at dnode_sync+0x6d0
  1144. #4 0xffffffff82959c3f at sync_dnodes_task+0x5f
  1145. #5 0xffffffff828d7b7f at taskq_run+0x1f
  1146. #6 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1147. #7 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1148. #8 0xffffffff80aa08d0 at fork_exit+0x80
  1149. #9 0xffffffff80fcffce at fork_trampoline+0xe
  1150. lock order reversal:
  1151. 1st 0xfffff810813fc248 zfs (zfs) @ /truenas-releng/freenas/_BE/os/sys/kern/vfs_lookup.c:693
  1152. 2nd 0xfffffe0167315bb0 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/os/freebsd/zfs/zfs_znode.c:925
  1153. stack backtrace:
  1154. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1155. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1156. #2 0xffffffff829035a6 at zfs_zget+0x56
  1157. #3 0xffffffff828e9bd9 at zfs_dirent_lookup+0x169
  1158. #4 0xffffffff828e9d3b at zfs_dirlook+0xdb
  1159. #5 0xffffffff82900620 at zfs_lookup+0x440
  1160. #6 0xffffffff828f8e12 at zfs_freebsd_cachedlookup+0x72
  1161. #7 0xffffffff812138fe at VOP_CACHEDLOOKUP_APV+0xce
  1162. #8 0xffffffff80b9f79c at vfs_cache_lookup+0xac
  1163. #9 0xffffffff812136fe at VOP_LOOKUP_APV+0xce
  1164. #10 0xffffffff80ba8b71 at lookup+0x491
  1165. #11 0xffffffff80ba8239 at namei+0x459
  1166. #12 0xffffffff80baf95a at vfs_mountroot+0x10ea
  1167. #13 0xffffffff80a77ff7 at start_init+0x27
  1168. #14 0xffffffff80aa08d0 at fork_exit+0x80
  1169. #15 0xffffffff80fcffce at fork_trampoline+0xe
  1170. lock order reversal:
  1171. 1st 0xfffff810e0d66d68 zio->io_lock (zio->io_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/zio.c:3864
  1172. 2nd 0xfffffe014f2b38f0 vq->vq_lock (vq->vq_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/vdev_queue.c:1033
  1173. stack backtrace:
  1174. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1175. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1176. #2 0xffffffff82a157e5 at vdev_queue_change_io_priority+0xe5
  1177. #3 0xffffffff82abe0b8 at zio_change_priority+0x68
  1178. #4 0xffffffff82abe1e1 at zio_change_priority+0x191
  1179. #5 0xffffffff829220f7 at arc_read+0x457
  1180. #6 0xffffffff82943c80 at dbuf_read_impl+0xd30
  1181. #7 0xffffffff82939862 at dbuf_read_+0x622
  1182. #8 0xffffffff829500dc at dmu_buf_set_init+0x40c
  1183. #9 0xffffffff8294fb6d at dmu_issue+0x10d
  1184. #10 0xffffffff829526d9 at dmu_read_uio_dbuf+0x79
  1185. #11 0xffffffff828fb979 at zfs_freebsd_read+0x6a9
  1186. #12 0xffffffff81214d9e at VOP_READ_APV+0xce
  1187. #13 0xffffffff80bc98f5 at vn_read+0x1f5
  1188. #14 0xffffffff80bc9653 at vn_io_fault_doio+0x43
  1189. #15 0xffffffff80bc739c at vn_io_fault1+0x16c
  1190. #16 0xffffffff80bc569d at vn_io_fault+0x19d
  1191. #17 0xffffffff80b4e0c5 at dofileread+0x95
  1192. lo0: link state changed to UP
  1193. warning: KLD '/boot/kernel-debug/vmm.ko' is newer than the linker.hints file
  1194. warning: KLD '/boot/kernel-debug/opensolaris.ko' is newer than the linker.hints file
  1195. warning: KLD '/boot/kernel-debug/sdt.ko' is newer than the linker.hints file
  1196. warning: KLD '/boot/kernel-debug/systrace.ko' is newer than the linker.hints file
  1197. warning: KLD '/boot/kernel-debug/systrace_freebsd32.ko' is newer than the linker.hints file
  1198. warning: KLD '/boot/kernel-debug/profile.ko' is newer than the linker.hints file
  1199. lock order reversal:
  1200. 1st 0xfffff801eea0f8b8 zilog->zl_issuer_lock (zilog->zl_issuer_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/zil.c:2459
  1201. 2nd 0xfffffe01673157b0 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/os/freebsd/zfs/zfs_znode.c:925
  1202. stack backtrace:
  1203. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1204. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1205. #2 0xffffffff829035a6 at zfs_zget+0x56
  1206. #3 0xffffffff828f33a3 at zfs_get_data+0xa3
  1207. #4 0xffffffff82ab4c50 at zil_commit_impl+0x1130
  1208. #5 0xffffffff828fcfa3 at zfs_freebsd_fsync+0xd3
  1209. #6 0xffffffff81215954 at VOP_FSYNC_APV+0xd4
  1210. #7 0xffffffff80ba4a44 at vop_stdfdatasync+0x34
  1211. #8 0xffffffff8121aea4 at VOP_FDATASYNC_APV+0xd4
  1212. #9 0xffffffff80bc33cd at kern_fsync+0x19d
  1213. #10 0xffffffff80ff934e at amd64_syscall+0x2be
  1214. #11 0xffffffff80fcf8be at fast_syscall_common+0xf8
  1215. lock order reversal:
  1216. 1st 0xfffff810f5884830 zcw->zcw_lock (zcw->zcw_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/zil.c:2523
  1217. 2nd 0xfffff801eea0f800 zilog->zl_lock (zilog->zl_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/zil.c:571
  1218. stack backtrace:
  1219. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1220. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1221. #2 0xffffffff82ab77c3 at zil_alloc_lwb+0x193
  1222. #3 0xffffffff82ab828d at zil_lwb_write_issue+0x3fd
  1223. #4 0xffffffff82ab55a9 at zil_commit_impl+0x1a89
  1224. #5 0xffffffff828fcfa3 at zfs_freebsd_fsync+0xd3
  1225. #6 0xffffffff81215954 at VOP_FSYNC_APV+0xd4
  1226. #7 0xffffffff80ba4a44 at vop_stdfdatasync+0x34
  1227. #8 0xffffffff8121aea4 at VOP_FDATASYNC_APV+0xd4
  1228. #9 0xffffffff80bc33cd at kern_fsync+0x19d
  1229. #10 0xffffffff80ff934e at amd64_syscall+0x2be
  1230. #11 0xffffffff80fcf8be at fast_syscall_common+0xf8
  1231. lock order reversal:
  1232. 1st 0xfffffe0167315520 zfsvfs)->z_teardown_inactive_lock (zfsvfs)->z_teardown_inactive_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/os/freebsd/zfs/zfs_vnops.c:5893
  1233. 2nd 0xfffffe01673157b0 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/os/freebsd/zfs/zfs_znode.c:1212
  1234. stack backtrace:
  1235. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1236. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1237. #2 0xffffffff829044be at zfs_zinactive+0x4e
  1238. #3 0xffffffff828fdf7b at zfs_freebsd_reclaim+0x5b
  1239. #4 0xffffffff81216ce4 at VOP_RECLAIM_APV+0xd4
  1240. #5 0xffffffff80bb6539 at vgonel+0x249
  1241. #6 0xffffffff80bb6abc at vrecycle+0x5c
  1242. #7 0xffffffff828fde51 at zfs_freebsd_inactive+0xb1
  1243. #8 0xffffffff81216b14 at VOP_INACTIVE_APV+0xd4
  1244. #9 0xffffffff80bb5228 at vinactive+0xe8
  1245. #10 0xffffffff80bb5862 at vputx+0x322
  1246. #11 0xffffffff80bc012c at kern_unlinkat+0x20c
  1247. #12 0xffffffff80ff934e at amd64_syscall+0x2be
  1248. #13 0xffffffff80fcf8be at fast_syscall_common+0xf8
  1249. lock order reversal:
  1250. 1st 0xfffff80110e0b280 mls->mls_lock (mls->mls_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/multilist.c:288
  1251. 2nd 0xfffff8035a53a8c8 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:5470
  1252. stack backtrace:
  1253. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1254. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1255. #2 0xffffffff82941e63 at dbuf_sync_indirect+0x2c3
  1256. #3 0xffffffff82941af0 at dbuf_sync_list+0x50
  1257. #4 0xffffffff82979a07 at dnode_sync+0xf17
  1258. #5 0xffffffff82959c3f at sync_dnodes_task+0x5f
  1259. #6 0xffffffff828d7b7f at taskq_run+0x1f
  1260. #7 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1261. #8 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1262. #9 0xffffffff80aa08d0 at fork_exit+0x80
  1263. #10 0xffffffff80fcffce at fork_trampoline+0xe
  1264. lock order reversal:
  1265. 1st 0xfffff81080655bb8 os->os_userused_lock (os->os_userused_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dmu_objset.c:1829
  1266. 2nd 0xfffff81081406410 dn->dn_struct_rwlock (dn->dn_struct_rwlock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dnode.c:275
  1267. stack backtrace:
  1268. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1269. #1 0xffffffff80aeaec7 at _sx_slock_int+0x67
  1270. #2 0xffffffff829723b8 at dnode_verify+0x98
  1271. #3 0xffffffff82974e24 at dnode_hold_impl+0x164
  1272. #4 0xffffffff82950c78 at dmu_buf_hold_noread+0x28
  1273. #5 0xffffffff82950e7c at dmu_buf_hold+0x1c
  1274. #6 0xffffffff82a8e8ce at zap_lockdir+0x2e
  1275. #7 0xffffffff82a9006c at zap_lookup_norm+0x3c
  1276. #8 0xffffffff82a90021 at zap_lookup+0x11
  1277. #9 0xffffffff82a89ff1 at zap_increment+0x41
  1278. #10 0xffffffff8295a5a4 at userquota_updates_task+0x524
  1279. #11 0xffffffff828d7b7f at taskq_run+0x1f
  1280. #12 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1281. #13 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1282. #14 0xffffffff80aa08d0 at fork_exit+0x80
  1283. #15 0xffffffff80fcffce at fork_trampoline+0xe
  1284. lock order reversal:
  1285. 1st 0xfffff81080655bb8 os->os_userused_lock (os->os_userused_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dmu_objset.c:1829
  1286. 2nd 0xfffff81081406528 dn->dn_mtx (dn->dn_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dnode.c:1601
  1287. stack backtrace:
  1288. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1289. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1290. #2 0xffffffff829764c7 at dnode_rele+0x27
  1291. #3 0xffffffff82950cfb at dmu_buf_hold_noread+0xab
  1292. #4 0xffffffff82950e7c at dmu_buf_hold+0x1c
  1293. #5 0xffffffff82a8e8ce at zap_lockdir+0x2e
  1294. #6 0xffffffff82a9006c at zap_lookup_norm+0x3c
  1295. #7 0xffffffff82a90021 at zap_lookup+0x11
  1296. #8 0xffffffff82a89ff1 at zap_increment+0x41
  1297. #9 0xffffffff8295a5a4 at userquota_updates_task+0x524
  1298. #10 0xffffffff828d7b7f at taskq_run+0x1f
  1299. #11 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1300. #12 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1301. #13 0xffffffff80aa08d0 at fork_exit+0x80
  1302. #14 0xffffffff80fcffce at fork_trampoline+0xe
  1303. lock order reversal:
  1304. 1st 0xfffff81080655bb8 os->os_userused_lock (os->os_userused_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dmu_objset.c:1829
  1305. 2nd 0xfffff8035a4805c0 db->db_mtx (db->db_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:2113
  1306. stack backtrace:
  1307. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1308. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1309. #2 0xffffffff82939334 at dbuf_read_+0xf4
  1310. #3 0xffffffff82950ea9 at dmu_buf_hold+0x49
  1311. #4 0xffffffff82a8e8ce at zap_lockdir+0x2e
  1312. #5 0xffffffff82a9006c at zap_lookup_norm+0x3c
  1313. #6 0xffffffff82a90021 at zap_lookup+0x11
  1314. #7 0xffffffff82a89ff1 at zap_increment+0x41
  1315. #8 0xffffffff8295a5a4 at userquota_updates_task+0x524
  1316. #9 0xffffffff828d7b7f at taskq_run+0x1f
  1317. #10 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1318. #11 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1319. #12 0xffffffff80aa08d0 at fork_exit+0x80
  1320. #13 0xffffffff80fcffce at fork_trampoline+0xe
  1321. lock order reversal:
  1322. 1st 0xfffff81080655bb8 os->os_userused_lock (os->os_userused_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dmu_objset.c:1829
  1323. 2nd 0xffffffff82c6ca48 buf_hash_table.ht_locks[i].ht_lock (buf_hash_table.ht_locks[i].ht_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/arc.c:1012
  1324. stack backtrace:
  1325. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1326. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1327. #2 0xffffffff82923d1a at buf_hash_find+0xba
  1328. #3 0xffffffff82921e53 at arc_read+0x1b3
  1329. #4 0xffffffff82943c80 at dbuf_read_impl+0xd30
  1330. #5 0xffffffff82939862 at dbuf_read_+0x622
  1331. #6 0xffffffff82950ea9 at dmu_buf_hold+0x49
  1332. #7 0xffffffff82a8e8ce at zap_lockdir+0x2e
  1333. #8 0xffffffff82a9006c at zap_lookup_norm+0x3c
  1334. #9 0xffffffff82a90021 at zap_lookup+0x11
  1335. #10 0xffffffff82a89ff1 at zap_increment+0x41
  1336. #11 0xffffffff8295a5a4 at userquota_updates_task+0x524
  1337. #12 0xffffffff828d7b7f at taskq_run+0x1f
  1338. #13 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1339. #14 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1340. #15 0xffffffff80aa08d0 at fork_exit+0x80
  1341. #16 0xffffffff80fcffce at fork_trampoline+0xe
  1342. lock order reversal:
  1343. 1st 0xfffff81080655bb8 os->os_userused_lock (os->os_userused_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dmu_objset.c:1829
  1344. 2nd 0xfffff8034e7778f0 zap->zap_rwlock (zap->zap_rwlock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/zap_micro.c:421
  1345. stack backtrace:
  1346. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1347. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1348. #2 0xffffffff82a8eae9 at zap_lockdir_impl+0x159
  1349. #3 0xffffffff82a8e95e at zap_lockdir+0xbe
  1350. #4 0xffffffff82a9006c at zap_lookup_norm+0x3c
  1351. #5 0xffffffff82a90021 at zap_lookup+0x11
  1352. #6 0xffffffff82a89ff1 at zap_increment+0x41
  1353. #7 0xffffffff8295a5a4 at userquota_updates_task+0x524
  1354. #8 0xffffffff828d7b7f at taskq_run+0x1f
  1355. #9 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1356. #10 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1357. #11 0xffffffff80aa08d0 at fork_exit+0x80
  1358. #12 0xffffffff80fcffce at fork_trampoline+0xe
  1359. lock order reversal:
  1360. 1st 0xfffff81080655bb8 os->os_userused_lock (os->os_userused_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dmu_objset.c:1852
  1361. 2nd 0xfffff81081406318 dn->dn_dbufs_mtx (dn->dn_dbufs_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:4166
  1362. stack backtrace:
  1363. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1364. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1365. #2 0xffffffff8293d258 at dbuf_destroy+0x278
  1366. #3 0xffffffff82a900f1 at zap_lookup_norm+0xc1
  1367. #4 0xffffffff82a90021 at zap_lookup+0x11
  1368. #5 0xffffffff82a89ff1 at zap_increment+0x41
  1369. #6 0xffffffff8295a794 at userquota_updates_task+0x714
  1370. #7 0xffffffff828d7b7f at taskq_run+0x1f
  1371. #8 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1372. #9 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1373. #10 0xffffffff80aa08d0 at fork_exit+0x80
  1374. #11 0xffffffff80fcffce at fork_trampoline+0xe
  1375. lock order reversal:
  1376. 1st 0xfffff81080655bb8 os->os_userused_lock (os->os_userused_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dmu_objset.c:1852
  1377. 2nd 0xffffffff82ce7dc0 h->hash_mutexes[i] (h->hash_mutexes[i]) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:508
  1378. stack backtrace:
  1379. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1380. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1381. #2 0xffffffff8293d33b at dbuf_destroy+0x35b
  1382. #3 0xffffffff82a900f1 at zap_lookup_norm+0xc1
  1383. #4 0xffffffff82a90021 at zap_lookup+0x11
  1384. #5 0xffffffff82a89ff1 at zap_increment+0x41
  1385. #6 0xffffffff8295a794 at userquota_updates_task+0x714
  1386. #7 0xffffffff828d7b7f at taskq_run+0x1f
  1387. #8 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1388. #9 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1389. #10 0xffffffff80aa08d0 at fork_exit+0x80
  1390. #11 0xffffffff80fcffce at fork_trampoline+0xe
  1391. lock order reversal:
  1392. 1st 0xfffffe01729ff9c8 vd->vdev_dtl_lock (vd->vdev_dtl_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/vdev.c:988
  1393. 2nd 0xfffff801eeda4408 db->db_mtx (db->db_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:5023
  1394. stack backtrace:
  1395. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1396. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1397. #2 0xffffffff82940ae7 at dmu_buf_rele+0x27
  1398. #3 0xffffffff829f75cf at space_map_close+0x1f
  1399. #4 0xffffffff829fd2d9 at vdev_free+0x3f9
  1400. #5 0xffffffff829fd0e9 at vdev_free+0x209
  1401. #6 0xffffffff829fd0e9 at vdev_free+0x209
  1402. #7 0xffffffff829da45a at spa_unload+0x3ea
  1403. #8 0xffffffff829dba1b at spa_tryimport+0x35b
  1404. #9 0xffffffff82aa878d at zfs_ioc_pool_tryimport+0x3d
  1405. #10 0xffffffff82aa09e9 at zfsdev_ioctl_common+0x489
  1406. #11 0xffffffff828dec96 at zfsdev_ioctl+0x146
  1407. #12 0xffffffff80972b95 at devfs_ioctl+0xb5
  1408. #13 0xffffffff81215214 at VOP_IOCTL_APV+0xd4
  1409. #14 0xffffffff80bc5a2d at vn_ioctl+0x13d
  1410. #15 0xffffffff8097320e at devfs_ioctl_f+0x1e
  1411. #16 0xffffffff80b4ed65 at kern_ioctl+0x2d5
  1412. #17 0xffffffff80b4e9e7 at sys_ioctl+0x127
  1413. lock order reversal:
  1414. 1st 0xfffffe01729ff9c8 vd->vdev_dtl_lock (vd->vdev_dtl_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/vdev.c:988
  1415. 2nd 0xfffff8112fbbc938 dn->dn_mtx (dn->dn_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dnode.c:1601
  1416. stack backtrace:
  1417. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1418. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1419. #2 0xffffffff829764c7 at dnode_rele+0x27
  1420. #3 0xffffffff829f75cf at space_map_close+0x1f
  1421. #4 0xffffffff829fd2d9 at vdev_free+0x3f9
  1422. #5 0xffffffff829fd0e9 at vdev_free+0x209
  1423. #6 0xffffffff829fd0e9 at vdev_free+0x209
  1424. #7 0xffffffff829da45a at spa_unload+0x3ea
  1425. #8 0xffffffff829dba1b at spa_tryimport+0x35b
  1426. #9 0xffffffff82aa878d at zfs_ioc_pool_tryimport+0x3d
  1427. #10 0xffffffff82aa09e9 at zfsdev_ioctl_common+0x489
  1428. #11 0xffffffff828dec96 at zfsdev_ioctl+0x146
  1429. #12 0xffffffff80972b95 at devfs_ioctl+0xb5
  1430. #13 0xffffffff81215214 at VOP_IOCTL_APV+0xd4
  1431. #14 0xffffffff80bc5a2d at vn_ioctl+0x13d
  1432. #15 0xffffffff8097320e at devfs_ioctl_f+0x1e
  1433. #16 0xffffffff80b4ed65 at kern_ioctl+0x2d5
  1434. #17 0xffffffff80b4e9e7 at sys_ioctl+0x127
  1435. lock order reversal:
  1436. 1st 0xfffff81145de7ba8 tmpfs (tmpfs) @ /truenas-releng/freenas/_BE/os/sys/kern/vfs_mount.c:930
  1437. 2nd 0xfffff8112fd2b248 zfs (zfs) @ /truenas-releng/freenas/_BE/os/sys/kern/vfs_mount.c:941
  1438. stack backtrace:
  1439. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1440. #1 0xffffffff80ab526d at lockmgr_lock_fast_path+0x18d
  1441. #2 0xffffffff81216ea8 at VOP_LOCK1_APV+0xb8
  1442. #3 0xffffffff80bc6cc5 at _vn_lock+0x65
  1443. #4 0xffffffff80bac355 at vfs_domount+0xda5
  1444. #5 0xffffffff80baafe7 at vfs_donmount+0x7d7
  1445. #6 0xffffffff80baa7e1 at sys_nmount+0x71
  1446. #7 0xffffffff80ff934e at amd64_syscall+0x2be
  1447. #8 0xffffffff80fcf8be at fast_syscall_common+0xf8
  1448. GEOM_MIRROR: Device mirror/swap0 launched (2/2).
  1449. GEOM_MIRROR: Device mirror/swap1 launched (2/2).
  1450. GEOM_MIRROR: Device mirror/swap2 launched (2/2).
  1451. GEOM_MIRROR: Device mirror/swap3 launched (2/2).
  1452. GEOM_MIRROR: Device mirror/swap4 launched (2/2).
  1453. GEOM_ELI: Device mirror/swap0.eli created.
  1454. GEOM_ELI: Encryption: AES-XTS 128
  1455. GEOM_ELI: Crypto: hardware
  1456. GEOM_ELI: Device mirror/swap1.eli created.
  1457. GEOM_ELI: Encryption: AES-XTS 128
  1458. GEOM_ELI: Crypto: hardware
  1459. GEOM_ELI: Device mirror/swap2.eli created.
  1460. GEOM_ELI: Encryption: AES-XTS 128
  1461. GEOM_ELI: Crypto: hardware
  1462. GEOM_ELI: Device mirror/swap3.eli created.
  1463. GEOM_ELI: Encryption: AES-XTS 128
  1464. GEOM_ELI: Crypto: hardware
  1465. GEOM_ELI: Device mirror/swap4.eli created.
  1466. GEOM_ELI: Encryption: AES-XTS 128
  1467. GEOM_ELI: Crypto: hardware
  1468. igb0: link state changed to UP
  1469. igb1: link state changed to UP
  1470. igb2: link state changed to UP
  1471. igb3: link state changed to UP
  1472. hwpmc: SOFT/16/64/0x67<INT,USR,SYS,REA,WRI> TSC/1/64/0x20<REA> IAP/8/48/0x3ff<INT,USR,SYS,EDG,THR,REA,WRI,INV,QUA,PRC> IAF/3/48/0x67<INT,USR,SYS,REA,WRI>
  1473. warning: KLD '/boot/kernel-debug/t4_tom.ko' is newer than the linker.hints file
  1474. warning: KLD '/boot/kernel-debug/toecore.ko' is newer than the linker.hints file
  1475. igb0: link state changed to DOWN
  1476. igb2: link state changed to DOWN
  1477. igb3: link state changed to DOWN
  1478. igb1: link state changed to DOWN
  1479. igb3: link state changed to UP
  1480. igb0: link state changed to UP
  1481. nfsd: can't register svc name
  1482. igb2: link state changed to UP
  1483. igb1: link state changed to UP
  1484. CPU: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz (2100.04-MHz K8-class CPU)
  1485. Origin="GenuineIntel" Id=0x306e4 Family=0x6 Model=0x3e Stepping=4
  1486. Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  1487. Features2=0x7fbee3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  1488. AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  1489. AMD Features2=0x1<LAHF>
  1490. Structured Extended Features=0x281<FSGSBASE,SMEP,ERMS>
  1491. Structured Extended Features3=0x9c000400<MD_CLEAR,IBPB,STIBP,L1DFL,SSBD>
  1492. XSAVE Features=0x1<XSAVEOPT>
  1493. VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
  1494. TSC: P-state invariant, performance statistics
  1495. NLM: can't start server - it appears to be running already
  1496. Security policy loaded: MAC/ntpd (mac_ntpd)
  1497. lock order reversal:
  1498. 1st 0xfffff804077af1c0 mls->mls_lock (mls->mls_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/multilist.c:288
  1499. 2nd 0xffffffff82cfb9e0 h->hash_mutexes[i] (h->hash_mutexes[i]) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:508
  1500. stack backtrace:
  1501. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1502. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1503. #2 0xffffffff8293d33b at dbuf_destroy+0x35b
  1504. #3 0xffffffff8295a293 at userquota_updates_task+0x213
  1505. #4 0xffffffff828d7b7f at taskq_run+0x1f
  1506. #5 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1507. #6 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1508. #7 0xffffffff80aa08d0 at fork_exit+0x80
  1509. #8 0xffffffff80fcffce at fork_trampoline+0xe
  1510. lock order reversal:
  1511. 1st 0xfffff8112f385250 db->db_mtx (db->db_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:2819
  1512. 2nd 0xfffff803f3a6c800 ms->ms_lock (ms->ms_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/metaslab.c:5992
  1513. stack backtrace:
  1514. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1515. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1516. #2 0xffffffff829bfac7 at metaslab_check_free_impl+0x107
  1517. #3 0xffffffff829c1bee at metaslab_check_free+0x5e
  1518. #4 0xffffffff82aba907 at zio_free+0x47
  1519. #5 0xffffffff82939ca8 at dbuf_unoverride+0xa8
  1520. #6 0xffffffff829462c0 at dbuf_dirty_leaf_common+0xc0
  1521. #7 0xffffffff8293b6bd at dbuf_dirty_leaf+0xbd
  1522. #8 0xffffffff82950ab8 at dmu_buf_transfer_write+0x58
  1523. #9 0xffffffff8294f1d6 at dmu_buf_set_transfer+0x76
  1524. #10 0xffffffff8294f953 at dmu_buf_set_ready+0x23
  1525. #11 0xffffffff8294fb9a at dmu_issue+0x13a
  1526. #12 0xffffffff829528dd at dmu_write_uio_dbuf+0x9d
  1527. #13 0xffffffff828fc3f4 at zfs_freebsd_write+0x7e4
  1528. #14 0xffffffff81214fe0 at VOP_WRITE_APV+0x140
  1529. #15 0xffffffff80bc96c5 at vn_io_fault_doio+0xb5
  1530. #16 0xffffffff80bc739c at vn_io_fault1+0x16c
  1531. #17 0xffffffff80bc712f at vn_rdwr+0x1df
  1532. lock order reversal:
  1533. 1st 0xfffff803ca952600 mls->mls_lock (mls->mls_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/multilist.c:288
  1534. 2nd 0xfffff8121796e908 db->db_rwlock (db->db_rwlock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:1256
  1535. stack backtrace:
  1536. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1537. #1 0xffffffff80aeaec7 at _sx_slock_int+0x67
  1538. #2 0xffffffff829390b6 at dmu_buf_lock_parent+0x36
  1539. #3 0xffffffff8297a729 at free_children+0x59
  1540. #4 0xffffffff8297a56c at dnode_sync_free_range+0x21c
  1541. #5 0xffffffff829cc252 at range_tree_walk+0x62
  1542. #6 0xffffffff829790ca at dnode_sync+0x5da
  1543. #7 0xffffffff82959c3f at sync_dnodes_task+0x5f
  1544. #8 0xffffffff828d7b7f at taskq_run+0x1f
  1545. #9 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1546. #10 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1547. #11 0xffffffff80aa08d0 at fork_exit+0x80
  1548. #12 0xffffffff80fcffce at fork_trampoline+0xe
  1549. lock order reversal:
  1550. 1st 0xfffff8035a472228 db->db_rwlock (db->db_rwlock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dnode_sync.c:327
  1551. 2nd 0xfffff810f0183778 db->db_mtx (db->db_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:3801
  1552. stack backtrace:
  1553. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1554. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1555. #2 0xffffffff8293ba77 at dmu_buf_will_dirty_impl+0x57
  1556. #3 0xffffffff8297bf31 at dsl_dataset_block_kill+0x1a1
  1557. #4 0xffffffff8297a072 at free_blocks+0xd2
  1558. #5 0xffffffff8297ae86 at free_children+0x7b6
  1559. #6 0xffffffff8297adbe at free_children+0x6ee
  1560. #7 0xffffffff8297a56c at dnode_sync_free_range+0x21c
  1561. #8 0xffffffff829cc252 at range_tree_walk+0x62
  1562. #9 0xffffffff829790ca at dnode_sync+0x5da
  1563. #10 0xffffffff82959c3f at sync_dnodes_task+0x5f
  1564. #11 0xffffffff828d7b7f at taskq_run+0x1f
  1565. #12 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1566. #13 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1567. #14 0xffffffff80aa08d0 at fork_exit+0x80
  1568. #15 0xffffffff80fcffce at fork_trampoline+0xe
  1569. lock order reversal:
  1570. 1st 0xfffff8035a472228 db->db_rwlock (db->db_rwlock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dnode_sync.c:327
  1571. 2nd 0xfffff812175ea2e8 dl->dl_lock (dl->dl_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dsl_deadlist.c:454
  1572. stack backtrace:
  1573. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1574. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1575. #2 0xffffffff82988c61 at dsl_deadlist_insert+0x61
  1576. #3 0xffffffff8297c1fb at dsl_dataset_block_kill+0x46b
  1577. #4 0xffffffff8297a072 at free_blocks+0xd2
  1578. #5 0xffffffff8297ae86 at free_children+0x7b6
  1579. #6 0xffffffff8297adbe at free_children+0x6ee
  1580. #7 0xffffffff8297a56c at dnode_sync_free_range+0x21c
  1581. #8 0xffffffff829cc252 at range_tree_walk+0x62
  1582. #9 0xffffffff829790ca at dnode_sync+0x5da
  1583. #10 0xffffffff82959c3f at sync_dnodes_task+0x5f
  1584. #11 0xffffffff828d7b7f at taskq_run+0x1f
  1585. #12 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1586. #13 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1587. #14 0xffffffff80aa08d0 at fork_exit+0x80
  1588. #15 0xffffffff80fcffce at fork_trampoline+0xe
  1589. lock order reversal:
  1590. 1st 0xfffff8035a472228 db->db_rwlock (db->db_rwlock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dnode_sync.c:327
  1591. 2nd 0xfffff809094320c8 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:3035
  1592. stack backtrace:
  1593. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1594. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1595. #2 0xffffffff829456b4 at dbuf_dirty_exit+0x454
  1596. #3 0xffffffff8293b565 at dbuf_dirty+0x2f5
  1597. #4 0xffffffff82972fbb at dnode_setdirty+0x24b
  1598. #5 0xffffffff829457a4 at dbuf_dirty_exit+0x544
  1599. #6 0xffffffff8293b3e7 at dbuf_dirty+0x177
  1600. #7 0xffffffff82988c79 at dsl_deadlist_insert+0x79
  1601. #8 0xffffffff8297c1fb at dsl_dataset_block_kill+0x46b
  1602. #9 0xffffffff8297a072 at free_blocks+0xd2
  1603. #10 0xffffffff8297ae86 at free_children+0x7b6
  1604. #11 0xffffffff8297adbe at free_children+0x6ee
  1605. #12 0xffffffff8297a56c at dnode_sync_free_range+0x21c
  1606. #13 0xffffffff829cc252 at range_tree_walk+0x62
  1607. #14 0xffffffff829790ca at dnode_sync+0x5da
  1608. #15 0xffffffff82959c3f at sync_dnodes_task+0x5f
  1609. #16 0xffffffff828d7b7f at taskq_run+0x1f
  1610. #17 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1611. lock order reversal:
  1612. 1st 0xfffff8035a472228 db->db_rwlock (db->db_rwlock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dnode_sync.c:327
  1613. 2nd 0xfffff80303083820 dn->dn_struct_rwlock (dn->dn_struct_rwlock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dnode.c:275
  1614. stack backtrace:
  1615. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1616. #1 0xffffffff80aeaec7 at _sx_slock_int+0x67
  1617. #2 0xffffffff829723b8 at dnode_verify+0x98
  1618. #3 0xffffffff82972da0 at dnode_setdirty+0x30
  1619. #4 0xffffffff829457a4 at dbuf_dirty_exit+0x544
  1620. #5 0xffffffff8293b3e7 at dbuf_dirty+0x177
  1621. #6 0xffffffff8297c27a at dsl_dataset_block_kill+0x4ea
  1622. #7 0xffffffff8297a072 at free_blocks+0xd2
  1623. #8 0xffffffff8297ae86 at free_children+0x7b6
  1624. #9 0xffffffff8297adbe at free_children+0x6ee
  1625. #10 0xffffffff8297a56c at dnode_sync_free_range+0x21c
  1626. #11 0xffffffff829cc252 at range_tree_walk+0x62
  1627. #12 0xffffffff829790ca at dnode_sync+0x5da
  1628. #13 0xffffffff82959c3f at sync_dnodes_task+0x5f
  1629. #14 0xffffffff828d7b7f at taskq_run+0x1f
  1630. #15 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1631. #16 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1632. #17 0xffffffff80aa08d0 at fork_exit+0x80
  1633. lock order reversal:
  1634. 1st 0xfffff8034eaf48c8 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:5470
  1635. 2nd 0xfffff8120495e118 dn->dn_mtx (dn->dn_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dnode.c:2323
  1636. stack backtrace:
  1637. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1638. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1639. #2 0xffffffff82977f64 at dnode_diduse_space+0x114
  1640. #3 0xffffffff82947de9 at dbuf_write_ready+0xd9
  1641. #4 0xffffffff82ac3295 at zio_ready+0xa5
  1642. #5 0xffffffff82abaf7c at zio_nowait+0x12c
  1643. #6 0xffffffff829428e2 at dbuf_sync_leaf+0x952
  1644. #7 0xffffffff82941b5b at dbuf_sync_list+0xbb
  1645. #8 0xffffffff82941e7d at dbuf_sync_indirect+0x2dd
  1646. #9 0xffffffff82941af0 at dbuf_sync_list+0x50
  1647. #10 0xffffffff82941e7d at dbuf_sync_indirect+0x2dd
  1648. #11 0xffffffff82941af0 at dbuf_sync_list+0x50
  1649. #12 0xffffffff82979a07 at dnode_sync+0xf17
  1650. #13 0xffffffff82959c3f at sync_dnodes_task+0x5f
  1651. #14 0xffffffff828d7b7f at taskq_run+0x1f
  1652. #15 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1653. #16 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1654. #17 0xffffffff80aa08d0 at fork_exit+0x80
  1655. lock order reversal:
  1656. 1st 0xfffff8034eaf48c8 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:5470
  1657. 2nd 0xfffff80311e1e000 dn->dn_struct_rwlock (dn->dn_struct_rwlock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dnode.c:275
  1658. stack backtrace:
  1659. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1660. #1 0xffffffff80aeaec7 at _sx_slock_int+0x67
  1661. #2 0xffffffff829723b8 at dnode_verify+0x98
  1662. #3 0xffffffff82972da0 at dnode_setdirty+0x30
  1663. #4 0xffffffff829457a4 at dbuf_dirty_exit+0x544
  1664. #5 0xffffffff8293b3e7 at dbuf_dirty+0x177
  1665. #6 0xffffffff8299354f at dsl_dir_diduse_space+0x6f
  1666. #7 0xffffffff82993708 at dsl_dir_diduse_space+0x228
  1667. #8 0xffffffff8297b6ce at dsl_dataset_block_born+0x4ee
  1668. #9 0xffffffff829484b8 at dbuf_write_done+0xb8
  1669. #10 0xffffffff82947b44 at dbuf_write_override_done+0x164
  1670. #11 0xffffffff82ac537d at zio_done+0xf7d
  1671. #12 0xffffffff82abaf7c at zio_nowait+0x12c
  1672. #13 0xffffffff829428e2 at dbuf_sync_leaf+0x952
  1673. #14 0xffffffff82941b5b at dbuf_sync_list+0xbb
  1674. #15 0xffffffff82941e7d at dbuf_sync_indirect+0x2dd
  1675. #16 0xffffffff82941af0 at dbuf_sync_list+0x50
  1676. #17 0xffffffff82941e7d at dbuf_sync_indirect+0x2dd
  1677. lock order reversal:
  1678. 1st 0xfffff8034eaf48c8 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:5470
  1679. 2nd 0xfffff8120495e318 dn->dn_dbufs_mtx (dn->dn_dbufs_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:4166
  1680. stack backtrace:
  1681. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1682. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1683. #2 0xffffffff8293d258 at dbuf_destroy+0x278
  1684. #3 0xffffffff82947b44 at dbuf_write_override_done+0x164
  1685. #4 0xffffffff82ac537d at zio_done+0xf7d
  1686. #5 0xffffffff82abaf7c at zio_nowait+0x12c
  1687. #6 0xffffffff829428e2 at dbuf_sync_leaf+0x952
  1688. #7 0xffffffff82941b5b at dbuf_sync_list+0xbb
  1689. #8 0xffffffff82941e7d at dbuf_sync_indirect+0x2dd
  1690. #9 0xffffffff82941af0 at dbuf_sync_list+0x50
  1691. #10 0xffffffff82941e7d at dbuf_sync_indirect+0x2dd
  1692. #11 0xffffffff82941af0 at dbuf_sync_list+0x50
  1693. #12 0xffffffff82979a07 at dnode_sync+0xf17
  1694. #13 0xffffffff82959c3f at sync_dnodes_task+0x5f
  1695. #14 0xffffffff828d7b7f at taskq_run+0x1f
  1696. #15 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1697. #16 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1698. #17 0xffffffff80aa08d0 at fork_exit+0x80
  1699. pid 3480 (syslog-ng), jid 0, uid 0: exited on signal 6 (core dumped)
  1700. lock order reversal:
  1701. 1st 0xfffff81204f582c8 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:5470
  1702. 2nd 0xfffff81202279000 ms->ms_lock (ms->ms_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/metaslab.c:5992
  1703. stack backtrace:
  1704. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1705. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1706. #2 0xffffffff829bfac7 at metaslab_check_free_impl+0x107
  1707. #3 0xffffffff829c1bee at metaslab_check_free+0x5e
  1708. #4 0xffffffff82aba907 at zio_free+0x47
  1709. #5 0xffffffff8297c03a at dsl_dataset_block_kill+0x2aa
  1710. #6 0xffffffff829484aa at dbuf_write_done+0xaa
  1711. #7 0xffffffff82947b44 at dbuf_write_override_done+0x164
  1712. #8 0xffffffff82ac537d at zio_done+0xf7d
  1713. #9 0xffffffff82abaf7c at zio_nowait+0x12c
  1714. #10 0xffffffff829428e2 at dbuf_sync_leaf+0x952
  1715. #11 0xffffffff82941b5b at dbuf_sync_list+0xbb
  1716. #12 0xffffffff82941e7d at dbuf_sync_indirect+0x2dd
  1717. #13 0xffffffff82941af0 at dbuf_sync_list+0x50
  1718. #14 0xffffffff82941e7d at dbuf_sync_indirect+0x2dd
  1719. #15 0xffffffff82941af0 at dbuf_sync_list+0x50
  1720. #16 0xffffffff82941e7d at dbuf_sync_indirect+0x2dd
  1721. #17 0xffffffff82941af0 at dbuf_sync_list+0x50
  1722. lock order reversal:
  1723. 1st 0xfffff8090363aca0 db->db_mtx (db->db_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:2458
  1724. 2nd 0xfffff80bdf6450c8 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dbuf.c:3750
  1725. stack backtrace:
  1726. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1727. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1728. #2 0xffffffff8293a283 at dbuf_free_range+0x4d3
  1729. #3 0xffffffff82977bf5 at dnode_free_range+0xca5
  1730. #4 0xffffffff82951da7 at dmu_free_range+0x57
  1731. #5 0xffffffff829f76ea at space_map_truncate+0xea
  1732. #6 0xffffffff829bbec7 at metaslab_condense+0x357
  1733. #7 0xffffffff829bcc1b at metaslab_sync+0x58b
  1734. #8 0xffffffff82a0381b at vdev_sync+0x16b
  1735. #9 0xffffffff829e111b at spa_sync+0x128b
  1736. #10 0xffffffff829f91e4 at txg_sync_thread+0x484
  1737. #11 0xffffffff80aa08d0 at fork_exit+0x80
  1738. #12 0xffffffff80fcffce at fork_trampoline+0xe
  1739. lock order reversal:
  1740. 1st 0xfffff812029e8e30 db->db_rwlock (db->db_rwlock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/dnode_sync.c:327
  1741. 2nd 0xfffff81202b28000 ms->ms_lock (ms->ms_lock) @ /wrkdirs/usr/ports/sysutils/openzfs-kmod/work/zfs-7ec2d8b9c/module/zfs/metaslab.c:5992
  1742. stack backtrace:
  1743. #0 0xffffffff80b49d41 at witness_debugger+0x71
  1744. #1 0xffffffff80ae9e07 at _sx_xlock+0x67
  1745. #2 0xffffffff829bfac7 at metaslab_check_free_impl+0x107
  1746. #3 0xffffffff829c1bee at metaslab_check_free+0x5e
  1747. #4 0xffffffff82aba907 at zio_free+0x47
  1748. #5 0xffffffff8297c4b5 at dsl_dataset_block_kill+0x725
  1749. #6 0xffffffff8297a072 at free_blocks+0xd2
  1750. #7 0xffffffff8297ae86 at free_children+0x7b6
  1751. #8 0xffffffff8297a56c at dnode_sync_free_range+0x21c
  1752. #9 0xffffffff829cc252 at range_tree_walk+0x62
  1753. #10 0xffffffff829790ca at dnode_sync+0x5da
  1754. #11 0xffffffff82959c3f at sync_dnodes_task+0x5f
  1755. #12 0xffffffff828d7b7f at taskq_run+0x1f
  1756. #13 0xffffffff80b3c868 at taskqueue_run_locked+0x168
  1757. #14 0xffffffff80b3d7e4 at taskqueue_thread_loop+0x94
  1758. #15 0xffffffff80aa08d0 at fork_exit+0x80
  1759. #16 0xffffffff80fcffce at fork_trampoline+0xe
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!