API tools faq
paste
Guest User

8DiskZFSStripe.txt

a guest
Jun 14th, 2024
74
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 24.24 KB | None | 0 0
raw download clone embed print report
  1. pool: tank
  2. state: ONLINE
  3. config:
  4.  
  5. NAME STATE READ WRITE CKSUM
  6. tank ONLINE 0 0 0
  7. nvme0n1 ONLINE 0 0 0
  8. nvme1n1 ONLINE 0 0 0
  9. nvme2n1 ONLINE 0 0 0
  10. nvme3n1 ONLINE 0 0 0
  11. nvme4n1 ONLINE 0 0 0
  12. nvme5n1 ONLINE 0 0 0
  13. nvme6n1 ONLINE 0 0 0
  14. nvme7n1 ONLINE 0 0 0
  15.  
  16. errors: No known data errors
  17. NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
  18. tank 13.9T 404K 13.9T - - 0% 0% 1.00x ONLINE -
  19. Version 2.00a ------Sequential Output------ --Sequential Input- --Random-
  20. -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
  21. Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
  22. rescue 16G::4 1.6g 90 1.6g 95 4.2g 99 +++++ +++
  23. Latency 7395us 1060us 74us 752us
  24. Version 2.00a ------Sequential Create------ --------Random Create--------
  25. rescue -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
  26. files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
  27. 1 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
  28. Latency 169us 4us 147us 121us 2us 74us
  29. 1.98,2.00a,rescue,4,1718505337,16G,,8192,5,,,1704153,90,1684764,95,,,4432213,99,+++++,+++,1,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,,7395us,1060us,,74us,752us,169us,4us,147us,121us,2us,74us
  30. sync_randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
  31. fio-3.33
  32. Starting 1 process
  33. sync_randwrite: Laying out IO file (1 file / 1024MiB)
  34.  
  35. sync_randwrite: (groupid=0, jobs=1): err= 0: pid=158201: Fri Jun 14 13:15:42 2024
  36. write: IOPS=9800, BW=38.3MiB/s (40.1MB/s)(1024MiB/26749msec); 0 zone resets
  37. clat (usec): min=37, max=21675, avg=99.65, stdev=142.24
  38. lat (usec): min=37, max=21675, avg=99.70, stdev=142.33
  39. clat percentiles (usec):
  40. | 1.00th=[ 47], 5.00th=[ 50], 10.00th=[ 52], 20.00th=[ 53],
  41. | 30.00th=[ 55], 40.00th=[ 57], 50.00th=[ 68], 60.00th=[ 73],
  42. | 70.00th=[ 80], 80.00th=[ 93], 90.00th=[ 123], 95.00th=[ 441],
  43. | 99.00th=[ 619], 99.50th=[ 644], 99.90th=[ 750], 99.95th=[ 799],
  44. | 99.99th=[ 1037]
  45. bw ( KiB/s): min= 6592, max=68944, per=99.57%, avg=39031.09, stdev=21745.30, samples=53
  46. iops : min= 1648, max=17236, avg=9757.77, stdev=5436.32, samples=53
  47. lat (usec) : 50=4.65%, 100=79.28%, 250=9.89%, 500=2.09%, 750=3.98%
  48. lat (usec) : 1000=0.09%
  49. lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  50. cpu : usr=2.69%, sys=32.23%, ctx=263768, majf=0, minf=9
  51. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  52. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  53. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  54. issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
  55. latency : target=0, window=0, percentile=100.00%, depth=1
  56.  
  57. Run status group 0 (all jobs):
  58. WRITE: bw=38.3MiB/s (40.1MB/s), 38.3MiB/s-38.3MiB/s (40.1MB/s-40.1MB/s), io=1024MiB (1074MB), run=26749-26749msec
  59. sync_randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
  60. fio-3.33
  61. Starting 1 process
  62. sync_randread: Laying out IO file (1 file / 1024MiB)
  63.  
  64. sync_randread: (groupid=0, jobs=1): err= 0: pid=158328: Fri Jun 14 13:16:18 2024
  65. read: IOPS=76.0k, BW=297MiB/s (311MB/s)(10.0GiB/34489msec)
  66. clat (nsec): min=1048, max=241924, avg=12940.09, stdev=7599.16
  67. lat (nsec): min=1058, max=242820, avg=12968.58, stdev=7600.15
  68. clat percentiles (nsec):
  69. | 1.00th=[ 1528], 5.00th=[ 1864], 10.00th=[ 1992], 20.00th=[ 2160],
  70. | 30.00th=[ 2416], 40.00th=[17280], 50.00th=[17792], 60.00th=[17792],
  71. | 70.00th=[18048], 80.00th=[18304], 90.00th=[19072], 95.00th=[19584],
  72. | 99.00th=[21120], 99.50th=[21888], 99.90th=[23168], 99.95th=[24192],
  73. | 99.99th=[28544]
  74. bw ( KiB/s): min=259256, max=490136, per=99.55%, avg=302674.59, stdev=47118.97, samples=68
  75. iops : min=64814, max=122534, avg=75668.65, stdev=11779.74, samples=68
  76. lat (usec) : 2=10.69%, 4=22.00%, 10=0.03%, 20=64.71%, 50=2.57%
  77. lat (usec) : 100=0.01%, 250=0.01%
  78. cpu : usr=2.11%, sys=97.89%, ctx=6, majf=0, minf=10
  79. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  80. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  81. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  82. issued rwts: total=2621440,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  83. latency : target=0, window=0, percentile=100.00%, depth=1
  84.  
  85. Run status group 0 (all jobs):
  86. READ: bw=297MiB/s (311MB/s), 297MiB/s-297MiB/s (311MB/s-311MB/s), io=10.0GiB (10.7GB), run=34489-34489msec
  87. sync_seqwrite: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1
  88. fio-3.33
  89. Starting 1 process
  90. sync_seqwrite: Laying out IO file (1 file / 5120MiB)
  91.  
  92. sync_seqwrite: (groupid=0, jobs=1): err= 0: pid=158486: Fri Jun 14 13:16:21 2024
  93. write: IOPS=458, BW=1833MiB/s (1922MB/s)(5120MiB/2793msec); 0 zone resets
  94. clat (usec): min=875, max=2676, avg=1180.60, stdev=294.69
  95. lat (usec): min=875, max=2677, avg=1180.76, stdev=294.87
  96. clat percentiles (usec):
  97. | 1.00th=[ 898], 5.00th=[ 938], 10.00th=[ 963], 20.00th=[ 988],
  98. | 30.00th=[ 1004], 40.00th=[ 1037], 50.00th=[ 1090], 60.00th=[ 1139],
  99. | 70.00th=[ 1188], 80.00th=[ 1254], 90.00th=[ 1762], 95.00th=[ 1844],
  100. | 99.00th=[ 2278], 99.50th=[ 2409], 99.90th=[ 2638], 99.95th=[ 2671],
  101. | 99.99th=[ 2671]
  102. bw ( MiB/s): min= 1264, max= 2088, per=97.84%, avg=1793.60, stdev=337.21, samples=5
  103. iops : min= 316, max= 522, avg=448.40, stdev=84.30, samples=5
  104. lat (usec) : 1000=29.84%
  105. lat (msec) : 2=68.20%, 4=1.95%
  106. cpu : usr=45.67%, sys=25.36%, ctx=3513, majf=0, minf=11
  107. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  108. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  109. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  110. issued rwts: total=0,1280,0,0 short=0,0,0,0 dropped=0,0,0,0
  111. latency : target=0, window=0, percentile=100.00%, depth=1
  112.  
  113. Run status group 0 (all jobs):
  114. WRITE: bw=1833MiB/s (1922MB/s), 1833MiB/s-1833MiB/s (1922MB/s-1922MB/s), io=5120MiB (5369MB), run=2793-2793msec
  115. sync_seqread: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1
  116. fio-3.33
  117. Starting 1 process
  118. sync_seqread: Laying out IO file (1 file / 1024MiB)
  119.  
  120. sync_seqread: (groupid=0, jobs=1): err= 0: pid=158553: Fri Jun 14 13:16:24 2024
  121. read: IOPS=1457, BW=5828MiB/s (6111MB/s)(10.0GiB/1757msec)
  122. clat (usec): min=351, max=2759, avg=680.65, stdev=194.11
  123. lat (usec): min=351, max=2760, avg=680.70, stdev=194.12
  124. clat percentiles (usec):
  125. | 1.00th=[ 355], 5.00th=[ 355], 10.00th=[ 355], 20.00th=[ 515],
  126. | 30.00th=[ 693], 40.00th=[ 701], 50.00th=[ 709], 60.00th=[ 709],
  127. | 70.00th=[ 717], 80.00th=[ 725], 90.00th=[ 938], 95.00th=[ 979],
  128. | 99.00th=[ 1156], 99.50th=[ 1237], 99.90th=[ 1319], 99.95th=[ 1336],
  129. | 99.99th=[ 2769]
  130. bw ( MiB/s): min= 4672, max= 6256, per=98.14%, avg=5720.00, stdev=907.67, samples=3
  131. iops : min= 1168, max= 1564, avg=1430.00, stdev=226.92, samples=3
  132. lat (usec) : 500=19.84%, 750=64.88%, 1000=11.48%
  133. lat (msec) : 2=3.75%, 4=0.04%
  134. cpu : usr=0.17%, sys=99.83%, ctx=2, majf=0, minf=1037
  135. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  136. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  137. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  138. issued rwts: total=2560,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  139. latency : target=0, window=0, percentile=100.00%, depth=1
  140.  
  141. Run status group 0 (all jobs):
  142. READ: bw=5828MiB/s (6111MB/s), 5828MiB/s-5828MiB/s (6111MB/s-6111MB/s), io=10.0GiB (10.7GB), run=1757-1757msec
  143. async_uncached_randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  144. ...
  145. fio-3.33
  146. Starting 4 processes
  147. async_uncached_randwrite: Laying out IO file (1 file / 1024MiB)
  148.  
  149. async_uncached_randwrite: (groupid=0, jobs=4): err= 0: pid=158606: Fri Jun 14 13:16:30 2024
  150. write: IOPS=176k, BW=686MiB/s (720MB/s)(4096MiB/5968msec); 0 zone resets
  151. slat (usec): min=2, max=1396, avg=19.85, stdev=19.33
  152. clat (nsec): min=675, max=3593.7k, avg=673486.52, stdev=375444.34
  153. lat (usec): min=4, max=3633, avg=693.34, stdev=386.71
  154. clat percentiles (usec):
  155. | 1.00th=[ 231], 5.00th=[ 273], 10.00th=[ 302], 20.00th=[ 351],
  156. | 30.00th=[ 400], 40.00th=[ 449], 50.00th=[ 506], 60.00th=[ 611],
  157. | 70.00th=[ 914], 80.00th=[ 1090], 90.00th=[ 1254], 95.00th=[ 1369],
  158. | 99.00th=[ 1549], 99.50th=[ 1631], 99.90th=[ 1827], 99.95th=[ 1958],
  159. | 99.99th=[ 3064]
  160. bw ( KiB/s): min=601104, max=779408, per=100.00%, avg=706989.60, stdev=13577.24, samples=42
  161. iops : min=150276, max=194852, avg=176747.40, stdev=3394.31, samples=42
  162. lat (nsec) : 750=0.01%, 1000=0.01%
  163. lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%
  164. lat (usec) : 250=2.13%, 500=47.15%, 750=15.72%, 1000=9.39%
  165. lat (msec) : 2=25.57%, 4=0.04%
  166. cpu : usr=8.20%, sys=91.41%, ctx=1715, majf=0, minf=43
  167. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  168. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  169. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  170. issued rwts: total=0,1048576,0,0 short=0,0,0,0 dropped=0,0,0,0
  171. latency : target=0, window=0, percentile=100.00%, depth=32
  172.  
  173. Run status group 0 (all jobs):
  174. WRITE: bw=686MiB/s (720MB/s), 686MiB/s-686MiB/s (720MB/s-720MB/s), io=4096MiB (4295MB), run=5968-5968msec
  175. async_cached_randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  176. ...
  177. fio-3.33
  178. Starting 4 processes
  179. async_cached_randwrite: Laying out IO file (1 file / 1024MiB)
  180.  
  181. async_cached_randwrite: (groupid=0, jobs=4): err= 0: pid=158667: Fri Jun 14 13:16:36 2024
  182. write: IOPS=178k, BW=695MiB/s (728MB/s)(4096MiB/5896msec); 0 zone resets
  183. slat (usec): min=2, max=2648, avg=19.91, stdev=20.34
  184. clat (nsec): min=656, max=4175.0k, avg=675678.11, stdev=388391.66
  185. lat (usec): min=3, max=4184, avg=695.59, stdev=399.95
  186. clat percentiles (usec):
  187. | 1.00th=[ 178], 5.00th=[ 241], 10.00th=[ 281], 20.00th=[ 330],
  188. | 30.00th=[ 383], 40.00th=[ 441], 50.00th=[ 519], 60.00th=[ 668],
  189. | 70.00th=[ 922], 80.00th=[ 1106], 90.00th=[ 1254], 95.00th=[ 1369],
  190. | 99.00th=[ 1565], 99.50th=[ 1647], 99.90th=[ 1860], 99.95th=[ 2212],
  191. | 99.99th=[ 3130]
  192. bw ( KiB/s): min=573445, max=859792, per=98.80%, avg=702847.09, stdev=21101.65, samples=43
  193. iops : min=143361, max=214948, avg=175711.68, stdev=5275.45, samples=43
  194. lat (nsec) : 750=0.01%, 1000=0.01%
  195. lat (usec) : 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%, 250=5.99%
  196. lat (usec) : 500=42.19%, 750=15.25%, 1000=10.48%
  197. lat (msec) : 2=26.02%, 4=0.06%, 10=0.01%
  198. cpu : usr=8.45%, sys=91.08%, ctx=1979, majf=0, minf=38
  199. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  200. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  201. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  202. issued rwts: total=0,1048576,0,0 short=0,0,0,0 dropped=0,0,0,0
  203. latency : target=0, window=0, percentile=100.00%, depth=32
  204.  
  205. Run status group 0 (all jobs):
  206. WRITE: bw=695MiB/s (728MB/s), 695MiB/s-695MiB/s (728MB/s-728MB/s), io=4096MiB (4295MB), run=5896-5896msec
  207. async_uncached_randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  208. ...
  209. fio-3.33
  210. Starting 4 processes
  211. async_uncached_randread: Laying out IO file (1 file / 1024MiB)
  212.  
  213. async_uncached_randread: (groupid=0, jobs=4): err= 0: pid=158726: Fri Jun 14 13:17:20 2024
  214. read: IOPS=248k, BW=969MiB/s (1016MB/s)(40.0GiB/42261msec)
  215. slat (nsec): min=1266, max=196124, avg=15514.56, stdev=8123.53
  216. clat (nsec): min=535, max=1537.7k, avg=493732.30, stdev=80380.17
  217. lat (usec): min=2, max=1570, avg=509.25, stdev=82.58
  218. clat percentiles (usec):
  219. | 1.00th=[ 198], 5.00th=[ 338], 10.00th=[ 392], 20.00th=[ 445],
  220. | 30.00th=[ 478], 40.00th=[ 494], 50.00th=[ 510], 60.00th=[ 529],
  221. | 70.00th=[ 537], 80.00th=[ 553], 90.00th=[ 578], 95.00th=[ 586],
  222. | 99.00th=[ 619], 99.50th=[ 627], 99.90th=[ 644], 99.95th=[ 660],
  223. | 99.99th=[ 824]
  224. bw ( KiB/s): min=919264, max=1436472, per=100.00%, avg=1001645.14, stdev=26892.00, samples=331
  225. iops : min=229816, max=359118, avg=250411.33, stdev=6722.99, samples=331
  226. lat (nsec) : 750=0.01%, 1000=0.01%
  227. lat (usec) : 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.02%
  228. lat (usec) : 250=2.01%, 500=40.93%, 750=57.03%, 1000=0.01%
  229. lat (msec) : 2=0.01%
  230. cpu : usr=2.59%, sys=97.38%, ctx=2990, majf=0, minf=170
  231. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  232. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  233. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  234. issued rwts: total=10485760,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  235. latency : target=0, window=0, percentile=100.00%, depth=32
  236.  
  237. Run status group 0 (all jobs):
  238. READ: bw=969MiB/s (1016MB/s), 969MiB/s-969MiB/s (1016MB/s-1016MB/s), io=40.0GiB (42.9GB), run=42261-42261msec
  239. async_cached_randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  240. ...
  241. fio-3.33
  242. Starting 4 processes
  243. async_cached_randread: Laying out IO file (1 file / 1024MiB)
  244.  
  245. async_cached_randread: (groupid=0, jobs=4): err= 0: pid=158879: Fri Jun 14 13:18:04 2024
  246. read: IOPS=250k, BW=977MiB/s (1024MB/s)(40.0GiB/41943msec)
  247. slat (nsec): min=1286, max=174337, avg=15351.36, stdev=8039.40
  248. clat (nsec): min=623, max=1357.0k, avg=489173.73, stdev=79819.36
  249. lat (usec): min=2, max=1442, avg=504.53, stdev=82.00
  250. clat percentiles (usec):
  251. | 1.00th=[ 196], 5.00th=[ 334], 10.00th=[ 388], 20.00th=[ 441],
  252. | 30.00th=[ 469], 40.00th=[ 490], 50.00th=[ 506], 60.00th=[ 519],
  253. | 70.00th=[ 537], 80.00th=[ 553], 90.00th=[ 570], 95.00th=[ 586],
  254. | 99.00th=[ 611], 99.50th=[ 619], 99.90th=[ 644], 99.95th=[ 652],
  255. | 99.99th=[ 906]
  256. bw ( KiB/s): min=930616, max=1502224, per=100.00%, avg=1010986.75, stdev=27450.94, samples=328
  257. iops : min=232654, max=375556, avg=252746.66, stdev=6862.72, samples=328
  258. lat (nsec) : 750=0.01%, 1000=0.01%
  259. lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  260. lat (usec) : 100=0.02%, 250=2.06%, 500=43.89%, 750=54.01%, 1000=0.01%
  261. lat (msec) : 2=0.01%
  262. cpu : usr=2.78%, sys=97.19%, ctx=2890, majf=0, minf=170
  263. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  264. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  265. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  266. issued rwts: total=10485760,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  267. latency : target=0, window=0, percentile=100.00%, depth=32
  268.  
  269. Run status group 0 (all jobs):
  270. READ: bw=977MiB/s (1024MB/s), 977MiB/s-977MiB/s (1024MB/s-1024MB/s), io=40.0GiB (42.9GB), run=41943-41943msec
  271. async_uncached_seqwrite: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  272. ...
  273. fio-3.33
  274. Starting 4 processes
  275. async_uncached_seqwrite: Laying out IO file (1 file / 2048MiB)
  276.  
  277. async_uncached_seqwrite: (groupid=0, jobs=4): err= 0: pid=159027: Fri Jun 14 13:18:05 2024
  278. write: IOPS=1906, BW=7628MiB/s (7998MB/s)(8192MiB/1074msec); 0 zone resets
  279. slat (usec): min=299, max=5485, avg=701.95, stdev=352.83
  280. clat (nsec): min=1773, max=102237k, avg=57462625.23, stdev=18003161.85
  281. lat (usec): min=435, max=103154, avg=58164.57, stdev=18145.68
  282. clat percentiles (msec):
  283. | 1.00th=[ 9], 5.00th=[ 35], 10.00th=[ 37], 20.00th=[ 45],
  284. | 30.00th=[ 52], 40.00th=[ 53], 50.00th=[ 55], 60.00th=[ 56],
  285. | 70.00th=[ 64], 80.00th=[ 73], 90.00th=[ 84], 95.00th=[ 91],
  286. | 99.00th=[ 101], 99.50th=[ 102], 99.90th=[ 103], 99.95th=[ 103],
  287. | 99.99th=[ 103]
  288. bw ( MiB/s): min= 5928, max= 6477, per=81.33%, avg=6203.41, stdev=194.11, samples=5
  289. iops : min= 1480, max= 1618, avg=1549.00, stdev=48.79, samples=5
  290. lat (usec) : 2=0.05%, 4=0.15%
  291. lat (msec) : 2=0.15%, 4=0.20%, 10=0.63%, 20=1.17%, 50=24.85%
  292. lat (msec) : 100=71.63%, 250=1.17%
  293. cpu : usr=64.85%, sys=26.51%, ctx=2497, majf=0, minf=46
  294. IO depths : 1=0.2%, 2=0.4%, 4=0.8%, 8=1.6%, 16=3.1%, 32=93.9%, >=64=0.0%
  295. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  296. complete : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.0%, 32=0.2%, 64=0.0%, >=64=0.0%
  297. issued rwts: total=0,2048,0,0 short=0,0,0,0 dropped=0,0,0,0
  298. latency : target=0, window=0, percentile=100.00%, depth=32
  299.  
  300. Run status group 0 (all jobs):
  301. WRITE: bw=7628MiB/s (7998MB/s), 7628MiB/s-7628MiB/s (7998MB/s-7998MB/s), io=8192MiB (8590MB), run=1074-1074msec
  302. async_cached_seqwrite: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  303. ...
  304. fio-3.33
  305. Starting 4 processes
  306. async_cached_seqwrite: Laying out IO file (1 file / 2048MiB)
  307.  
  308. async_cached_seqwrite: (groupid=0, jobs=4): err= 0: pid=159091: Fri Jun 14 13:18:07 2024
  309. write: IOPS=1894, BW=7578MiB/s (7946MB/s)(8192MiB/1081msec); 0 zone resets
  310. slat (usec): min=330, max=3582, avg=634.72, stdev=319.82
  311. clat (nsec): min=1847, max=88456k, avg=58020002.65, stdev=15586150.84
  312. lat (usec): min=458, max=89104, avg=58654.73, stdev=15630.27
  313. clat percentiles (usec):
  314. | 1.00th=[ 6915], 5.00th=[34866], 10.00th=[36439], 20.00th=[49021],
  315. | 30.00th=[55837], 40.00th=[55837], 50.00th=[56361], 60.00th=[58459],
  316. | 70.00th=[66323], 80.00th=[73925], 90.00th=[74974], 95.00th=[80217],
  317. | 99.00th=[86508], 99.50th=[87557], 99.90th=[88605], 99.95th=[88605],
  318. | 99.99th=[88605]
  319. bw ( MiB/s): min= 7144, max= 7616, per=97.39%, avg=7380.00, stdev=149.26, samples=6
  320. iops : min= 1786, max= 1904, avg=1845.00, stdev=37.31, samples=6
  321. lat (usec) : 2=0.10%, 4=0.10%
  322. lat (msec) : 2=0.20%, 4=0.29%, 10=0.78%, 20=1.32%, 50=17.92%
  323. lat (msec) : 100=79.30%
  324. cpu : usr=66.61%, sys=30.17%, ctx=1643, majf=0, minf=53
  325. IO depths : 1=0.2%, 2=0.4%, 4=0.8%, 8=1.6%, 16=3.1%, 32=93.9%, >=64=0.0%
  326. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  327. complete : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.0%, 32=0.2%, 64=0.0%, >=64=0.0%
  328. issued rwts: total=0,2048,0,0 short=0,0,0,0 dropped=0,0,0,0
  329. latency : target=0, window=0, percentile=100.00%, depth=32
  330.  
  331. Run status group 0 (all jobs):
  332. WRITE: bw=7578MiB/s (7946MB/s), 7578MiB/s-7578MiB/s (7946MB/s-7946MB/s), io=8192MiB (8590MB), run=1081-1081msec
  333. async_uncached_seqread: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  334. ...
  335. fio-3.33
  336. Starting 4 processes
  337. async_uncached_seqread: Laying out IO file (1 file / 1024MiB)
  338.  
  339. async_uncached_seqread: (groupid=0, jobs=4): err= 0: pid=159152: Fri Jun 14 13:18:15 2024
  340. read: IOPS=6591, BW=25.7GiB/s (27.6GB/s)(200GiB/7768msec)
  341. slat (usec): min=356, max=3065, avg=596.24, stdev=81.11
  342. clat (nsec): min=1478, max=65558k, avg=17255769.70, stdev=4033272.33
  343. lat (usec): min=508, max=67721, avg=17852.01, stdev=4070.73
  344. clat percentiles (usec):
  345. | 1.00th=[ 1045], 5.00th=[ 6718], 10.00th=[13698], 20.00th=[17171],
  346. | 30.00th=[18482], 40.00th=[18482], 50.00th=[18482], 60.00th=[18482],
  347. | 70.00th=[18482], 80.00th=[18482], 90.00th=[18744], 95.00th=[19792],
  348. | 99.00th=[20579], 99.50th=[21890], 99.90th=[43254], 99.95th=[51119],
  349. | 99.99th=[62653]
  350. bw ( MiB/s): min=21536, max=27616, per=99.75%, avg=26297.60, stdev=351.57, samples=60
  351. iops : min= 5384, max= 6904, avg=6574.40, stdev=87.89, samples=60
  352. lat (usec) : 2=0.30%, 4=0.09%, 750=0.39%
  353. lat (msec) : 2=0.78%, 4=1.41%, 10=4.35%, 20=88.61%, 50=4.01%
  354. lat (msec) : 100=0.05%
  355. cpu : usr=0.31%, sys=98.21%, ctx=116720, majf=0, minf=131125
  356. IO depths : 1=0.4%, 2=0.8%, 4=1.6%, 8=3.1%, 16=6.2%, 32=87.9%, >=64=0.0%
  357. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  358. complete : 0=0.0%, 4=99.6%, 8=0.0%, 16=0.0%, 32=0.4%, 64=0.0%, >=64=0.0%
  359. issued rwts: total=51200,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  360. latency : target=0, window=0, percentile=100.00%, depth=32
  361.  
  362. Run status group 0 (all jobs):
  363. READ: bw=25.7GiB/s (27.6GB/s), 25.7GiB/s-25.7GiB/s (27.6GB/s-27.6GB/s), io=200GiB (215GB), run=7768-7768msec
  364. async_cached_seqread: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  365. ...
  366. fio-3.33
  367. Starting 4 processes
  368. async_cached_seqread: Laying out IO file (1 file / 1024MiB)
  369.  
  370. async_cached_seqread: (groupid=0, jobs=4): err= 0: pid=159221: Fri Jun 14 13:18:24 2024
  371. read: IOPS=6328, BW=24.7GiB/s (26.5GB/s)(200GiB/8091msec)
  372. slat (usec): min=355, max=2387, avg=620.55, stdev=92.60
  373. clat (nsec): min=1372, max=69668k, avg=17961846.49, stdev=4312487.65
  374. lat (usec): min=508, max=72021, avg=18582.40, stdev=4359.95
  375. clat percentiles (usec):
  376. | 1.00th=[ 1074], 5.00th=[ 6849], 10.00th=[14222], 20.00th=[17957],
  377. | 30.00th=[19006], 40.00th=[19006], 50.00th=[19006], 60.00th=[19268],
  378. | 70.00th=[19268], 80.00th=[19530], 90.00th=[20317], 95.00th=[20579],
  379. | 99.00th=[22152], 99.50th=[22414], 99.90th=[48497], 99.95th=[57934],
  380. | 99.99th=[67634]
  381. bw ( MiB/s): min=19808, max=27200, per=99.79%, avg=25258.00, stdev=381.77, samples=64
  382. iops : min= 4952, max= 6800, avg=6314.50, stdev=95.44, samples=64
  383. lat (usec) : 2=0.27%, 4=0.12%, 750=0.39%
  384. lat (msec) : 2=0.78%, 4=1.40%, 10=4.25%, 20=74.90%, 50=17.79%
  385. lat (msec) : 100=0.09%
  386. cpu : usr=0.36%, sys=97.97%, ctx=128564, majf=0, minf=131121
  387. IO depths : 1=0.4%, 2=0.8%, 4=1.6%, 8=3.1%, 16=6.2%, 32=87.9%, >=64=0.0%
  388. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  389. complete : 0=0.0%, 4=99.6%, 8=0.0%, 16=0.0%, 32=0.4%, 64=0.0%, >=64=0.0%
  390. issued rwts: total=51200,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  391. latency : target=0, window=0, percentile=100.00%, depth=32
  392.  
  393. Run status group 0 (all jobs):
  394. READ: bw=24.7GiB/s (26.5GB/s), 24.7GiB/s-24.7GiB/s (26.5GB/s-26.5GB/s), io=200GiB (215GB), run=8091-8091msec
  395. Linux 6.7.4 (rescue) 06/14/2024 _x86_64_ (48 CPU)
  396.  
  397. avg-cpu: %user %nice %system %iowait %steal %idle
  398. 0.11 0.00 0.59 0.00 0.00 99.29
  399.  
  400. Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
  401. loop0 0.40 9.02 0.00 0.00 440029 0 0
  402. nvme0n1 126.24 6.60 5614.52 0.00 322305 273995225 0
  403. nvme1n1 90.79 5.43 3446.76 0.00 264763 168205763 0
  404. nvme2n1 87.03 5.11 3383.66 0.00 249559 165126508 0
  405. nvme3n1 73.03 4.52 2706.36 0.00 220380 132073349 0
  406. nvme4n1 59.56 3.70 2101.69 0.00 180701 102565014 0
  407. nvme5n1 42.63 3.00 1446.83 0.00 146396 70606929 0
  408. nvme6n1 25.80 1.81 881.13 0.00 88545 42999950 0
  409. nvme7n1 13.88 1.13 431.80 0.00 54940 21072321 0
  410.  
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!