API tools faq
paste
Guest User

8DiskZFSRaidZ1.txt

a guest
Jun 14th, 2024
68
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 24.26 KB | None | 0 0
raw download clone embed print report
  1. pool: tank
  2. state: ONLINE
  3. config:
  4.  
  5. NAME STATE READ WRITE CKSUM
  6. tank ONLINE 0 0 0
  7. raidz1-0 ONLINE 0 0 0
  8. nvme0n1 ONLINE 0 0 0
  9. nvme1n1 ONLINE 0 0 0
  10. nvme2n1 ONLINE 0 0 0
  11. nvme3n1 ONLINE 0 0 0
  12. nvme4n1 ONLINE 0 0 0
  13. nvme5n1 ONLINE 0 0 0
  14. nvme6n1 ONLINE 0 0 0
  15. nvme7n1 ONLINE 0 0 0
  16.  
  17. errors: No known data errors
  18. NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
  19. tank 14.0T 664K 14.0T - - 0% 0% 1.00x ONLINE -
  20. Version 2.00a ------Sequential Output------ --Sequential Input- --Random-
  21. -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
  22. Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
  23. rescue 16G::4 1.5g 91 1.6g 94 4.2g 99 +++++ +++
  24. Latency 7252us 642us 97us 1350us
  25. Version 2.00a ------Sequential Create------ --------Random Create--------
  26. rescue -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
  27. files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
  28. 1 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
  29. Latency 789us 4us 269us 182us 2us 92us
  30. 1.98,2.00a,rescue,4,1718520183,16G,,8192,5,,,1523178,91,1728494,94,,,4403884,99,+++++,+++,1,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,,7252us,642us,,97us,1350us,789us,4us,269us,182us,2us,92us
  31. sync_randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
  32. fio-3.33
  33. Starting 1 process
  34. sync_randwrite: Laying out IO file (1 file / 1024MiB)
  35.  
  36. sync_randwrite: (groupid=0, jobs=1): err= 0: pid=179442: Fri Jun 14 14:19:04 2024
  37. write: IOPS=12.2k, BW=47.5MiB/s (49.8MB/s)(1024MiB/21552msec); 0 zone resets
  38. clat (usec): min=43, max=12194, avg=80.76, stdev=83.77
  39. lat (usec): min=43, max=12194, avg=80.78, stdev=83.80
  40. clat percentiles (usec):
  41. | 1.00th=[ 55], 5.00th=[ 59], 10.00th=[ 61], 20.00th=[ 63],
  42. | 30.00th=[ 65], 40.00th=[ 67], 50.00th=[ 69], 60.00th=[ 72],
  43. | 70.00th=[ 82], 80.00th=[ 89], 90.00th=[ 101], 95.00th=[ 120],
  44. | 99.00th=[ 243], 99.50th=[ 515], 99.90th=[ 693], 99.95th=[ 750],
  45. | 99.99th=[ 2008]
  46. bw ( KiB/s): min= 6464, max=60976, per=99.99%, avg=48650.98, stdev=10131.24, samples=43
  47. iops : min= 1616, max=15244, avg=12162.74, stdev=2532.80, samples=43
  48. lat (usec) : 50=0.08%, 100=89.46%, 250=9.51%, 500=0.43%, 750=0.47%
  49. lat (usec) : 1000=0.03%
  50. lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%
  51. cpu : usr=3.07%, sys=29.39%, ctx=263785, majf=0, minf=9
  52. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  53. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  54. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  55. issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
  56. latency : target=0, window=0, percentile=100.00%, depth=1
  57.  
  58. Run status group 0 (all jobs):
  59. WRITE: bw=47.5MiB/s (49.8MB/s), 47.5MiB/s-47.5MiB/s (49.8MB/s-49.8MB/s), io=1024MiB (1074MB), run=21552-21552msec
  60. sync_randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
  61. fio-3.33
  62. Starting 1 process
  63. sync_randread: Laying out IO file (1 file / 1024MiB)
  64.  
  65. sync_randread: (groupid=0, jobs=1): err= 0: pid=179547: Fri Jun 14 14:19:39 2024
  66. read: IOPS=78.0k, BW=305MiB/s (319MB/s)(10.0GiB/33618msec)
  67. clat (nsec): min=1058, max=125936, avg=12615.94, stdev=7376.37
  68. lat (nsec): min=1072, max=126859, avg=12641.47, stdev=7376.88
  69. clat percentiles (nsec):
  70. | 1.00th=[ 1512], 5.00th=[ 1848], 10.00th=[ 1960], 20.00th=[ 2128],
  71. | 30.00th=[ 2352], 40.00th=[17024], 50.00th=[17280], 60.00th=[17536],
  72. | 70.00th=[17792], 80.00th=[18048], 90.00th=[18560], 95.00th=[18816],
  73. | 99.00th=[19840], 99.50th=[20352], 99.90th=[21632], 99.95th=[22400],
  74. | 99.99th=[28800]
  75. bw ( KiB/s): min=279368, max=482312, per=99.68%, avg=310922.51, stdev=44031.02, samples=67
  76. iops : min=69842, max=120578, avg=77730.66, stdev=11007.81, samples=67
  77. lat (usec) : 2=12.25%, 4=20.32%, 10=0.03%, 20=66.60%, 50=0.80%
  78. lat (usec) : 100=0.01%, 250=0.01%
  79. cpu : usr=2.20%, sys=97.79%, ctx=12, majf=0, minf=10
  80. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  81. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  82. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  83. issued rwts: total=2621440,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  84. latency : target=0, window=0, percentile=100.00%, depth=1
  85.  
  86. Run status group 0 (all jobs):
  87. READ: bw=305MiB/s (319MB/s), 305MiB/s-305MiB/s (319MB/s-319MB/s), io=10.0GiB (10.7GB), run=33618-33618msec
  88. sync_seqwrite: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1
  89. fio-3.33
  90. Starting 1 process
  91. sync_seqwrite: Laying out IO file (1 file / 5120MiB)
  92.  
  93. sync_seqwrite: (groupid=0, jobs=1): err= 0: pid=179613: Fri Jun 14 14:19:42 2024
  94. write: IOPS=482, BW=1928MiB/s (2022MB/s)(5120MiB/2655msec); 0 zone resets
  95. clat (usec): min=996, max=2764, avg=1248.17, stdev=158.91
  96. lat (usec): min=997, max=2765, avg=1248.31, stdev=158.93
  97. clat percentiles (usec):
  98. | 1.00th=[ 1020], 5.00th=[ 1045], 10.00th=[ 1057], 20.00th=[ 1090],
  99. | 30.00th=[ 1139], 40.00th=[ 1188], 50.00th=[ 1237], 60.00th=[ 1287],
  100. | 70.00th=[ 1319], 80.00th=[ 1385], 90.00th=[ 1467], 95.00th=[ 1532],
  101. | 99.00th=[ 1614], 99.50th=[ 1663], 99.90th=[ 1926], 99.95th=[ 2769],
  102. | 99.99th=[ 2769]
  103. bw ( MiB/s): min= 1656, max= 2072, per=99.65%, avg=1921.60, stdev=198.51, samples=5
  104. iops : min= 414, max= 518, avg=480.40, stdev=49.63, samples=5
  105. lat (usec) : 1000=0.08%
  106. lat (msec) : 2=99.84%, 4=0.08%
  107. cpu : usr=39.49%, sys=26.04%, ctx=3510, majf=0, minf=11
  108. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  109. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  110. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  111. issued rwts: total=0,1280,0,0 short=0,0,0,0 dropped=0,0,0,0
  112. latency : target=0, window=0, percentile=100.00%, depth=1
  113.  
  114. Run status group 0 (all jobs):
  115. WRITE: bw=1928MiB/s (2022MB/s), 1928MiB/s-1928MiB/s (2022MB/s-2022MB/s), io=5120MiB (5369MB), run=2655-2655msec
  116. sync_seqread: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1
  117. fio-3.33
  118. Starting 1 process
  119. sync_seqread: Laying out IO file (1 file / 1024MiB)
  120.  
  121. sync_seqread: (groupid=0, jobs=1): err= 0: pid=179673: Fri Jun 14 14:19:45 2024
  122. read: IOPS=1447, BW=5789MiB/s (6070MB/s)(10.0GiB/1769msec)
  123. clat (usec): min=350, max=2399, avg=684.83, stdev=186.34
  124. lat (usec): min=350, max=2399, avg=684.87, stdev=186.35
  125. clat percentiles (usec):
  126. | 1.00th=[ 351], 5.00th=[ 355], 10.00th=[ 355], 20.00th=[ 515],
  127. | 30.00th=[ 685], 40.00th=[ 693], 50.00th=[ 701], 60.00th=[ 709],
  128. | 70.00th=[ 717], 80.00th=[ 742], 90.00th=[ 947], 95.00th=[ 963],
  129. | 99.00th=[ 988], 99.50th=[ 1004], 99.90th=[ 1020], 99.95th=[ 1029],
  130. | 99.99th=[ 2409]
  131. bw ( MiB/s): min= 4560, max= 6320, per=97.89%, avg=5666.67, stdev=963.60, samples=3
  132. iops : min= 1140, max= 1580, avg=1416.67, stdev=240.90, samples=3
  133. lat (usec) : 500=19.84%, 750=60.39%, 1000=19.18%
  134. lat (msec) : 2=0.55%, 4=0.04%
  135. cpu : usr=0.00%, sys=99.83%, ctx=2, majf=0, minf=1035
  136. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  137. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  138. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  139. issued rwts: total=2560,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  140. latency : target=0, window=0, percentile=100.00%, depth=1
  141.  
  142. Run status group 0 (all jobs):
  143. READ: bw=5789MiB/s (6070MB/s), 5789MiB/s-5789MiB/s (6070MB/s-6070MB/s), io=10.0GiB (10.7GB), run=1769-1769msec
  144. async_uncached_randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  145. ...
  146. fio-3.33
  147. Starting 4 processes
  148. async_uncached_randwrite: Laying out IO file (1 file / 1024MiB)
  149.  
  150. async_uncached_randwrite: (groupid=0, jobs=4): err= 0: pid=179726: Fri Jun 14 14:19:51 2024
  151. write: IOPS=183k, BW=714MiB/s (749MB/s)(4096MiB/5738msec); 0 zone resets
  152. slat (usec): min=2, max=6845, avg=19.28, stdev=26.27
  153. clat (nsec): min=615, max=8403.0k, avg=664448.18, stdev=333144.55
  154. lat (usec): min=6, max=8443, avg=683.72, stdev=342.43
  155. clat percentiles (usec):
  156. | 1.00th=[ 212], 5.00th=[ 285], 10.00th=[ 330], 20.00th=[ 388],
  157. | 30.00th=[ 433], 40.00th=[ 482], 50.00th=[ 562], 60.00th=[ 717],
  158. | 70.00th=[ 857], 80.00th=[ 971], 90.00th=[ 1106], 95.00th=[ 1205],
  159. | 99.00th=[ 1401], 99.50th=[ 1483], 99.90th=[ 2245], 99.95th=[ 4047],
  160. | 99.99th=[ 7570]
  161. bw ( KiB/s): min=646768, max=895128, per=100.00%, avg=732701.82, stdev=18357.76, samples=44
  162. iops : min=161692, max=223782, avg=183175.45, stdev=4589.44, samples=44
  163. lat (nsec) : 750=0.01%
  164. lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%
  165. lat (usec) : 250=2.47%, 500=40.77%, 750=18.96%, 1000=20.21%
  166. lat (msec) : 2=17.46%, 4=0.07%, 10=0.05%
  167. cpu : usr=9.54%, sys=89.54%, ctx=1788, majf=0, minf=44
  168. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  169. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  170. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  171. issued rwts: total=0,1048576,0,0 short=0,0,0,0 dropped=0,0,0,0
  172. latency : target=0, window=0, percentile=100.00%, depth=32
  173.  
  174. Run status group 0 (all jobs):
  175. WRITE: bw=714MiB/s (749MB/s), 714MiB/s-714MiB/s (749MB/s-749MB/s), io=4096MiB (4295MB), run=5738-5738msec
  176. async_cached_randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  177. ...
  178. fio-3.33
  179. Starting 4 processes
  180. async_cached_randwrite: Laying out IO file (1 file / 1024MiB)
  181.  
  182. async_cached_randwrite: (groupid=0, jobs=4): err= 0: pid=179792: Fri Jun 14 14:19:57 2024
  183. write: IOPS=186k, BW=728MiB/s (763MB/s)(4096MiB/5628msec); 0 zone resets
  184. slat (usec): min=2, max=4901, avg=19.12, stdev=18.91
  185. clat (nsec): min=694, max=6138.4k, avg=657879.49, stdev=321720.82
  186. lat (usec): min=5, max=6160, avg=677.00, stdev=331.11
  187. clat percentiles (usec):
  188. | 1.00th=[ 174], 5.00th=[ 277], 10.00th=[ 318], 20.00th=[ 375],
  189. | 30.00th=[ 420], 40.00th=[ 469], 50.00th=[ 537], 60.00th=[ 709],
  190. | 70.00th=[ 865], 80.00th=[ 979], 90.00th=[ 1123], 95.00th=[ 1221],
  191. | 99.00th=[ 1385], 99.50th=[ 1450], 99.90th=[ 1631], 99.95th=[ 1844],
  192. | 99.99th=[ 4752]
  193. bw ( KiB/s): min=661608, max=928872, per=98.75%, avg=735917.60, stdev=18320.74, samples=43
  194. iops : min=165402, max=232218, avg=183979.40, stdev=4580.18, samples=43
  195. lat (nsec) : 750=0.01%
  196. lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%
  197. lat (usec) : 250=2.73%, 500=42.38%, 750=17.04%, 1000=19.32%
  198. lat (msec) : 2=18.49%, 4=0.02%, 10=0.02%
  199. cpu : usr=9.68%, sys=89.86%, ctx=1610, majf=0, minf=41
  200. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  201. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  202. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  203. issued rwts: total=0,1048576,0,0 short=0,0,0,0 dropped=0,0,0,0
  204. latency : target=0, window=0, percentile=100.00%, depth=32
  205.  
  206. Run status group 0 (all jobs):
  207. WRITE: bw=728MiB/s (763MB/s), 728MiB/s-728MiB/s (763MB/s-763MB/s), io=4096MiB (4295MB), run=5628-5628msec
  208. async_uncached_randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  209. ...
  210. fio-3.33
  211. Starting 4 processes
  212. async_uncached_randread: Laying out IO file (1 file / 1024MiB)
  213.  
  214. async_uncached_randread: (groupid=0, jobs=4): err= 0: pid=179852: Fri Jun 14 14:20:41 2024
  215. read: IOPS=253k, BW=989MiB/s (1037MB/s)(40.0GiB/41409msec)
  216. slat (nsec): min=1278, max=321483, avg=15252.20, stdev=7974.66
  217. clat (nsec): min=579, max=1309.8k, avg=485908.64, stdev=78576.42
  218. lat (usec): min=2, max=1450, avg=501.16, stdev=80.72
  219. clat percentiles (usec):
  220. | 1.00th=[ 194], 5.00th=[ 334], 10.00th=[ 388], 20.00th=[ 441],
  221. | 30.00th=[ 469], 40.00th=[ 486], 50.00th=[ 502], 60.00th=[ 515],
  222. | 70.00th=[ 529], 80.00th=[ 545], 90.00th=[ 562], 95.00th=[ 578],
  223. | 99.00th=[ 611], 99.50th=[ 619], 99.90th=[ 652], 99.95th=[ 668],
  224. | 99.99th=[ 824]
  225. bw ( KiB/s): min=904136, max=1491592, per=100.00%, avg=1018031.37, stdev=27356.46, samples=326
  226. iops : min=226034, max=372898, avg=254507.94, stdev=6839.12, samples=326
  227. lat (nsec) : 750=0.01%, 1000=0.01%
  228. lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  229. lat (usec) : 100=0.02%, 250=2.11%, 500=46.34%, 750=51.50%, 1000=0.01%
  230. lat (msec) : 2=0.01%
  231. cpu : usr=2.69%, sys=97.28%, ctx=2998, majf=0, minf=177
  232. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  233. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  234. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  235. issued rwts: total=10485760,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  236. latency : target=0, window=0, percentile=100.00%, depth=32
  237.  
  238. Run status group 0 (all jobs):
  239. READ: bw=989MiB/s (1037MB/s), 989MiB/s-989MiB/s (1037MB/s-1037MB/s), io=40.0GiB (42.9GB), run=41409-41409msec
  240. async_cached_randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  241. ...
  242. fio-3.33
  243. Starting 4 processes
  244. async_cached_randread: Laying out IO file (1 file / 1024MiB)
  245.  
  246. async_cached_randread: (groupid=0, jobs=4): err= 0: pid=179972: Fri Jun 14 14:21:24 2024
  247. read: IOPS=249k, BW=973MiB/s (1020MB/s)(40.0GiB/42100msec)
  248. slat (nsec): min=1288, max=340922, avg=15455.06, stdev=8099.55
  249. clat (nsec): min=704, max=1264.3k, avg=492071.05, stdev=80993.44
  250. lat (usec): min=2, max=1375, avg=507.53, stdev=83.21
  251. clat percentiles (usec):
  252. | 1.00th=[ 198], 5.00th=[ 334], 10.00th=[ 388], 20.00th=[ 445],
  253. | 30.00th=[ 474], 40.00th=[ 494], 50.00th=[ 510], 60.00th=[ 523],
  254. | 70.00th=[ 537], 80.00th=[ 553], 90.00th=[ 570], 95.00th=[ 586],
  255. | 99.00th=[ 619], 99.50th=[ 627], 99.90th=[ 652], 99.95th=[ 668],
  256. | 99.99th=[ 742]
  257. bw ( KiB/s): min=904752, max=1496136, per=100.00%, avg=1004867.32, stdev=27759.34, samples=329
  258. iops : min=226188, max=374036, avg=251216.88, stdev=6939.86, samples=329
  259. lat (nsec) : 750=0.01%, 1000=0.01%
  260. lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  261. lat (usec) : 100=0.02%, 250=2.00%, 500=42.05%, 750=55.91%, 1000=0.01%
  262. lat (msec) : 2=0.01%
  263. cpu : usr=2.44%, sys=97.54%, ctx=2919, majf=0, minf=164
  264. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  265. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  266. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  267. issued rwts: total=10485760,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  268. latency : target=0, window=0, percentile=100.00%, depth=32
  269.  
  270. Run status group 0 (all jobs):
  271. READ: bw=973MiB/s (1020MB/s), 973MiB/s-973MiB/s (1020MB/s-1020MB/s), io=40.0GiB (42.9GB), run=42100-42100msec
  272. async_uncached_seqwrite: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  273. ...
  274. fio-3.33
  275. Starting 4 processes
  276. async_uncached_seqwrite: Laying out IO file (1 file / 2048MiB)
  277.  
  278. async_uncached_seqwrite: (groupid=0, jobs=4): err= 0: pid=180081: Fri Jun 14 14:21:26 2024
  279. write: IOPS=2174, BW=8696MiB/s (9119MB/s)(8192MiB/942msec); 0 zone resets
  280. slat (usec): min=309, max=2397, avg=603.39, stdev=204.91
  281. clat (usec): min=2, max=77817, avg=52763.46, stdev=13413.35
  282. lat (usec): min=510, max=78350, avg=53366.85, stdev=13447.82
  283. clat percentiles (usec):
  284. | 1.00th=[ 8848], 5.00th=[34341], 10.00th=[34866], 20.00th=[43779],
  285. | 30.00th=[49021], 40.00th=[51643], 50.00th=[54789], 60.00th=[55837],
  286. | 70.00th=[57934], 80.00th=[63701], 90.00th=[72877], 95.00th=[72877],
  287. | 99.00th=[77071], 99.50th=[77071], 99.90th=[78119], 99.95th=[78119],
  288. | 99.99th=[78119]
  289. bw ( MiB/s): min= 8156, max= 8156, per=93.79%, avg=8156.04, stdev= 0.00, samples=4
  290. iops : min= 2039, max= 2039, avg=2039.00, stdev= 0.00, samples=4
  291. lat (usec) : 4=0.20%
  292. lat (msec) : 2=0.20%, 4=0.20%, 10=0.59%, 20=1.17%, 50=28.81%
  293. lat (msec) : 100=68.85%
  294. cpu : usr=67.37%, sys=26.70%, ctx=2112, majf=0, minf=44
  295. IO depths : 1=0.2%, 2=0.4%, 4=0.8%, 8=1.6%, 16=3.1%, 32=93.9%, >=64=0.0%
  296. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  297. complete : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.0%, 32=0.2%, 64=0.0%, >=64=0.0%
  298. issued rwts: total=0,2048,0,0 short=0,0,0,0 dropped=0,0,0,0
  299. latency : target=0, window=0, percentile=100.00%, depth=32
  300.  
  301. Run status group 0 (all jobs):
  302. WRITE: bw=8696MiB/s (9119MB/s), 8696MiB/s-8696MiB/s (9119MB/s-9119MB/s), io=8192MiB (8590MB), run=942-942msec
  303. async_cached_seqwrite: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  304. ...
  305. fio-3.33
  306. Starting 4 processes
  307. async_cached_seqwrite: Laying out IO file (1 file / 2048MiB)
  308.  
  309. async_cached_seqwrite: (groupid=0, jobs=4): err= 0: pid=180143: Fri Jun 14 14:21:27 2024
  310. write: IOPS=2167, BW=8669MiB/s (9090MB/s)(8192MiB/945msec); 0 zone resets
  311. slat (usec): min=307, max=3227, avg=605.25, stdev=251.15
  312. clat (nsec): min=1708, max=75415k, avg=52508764.23, stdev=13097546.24
  313. lat (usec): min=441, max=75938, avg=53114.01, stdev=13141.25
  314. clat percentiles (usec):
  315. | 1.00th=[ 7242], 5.00th=[34866], 10.00th=[35390], 20.00th=[42206],
  316. | 30.00th=[46400], 40.00th=[52167], 50.00th=[54264], 60.00th=[56361],
  317. | 70.00th=[58459], 80.00th=[63177], 90.00th=[68682], 95.00th=[73925],
  318. | 99.00th=[73925], 99.50th=[73925], 99.90th=[73925], 99.95th=[74974],
  319. | 99.99th=[74974]
  320. bw ( MiB/s): min= 8008, max= 8008, per=92.38%, avg=8008.00, stdev= 0.00, samples=4
  321. iops : min= 2002, max= 2002, avg=2002.00, stdev= 0.00, samples=4
  322. lat (usec) : 2=0.05%, 4=0.15%
  323. lat (msec) : 2=0.15%, 4=0.24%, 10=0.68%, 20=1.22%, 50=31.98%
  324. lat (msec) : 100=65.53%
  325. cpu : usr=65.85%, sys=30.23%, ctx=1939, majf=0, minf=55
  326. IO depths : 1=0.2%, 2=0.4%, 4=0.8%, 8=1.6%, 16=3.1%, 32=93.9%, >=64=0.0%
  327. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  328. complete : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.0%, 32=0.2%, 64=0.0%, >=64=0.0%
  329. issued rwts: total=0,2048,0,0 short=0,0,0,0 dropped=0,0,0,0
  330. latency : target=0, window=0, percentile=100.00%, depth=32
  331.  
  332. Run status group 0 (all jobs):
  333. WRITE: bw=8669MiB/s (9090MB/s), 8669MiB/s-8669MiB/s (9090MB/s-9090MB/s), io=8192MiB (8590MB), run=945-945msec
  334. async_uncached_seqread: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  335. ...
  336. fio-3.33
  337. Starting 4 processes
  338. async_uncached_seqread: Laying out IO file (1 file / 1024MiB)
  339.  
  340. async_uncached_seqread: (groupid=0, jobs=4): err= 0: pid=180202: Fri Jun 14 14:21:36 2024
  341. read: IOPS=6469, BW=25.3GiB/s (27.1GB/s)(200GiB/7914msec)
  342. slat (usec): min=326, max=3150, avg=606.32, stdev=125.71
  343. clat (nsec): min=1338, max=86629k, avg=17542612.08, stdev=4715128.31
  344. lat (usec): min=504, max=89410, avg=18148.93, stdev=4795.21
  345. clat percentiles (usec):
  346. | 1.00th=[ 1139], 5.00th=[ 6652], 10.00th=[13829], 20.00th=[17695],
  347. | 30.00th=[18220], 40.00th=[18220], 50.00th=[18482], 60.00th=[18482],
  348. | 70.00th=[18482], 80.00th=[19006], 90.00th=[19792], 95.00th=[22938],
  349. | 99.00th=[24511], 99.50th=[27919], 99.90th=[68682], 99.95th=[78119],
  350. | 99.99th=[85459]
  351. bw ( MiB/s): min=16768, max=27136, per=99.52%, avg=25753.60, stdev=633.31, samples=60
  352. iops : min= 4192, max= 6784, avg=6438.40, stdev=158.33, samples=60
  353. lat (usec) : 2=0.28%, 4=0.12%, 750=0.39%
  354. lat (msec) : 2=0.78%, 4=1.38%, 10=4.41%, 20=86.23%, 50=6.23%
  355. lat (msec) : 100=0.18%
  356. cpu : usr=0.24%, sys=98.03%, ctx=124062, majf=0, minf=131122
  357. IO depths : 1=0.4%, 2=0.8%, 4=1.6%, 8=3.1%, 16=6.2%, 32=87.9%, >=64=0.0%
  358. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  359. complete : 0=0.0%, 4=99.6%, 8=0.0%, 16=0.0%, 32=0.4%, 64=0.0%, >=64=0.0%
  360. issued rwts: total=51200,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  361. latency : target=0, window=0, percentile=100.00%, depth=32
  362.  
  363. Run status group 0 (all jobs):
  364. READ: bw=25.3GiB/s (27.1GB/s), 25.3GiB/s-25.3GiB/s (27.1GB/s-27.1GB/s), io=200GiB (215GB), run=7914-7914msec
  365. async_cached_seqread: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  366. ...
  367. fio-3.33
  368. Starting 4 processes
  369. async_cached_seqread: Laying out IO file (1 file / 1024MiB)
  370.  
  371. async_cached_seqread: (groupid=0, jobs=4): err= 0: pid=180265: Fri Jun 14 14:21:45 2024
  372. read: IOPS=6384, BW=24.9GiB/s (26.8GB/s)(200GiB/8019msec)
  373. slat (usec): min=341, max=2384, avg=615.79, stdev=86.10
  374. clat (nsec): min=1264, max=68198k, avg=17822811.11, stdev=4221717.22
  375. lat (usec): min=510, max=70414, avg=18438.60, stdev=4263.77
  376. clat percentiles (usec):
  377. | 1.00th=[ 1057], 5.00th=[ 6783], 10.00th=[14091], 20.00th=[17695],
  378. | 30.00th=[19006], 40.00th=[19006], 50.00th=[19006], 60.00th=[19006],
  379. | 70.00th=[19006], 80.00th=[19268], 90.00th=[20317], 95.00th=[20579],
  380. | 99.00th=[20841], 99.50th=[22414], 99.90th=[44827], 99.95th=[53216],
  381. | 99.99th=[65274]
  382. bw ( MiB/s): min=20832, max=26912, per=99.82%, avg=25494.00, stdev=337.73, samples=64
  383. iops : min= 5208, max= 6728, avg=6373.50, stdev=84.43, samples=64
  384. lat (usec) : 2=0.27%, 4=0.12%, 750=0.39%
  385. lat (msec) : 2=0.77%, 4=1.40%, 10=4.34%, 20=76.44%, 50=16.20%
  386. lat (msec) : 100=0.06%
  387. cpu : usr=0.15%, sys=98.04%, ctx=126221, majf=0, minf=131124
  388. IO depths : 1=0.4%, 2=0.8%, 4=1.6%, 8=3.1%, 16=6.2%, 32=87.9%, >=64=0.0%
  389. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  390. complete : 0=0.0%, 4=99.6%, 8=0.0%, 16=0.0%, 32=0.4%, 64=0.0%, >=64=0.0%
  391. issued rwts: total=51200,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  392. latency : target=0, window=0, percentile=100.00%, depth=32
  393.  
  394. Run status group 0 (all jobs):
  395. READ: bw=24.9GiB/s (26.8GB/s), 24.9GiB/s-24.9GiB/s (26.8GB/s-26.8GB/s), io=200GiB (215GB), run=8019-8019msec
  396. Linux 6.7.4 (rescue) 06/14/2024 _x86_64_ (48 CPU)
  397.  
  398. avg-cpu: %user %nice %system %iowait %steal %idle
  399. 0.12 0.00 0.93 0.01 0.00 98.94
  400.  
  401. Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
  402. loop0 0.37 8.37 0.00 0.00 440029 0 0
  403. nvme0n1 287.44 7.82 16689.99 0.00 411350 877923890 0
  404. nvme1n1 254.39 6.59 14678.51 0.00 346552 772116456 0
  405. nvme2n1 222.29 6.08 12701.99 0.00 319753 668147846 0
  406. nvme3n1 191.65 5.42 11037.89 0.00 284960 580613133 0
  407. nvme4n1 162.40 4.45 8826.86 0.00 234019 464308704 0
  408. nvme5n1 127.70 3.69 7272.60 0.00 193976 382551781 0
  409. nvme6n1 81.17 2.26 4492.77 0.00 118885 236327806 0
  410. nvme7n1 54.34 1.52 3023.27 0.00 79798 159029295 0
  411.  
  412.  
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!