API tools faq
paste
Guest User

8DiskZFSRaidz2.txt

a guest
Jun 14th, 2024
67
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 24.32 KB | None | 0 0
raw download clone embed print report
  1. pool: tank
  2. state: ONLINE
  3. config:
  4.  
  5. NAME STATE READ WRITE CKSUM
  6. tank ONLINE 0 0 0
  7. raidz2-0 ONLINE 0 0 0
  8. nvme0n1 ONLINE 0 0 0
  9. nvme1n1 ONLINE 0 0 0
  10. nvme2n1 ONLINE 0 0 0
  11. nvme3n1 ONLINE 0 0 0
  12. nvme4n1 ONLINE 0 0 0
  13. nvme5n1 ONLINE 0 0 0
  14. nvme6n1 ONLINE 0 0 0
  15. nvme7n1 ONLINE 0 0 0
  16.  
  17. errors: No known data errors
  18. NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
  19. tank 14.0T 1.37M 14.0T - - 0% 0% 1.00x ONLINE -
  20. Version 2.00a ------Sequential Output------ --Sequential Input- --Random-
  21. -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
  22. Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
  23. rescue 16G::4 1.5g 91 1.6g 96 4.2g 99 +++++ +++
  24. Latency 3942us 907us 61us 1509us
  25. Version 2.00a ------Sequential Create------ --------Random Create--------
  26. rescue -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
  27. files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
  28. 1 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
  29. Latency 312us 2us 159us 144us 1us 311us
  30. 1.98,2.00a,rescue,4,1718543627,16G,,8192,5,,,1602650,91,1721674,96,,,4417571,99,+++++,+++,1,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,,3942us,907us,,61us,1509us,312us,2us,159us,144us,1us,311us
  31. sync_randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
  32. fio-3.33
  33. Starting 1 process
  34. sync_randwrite: Laying out IO file (1 file / 1024MiB)
  35.  
  36. sync_randwrite: (groupid=0, jobs=1): err= 0: pid=188140: Fri Jun 14 14:46:04 2024
  37. write: IOPS=12.4k, BW=48.4MiB/s (50.8MB/s)(1024MiB/21143msec); 0 zone resets
  38. clat (usec): min=47, max=11682, avg=79.07, stdev=79.67
  39. lat (usec): min=47, max=11682, avg=79.09, stdev=79.68
  40. clat percentiles (usec):
  41. | 1.00th=[ 57], 5.00th=[ 61], 10.00th=[ 62], 20.00th=[ 64],
  42. | 30.00th=[ 66], 40.00th=[ 68], 50.00th=[ 70], 60.00th=[ 73],
  43. | 70.00th=[ 82], 80.00th=[ 89], 90.00th=[ 99], 95.00th=[ 114],
  44. | 99.00th=[ 204], 99.50th=[ 243], 99.90th=[ 553], 99.95th=[ 594],
  45. | 99.99th=[ 1369]
  46. bw ( KiB/s): min=29256, max=60360, per=99.89%, avg=49539.62, stdev=5922.09, samples=42
  47. iops : min= 7314, max=15090, avg=12384.86, stdev=1480.49, samples=42
  48. lat (usec) : 50=0.04%, 100=90.70%, 250=8.80%, 500=0.31%, 750=0.14%
  49. lat (usec) : 1000=0.01%
  50. lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%
  51. cpu : usr=2.71%, sys=30.22%, ctx=263854, majf=0, minf=10
  52. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  53. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  54. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  55. issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
  56. latency : target=0, window=0, percentile=100.00%, depth=1
  57.  
  58. Run status group 0 (all jobs):
  59. WRITE: bw=48.4MiB/s (50.8MB/s), 48.4MiB/s-48.4MiB/s (50.8MB/s-50.8MB/s), io=1024MiB (1074MB), run=21143-21143msec
  60. sync_randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
  61. fio-3.33
  62. Starting 1 process
  63. sync_randread: Laying out IO file (1 file / 1024MiB)
  64.  
  65. sync_randread: (groupid=0, jobs=1): err= 0: pid=188218: Fri Jun 14 14:46:40 2024
  66. read: IOPS=76.5k, BW=299MiB/s (313MB/s)(10.0GiB/34264msec)
  67. clat (nsec): min=1048, max=212040, avg=12856.75, stdev=7504.91
  68. lat (nsec): min=1066, max=212892, avg=12884.09, stdev=7505.85
  69. clat percentiles (nsec):
  70. | 1.00th=[ 1512], 5.00th=[ 1880], 10.00th=[ 1992], 20.00th=[ 2160],
  71. | 30.00th=[ 2416], 40.00th=[17280], 50.00th=[17536], 60.00th=[17792],
  72. | 70.00th=[18048], 80.00th=[18304], 90.00th=[18816], 95.00th=[19328],
  73. | 99.00th=[21120], 99.50th=[21632], 99.90th=[23168], 99.95th=[24192],
  74. | 99.99th=[29824]
  75. bw ( KiB/s): min=262432, max=490712, per=99.57%, avg=304704.71, stdev=44831.80, samples=68
  76. iops : min=65608, max=122678, avg=76176.24, stdev=11207.96, samples=68
  77. lat (usec) : 2=10.58%, 4=21.89%, 10=0.03%, 20=65.31%, 50=2.19%
  78. lat (usec) : 100=0.01%, 250=0.01%
  79. cpu : usr=2.06%, sys=97.93%, ctx=5, majf=0, minf=9
  80. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  81. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  82. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  83. issued rwts: total=2621440,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  84. latency : target=0, window=0, percentile=100.00%, depth=1
  85.  
  86. Run status group 0 (all jobs):
  87. READ: bw=299MiB/s (313MB/s), 299MiB/s-299MiB/s (313MB/s-313MB/s), io=10.0GiB (10.7GB), run=34264-34264msec
  88. sync_seqwrite: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1
  89. fio-3.33
  90. Starting 1 process
  91. sync_seqwrite: Laying out IO file (1 file / 5120MiB)
  92.  
  93. sync_seqwrite: (groupid=0, jobs=1): err= 0: pid=188347: Fri Jun 14 14:46:43 2024
  94. write: IOPS=398, BW=1595MiB/s (1672MB/s)(5120MiB/3210msec); 0 zone resets
  95. clat (usec): min=1130, max=4894, avg=1559.72, stdev=563.06
  96. lat (usec): min=1130, max=4895, avg=1559.97, stdev=563.34
  97. clat percentiles (usec):
  98. | 1.00th=[ 1156], 5.00th=[ 1172], 10.00th=[ 1188], 20.00th=[ 1237],
  99. | 30.00th=[ 1287], 40.00th=[ 1319], 50.00th=[ 1401], 60.00th=[ 1434],
  100. | 70.00th=[ 1483], 80.00th=[ 1565], 90.00th=[ 2606], 95.00th=[ 2769],
  101. | 99.00th=[ 3752], 99.50th=[ 4080], 99.90th=[ 4293], 99.95th=[ 4883],
  102. | 99.99th=[ 4883]
  103. bw ( MiB/s): min= 808, max= 1952, per=98.56%, avg=1572.00, stdev=459.42, samples=6
  104. iops : min= 202, max= 488, avg=393.00, stdev=114.85, samples=6
  105. lat (msec) : 2=88.13%, 4=11.33%, 10=0.55%
  106. cpu : usr=39.58%, sys=22.22%, ctx=3629, majf=0, minf=11
  107. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  108. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  109. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  110. issued rwts: total=0,1280,0,0 short=0,0,0,0 dropped=0,0,0,0
  111. latency : target=0, window=0, percentile=100.00%, depth=1
  112.  
  113. Run status group 0 (all jobs):
  114. WRITE: bw=1595MiB/s (1672MB/s), 1595MiB/s-1595MiB/s (1672MB/s-1672MB/s), io=5120MiB (5369MB), run=3210-3210msec
  115. sync_seqread: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1
  116. fio-3.33
  117. Starting 1 process
  118. sync_seqread: Laying out IO file (1 file / 1024MiB)
  119.  
  120. sync_seqread: (groupid=0, jobs=1): err= 0: pid=188413: Fri Jun 14 14:46:46 2024
  121. read: IOPS=1421, BW=5686MiB/s (5962MB/s)(10.0GiB/1801msec)
  122. clat (usec): min=359, max=2830, avg=696.78, stdev=188.84
  123. lat (usec): min=359, max=2830, avg=696.82, stdev=188.84
  124. clat percentiles (usec):
  125. | 1.00th=[ 359], 5.00th=[ 363], 10.00th=[ 363], 20.00th=[ 519],
  126. | 30.00th=[ 701], 40.00th=[ 709], 50.00th=[ 717], 60.00th=[ 725],
  127. | 70.00th=[ 734], 80.00th=[ 750], 90.00th=[ 963], 95.00th=[ 971],
  128. | 99.00th=[ 988], 99.50th=[ 996], 99.90th=[ 1020], 99.95th=[ 1029],
  129. | 99.99th=[ 2835]
  130. bw ( MiB/s): min= 4496, max= 6168, per=97.88%, avg=5565.33, stdev=928.56, samples=3
  131. iops : min= 1124, max= 1542, avg=1391.33, stdev=232.14, samples=3
  132. lat (usec) : 500=19.77%, 750=60.08%, 1000=19.73%
  133. lat (msec) : 2=0.39%, 4=0.04%
  134. cpu : usr=0.00%, sys=99.83%, ctx=2, majf=0, minf=1034
  135. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  136. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  137. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  138. issued rwts: total=2560,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  139. latency : target=0, window=0, percentile=100.00%, depth=1
  140.  
  141. Run status group 0 (all jobs):
  142. READ: bw=5686MiB/s (5962MB/s), 5686MiB/s-5686MiB/s (5962MB/s-5962MB/s), io=10.0GiB (10.7GB), run=1801-1801msec
  143. async_uncached_randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  144. ...
  145. fio-3.33
  146. Starting 4 processes
  147. async_uncached_randwrite: Laying out IO file (1 file / 1024MiB)
  148.  
  149. async_uncached_randwrite: (groupid=0, jobs=4): err= 0: pid=188468: Fri Jun 14 14:46:52 2024
  150. write: IOPS=180k, BW=704MiB/s (738MB/s)(4096MiB/5818msec); 0 zone resets
  151. slat (usec): min=2, max=10108, avg=19.61, stdev=28.03
  152. clat (nsec): min=922, max=14861k, avg=678433.35, stdev=330757.19
  153. lat (usec): min=4, max=14891, avg=698.04, stdev=339.96
  154. clat percentiles (usec):
  155. | 1.00th=[ 258], 5.00th=[ 306], 10.00th=[ 343], 20.00th=[ 396],
  156. | 30.00th=[ 441], 40.00th=[ 506], 50.00th=[ 619], 60.00th=[ 742],
  157. | 70.00th=[ 848], 80.00th=[ 955], 90.00th=[ 1090], 95.00th=[ 1205],
  158. | 99.00th=[ 1385], 99.50th=[ 1483], 99.90th=[ 2540], 99.95th=[ 3556],
  159. | 99.99th=[ 8848]
  160. bw ( KiB/s): min=635696, max=832184, per=98.87%, avg=712749.09, stdev=14086.73, samples=44
  161. iops : min=158924, max=208046, avg=178187.27, stdev=3521.68, samples=44
  162. lat (nsec) : 1000=0.01%
  163. lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%
  164. lat (usec) : 250=0.78%, 500=38.71%, 750=21.22%, 1000=22.73%
  165. lat (msec) : 2=16.40%, 4=0.12%, 10=0.03%, 20=0.01%
  166. cpu : usr=10.05%, sys=88.90%, ctx=2035, majf=0, minf=44
  167. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  168. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  169. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  170. issued rwts: total=0,1048576,0,0 short=0,0,0,0 dropped=0,0,0,0
  171. latency : target=0, window=0, percentile=100.00%, depth=32
  172.  
  173. Run status group 0 (all jobs):
  174. WRITE: bw=704MiB/s (738MB/s), 704MiB/s-704MiB/s (738MB/s-738MB/s), io=4096MiB (4295MB), run=5818-5818msec
  175. async_cached_randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  176. ...
  177. fio-3.33
  178. Starting 4 processes
  179. async_cached_randwrite: Laying out IO file (1 file / 1024MiB)
  180.  
  181. async_cached_randwrite: (groupid=0, jobs=4): err= 0: pid=188532: Fri Jun 14 14:46:59 2024
  182. write: IOPS=181k, BW=705MiB/s (740MB/s)(4096MiB/5807msec); 0 zone resets
  183. slat (usec): min=2, max=14971, avg=19.56, stdev=28.55
  184. clat (nsec): min=946, max=16506k, avg=679006.99, stdev=329030.04
  185. lat (usec): min=6, max=16539, avg=698.57, stdev=338.00
  186. clat percentiles (usec):
  187. | 1.00th=[ 245], 5.00th=[ 310], 10.00th=[ 347], 20.00th=[ 396],
  188. | 30.00th=[ 441], 40.00th=[ 498], 50.00th=[ 619], 60.00th=[ 750],
  189. | 70.00th=[ 857], 80.00th=[ 963], 90.00th=[ 1106], 95.00th=[ 1205],
  190. | 99.00th=[ 1401], 99.50th=[ 1483], 99.90th=[ 1795], 99.95th=[ 2376],
  191. | 99.99th=[ 5014]
  192. bw ( KiB/s): min=657992, max=835272, per=98.95%, avg=714708.36, stdev=13319.60, samples=44
  193. iops : min=164498, max=208818, avg=178677.09, stdev=3329.90, samples=44
  194. lat (nsec) : 1000=0.01%
  195. lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%
  196. lat (usec) : 250=1.15%, 500=39.01%, 750=20.01%, 1000=22.86%
  197. lat (msec) : 2=16.89%, 4=0.07%, 10=0.01%, 20=0.01%
  198. cpu : usr=9.07%, sys=90.24%, ctx=1622, majf=0, minf=42
  199. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  200. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  201. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  202. issued rwts: total=0,1048576,0,0 short=0,0,0,0 dropped=0,0,0,0
  203. latency : target=0, window=0, percentile=100.00%, depth=32
  204.  
  205. Run status group 0 (all jobs):
  206. WRITE: bw=705MiB/s (740MB/s), 705MiB/s-705MiB/s (740MB/s-740MB/s), io=4096MiB (4295MB), run=5807-5807msec
  207. async_uncached_randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  208. ...
  209. fio-3.33
  210. Starting 4 processes
  211. async_uncached_randread: Laying out IO file (1 file / 1024MiB)
  212.  
  213. async_uncached_randread: (groupid=0, jobs=4): err= 0: pid=188595: Fri Jun 14 14:47:42 2024
  214. read: IOPS=253k, BW=989MiB/s (1037MB/s)(40.0GiB/41404msec)
  215. slat (nsec): min=1286, max=435797, avg=15323.05, stdev=8011.03
  216. clat (nsec): min=593, max=1348.3k, avg=487898.01, stdev=78681.69
  217. lat (usec): min=2, max=1385, avg=503.22, stdev=80.82
  218. clat percentiles (usec):
  219. | 1.00th=[ 196], 5.00th=[ 334], 10.00th=[ 388], 20.00th=[ 445],
  220. | 30.00th=[ 469], 40.00th=[ 490], 50.00th=[ 506], 60.00th=[ 519],
  221. | 70.00th=[ 529], 80.00th=[ 545], 90.00th=[ 570], 95.00th=[ 578],
  222. | 99.00th=[ 611], 99.50th=[ 619], 99.90th=[ 644], 99.95th=[ 652],
  223. | 99.99th=[ 824]
  224. bw ( KiB/s): min=919960, max=1469496, per=100.00%, avg=1013633.88, stdev=26998.03, samples=327
  225. iops : min=229990, max=367374, avg=253408.49, stdev=6749.50, samples=327
  226. lat (nsec) : 750=0.01%, 1000=0.01%
  227. lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  228. lat (usec) : 100=0.02%, 250=2.05%, 500=44.78%, 750=53.13%, 1000=0.01%
  229. lat (msec) : 2=0.01%
  230. cpu : usr=2.66%, sys=97.31%, ctx=2921, majf=0, minf=172
  231. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  232. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  233. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  234. issued rwts: total=10485760,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  235. latency : target=0, window=0, percentile=100.00%, depth=32
  236.  
  237. Run status group 0 (all jobs):
  238. READ: bw=989MiB/s (1037MB/s), 989MiB/s-989MiB/s (1037MB/s-1037MB/s), io=40.0GiB (42.9GB), run=41404-41404msec
  239. async_cached_randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  240. ...
  241. fio-3.33
  242. Starting 4 processes
  243. async_cached_randread: Laying out IO file (1 file / 1024MiB)
  244.  
  245. async_cached_randread: (groupid=0, jobs=4): err= 0: pid=188737: Fri Jun 14 14:48:26 2024
  246. read: IOPS=249k, BW=974MiB/s (1022MB/s)(40.0GiB/42036msec)
  247. slat (nsec): min=1303, max=298561, avg=15460.96, stdev=8080.40
  248. clat (nsec): min=597, max=1365.7k, avg=491962.88, stdev=79784.28
  249. lat (usec): min=2, max=1427, avg=507.42, stdev=81.96
  250. clat percentiles (usec):
  251. | 1.00th=[ 196], 5.00th=[ 338], 10.00th=[ 392], 20.00th=[ 445],
  252. | 30.00th=[ 474], 40.00th=[ 494], 50.00th=[ 510], 60.00th=[ 523],
  253. | 70.00th=[ 537], 80.00th=[ 553], 90.00th=[ 570], 95.00th=[ 586],
  254. | 99.00th=[ 611], 99.50th=[ 627], 99.90th=[ 644], 99.95th=[ 652],
  255. | 99.99th=[ 783]
  256. bw ( KiB/s): min=937464, max=1494624, per=100.00%, avg=1006319.32, stdev=27728.14, samples=331
  257. iops : min=234366, max=373656, avg=251579.80, stdev=6932.03, samples=331
  258. lat (nsec) : 750=0.01%, 1000=0.01%
  259. lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  260. lat (usec) : 100=0.02%, 250=2.02%, 500=42.14%, 750=55.80%, 1000=0.01%
  261. lat (msec) : 2=0.01%
  262. cpu : usr=2.50%, sys=97.48%, ctx=2871, majf=0, minf=167
  263. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  264. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  265. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  266. issued rwts: total=10485760,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  267. latency : target=0, window=0, percentile=100.00%, depth=32
  268.  
  269. Run status group 0 (all jobs):
  270. READ: bw=974MiB/s (1022MB/s), 974MiB/s-974MiB/s (1022MB/s-1022MB/s), io=40.0GiB (42.9GB), run=42036-42036msec
  271. async_uncached_seqwrite: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  272. ...
  273. fio-3.33
  274. Starting 4 processes
  275. async_uncached_seqwrite: Laying out IO file (1 file / 2048MiB)
  276.  
  277. async_uncached_seqwrite: (groupid=0, jobs=4): err= 0: pid=188800: Fri Jun 14 14:48:27 2024
  278. write: IOPS=2045, BW=8184MiB/s (8581MB/s)(8192MiB/1001msec); 0 zone resets
  279. slat (usec): min=330, max=5109, avg=624.93, stdev=297.65
  280. clat (nsec): min=1684, max=80418k, avg=54932356.88, stdev=15645588.64
  281. lat (usec): min=424, max=81332, avg=55557.29, stdev=15707.07
  282. clat percentiles (usec):
  283. | 1.00th=[ 6718], 5.00th=[34341], 10.00th=[34866], 20.00th=[40109],
  284. | 30.00th=[48497], 40.00th=[52167], 50.00th=[55837], 60.00th=[57934],
  285. | 70.00th=[65274], 80.00th=[72877], 90.00th=[72877], 95.00th=[74974],
  286. | 99.00th=[80217], 99.50th=[80217], 99.90th=[80217], 99.95th=[80217],
  287. | 99.99th=[80217]
  288. bw ( MiB/s): min= 7291, max= 7291, per=89.09%, avg=7291.16, stdev= 0.00, samples=4
  289. iops : min= 1822, max= 1822, avg=1822.00, stdev= 0.00, samples=4
  290. lat (usec) : 2=0.10%, 4=0.05%, 10=0.05%
  291. lat (msec) : 2=0.20%, 4=0.29%, 10=0.78%, 20=1.46%, 50=30.27%
  292. lat (msec) : 100=66.80%
  293. cpu : usr=67.65%, sys=29.22%, ctx=1813, majf=0, minf=56
  294. IO depths : 1=0.2%, 2=0.4%, 4=0.8%, 8=1.6%, 16=3.1%, 32=93.9%, >=64=0.0%
  295. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  296. complete : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.0%, 32=0.2%, 64=0.0%, >=64=0.0%
  297. issued rwts: total=0,2048,0,0 short=0,0,0,0 dropped=0,0,0,0
  298. latency : target=0, window=0, percentile=100.00%, depth=32
  299.  
  300. Run status group 0 (all jobs):
  301. WRITE: bw=8184MiB/s (8581MB/s), 8184MiB/s-8184MiB/s (8581MB/s-8581MB/s), io=8192MiB (8590MB), run=1001-1001msec
  302. async_cached_seqwrite: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  303. ...
  304. fio-3.33
  305. Starting 4 processes
  306. async_cached_seqwrite: Laying out IO file (1 file / 2048MiB)
  307.  
  308. async_cached_seqwrite: (groupid=0, jobs=4): err= 0: pid=188865: Fri Jun 14 14:48:29 2024
  309. write: IOPS=1891, BW=7564MiB/s (7932MB/s)(8192MiB/1083msec); 0 zone resets
  310. slat (usec): min=302, max=4018, avg=651.01, stdev=317.34
  311. clat (usec): min=2, max=82287, avg=55305.70, stdev=15162.27
  312. lat (usec): min=539, max=82827, avg=55956.71, stdev=15222.81
  313. clat percentiles (usec):
  314. | 1.00th=[ 9110], 5.00th=[35390], 10.00th=[36439], 20.00th=[42730],
  315. | 30.00th=[45876], 40.00th=[52691], 50.00th=[56361], 60.00th=[58983],
  316. | 70.00th=[64750], 80.00th=[71828], 90.00th=[73925], 95.00th=[77071],
  317. | 99.00th=[81265], 99.50th=[82314], 99.90th=[82314], 99.95th=[82314],
  318. | 99.99th=[82314]
  319. bw ( MiB/s): min= 7772, max= 8260, per=100.00%, avg=8016.47, stdev=172.53, samples=5
  320. iops : min= 1943, max= 2065, avg=2004.00, stdev=43.13, samples=5
  321. lat (usec) : 4=0.15%, 10=0.05%
  322. lat (msec) : 2=0.15%, 4=0.20%, 10=0.59%, 20=0.98%, 50=34.86%
  323. lat (msec) : 100=63.04%
  324. cpu : usr=64.96%, sys=31.10%, ctx=1784, majf=0, minf=54
  325. IO depths : 1=0.2%, 2=0.4%, 4=0.8%, 8=1.6%, 16=3.1%, 32=93.9%, >=64=0.0%
  326. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  327. complete : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.0%, 32=0.2%, 64=0.0%, >=64=0.0%
  328. issued rwts: total=0,2048,0,0 short=0,0,0,0 dropped=0,0,0,0
  329. latency : target=0, window=0, percentile=100.00%, depth=32
  330.  
  331. Run status group 0 (all jobs):
  332. WRITE: bw=7564MiB/s (7932MB/s), 7564MiB/s-7564MiB/s (7932MB/s-7932MB/s), io=8192MiB (8590MB), run=1083-1083msec
  333. async_uncached_seqread: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  334. ...
  335. fio-3.33
  336. Starting 4 processes
  337. async_uncached_seqread: Laying out IO file (1 file / 1024MiB)
  338.  
  339. async_uncached_seqread: (groupid=0, jobs=4): err= 0: pid=188923: Fri Jun 14 14:48:38 2024
  340. read: IOPS=6479, BW=25.3GiB/s (27.2GB/s)(200GiB/7902msec)
  341. slat (usec): min=326, max=2402, avg=606.58, stdev=92.70
  342. clat (nsec): min=1334, max=67419k, avg=17572432.08, stdev=4244673.16
  343. lat (usec): min=472, max=69809, avg=18179.01, stdev=4292.44
  344. clat percentiles (usec):
  345. | 1.00th=[ 1057], 5.00th=[ 6718], 10.00th=[13829], 20.00th=[17695],
  346. | 30.00th=[18220], 40.00th=[18482], 50.00th=[18482], 60.00th=[18482],
  347. | 70.00th=[18744], 80.00th=[19530], 90.00th=[20055], 95.00th=[21365],
  348. | 99.00th=[21627], 99.50th=[21890], 99.90th=[47449], 99.95th=[56886],
  349. | 99.99th=[65274]
  350. bw ( MiB/s): min=20151, max=27392, per=99.63%, avg=25821.31, stdev=427.35, samples=60
  351. iops : min= 5036, max= 6848, avg=6455.20, stdev=106.95, samples=60
  352. lat (usec) : 2=0.21%, 4=0.18%, 500=0.01%, 750=0.38%, 1000=0.01%
  353. lat (msec) : 2=0.78%, 4=1.40%, 10=4.34%, 20=81.73%, 50=10.87%
  354. lat (msec) : 100=0.09%
  355. cpu : usr=0.16%, sys=97.97%, ctx=128965, majf=0, minf=131126
  356. IO depths : 1=0.4%, 2=0.8%, 4=1.6%, 8=3.1%, 16=6.2%, 32=87.9%, >=64=0.0%
  357. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  358. complete : 0=0.0%, 4=99.6%, 8=0.0%, 16=0.0%, 32=0.4%, 64=0.0%, >=64=0.0%
  359. issued rwts: total=51200,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  360. latency : target=0, window=0, percentile=100.00%, depth=32
  361.  
  362. Run status group 0 (all jobs):
  363. READ: bw=25.3GiB/s (27.2GB/s), 25.3GiB/s-25.3GiB/s (27.2GB/s-27.2GB/s), io=200GiB (215GB), run=7902-7902msec
  364. async_cached_seqread: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  365. ...
  366. fio-3.33
  367. Starting 4 processes
  368. async_cached_seqread: Laying out IO file (1 file / 1024MiB)
  369.  
  370. async_cached_seqread: (groupid=0, jobs=4): err= 0: pid=188990: Fri Jun 14 14:48:47 2024
  371. read: IOPS=6288, BW=24.6GiB/s (26.4GB/s)(200GiB/8142msec)
  372. slat (usec): min=365, max=2645, avg=624.15, stdev=107.57
  373. clat (nsec): min=1482, max=77861k, avg=18057606.29, stdev=4503209.37
  374. lat (usec): min=515, max=80370, avg=18681.76, stdev=4563.50
  375. clat percentiles (usec):
  376. | 1.00th=[ 1205], 5.00th=[ 6849], 10.00th=[14222], 20.00th=[18220],
  377. | 30.00th=[19006], 40.00th=[19006], 50.00th=[19006], 60.00th=[19006],
  378. | 70.00th=[19268], 80.00th=[20055], 90.00th=[20317], 95.00th=[20579],
  379. | 99.00th=[24249], 99.50th=[25297], 99.90th=[58983], 99.95th=[68682],
  380. | 99.99th=[76022]
  381. bw ( MiB/s): min=18176, max=26560, per=99.75%, avg=25090.00, stdev=466.91, samples=64
  382. iops : min= 4544, max= 6640, avg=6272.50, stdev=116.73, samples=64
  383. lat (usec) : 2=0.17%, 4=0.22%, 750=0.39%
  384. lat (msec) : 2=0.78%, 4=1.38%, 10=4.24%, 20=72.12%, 50=20.55%
  385. lat (msec) : 100=0.14%
  386. cpu : usr=0.27%, sys=98.01%, ctx=127037, majf=0, minf=131123
  387. IO depths : 1=0.4%, 2=0.8%, 4=1.6%, 8=3.1%, 16=6.2%, 32=87.9%, >=64=0.0%
  388. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  389. complete : 0=0.0%, 4=99.6%, 8=0.0%, 16=0.0%, 32=0.4%, 64=0.0%, >=64=0.0%
  390. issued rwts: total=51200,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  391. latency : target=0, window=0, percentile=100.00%, depth=32
  392.  
  393. Run status group 0 (all jobs):
  394. READ: bw=24.6GiB/s (26.4GB/s), 24.6GiB/s-24.6GiB/s (26.4GB/s-26.4GB/s), io=200GiB (215GB), run=8142-8142msec
  395. Linux 6.7.4 (rescue) 06/14/2024 _x86_64_ (48 CPU)
  396.  
  397. avg-cpu: %user %nice %system %iowait %steal %idle
  398. 0.13 0.00 1.09 0.01 0.00 98.77
  399.  
  400. Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
  401. loop0 0.36 8.12 0.00 0.00 440029 0 0
  402. nvme0n1 348.88 8.24 19354.78 0.00 446643 1049488851 0
  403. nvme1n1 315.16 7.02 17396.65 0.00 380809 943311409 0
  404. nvme2n1 285.71 6.53 15486.05 0.00 353906 839711923 0
  405. nvme3n1 236.78 5.78 13189.86 0.00 313456 715203609 0
  406. nvme4n1 210.14 4.73 10481.40 0.00 256737 568340790 0
  407. nvme5n1 154.87 3.89 8393.07 0.00 211057 455103510 0
  408. nvme6n1 99.74 2.40 5195.63 0.00 130232 281726789 0
  409. nvme7n1 65.32 1.58 3361.30 0.00 85492 182262457 0
  410.  
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!