API tools faq
paste
Guest User

8DiskZFSRaidz3.txt

a guest
Jun 14th, 2024
68
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 24.32 KB | None | 0 0
raw download clone embed print report
  1. pool: tank
  2. state: ONLINE
  3. config:
  4.  
  5. NAME STATE READ WRITE CKSUM
  6. tank ONLINE 0 0 0
  7. raidz3-0 ONLINE 0 0 0
  8. nvme0n1 ONLINE 0 0 0
  9. nvme1n1 ONLINE 0 0 0
  10. nvme2n1 ONLINE 0 0 0
  11. nvme3n1 ONLINE 0 0 0
  12. nvme4n1 ONLINE 0 0 0
  13. nvme5n1 ONLINE 0 0 0
  14. nvme6n1 ONLINE 0 0 0
  15. nvme7n1 ONLINE 0 0 0
  16.  
  17. errors: No known data errors
  18. NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
  19. tank 14.0T 1.30M 14.0T - - 0% 0% 1.00x ONLINE -
  20. Version 2.00a ------Sequential Output------ --Sequential Input- --Random-
  21. -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
  22. Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
  23. rescue 16G::4 1.5g 90 1.7g 95 3.9g 99 +++++ +++
  24. Latency 7488us 779us 97us 1248us
  25. Version 2.00a ------Sequential Create------ --------Random Create--------
  26. rescue -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
  27. files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
  28. 1 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
  29. Latency 411us 2us 183us 271us 4us 96us
  30. 1.98,2.00a,rescue,4,1718533739,16G,,8192,5,,,1567349,90,1772846,95,,,4100462,99,+++++,+++,1,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,,7488us,779us,,97us,1248us,411us,2us,183us,271us,4us,96us
  31. sync_randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
  32. fio-3.33
  33. Starting 1 process
  34. sync_randwrite: Laying out IO file (1 file / 1024MiB)
  35.  
  36. sync_randwrite: (groupid=0, jobs=1): err= 0: pid=196692: Fri Jun 14 15:13:11 2024
  37. write: IOPS=11.9k, BW=46.5MiB/s (48.8MB/s)(1024MiB/22001msec); 0 zone resets
  38. clat (usec): min=47, max=11985, avg=82.52, stdev=75.12
  39. lat (usec): min=47, max=11985, avg=82.54, stdev=75.13
  40. clat percentiles (usec):
  41. | 1.00th=[ 59], 5.00th=[ 62], 10.00th=[ 64], 20.00th=[ 66],
  42. | 30.00th=[ 68], 40.00th=[ 71], 50.00th=[ 76], 60.00th=[ 81],
  43. | 70.00th=[ 86], 80.00th=[ 94], 90.00th=[ 105], 95.00th=[ 118],
  44. | 99.00th=[ 194], 99.50th=[ 239], 99.90th=[ 404], 99.95th=[ 494],
  45. | 99.99th=[ 1156]
  46. bw ( KiB/s): min=36656, max=59704, per=99.45%, avg=47399.07, stdev=5139.13, samples=43
  47. iops : min= 9164, max=14926, avg=11849.77, stdev=1284.78, samples=43
  48. lat (usec) : 50=0.01%, 100=86.51%, 250=13.07%, 500=0.37%, 750=0.03%
  49. lat (usec) : 1000=0.01%
  50. lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%
  51. cpu : usr=2.40%, sys=28.30%, ctx=263725, majf=0, minf=9
  52. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  53. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  54. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  55. issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
  56. latency : target=0, window=0, percentile=100.00%, depth=1
  57.  
  58. Run status group 0 (all jobs):
  59. WRITE: bw=46.5MiB/s (48.8MB/s), 46.5MiB/s-46.5MiB/s (48.8MB/s-48.8MB/s), io=1024MiB (1074MB), run=22001-22001msec
  60. sync_randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
  61. fio-3.33
  62. Starting 1 process
  63. sync_randread: Laying out IO file (1 file / 1024MiB)
  64.  
  65. sync_randread: (groupid=0, jobs=1): err= 0: pid=196809: Fri Jun 14 15:13:47 2024
  66. read: IOPS=76.8k, BW=300MiB/s (314MB/s)(10.0GiB/34153msec)
  67. clat (nsec): min=1051, max=129155, avg=12815.41, stdev=7508.96
  68. lat (nsec): min=1069, max=130136, avg=12842.49, stdev=7509.83
  69. clat percentiles (nsec):
  70. | 1.00th=[ 1544], 5.00th=[ 1880], 10.00th=[ 1992], 20.00th=[ 2160],
  71. | 30.00th=[ 2416], 40.00th=[17280], 50.00th=[17536], 60.00th=[17792],
  72. | 70.00th=[18048], 80.00th=[18304], 90.00th=[18816], 95.00th=[19328],
  73. | 99.00th=[20864], 99.50th=[21632], 99.90th=[23168], 99.95th=[24192],
  74. | 99.99th=[30080]
  75. bw ( KiB/s): min=275000, max=488848, per=99.63%, avg=305873.53, stdev=46961.24, samples=68
  76. iops : min=68750, max=122210, avg=76468.38, stdev=11740.24, samples=68
  77. lat (usec) : 2=10.61%, 4=22.05%, 10=0.03%, 20=65.39%, 50=1.92%
  78. lat (usec) : 100=0.01%, 250=0.01%
  79. cpu : usr=2.25%, sys=97.75%, ctx=3, majf=0, minf=11
  80. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  81. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  82. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  83. issued rwts: total=2621440,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  84. latency : target=0, window=0, percentile=100.00%, depth=1
  85.  
  86. Run status group 0 (all jobs):
  87. READ: bw=300MiB/s (314MB/s), 300MiB/s-300MiB/s (314MB/s-314MB/s), io=10.0GiB (10.7GB), run=34153-34153msec
  88. sync_seqwrite: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1
  89. fio-3.33
  90. Starting 1 process
  91. sync_seqwrite: Laying out IO file (1 file / 5120MiB)
  92.  
  93. sync_seqwrite: (groupid=0, jobs=1): err= 0: pid=196900: Fri Jun 14 15:13:51 2024
  94. write: IOPS=370, BW=1482MiB/s (1554MB/s)(5120MiB/3454msec); 0 zone resets
  95. clat (usec): min=1143, max=4273, avg=1658.53, stdev=539.67
  96. lat (usec): min=1143, max=4274, avg=1658.77, stdev=539.91
  97. clat percentiles (usec):
  98. | 1.00th=[ 1188], 5.00th=[ 1237], 10.00th=[ 1303], 20.00th=[ 1352],
  99. | 30.00th=[ 1401], 40.00th=[ 1434], 50.00th=[ 1483], 60.00th=[ 1516],
  100. | 70.00th=[ 1582], 80.00th=[ 1663], 90.00th=[ 2638], 95.00th=[ 3032],
  101. | 99.00th=[ 3556], 99.50th=[ 3818], 99.90th=[ 4228], 99.95th=[ 4293],
  102. | 99.99th=[ 4293]
  103. bw ( MiB/s): min= 832, max= 1800, per=96.51%, avg=1430.67, stdev=387.76, samples=6
  104. iops : min= 208, max= 450, avg=357.67, stdev=96.94, samples=6
  105. lat (msec) : 2=86.25%, 4=13.52%, 10=0.23%
  106. cpu : usr=38.26%, sys=22.18%, ctx=3787, majf=0, minf=11
  107. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  108. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  109. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  110. issued rwts: total=0,1280,0,0 short=0,0,0,0 dropped=0,0,0,0
  111. latency : target=0, window=0, percentile=100.00%, depth=1
  112.  
  113. Run status group 0 (all jobs):
  114. WRITE: bw=1482MiB/s (1554MB/s), 1482MiB/s-1482MiB/s (1554MB/s-1554MB/s), io=5120MiB (5369MB), run=3454-3454msec
  115. sync_seqread: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1
  116. fio-3.33
  117. Starting 1 process
  118. sync_seqread: Laying out IO file (1 file / 1024MiB)
  119.  
  120. sync_seqread: (groupid=0, jobs=1): err= 0: pid=196960: Fri Jun 14 15:13:53 2024
  121. read: IOPS=1449, BW=5798MiB/s (6080MB/s)(10.0GiB/1766msec)
  122. clat (usec): min=353, max=2438, avg=683.71, stdev=185.94
  123. lat (usec): min=353, max=2439, avg=683.75, stdev=185.95
  124. clat percentiles (usec):
  125. | 1.00th=[ 355], 5.00th=[ 355], 10.00th=[ 359], 20.00th=[ 506],
  126. | 30.00th=[ 701], 40.00th=[ 709], 50.00th=[ 717], 60.00th=[ 717],
  127. | 70.00th=[ 725], 80.00th=[ 734], 90.00th=[ 947], 95.00th=[ 979],
  128. | 99.00th=[ 1020], 99.50th=[ 1029], 99.90th=[ 1057], 99.95th=[ 1057],
  129. | 99.99th=[ 2442]
  130. bw ( MiB/s): min= 4768, max= 6200, per=98.33%, avg=5701.33, stdev=808.92, samples=3
  131. iops : min= 1192, max= 1550, avg=1425.33, stdev=202.23, samples=3
  132. lat (usec) : 500=19.84%, 750=64.73%, 1000=12.85%
  133. lat (msec) : 2=2.54%, 4=0.04%
  134. cpu : usr=0.17%, sys=99.66%, ctx=2, majf=0, minf=1034
  135. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  136. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  137. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  138. issued rwts: total=2560,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  139. latency : target=0, window=0, percentile=100.00%, depth=1
  140.  
  141. Run status group 0 (all jobs):
  142. READ: bw=5798MiB/s (6080MB/s), 5798MiB/s-5798MiB/s (6080MB/s-6080MB/s), io=10.0GiB (10.7GB), run=1766-1766msec
  143. async_uncached_randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  144. ...
  145. fio-3.33
  146. Starting 4 processes
  147. async_uncached_randwrite: Laying out IO file (1 file / 1024MiB)
  148.  
  149. async_uncached_randwrite: (groupid=0, jobs=4): err= 0: pid=197013: Fri Jun 14 15:14:00 2024
  150. write: IOPS=180k, BW=702MiB/s (736MB/s)(4096MiB/5834msec); 0 zone resets
  151. slat (usec): min=2, max=6770, avg=19.72, stdev=22.40
  152. clat (nsec): min=797, max=9091.5k, avg=685199.44, stdev=290870.48
  153. lat (usec): min=6, max=9136, avg=704.92, stdev=299.00
  154. clat percentiles (usec):
  155. | 1.00th=[ 225], 5.00th=[ 310], 10.00th=[ 347], 20.00th=[ 408],
  156. | 30.00th=[ 502], 40.00th=[ 594], 50.00th=[ 668], 60.00th=[ 742],
  157. | 70.00th=[ 824], 80.00th=[ 922], 90.00th=[ 1057], 95.00th=[ 1156],
  158. | 99.00th=[ 1352], 99.50th=[ 1434], 99.90th=[ 2278], 99.95th=[ 3130],
  159. | 99.99th=[ 5866]
  160. bw ( KiB/s): min=628208, max=792344, per=97.57%, avg=701477.82, stdev=11461.67, samples=44
  161. iops : min=157052, max=198086, avg=175369.27, stdev=2865.39, samples=44
  162. lat (nsec) : 1000=0.01%
  163. lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%
  164. lat (usec) : 250=1.55%, 500=28.19%, 750=31.84%, 1000=24.78%
  165. lat (msec) : 2=13.52%, 4=0.08%, 10=0.04%
  166. cpu : usr=11.52%, sys=87.62%, ctx=1972, majf=0, minf=43
  167. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  168. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  169. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  170. issued rwts: total=0,1048576,0,0 short=0,0,0,0 dropped=0,0,0,0
  171. latency : target=0, window=0, percentile=100.00%, depth=32
  172.  
  173. Run status group 0 (all jobs):
  174. WRITE: bw=702MiB/s (736MB/s), 702MiB/s-702MiB/s (736MB/s-736MB/s), io=4096MiB (4295MB), run=5834-5834msec
  175. async_cached_randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  176. ...
  177. fio-3.33
  178. Starting 4 processes
  179. async_cached_randwrite: Laying out IO file (1 file / 1024MiB)
  180.  
  181. async_cached_randwrite: (groupid=0, jobs=4): err= 0: pid=197079: Fri Jun 14 15:14:06 2024
  182. write: IOPS=181k, BW=707MiB/s (741MB/s)(4096MiB/5795msec); 0 zone resets
  183. slat (usec): min=2, max=10546, avg=19.55, stdev=21.50
  184. clat (nsec): min=626, max=11671k, avg=677281.94, stdev=296576.42
  185. lat (usec): min=3, max=11701, avg=696.83, stdev=304.95
  186. clat percentiles (usec):
  187. | 1.00th=[ 184], 5.00th=[ 293], 10.00th=[ 334], 20.00th=[ 400],
  188. | 30.00th=[ 482], 40.00th=[ 578], 50.00th=[ 660], 60.00th=[ 742],
  189. | 70.00th=[ 824], 80.00th=[ 922], 90.00th=[ 1057], 95.00th=[ 1156],
  190. | 99.00th=[ 1369], 99.50th=[ 1467], 99.90th=[ 1958], 99.95th=[ 2835],
  191. | 99.99th=[ 5080]
  192. bw ( KiB/s): min=624512, max=791560, per=97.90%, avg=708586.18, stdev=12859.61, samples=44
  193. iops : min=156128, max=197890, avg=177146.55, stdev=3214.90, samples=44
  194. lat (nsec) : 750=0.01%, 1000=0.01%
  195. lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%
  196. lat (usec) : 250=3.06%, 500=28.99%, 750=29.39%, 1000=24.85%
  197. lat (msec) : 2=13.61%, 4=0.08%, 10=0.02%, 20=0.01%
  198. cpu : usr=10.25%, sys=89.05%, ctx=1754, majf=0, minf=39
  199. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  200. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  201. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  202. issued rwts: total=0,1048576,0,0 short=0,0,0,0 dropped=0,0,0,0
  203. latency : target=0, window=0, percentile=100.00%, depth=32
  204.  
  205. Run status group 0 (all jobs):
  206. WRITE: bw=707MiB/s (741MB/s), 707MiB/s-707MiB/s (741MB/s-741MB/s), io=4096MiB (4295MB), run=5795-5795msec
  207. async_uncached_randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  208. ...
  209. fio-3.33
  210. Starting 4 processes
  211. async_uncached_randread: Laying out IO file (1 file / 1024MiB)
  212.  
  213. async_uncached_randread: (groupid=0, jobs=4): err= 0: pid=197180: Fri Jun 14 15:14:50 2024
  214. read: IOPS=249k, BW=974MiB/s (1021MB/s)(40.0GiB/42061msec)
  215. slat (nsec): min=1288, max=272322, avg=15392.66, stdev=8057.87
  216. clat (nsec): min=596, max=1422.3k, avg=489888.26, stdev=79687.26
  217. lat (usec): min=2, max=1462, avg=505.28, stdev=81.86
  218. clat percentiles (usec):
  219. | 1.00th=[ 194], 5.00th=[ 334], 10.00th=[ 392], 20.00th=[ 445],
  220. | 30.00th=[ 474], 40.00th=[ 490], 50.00th=[ 506], 60.00th=[ 523],
  221. | 70.00th=[ 537], 80.00th=[ 553], 90.00th=[ 570], 95.00th=[ 586],
  222. | 99.00th=[ 611], 99.50th=[ 619], 99.90th=[ 644], 99.95th=[ 660],
  223. | 99.99th=[ 889]
  224. bw ( KiB/s): min=937240, max=1466264, per=100.00%, avg=1009990.43, stdev=27335.89, samples=329
  225. iops : min=234310, max=366566, avg=252497.63, stdev=6833.96, samples=329
  226. lat (nsec) : 750=0.01%, 1000=0.01%
  227. lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  228. lat (usec) : 100=0.02%, 250=2.09%, 500=43.39%, 750=54.47%, 1000=0.02%
  229. lat (msec) : 2=0.01%
  230. cpu : usr=2.47%, sys=97.51%, ctx=2973, majf=0, minf=175
  231. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  232. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  233. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  234. issued rwts: total=10485760,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  235. latency : target=0, window=0, percentile=100.00%, depth=32
  236.  
  237. Run status group 0 (all jobs):
  238. READ: bw=974MiB/s (1021MB/s), 974MiB/s-974MiB/s (1021MB/s-1021MB/s), io=40.0GiB (42.9GB), run=42061-42061msec
  239. async_cached_randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  240. ...
  241. fio-3.33
  242. Starting 4 processes
  243. async_cached_randread: Laying out IO file (1 file / 1024MiB)
  244.  
  245. async_cached_randread: (groupid=0, jobs=4): err= 0: pid=197266: Fri Jun 14 15:15:32 2024
  246. read: IOPS=254k, BW=991MiB/s (1039MB/s)(40.0GiB/41322msec)
  247. slat (nsec): min=1283, max=214296, avg=15223.09, stdev=7967.85
  248. clat (nsec): min=679, max=1357.9k, avg=484693.18, stdev=79045.11
  249. lat (usec): min=2, max=1427, avg=499.92, stdev=81.21
  250. clat percentiles (usec):
  251. | 1.00th=[ 194], 5.00th=[ 330], 10.00th=[ 383], 20.00th=[ 437],
  252. | 30.00th=[ 465], 40.00th=[ 486], 50.00th=[ 502], 60.00th=[ 515],
  253. | 70.00th=[ 529], 80.00th=[ 545], 90.00th=[ 562], 95.00th=[ 578],
  254. | 99.00th=[ 603], 99.50th=[ 611], 99.90th=[ 635], 99.95th=[ 652],
  255. | 99.99th=[ 873]
  256. bw ( KiB/s): min=919576, max=1504936, per=100.00%, avg=1020461.54, stdev=27755.21, samples=325
  257. iops : min=229894, max=376234, avg=255115.43, stdev=6938.80, samples=325
  258. lat (nsec) : 750=0.01%, 1000=0.01%
  259. lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  260. lat (usec) : 100=0.02%, 250=2.10%, 500=46.85%, 750=50.99%, 1000=0.02%
  261. lat (msec) : 2=0.01%
  262. cpu : usr=2.51%, sys=97.47%, ctx=2927, majf=0, minf=176
  263. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  264. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  265. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  266. issued rwts: total=10485760,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  267. latency : target=0, window=0, percentile=100.00%, depth=32
  268.  
  269. Run status group 0 (all jobs):
  270. READ: bw=991MiB/s (1039MB/s), 991MiB/s-991MiB/s (1039MB/s-1039MB/s), io=40.0GiB (42.9GB), run=41322-41322msec
  271. async_uncached_seqwrite: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  272. ...
  273. fio-3.33
  274. Starting 4 processes
  275. async_uncached_seqwrite: Laying out IO file (1 file / 2048MiB)
  276.  
  277. async_uncached_seqwrite: (groupid=0, jobs=4): err= 0: pid=197390: Fri Jun 14 15:15:34 2024
  278. write: IOPS=2007, BW=8031MiB/s (8422MB/s)(8192MiB/1020msec); 0 zone resets
  279. slat (usec): min=311, max=5829, avg=633.17, stdev=264.50
  280. clat (nsec): min=1784, max=85982k, avg=57702875.46, stdev=14966363.75
  281. lat (usec): min=424, max=87063, avg=58336.05, stdev=15053.13
  282. clat percentiles (usec):
  283. | 1.00th=[ 6783], 5.00th=[34866], 10.00th=[37487], 20.00th=[47449],
  284. | 30.00th=[51643], 40.00th=[55837], 50.00th=[58459], 60.00th=[62129],
  285. | 70.00th=[66847], 80.00th=[72877], 90.00th=[74974], 95.00th=[76022],
  286. | 99.00th=[80217], 99.50th=[83362], 99.90th=[85459], 99.95th=[85459],
  287. | 99.99th=[85459]
  288. bw ( MiB/s): min= 6936, max= 7592, per=90.45%, avg=7264.00, stdev=150.18, samples=6
  289. iops : min= 1734, max= 1898, avg=1816.00, stdev=37.54, samples=6
  290. lat (usec) : 2=0.05%, 4=0.15%
  291. lat (msec) : 2=0.20%, 4=0.29%, 10=0.78%, 20=1.42%, 50=20.70%
  292. lat (msec) : 100=76.42%
  293. cpu : usr=67.05%, sys=28.09%, ctx=1886, majf=0, minf=45
  294. IO depths : 1=0.2%, 2=0.4%, 4=0.8%, 8=1.6%, 16=3.1%, 32=93.9%, >=64=0.0%
  295. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  296. complete : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.0%, 32=0.2%, 64=0.0%, >=64=0.0%
  297. issued rwts: total=0,2048,0,0 short=0,0,0,0 dropped=0,0,0,0
  298. latency : target=0, window=0, percentile=100.00%, depth=32
  299.  
  300. Run status group 0 (all jobs):
  301. WRITE: bw=8031MiB/s (8422MB/s), 8031MiB/s-8031MiB/s (8422MB/s-8422MB/s), io=8192MiB (8590MB), run=1020-1020msec
  302. async_cached_seqwrite: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  303. ...
  304. fio-3.33
  305. Starting 4 processes
  306. async_cached_seqwrite: Laying out IO file (1 file / 2048MiB)
  307.  
  308. async_cached_seqwrite: (groupid=0, jobs=4): err= 0: pid=197455: Fri Jun 14 15:15:35 2024
  309. write: IOPS=2000, BW=8000MiB/s (8389MB/s)(8192MiB/1024msec); 0 zone resets
  310. slat (usec): min=319, max=3438, avg=666.19, stdev=324.73
  311. clat (nsec): min=1753, max=84855k, avg=55271374.96, stdev=16515342.59
  312. lat (usec): min=454, max=85395, avg=55937.56, stdev=16618.65
  313. clat percentiles (usec):
  314. | 1.00th=[ 7242], 5.00th=[35390], 10.00th=[35390], 20.00th=[37487],
  315. | 30.00th=[45351], 40.00th=[54264], 50.00th=[56886], 60.00th=[58983],
  316. | 70.00th=[65799], 80.00th=[72877], 90.00th=[74974], 95.00th=[80217],
  317. | 99.00th=[82314], 99.50th=[83362], 99.90th=[84411], 99.95th=[84411],
  318. | 99.99th=[84411]
  319. bw ( MiB/s): min= 7435, max= 7435, per=92.95%, avg=7435.85, stdev= 0.00, samples=4
  320. iops : min= 1858, max= 1858, avg=1858.00, stdev= 0.00, samples=4
  321. lat (usec) : 2=0.10%, 4=0.10%
  322. lat (msec) : 2=0.20%, 4=0.24%, 10=0.68%, 20=1.27%, 50=33.64%
  323. lat (msec) : 100=63.77%
  324. cpu : usr=65.24%, sys=31.43%, ctx=1450, majf=0, minf=45
  325. IO depths : 1=0.2%, 2=0.4%, 4=0.8%, 8=1.6%, 16=3.1%, 32=93.9%, >=64=0.0%
  326. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  327. complete : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.0%, 32=0.2%, 64=0.0%, >=64=0.0%
  328. issued rwts: total=0,2048,0,0 short=0,0,0,0 dropped=0,0,0,0
  329. latency : target=0, window=0, percentile=100.00%, depth=32
  330.  
  331. Run status group 0 (all jobs):
  332. WRITE: bw=8000MiB/s (8389MB/s), 8000MiB/s-8000MiB/s (8389MB/s-8389MB/s), io=8192MiB (8590MB), run=1024-1024msec
  333. async_uncached_seqread: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  334. ...
  335. fio-3.33
  336. Starting 4 processes
  337. async_uncached_seqread: Laying out IO file (1 file / 1024MiB)
  338.  
  339. async_uncached_seqread: (groupid=0, jobs=4): err= 0: pid=197514: Fri Jun 14 15:15:44 2024
  340. read: IOPS=6542, BW=25.6GiB/s (27.4GB/s)(200GiB/7826msec)
  341. slat (usec): min=351, max=2496, avg=600.37, stdev=88.53
  342. clat (nsec): min=1302, max=66414k, avg=17369579.56, stdev=4148625.73
  343. lat (usec): min=505, max=68650, avg=17969.95, stdev=4193.11
  344. clat percentiles (usec):
  345. | 1.00th=[ 1045], 5.00th=[ 6718], 10.00th=[13698], 20.00th=[17171],
  346. | 30.00th=[18482], 40.00th=[18482], 50.00th=[18482], 60.00th=[18482],
  347. | 70.00th=[18482], 80.00th=[18744], 90.00th=[19792], 95.00th=[20317],
  348. | 99.00th=[20579], 99.50th=[21627], 99.90th=[46400], 99.95th=[55313],
  349. | 99.99th=[64750]
  350. bw ( MiB/s): min=21024, max=27712, per=99.79%, avg=26114.13, stdev=374.47, samples=60
  351. iops : min= 5256, max= 6928, avg=6528.53, stdev=93.62, samples=60
  352. lat (usec) : 2=0.29%, 4=0.10%, 750=0.39%
  353. lat (msec) : 2=0.78%, 4=1.41%, 10=4.38%, 20=82.79%, 50=9.78%
  354. lat (msec) : 100=0.08%
  355. cpu : usr=0.22%, sys=98.24%, ctx=120454, majf=0, minf=131121
  356. IO depths : 1=0.4%, 2=0.8%, 4=1.6%, 8=3.1%, 16=6.2%, 32=87.9%, >=64=0.0%
  357. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  358. complete : 0=0.0%, 4=99.6%, 8=0.0%, 16=0.0%, 32=0.4%, 64=0.0%, >=64=0.0%
  359. issued rwts: total=51200,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  360. latency : target=0, window=0, percentile=100.00%, depth=32
  361.  
  362. Run status group 0 (all jobs):
  363. READ: bw=25.6GiB/s (27.4GB/s), 25.6GiB/s-25.6GiB/s (27.4GB/s-27.4GB/s), io=200GiB (215GB), run=7826-7826msec
  364. async_cached_seqread: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  365. ...
  366. fio-3.33
  367. Starting 4 processes
  368. async_cached_seqread: Laying out IO file (1 file / 1024MiB)
  369.  
  370. async_cached_seqread: (groupid=0, jobs=4): err= 0: pid=197578: Fri Jun 14 15:15:53 2024
  371. read: IOPS=6373, BW=24.9GiB/s (26.7GB/s)(200GiB/8033msec)
  372. slat (usec): min=352, max=3040, avg=615.64, stdev=110.80
  373. clat (nsec): min=1405, max=79002k, avg=17818115.88, stdev=4508965.90
  374. lat (usec): min=503, max=81522, avg=18433.75, stdev=4572.48
  375. clat percentiles (usec):
  376. | 1.00th=[ 1057], 5.00th=[ 6783], 10.00th=[14091], 20.00th=[17957],
  377. | 30.00th=[18744], 40.00th=[18744], 50.00th=[18744], 60.00th=[19006],
  378. | 70.00th=[19006], 80.00th=[19006], 90.00th=[20055], 95.00th=[20317],
  379. | 99.00th=[24249], 99.50th=[25822], 99.90th=[60556], 99.95th=[69731],
  380. | 99.99th=[77071]
  381. bw ( MiB/s): min=17472, max=26656, per=99.80%, avg=25444.00, stdev=533.29, samples=64
  382. iops : min= 4368, max= 6664, avg=6361.00, stdev=133.32, samples=64
  383. lat (usec) : 2=0.28%, 4=0.11%, 10=0.01%, 750=0.39%, 1000=0.01%
  384. lat (msec) : 2=0.78%, 4=1.39%, 10=4.32%, 20=78.63%, 50=13.95%
  385. lat (msec) : 100=0.15%
  386. cpu : usr=0.36%, sys=97.94%, ctx=124165, majf=0, minf=131120
  387. IO depths : 1=0.4%, 2=0.8%, 4=1.6%, 8=3.1%, 16=6.2%, 32=87.9%, >=64=0.0%
  388. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  389. complete : 0=0.0%, 4=99.6%, 8=0.0%, 16=0.0%, 32=0.4%, 64=0.0%, >=64=0.0%
  390. issued rwts: total=51200,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  391. latency : target=0, window=0, percentile=100.00%, depth=32
  392.  
  393. Run status group 0 (all jobs):
  394. READ: bw=24.9GiB/s (26.7GB/s), 24.9GiB/s-24.9GiB/s (26.7GB/s-26.7GB/s), io=200GiB (215GB), run=8033-8033msec
  395. Linux 6.7.4 (rescue) 06/14/2024 _x86_64_ (48 CPU)
  396.  
  397. avg-cpu: %user %nice %system %iowait %steal %idle
  398. 0.13 0.00 1.24 0.01 0.00 98.62
  399.  
  400. Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
  401. loop0 0.35 7.88 0.00 0.00 440029 0 0
  402. nvme0n1 417.23 8.70 22599.94 0.00 485828 1262208420 0
  403. nvme1n1 380.39 7.43 20681.87 0.00 415030 1155083874 0
  404. nvme2n1 351.67 6.95 18826.61 0.00 388043 1051467540 0
  405. nvme3n1 290.05 6.12 15672.85 0.00 341851 875329976 0
  406. nvme4n1 258.67 5.01 12365.79 0.00 279555 690630352 0
  407. nvme5n1 182.50 4.08 9715.94 0.00 228121 542636110 0
  408. nvme6n1 120.60 2.54 6057.77 0.00 141619 338326816 0
  409. nvme7n1 78.29 1.63 3778.09 0.00 91241 211006542 0
  410.  
  411.  
  412.  
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!