API tools faq
paste
Guest User

2x4DiskZFSMirror.txt

a guest
Jun 14th, 2024
77
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 24.30 KB | None | 0 0
raw download clone embed print report
  1. pool: tank
  2. state: ONLINE
  3. config:
  4.  
  5. NAME STATE READ WRITE CKSUM
  6. tank ONLINE 0 0 0
  7. mirror-0 ONLINE 0 0 0
  8. nvme0n1 ONLINE 0 0 0
  9. nvme1n1 ONLINE 0 0 0
  10. nvme2n1 ONLINE 0 0 0
  11. nvme3n1 ONLINE 0 0 0
  12. mirror-1 ONLINE 0 0 0
  13. nvme4n1 ONLINE 0 0 0
  14. nvme5n1 ONLINE 0 0 0
  15. nvme6n1 ONLINE 0 0 0
  16. nvme7n1 ONLINE 0 0 0
  17.  
  18. errors: No known data errors
  19. NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
  20. tank 3.47T 356K 3.47T - - 0% 0% 1.00x ONLINE -
  21. Version 2.00a ------Sequential Output------ --Sequential Input- --Random-
  22. -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
  23. Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
  24. rescue 16G::4 1.5g 92 1.6g 96 4.2g 99 +++++ +++
  25. Latency 12965us 602us 63us 761us
  26. Version 2.00a ------Sequential Create------ --------Random Create--------
  27. rescue -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
  28. files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
  29. 1 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
  30. Latency 321us 5us 159us 118us 2us 164us
  31. 1.98,2.00a,rescue,4,1718519187,16G,,8192,5,,,1611489,92,1669309,96,,,4449813,99,+++++,+++,1,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,,12965us,602us,,63us,761us,321us,5us,159us,118us,2us,164us
  32. sync_randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
  33. fio-3.33
  34. Starting 1 process
  35. sync_randwrite: Laying out IO file (1 file / 1024MiB)
  36.  
  37. sync_randwrite: (groupid=0, jobs=1): err= 0: pid=177921: Fri Jun 14 14:14:35 2024
  38. write: IOPS=13.2k, BW=51.5MiB/s (54.0MB/s)(1024MiB/19868msec); 0 zone resets
  39. clat (usec): min=41, max=13667, avg=74.17, stdev=88.45
  40. lat (usec): min=41, max=13667, avg=74.20, stdev=88.45
  41. clat percentiles (usec):
  42. | 1.00th=[ 53], 5.00th=[ 56], 10.00th=[ 58], 20.00th=[ 60],
  43. | 30.00th=[ 61], 40.00th=[ 62], 50.00th=[ 64], 60.00th=[ 67],
  44. | 70.00th=[ 77], 80.00th=[ 83], 90.00th=[ 93], 95.00th=[ 109],
  45. | 99.00th=[ 221], 99.50th=[ 330], 99.90th=[ 383], 99.95th=[ 424],
  46. | 99.99th=[ 676]
  47. bw ( KiB/s): min=38776, max=62312, per=99.67%, avg=52605.54, stdev=7046.49, samples=39
  48. iops : min= 9694, max=15578, avg=13151.38, stdev=1761.62, samples=39
  49. lat (usec) : 50=0.21%, 100=92.93%, 250=6.00%, 500=0.83%, 750=0.01%
  50. lat (usec) : 1000=0.01%
  51. lat (msec) : 4=0.01%, 10=0.01%, 20=0.01%
  52. cpu : usr=3.09%, sys=30.79%, ctx=263155, majf=0, minf=9
  53. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  54. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  55. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  56. issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
  57. latency : target=0, window=0, percentile=100.00%, depth=1
  58.  
  59. Run status group 0 (all jobs):
  60. WRITE: bw=51.5MiB/s (54.0MB/s), 51.5MiB/s-51.5MiB/s (54.0MB/s-54.0MB/s), io=1024MiB (1074MB), run=19868-19868msec
  61. sync_randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
  62. fio-3.33
  63. Starting 1 process
  64. sync_randread: Laying out IO file (1 file / 1024MiB)
  65.  
  66. sync_randread: (groupid=0, jobs=1): err= 0: pid=177987: Fri Jun 14 14:15:11 2024
  67. read: IOPS=76.8k, BW=300MiB/s (314MB/s)(10.0GiB/34153msec)
  68. clat (nsec): min=1062, max=128125, avg=12815.53, stdev=7498.67
  69. lat (nsec): min=1074, max=129038, avg=12843.14, stdev=7499.52
  70. clat percentiles (nsec):
  71. | 1.00th=[ 1528], 5.00th=[ 1864], 10.00th=[ 1976], 20.00th=[ 2128],
  72. | 30.00th=[ 2384], 40.00th=[17280], 50.00th=[17536], 60.00th=[17792],
  73. | 70.00th=[18048], 80.00th=[18304], 90.00th=[18816], 95.00th=[19328],
  74. | 99.00th=[20352], 99.50th=[20864], 99.90th=[22400], 99.95th=[23424],
  75. | 99.99th=[27008]
  76. bw ( KiB/s): min=258176, max=481552, per=99.63%, avg=305891.53, stdev=46508.25, samples=68
  77. iops : min=64544, max=120388, avg=76473.00, stdev=11626.99, samples=68
  78. lat (usec) : 2=11.36%, 4=21.18%, 10=0.02%, 20=66.00%, 50=1.44%
  79. lat (usec) : 100=0.01%, 250=0.01%
  80. cpu : usr=2.18%, sys=97.82%, ctx=5, majf=0, minf=9
  81. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  82. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  83. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  84. issued rwts: total=2621440,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  85. latency : target=0, window=0, percentile=100.00%, depth=1
  86.  
  87. Run status group 0 (all jobs):
  88. READ: bw=300MiB/s (314MB/s), 300MiB/s-300MiB/s (314MB/s-314MB/s), io=10.0GiB (10.7GB), run=34153-34153msec
  89. sync_seqwrite: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1
  90. fio-3.33
  91. Starting 1 process
  92. sync_seqwrite: Laying out IO file (1 file / 5120MiB)
  93.  
  94. sync_seqwrite: (groupid=0, jobs=1): err= 0: pid=178102: Fri Jun 14 14:15:14 2024
  95. write: IOPS=437, BW=1749MiB/s (1834MB/s)(5120MiB/2928msec); 0 zone resets
  96. clat (usec): min=1042, max=3514, avg=1353.70, stdev=324.99
  97. lat (usec): min=1042, max=3515, avg=1353.91, stdev=325.15
  98. clat percentiles (usec):
  99. | 1.00th=[ 1057], 5.00th=[ 1090], 10.00th=[ 1106], 20.00th=[ 1139],
  100. | 30.00th=[ 1172], 40.00th=[ 1221], 50.00th=[ 1270], 60.00th=[ 1319],
  101. | 70.00th=[ 1369], 80.00th=[ 1434], 90.00th=[ 1827], 95.00th=[ 1991],
  102. | 99.00th=[ 2737], 99.50th=[ 2900], 99.90th=[ 3392], 99.95th=[ 3523],
  103. | 99.99th=[ 3523]
  104. bw ( MiB/s): min= 1016, max= 2032, per=97.36%, avg=1702.40, stdev=429.56, samples=5
  105. iops : min= 254, max= 508, avg=425.60, stdev=107.39, samples=5
  106. lat (msec) : 2=95.31%, 4=4.69%
  107. cpu : usr=43.18%, sys=23.57%, ctx=3555, majf=0, minf=11
  108. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  109. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  110. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  111. issued rwts: total=0,1280,0,0 short=0,0,0,0 dropped=0,0,0,0
  112. latency : target=0, window=0, percentile=100.00%, depth=1
  113.  
  114. Run status group 0 (all jobs):
  115. WRITE: bw=1749MiB/s (1834MB/s), 1749MiB/s-1749MiB/s (1834MB/s-1834MB/s), io=5120MiB (5369MB), run=2928-2928msec
  116. sync_seqread: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1
  117. fio-3.33
  118. Starting 1 process
  119. sync_seqread: Laying out IO file (1 file / 1024MiB)
  120.  
  121. sync_seqread: (groupid=0, jobs=1): err= 0: pid=178165: Fri Jun 14 14:15:17 2024
  122. read: IOPS=1352, BW=5409MiB/s (5672MB/s)(10.0GiB/1893msec)
  123. clat (usec): min=346, max=3061, avg=732.68, stdev=204.24
  124. lat (usec): min=346, max=3062, avg=732.73, stdev=204.25
  125. clat percentiles (usec):
  126. | 1.00th=[ 351], 5.00th=[ 355], 10.00th=[ 359], 20.00th=[ 465],
  127. | 30.00th=[ 685], 40.00th=[ 693], 50.00th=[ 701], 60.00th=[ 857],
  128. | 70.00th=[ 930], 80.00th=[ 938], 90.00th=[ 947], 95.00th=[ 955],
  129. | 99.00th=[ 963], 99.50th=[ 971], 99.90th=[ 996], 99.95th=[ 1012],
  130. | 99.99th=[ 3064]
  131. bw ( MiB/s): min= 4584, max= 6072, per=94.65%, avg=5120.00, stdev=826.64, samples=3
  132. iops : min= 1146, max= 1518, avg=1280.00, stdev=206.66, samples=3
  133. lat (usec) : 500=20.31%, 750=39.61%, 1000=40.00%
  134. lat (msec) : 2=0.04%, 4=0.04%
  135. cpu : usr=0.16%, sys=99.79%, ctx=2, majf=0, minf=1034
  136. IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  137. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  138. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  139. issued rwts: total=2560,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  140. latency : target=0, window=0, percentile=100.00%, depth=1
  141.  
  142. Run status group 0 (all jobs):
  143. READ: bw=5409MiB/s (5672MB/s), 5409MiB/s-5409MiB/s (5672MB/s-5672MB/s), io=10.0GiB (10.7GB), run=1893-1893msec
  144. async_uncached_randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  145. ...
  146. fio-3.33
  147. Starting 4 processes
  148. async_uncached_randwrite: Laying out IO file (1 file / 1024MiB)
  149.  
  150. async_uncached_randwrite: (groupid=0, jobs=4): err= 0: pid=178219: Fri Jun 14 14:15:22 2024
  151. write: IOPS=244k, BW=953MiB/s (1000MB/s)(4096MiB/4297msec); 0 zone resets
  152. slat (usec): min=2, max=2916, avg=14.40, stdev=14.55
  153. clat (nsec): min=787, max=5829.9k, avg=502807.45, stdev=228491.28
  154. lat (usec): min=3, max=5886, avg=517.21, stdev=235.11
  155. clat percentiles (usec):
  156. | 1.00th=[ 262], 5.00th=[ 289], 10.00th=[ 306], 20.00th=[ 330],
  157. | 30.00th=[ 359], 40.00th=[ 388], 50.00th=[ 424], 60.00th=[ 478],
  158. | 70.00th=[ 545], 80.00th=[ 660], 90.00th=[ 832], 95.00th=[ 963],
  159. | 99.00th=[ 1205], 99.50th=[ 1287], 99.90th=[ 1729], 99.95th=[ 2376],
  160. | 99.99th=[ 4146]
  161. bw ( KiB/s): min=670232, max=1253296, per=98.86%, avg=964986.00, stdev=53936.87, samples=32
  162. iops : min=167558, max=313324, avg=241246.50, stdev=13484.22, samples=32
  163. lat (nsec) : 1000=0.01%
  164. lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%
  165. lat (usec) : 250=0.62%, 500=63.22%, 750=22.01%, 1000=10.24%
  166. lat (msec) : 2=3.84%, 4=0.06%, 10=0.01%
  167. cpu : usr=10.67%, sys=88.44%, ctx=2952, majf=0, minf=43
  168. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  169. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  170. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  171. issued rwts: total=0,1048576,0,0 short=0,0,0,0 dropped=0,0,0,0
  172. latency : target=0, window=0, percentile=100.00%, depth=32
  173.  
  174. Run status group 0 (all jobs):
  175. WRITE: bw=953MiB/s (1000MB/s), 953MiB/s-953MiB/s (1000MB/s-1000MB/s), io=4096MiB (4295MB), run=4297-4297msec
  176. async_cached_randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  177. ...
  178. fio-3.33
  179. Starting 4 processes
  180. async_cached_randwrite: Laying out IO file (1 file / 1024MiB)
  181.  
  182. async_cached_randwrite: (groupid=0, jobs=4): err= 0: pid=178279: Fri Jun 14 14:15:26 2024
  183. write: IOPS=251k, BW=980MiB/s (1028MB/s)(4096MiB/4178msec); 0 zone resets
  184. slat (usec): min=2, max=4956, avg=14.07, stdev=15.08
  185. clat (nsec): min=779, max=6293.5k, avg=491186.57, stdev=225919.84
  186. lat (usec): min=4, max=6342, avg=505.26, stdev=232.39
  187. clat percentiles (usec):
  188. | 1.00th=[ 258], 5.00th=[ 285], 10.00th=[ 302], 20.00th=[ 326],
  189. | 30.00th=[ 351], 40.00th=[ 379], 50.00th=[ 412], 60.00th=[ 457],
  190. | 70.00th=[ 523], 80.00th=[ 635], 90.00th=[ 824], 95.00th=[ 955],
  191. | 99.00th=[ 1205], 99.50th=[ 1303], 99.90th=[ 1631], 99.95th=[ 2008],
  192. | 99.99th=[ 4113]
  193. bw ( KiB/s): min=672880, max=1310952, per=99.12%, avg=995044.00, stdev=57178.84, samples=32
  194. iops : min=168220, max=327738, avg=248761.00, stdev=14294.71, samples=32
  195. lat (nsec) : 1000=0.01%
  196. lat (usec) : 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%, 250=0.84%
  197. lat (usec) : 500=66.21%, 750=19.39%, 1000=9.79%
  198. lat (msec) : 2=3.72%, 4=0.04%, 10=0.01%
  199. cpu : usr=10.10%, sys=89.09%, ctx=2417, majf=0, minf=39
  200. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  201. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  202. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  203. issued rwts: total=0,1048576,0,0 short=0,0,0,0 dropped=0,0,0,0
  204. latency : target=0, window=0, percentile=100.00%, depth=32
  205.  
  206. Run status group 0 (all jobs):
  207. WRITE: bw=980MiB/s (1028MB/s), 980MiB/s-980MiB/s (1028MB/s-1028MB/s), io=4096MiB (4295MB), run=4178-4178msec
  208. async_uncached_randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  209. ...
  210. fio-3.33
  211. Starting 4 processes
  212. async_uncached_randread: Laying out IO file (1 file / 1024MiB)
  213.  
  214. async_uncached_randread: (groupid=0, jobs=4): err= 0: pid=178336: Fri Jun 14 14:16:09 2024
  215. read: IOPS=254k, BW=994MiB/s (1042MB/s)(40.0GiB/41216msec)
  216. slat (nsec): min=1283, max=254799, avg=15198.28, stdev=7973.05
  217. clat (nsec): min=574, max=1303.7k, avg=483918.82, stdev=78779.10
  218. lat (usec): min=2, max=1469, avg=499.12, stdev=80.93
  219. clat percentiles (usec):
  220. | 1.00th=[ 194], 5.00th=[ 330], 10.00th=[ 383], 20.00th=[ 437],
  221. | 30.00th=[ 465], 40.00th=[ 486], 50.00th=[ 502], 60.00th=[ 515],
  222. | 70.00th=[ 529], 80.00th=[ 545], 90.00th=[ 562], 95.00th=[ 578],
  223. | 99.00th=[ 603], 99.50th=[ 611], 99.90th=[ 635], 99.95th=[ 652],
  224. | 99.99th=[ 693]
  225. bw ( KiB/s): min=902936, max=1511152, per=100.00%, avg=1023437.48, stdev=28147.18, samples=326
  226. iops : min=225734, max=377788, avg=255859.37, stdev=7036.81, samples=326
  227. lat (nsec) : 750=0.01%, 1000=0.01%
  228. lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  229. lat (usec) : 100=0.02%, 250=2.09%, 500=47.45%, 750=50.43%, 1000=0.01%
  230. lat (msec) : 2=0.01%
  231. cpu : usr=2.61%, sys=97.36%, ctx=2867, majf=0, minf=172
  232. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  233. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  234. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  235. issued rwts: total=10485760,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  236. latency : target=0, window=0, percentile=100.00%, depth=32
  237.  
  238. Run status group 0 (all jobs):
  239. READ: bw=994MiB/s (1042MB/s), 994MiB/s-994MiB/s (1042MB/s-1042MB/s), io=40.0GiB (42.9GB), run=41216-41216msec
  240. async_cached_randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
  241. ...
  242. fio-3.33
  243. Starting 4 processes
  244. async_cached_randread: Laying out IO file (1 file / 1024MiB)
  245.  
  246. async_cached_randread: (groupid=0, jobs=4): err= 0: pid=178452: Fri Jun 14 14:16:52 2024
  247. read: IOPS=253k, BW=989MiB/s (1038MB/s)(40.0GiB/41397msec)
  248. slat (nsec): min=1279, max=130721, avg=15240.48, stdev=7973.58
  249. clat (nsec): min=657, max=1430.1k, avg=484989.40, stdev=79146.86
  250. lat (nsec): min=1993, max=1526.6k, avg=500229.88, stdev=81311.54
  251. clat percentiles (usec):
  252. | 1.00th=[ 194], 5.00th=[ 330], 10.00th=[ 383], 20.00th=[ 437],
  253. | 30.00th=[ 469], 40.00th=[ 486], 50.00th=[ 502], 60.00th=[ 515],
  254. | 70.00th=[ 529], 80.00th=[ 545], 90.00th=[ 562], 95.00th=[ 578],
  255. | 99.00th=[ 603], 99.50th=[ 611], 99.90th=[ 635], 99.95th=[ 652],
  256. | 99.99th=[ 750]
  257. bw ( KiB/s): min=950160, max=1508440, per=100.00%, avg=1019919.32, stdev=28030.00, samples=325
  258. iops : min=237540, max=377110, avg=254979.88, stdev=7007.51, samples=325
  259. lat (nsec) : 750=0.01%, 1000=0.01%
  260. lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  261. lat (usec) : 100=0.02%, 250=2.09%, 500=46.48%, 750=51.39%, 1000=0.01%
  262. lat (msec) : 2=0.01%
  263. cpu : usr=2.56%, sys=97.42%, ctx=2903, majf=0, minf=163
  264. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
  265. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  266. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
  267. issued rwts: total=10485760,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  268. latency : target=0, window=0, percentile=100.00%, depth=32
  269.  
  270. Run status group 0 (all jobs):
  271. READ: bw=989MiB/s (1038MB/s), 989MiB/s-989MiB/s (1038MB/s-1038MB/s), io=40.0GiB (42.9GB), run=41397-41397msec
  272. async_uncached_seqwrite: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  273. ...
  274. fio-3.33
  275. Starting 4 processes
  276. async_uncached_seqwrite: Laying out IO file (1 file / 2048MiB)
  277.  
  278. async_uncached_seqwrite: (groupid=0, jobs=4): err= 0: pid=178537: Fri Jun 14 14:16:54 2024
  279. write: IOPS=2058, BW=8233MiB/s (8633MB/s)(8192MiB/995msec); 0 zone resets
  280. slat (usec): min=317, max=5125, avg=674.92, stdev=373.15
  281. clat (nsec): min=1962, max=82856k, avg=52844226.65, stdev=13988193.71
  282. lat (usec): min=426, max=83870, avg=53519.15, stdev=14132.49
  283. clat percentiles (usec):
  284. | 1.00th=[ 8029], 5.00th=[34341], 10.00th=[34866], 20.00th=[40633],
  285. | 30.00th=[46400], 40.00th=[51643], 50.00th=[55313], 60.00th=[55837],
  286. | 70.00th=[57934], 80.00th=[63177], 90.00th=[70779], 95.00th=[76022],
  287. | 99.00th=[82314], 99.50th=[82314], 99.90th=[82314], 99.95th=[82314],
  288. | 99.99th=[83362]
  289. bw ( MiB/s): min= 7992, max= 7992, per=97.07%, avg=7992.00, stdev= 0.00, samples=4
  290. iops : min= 1998, max= 1998, avg=1998.00, stdev= 0.00, samples=4
  291. lat (usec) : 2=0.05%, 4=0.15%
  292. lat (msec) : 2=0.15%, 4=0.20%, 10=0.68%, 20=1.12%, 50=33.74%
  293. lat (msec) : 100=63.92%
  294. cpu : usr=60.50%, sys=33.78%, ctx=1842, majf=0, minf=49
  295. IO depths : 1=0.2%, 2=0.4%, 4=0.8%, 8=1.6%, 16=3.1%, 32=93.9%, >=64=0.0%
  296. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  297. complete : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.0%, 32=0.2%, 64=0.0%, >=64=0.0%
  298. issued rwts: total=0,2048,0,0 short=0,0,0,0 dropped=0,0,0,0
  299. latency : target=0, window=0, percentile=100.00%, depth=32
  300.  
  301. Run status group 0 (all jobs):
  302. WRITE: bw=8233MiB/s (8633MB/s), 8233MiB/s-8233MiB/s (8633MB/s-8633MB/s), io=8192MiB (8590MB), run=995-995msec
  303. async_cached_seqwrite: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  304. ...
  305. fio-3.33
  306. Starting 4 processes
  307. async_cached_seqwrite: Laying out IO file (1 file / 2048MiB)
  308.  
  309. async_cached_seqwrite: (groupid=0, jobs=4): err= 0: pid=178603: Fri Jun 14 14:16:55 2024
  310. write: IOPS=1848, BW=7394MiB/s (7753MB/s)(8192MiB/1108msec); 0 zone resets
  311. slat (usec): min=306, max=8676, avg=713.24, stdev=444.76
  312. clat (usec): min=2, max=89175, avg=55367.92, stdev=14440.98
  313. lat (usec): min=323, max=90358, avg=56081.17, stdev=14571.66
  314. clat percentiles (usec):
  315. | 1.00th=[ 7570], 5.00th=[37487], 10.00th=[41681], 20.00th=[43779],
  316. | 30.00th=[46400], 40.00th=[49021], 50.00th=[57410], 60.00th=[60031],
  317. | 70.00th=[62653], 80.00th=[67634], 90.00th=[73925], 95.00th=[77071],
  318. | 99.00th=[84411], 99.50th=[85459], 99.90th=[87557], 99.95th=[88605],
  319. | 99.99th=[89654]
  320. bw ( MiB/s): min= 8312, max= 8441, per=100.00%, avg=8377.08, stdev=45.49, samples=5
  321. iops : min= 2078, max= 2110, avg=2094.00, stdev=11.31, samples=5
  322. lat (usec) : 4=0.20%
  323. lat (msec) : 2=0.20%, 4=0.24%, 10=0.73%, 20=1.12%, 50=39.55%
  324. lat (msec) : 100=57.96%
  325. cpu : usr=62.33%, sys=31.09%, ctx=2015, majf=0, minf=47
  326. IO depths : 1=0.2%, 2=0.4%, 4=0.8%, 8=1.6%, 16=3.1%, 32=93.9%, >=64=0.0%
  327. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  328. complete : 0=0.0%, 4=99.8%, 8=0.0%, 16=0.0%, 32=0.2%, 64=0.0%, >=64=0.0%
  329. issued rwts: total=0,2048,0,0 short=0,0,0,0 dropped=0,0,0,0
  330. latency : target=0, window=0, percentile=100.00%, depth=32
  331.  
  332. Run status group 0 (all jobs):
  333. WRITE: bw=7394MiB/s (7753MB/s), 7394MiB/s-7394MiB/s (7753MB/s-7753MB/s), io=8192MiB (8590MB), run=1108-1108msec
  334. async_uncached_seqread: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  335. ...
  336. fio-3.33
  337. Starting 4 processes
  338. async_uncached_seqread: Laying out IO file (1 file / 1024MiB)
  339.  
  340. async_uncached_seqread: (groupid=0, jobs=4): err= 0: pid=178666: Fri Jun 14 14:17:04 2024
  341. read: IOPS=6281, BW=24.5GiB/s (26.3GB/s)(200GiB/8151msec)
  342. slat (usec): min=348, max=2639, avg=618.25, stdev=93.79
  343. clat (nsec): min=1309, max=67675k, avg=17904670.30, stdev=4418360.52
  344. lat (usec): min=507, max=69922, avg=18522.92, stdev=4470.03
  345. clat percentiles (usec):
  346. | 1.00th=[ 1188], 5.00th=[ 6783], 10.00th=[13960], 20.00th=[17957],
  347. | 30.00th=[18482], 40.00th=[18744], 50.00th=[18744], 60.00th=[18744],
  348. | 70.00th=[19006], 80.00th=[19792], 90.00th=[20579], 95.00th=[21365],
  349. | 99.00th=[26608], 99.50th=[26608], 99.90th=[44303], 99.95th=[52167],
  350. | 99.99th=[64750]
  351. bw ( MiB/s): min=19616, max=26688, per=100.00%, avg=25354.00, stdev=471.37, samples=62
  352. iops : min= 4904, max= 6672, avg=6338.50, stdev=117.84, samples=62
  353. lat (usec) : 2=0.22%, 4=0.17%, 750=0.39%, 1000=0.01%
  354. lat (msec) : 2=0.77%, 4=1.38%, 10=4.28%, 20=77.58%, 50=15.15%
  355. lat (msec) : 100=0.06%
  356. cpu : usr=0.27%, sys=98.06%, ctx=117167, majf=0, minf=131121
  357. IO depths : 1=0.4%, 2=0.8%, 4=1.6%, 8=3.1%, 16=6.2%, 32=87.9%, >=64=0.0%
  358. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  359. complete : 0=0.0%, 4=99.6%, 8=0.0%, 16=0.0%, 32=0.4%, 64=0.0%, >=64=0.0%
  360. issued rwts: total=51200,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  361. latency : target=0, window=0, percentile=100.00%, depth=32
  362.  
  363. Run status group 0 (all jobs):
  364. READ: bw=24.5GiB/s (26.3GB/s), 24.5GiB/s-24.5GiB/s (26.3GB/s-26.3GB/s), io=200GiB (215GB), run=8151-8151msec
  365. async_cached_seqread: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
  366. ...
  367. fio-3.33
  368. Starting 4 processes
  369. async_cached_seqread: Laying out IO file (1 file / 1024MiB)
  370.  
  371. async_cached_seqread: (groupid=0, jobs=4): err= 0: pid=178781: Fri Jun 14 14:17:13 2024
  372. read: IOPS=6495, BW=25.4GiB/s (27.2GB/s)(200GiB/7882msec)
  373. slat (usec): min=329, max=2532, avg=605.24, stdev=90.34
  374. clat (nsec): min=1408, max=68935k, avg=17559074.64, stdev=4230383.47
  375. lat (usec): min=509, max=71196, avg=18164.31, stdev=4276.56
  376. clat percentiles (usec):
  377. | 1.00th=[ 1156], 5.00th=[ 6718], 10.00th=[13960], 20.00th=[17695],
  378. | 30.00th=[18482], 40.00th=[18482], 50.00th=[18744], 60.00th=[18744],
  379. | 70.00th=[18744], 80.00th=[18744], 90.00th=[19530], 95.00th=[21627],
  380. | 99.00th=[21890], 99.50th=[21890], 99.90th=[47449], 99.95th=[57410],
  381. | 99.99th=[66847]
  382. bw ( MiB/s): min=20096, max=28128, per=99.75%, avg=25917.87, stdev=447.11, samples=60
  383. iops : min= 5024, max= 7032, avg=6479.47, stdev=111.78, samples=60
  384. lat (usec) : 2=0.27%, 4=0.12%, 750=0.39%
  385. lat (msec) : 2=0.78%, 4=1.38%, 10=4.33%, 20=83.37%, 50=9.27%
  386. lat (msec) : 100=0.09%
  387. cpu : usr=0.22%, sys=98.25%, ctx=120192, majf=0, minf=131122
  388. IO depths : 1=0.4%, 2=0.8%, 4=1.6%, 8=3.1%, 16=6.2%, 32=87.9%, >=64=0.0%
  389. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  390. complete : 0=0.0%, 4=99.6%, 8=0.0%, 16=0.0%, 32=0.4%, 64=0.0%, >=64=0.0%
  391. issued rwts: total=51200,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  392. latency : target=0, window=0, percentile=100.00%, depth=32
  393.  
  394. Run status group 0 (all jobs):
  395. READ: bw=25.4GiB/s (27.2GB/s), 25.4GiB/s-25.4GiB/s (27.2GB/s-27.2GB/s), io=200GiB (215GB), run=7882-7882msec
  396. Linux 6.7.4 (rescue) 06/14/2024 _x86_64_ (48 CPU)
  397.  
  398. avg-cpu: %user %nice %system %iowait %steal %idle
  399. 0.12 0.00 0.90 0.01 0.00 98.97
  400.  
  401. Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
  402. loop0 0.37 8.41 0.00 0.00 440029 0 0
  403. nvme0n1 277.59 7.75 16406.11 0.00 405697 858537297 0
  404. nvme1n1 244.51 6.51 14384.54 0.00 340839 752747819 0
  405. nvme2n1 212.10 6.00 12397.41 0.00 314100 648760773 0
  406. nvme3n1 183.97 5.34 10735.09 0.00 279207 561771036 0
  407. nvme4n1 150.60 4.36 8502.19 0.00 228365 444922690 0
  408. nvme5n1 117.18 3.60 6940.21 0.00 188323 363183656 0
  409. nvme6n1 70.25 2.16 4145.60 0.00 113115 216940932 0
  410. nvme7n1 44.68 1.41 2678.90 0.00 73801 140187682 0
  411.  
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!