Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- root@pve ~ # cat bench-4k-IOD4.txt
- seqRead: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
- seqWrite: (g=1): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
- seqWrite/sync: (g=2): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
- randRead: (g=3): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
- randWrite: (g=4): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
- randWrite/sync: (g=5): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
- randReadWrite: (g=6): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
- randReadWrite/sync: (g=7): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
- fio-3.12
- Starting 8 processes
- seqRead: (groupid=0, jobs=1): err= 0: pid=31623: Wed Apr 1 11:38:27 2020
- read: IOPS=664k, BW=2592MiB/s (2718MB/s)(1024MiB/395msec)
- slat (nsec): min=628, max=4325.0k, avg=1122.77, stdev=9103.47
- clat (nsec): min=708, max=4345.2k, avg=4758.49, stdev=15804.41
- lat (nsec): min=1395, max=4346.1k, avg=5902.97, stdev=18236.53
- clat percentiles (nsec):
- | 1.00th=[ 3312], 5.00th=[ 3312], 10.00th=[ 3312], 20.00th=[ 3344],
- | 30.00th=[ 3344], 40.00th=[ 3344], 50.00th=[ 3376], 60.00th=[ 3376],
- | 70.00th=[ 3408], 80.00th=[ 3440], 90.00th=[ 3632], 95.00th=[ 4080],
- | 99.00th=[31360], 99.50th=[32128], 99.90th=[34048], 99.95th=[35072],
- | 99.99th=[58112]
- lat (nsec) : 750=0.01%
- lat (usec) : 2=0.01%, 4=94.86%, 10=0.38%, 20=0.04%, 50=4.69%
- lat (usec) : 100=0.01%
- lat (msec) : 10=0.01%
- cpu : usr=35.03%, sys=63.20%, ctx=4, majf=0, minf=13
- IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=262144,0,0,0 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=4
- seqWrite: (groupid=1, jobs=1): err= 0: pid=31624: Wed Apr 1 11:38:27 2020
- write: IOPS=501k, BW=1958MiB/s (2053MB/s)(1024MiB/523msec); 0 zone resets
- slat (nsec): min=974, max=25889, avg=1171.37, stdev=365.10
- clat (nsec): min=770, max=35291, avg=6237.42, stdev=810.25
- lat (nsec): min=1854, max=36393, avg=7430.17, stdev=887.49
- clat percentiles (nsec):
- | 1.00th=[ 5792], 5.00th=[ 5856], 10.00th=[ 5856], 20.00th=[ 5920],
- | 30.00th=[ 5920], 40.00th=[ 5984], 50.00th=[ 6048], 60.00th=[ 6112],
- | 70.00th=[ 6432], 80.00th=[ 6560], 90.00th=[ 6688], 95.00th=[ 6880],
- | 99.00th=[ 7520], 99.50th=[ 7968], 99.90th=[21120], 99.95th=[22144],
- | 99.99th=[30592]
- lat (nsec) : 1000=0.01%
- lat (usec) : 4=0.01%, 10=99.79%, 20=0.08%, 50=0.14%
- cpu : usr=45.02%, sys=54.98%, ctx=1, majf=0, minf=11
- IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=4
- seqWrite/sync: (groupid=2, jobs=1): err= 0: pid=31638: Wed Apr 1 11:38:27 2020
- write: IOPS=1987, BW=7951KiB/s (8142kB/s)(466MiB/60001msec); 0 zone resets
- slat (nsec): min=1228, max=48404, avg=9948.26, stdev=5274.18
- clat (usec): min=422, max=7510, avg=1510.18, stdev=327.67
- lat (usec): min=427, max=7516, avg=1520.39, stdev=328.52
- clat percentiles (usec):
- | 1.00th=[ 1057], 5.00th=[ 1123], 10.00th=[ 1188], 20.00th=[ 1319],
- | 30.00th=[ 1369], 40.00th=[ 1434], 50.00th=[ 1549], 60.00th=[ 1582],
- | 70.00th=[ 1647], 80.00th=[ 1696], 90.00th=[ 1762], 95.00th=[ 1795],
- | 99.00th=[ 1893], 99.50th=[ 2008], 99.90th=[ 5866], 99.95th=[ 5932],
- | 99.99th=[ 7308]
- bw ( KiB/s): min= 5620, max= 8502, per=78.61%, avg=6249.63, stdev=356.11, samples=119
- iops : min= 1405, max= 2125, avg=1561.99, stdev=89.00, samples=119
- lat (usec) : 500=0.01%, 1000=0.16%
- lat (msec) : 2=99.34%, 4=0.16%, 10=0.34%
- fsync/fdatasync/sync_file_range:
- sync (nsec): min=14, max=18558, avg=331.43, stdev=386.17
- sync percentiles (nsec):
- | 1.00th=[ 94], 5.00th=[ 131], 10.00th=[ 133], 20.00th=[ 137],
- | 30.00th=[ 139], 40.00th=[ 147], 50.00th=[ 193], 60.00th=[ 470],
- | 70.00th=[ 478], 80.00th=[ 486], 90.00th=[ 498], 95.00th=[ 740],
- | 99.00th=[ 820], 99.50th=[ 828], 99.90th=[ 876], 99.95th=[13248],
- | 99.99th=[15552]
- cpu : usr=2.46%, sys=11.17%, ctx=356121, majf=0, minf=13
- IO depths : 1=0.1%, 2=0.1%, 4=200.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=0,119265,0,119263 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=4
- randRead: (groupid=3, jobs=1): err= 0: pid=31918: Wed Apr 1 11:38:27 2020
- read: IOPS=12.9k, BW=50.4MiB/s (52.8MB/s)(1024MiB/20317msec)
- slat (usec): min=35, max=5050, avg=76.59, stdev=54.26
- clat (nsec): min=868, max=13191k, avg=232813.01, stdev=127722.94
- lat (usec): min=71, max=18250, avg=309.50, stdev=163.71
- clat percentiles (usec):
- | 1.00th=[ 210], 5.00th=[ 210], 10.00th=[ 212], 20.00th=[ 212],
- | 30.00th=[ 212], 40.00th=[ 212], 50.00th=[ 212], 60.00th=[ 215],
- | 70.00th=[ 215], 80.00th=[ 215], 90.00th=[ 217], 95.00th=[ 221],
- | 99.00th=[ 955], 99.50th=[ 1037], 99.90th=[ 1057], 99.95th=[ 1074],
- | 99.99th=[ 4293]
- bw ( KiB/s): min=40060, max=47418, per=85.01%, avg=43873.45, stdev=1467.80, samples=40
- iops : min=10015, max=11854, avg=10968.02, stdev=366.83, samples=40
- lat (nsec) : 1000=0.01%
- lat (usec) : 100=0.01%, 250=95.96%, 500=1.21%, 750=1.29%, 1000=0.68%
- lat (msec) : 2=0.83%, 10=0.02%, 20=0.01%
- cpu : usr=1.92%, sys=5.39%, ctx=262146, majf=0, minf=13
- IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=262144,0,0,0 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=4
- randWrite: (groupid=4, jobs=1): err= 0: pid=32010: Wed Apr 1 11:38:27 2020
- write: IOPS=474k, BW=1852MiB/s (1942MB/s)(1024MiB/553msec); 0 zone resets
- slat (nsec): min=995, max=31701, avg=1255.54, stdev=325.96
- clat (nsec): min=767, max=37156, avg=6570.55, stdev=648.64
- lat (nsec): min=1842, max=38555, avg=7847.46, stdev=727.43
- clat percentiles (nsec):
- | 1.00th=[ 6048], 5.00th=[ 6112], 10.00th=[ 6176], 20.00th=[ 6240],
- | 30.00th=[ 6304], 40.00th=[ 6304], 50.00th=[ 6368], 60.00th=[ 6496],
- | 70.00th=[ 6752], 80.00th=[ 6944], 90.00th=[ 7072], 95.00th=[ 7328],
- | 99.00th=[ 7968], 99.50th=[ 8384], 99.90th=[11200], 99.95th=[20864],
- | 99.99th=[27520]
- lat (nsec) : 1000=0.01%
- lat (usec) : 4=0.01%, 10=99.85%, 20=0.08%, 50=0.07%
- cpu : usr=49.82%, sys=49.82%, ctx=2, majf=0, minf=9
- IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=4
- randWrite/sync: (groupid=5, jobs=1): err= 0: pid=32014: Wed Apr 1 11:38:27 2020
- write: IOPS=2009, BW=8037KiB/s (8230kB/s)(471MiB/60001msec); 0 zone resets
- slat (nsec): min=1213, max=48111, avg=9870.68, stdev=5193.04
- clat (usec): min=373, max=7835, avg=1493.98, stdev=337.61
- lat (usec): min=385, max=7841, avg=1504.12, stdev=338.33
- clat percentiles (usec):
- | 1.00th=[ 1057], 5.00th=[ 1123], 10.00th=[ 1188], 20.00th=[ 1319],
- | 30.00th=[ 1369], 40.00th=[ 1401], 50.00th=[ 1483], 60.00th=[ 1565],
- | 70.00th=[ 1598], 80.00th=[ 1663], 90.00th=[ 1745], 95.00th=[ 1778],
- | 99.00th=[ 1876], 99.50th=[ 1991], 99.90th=[ 5866], 99.95th=[ 6849],
- | 99.99th=[ 7177]
- bw ( KiB/s): min= 6261, max= 8501, per=88.45%, avg=7108.47, stdev=287.97, samples=119
- iops : min= 1565, max= 2125, avg=1776.78, stdev=72.00, samples=119
- lat (usec) : 500=0.01%, 1000=0.04%
- lat (msec) : 2=99.47%, 4=0.12%, 10=0.37%
- fsync/fdatasync/sync_file_range:
- sync (nsec): min=41, max=22013, avg=338.77, stdev=374.26
- sync percentiles (nsec):
- | 1.00th=[ 98], 5.00th=[ 131], 10.00th=[ 133], 20.00th=[ 137],
- | 30.00th=[ 141], 40.00th=[ 143], 50.00th=[ 169], 60.00th=[ 498],
- | 70.00th=[ 516], 80.00th=[ 524], 90.00th=[ 572], 95.00th=[ 820],
- | 99.00th=[ 852], 99.50th=[ 876], 99.90th=[ 924], 99.95th=[ 948],
- | 99.99th=[15040]
- cpu : usr=2.60%, sys=11.23%, ctx=360515, majf=0, minf=11
- IO depths : 1=0.1%, 2=0.1%, 4=200.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=0,120564,0,120562 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=4
- randReadWrite: (groupid=6, jobs=1): err= 0: pid=32265: Wed Apr 1 11:38:27 2020
- read: IOPS=12.8k, BW=49.0MiB/s (52.4MB/s)(511MiB/10219msec)
- slat (usec): min=66, max=4247, avg=74.66, stdev=45.88
- clat (nsec): min=888, max=4381.1k, avg=117441.77, stdev=91759.88
- lat (usec): min=70, max=4686, avg=192.20, stdev=113.03
- clat percentiles (usec):
- | 1.00th=[ 8], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 77],
- | 30.00th=[ 77], 40.00th=[ 78], 50.00th=[ 143], 60.00th=[ 145],
- | 70.00th=[ 145], 80.00th=[ 147], 90.00th=[ 212], 95.00th=[ 215],
- | 99.00th=[ 371], 99.50th=[ 562], 99.90th=[ 824], 99.95th=[ 1029],
- | 99.99th=[ 4228]
- bw ( KiB/s): min=36451, max=49928, per=75.55%, avg=38669.75, stdev=2807.57, samples=20
- iops : min= 9112, max=12482, avg=9667.00, stdev=702.02, samples=20
- write: IOPS=12.9k, BW=50.2MiB/s (52.7MB/s)(513MiB/10219msec); 0 zone resets
- slat (nsec): min=1032, max=25553, avg=1644.67, stdev=929.99
- clat (usec): min=6, max=4474, avg=117.10, stdev=91.72
- lat (usec): min=7, max=4476, avg=118.77, stdev=91.97
- clat percentiles (usec):
- | 1.00th=[ 7], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 76],
- | 30.00th=[ 77], 40.00th=[ 78], 50.00th=[ 143], 60.00th=[ 145],
- | 70.00th=[ 145], 80.00th=[ 147], 90.00th=[ 212], 95.00th=[ 215],
- | 99.00th=[ 359], 99.50th=[ 553], 99.90th=[ 914], 99.95th=[ 1037],
- | 99.99th=[ 4228]
- bw ( KiB/s): min=36588, max=51323, per=75.57%, avg=38862.15, stdev=3086.88, samples=20
- iops : min= 9147, max=12830, avg=9715.25, stdev=771.62, samples=20
- lat (nsec) : 1000=0.01%
- lat (usec) : 10=12.32%, 20=0.15%, 50=0.08%, 100=36.58%, 250=49.20%
- lat (usec) : 500=1.03%, 750=0.50%, 1000=0.06%
- lat (msec) : 2=0.05%, 10=0.01%
- cpu : usr=3.43%, sys=6.85%, ctx=130770, majf=0, minf=11
- IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=130767,131377,0,0 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=4
- randReadWrite/sync: (groupid=7, jobs=1): err= 0: pid=32310: Wed Apr 1 11:38:27 2020
- read: IOPS=1713, BW=6852KiB/s (7017kB/s)(402MiB/60001msec)
- slat (usec): min=39, max=4653, avg=94.77, stdev=65.26
- clat (usec): min=359, max=10056, avg=744.00, stdev=329.01
- lat (usec): min=433, max=10129, avg=838.93, stdev=362.86
- clat percentiles (usec):
- | 1.00th=[ 392], 5.00th=[ 396], 10.00th=[ 461], 20.00th=[ 545],
- | 30.00th=[ 586], 40.00th=[ 652], 50.00th=[ 660], 60.00th=[ 766],
- | 70.00th=[ 816], 80.00th=[ 898], 90.00th=[ 1074], 95.00th=[ 1205],
- | 99.00th=[ 1663], 99.50th=[ 1860], 99.90th=[ 5407], 99.95th=[ 5932],
- | 99.99th=[ 6849]
- bw ( KiB/s): min= 4584, max= 8488, per=99.98%, avg=6850.90, stdev=976.02, samples=120
- iops : min= 1146, max= 2122, avg=1712.69, stdev=243.99, samples=120
- write: IOPS=1715, BW=6860KiB/s (7025kB/s)(402MiB/60001msec); 0 zone resets
- slat (nsec): min=2008, max=40902, avg=3444.33, stdev=3078.88
- clat (usec): min=361, max=10963, avg=1007.08, stdev=352.78
- lat (usec): min=364, max=10966, avg=1010.58, stdev=353.84
- clat percentiles (usec):
- | 1.00th=[ 619], 5.00th=[ 652], 10.00th=[ 717], 20.00th=[ 807],
- | 30.00th=[ 840], 40.00th=[ 889], 50.00th=[ 922], 60.00th=[ 1020],
- | 70.00th=[ 1074], 80.00th=[ 1188], 90.00th=[ 1352], 95.00th=[ 1500],
- | 99.00th=[ 1942], 99.50th=[ 2147], 99.90th=[ 5735], 99.95th=[ 6325],
- | 99.99th=[ 7439]
- bw ( KiB/s): min= 4942, max= 8264, per=99.98%, avg=6858.70, stdev=917.26, samples=120
- iops : min= 1235, max= 2066, avg=1714.65, stdev=229.32, samples=120
- lat (usec) : 500=8.22%, 750=29.28%, 1000=34.77%
- lat (msec) : 2=27.16%, 4=0.37%, 10=0.20%, 20=0.01%
- fsync/fdatasync/sync_file_range:
- sync (nsec): min=10, max=21039, avg=163.03, stdev=933.38
- sync percentiles (nsec):
- | 1.00th=[ 34], 5.00th=[ 40], 10.00th=[ 42], 20.00th=[ 48],
- | 30.00th=[ 53], 40.00th=[ 58], 50.00th=[ 73], 60.00th=[ 90],
- | 70.00th=[ 113], 80.00th=[ 135], 90.00th=[ 159], 95.00th=[ 346],
- | 99.00th=[ 748], 99.50th=[ 836], 99.90th=[15808], 99.95th=[16064],
- | 99.99th=[17024]
- cpu : usr=3.42%, sys=6.96%, ctx=612711, majf=0, minf=12
- IO depths : 1=0.1%, 2=0.1%, 4=200.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=102788,102904,0,205690 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=4
- Run status group 0 (all jobs):
- READ: bw=2592MiB/s (2718MB/s), 2592MiB/s-2592MiB/s (2718MB/s-2718MB/s), io=1024MiB (1074MB), run=395-395msec
- Run status group 1 (all jobs):
- WRITE: bw=1958MiB/s (2053MB/s), 1958MiB/s-1958MiB/s (2053MB/s-2053MB/s), io=1024MiB (1074MB), run=523-523msec
- Run status group 2 (all jobs):
- WRITE: bw=7951KiB/s (8142kB/s), 7951KiB/s-7951KiB/s (8142kB/s-8142kB/s), io=466MiB (489MB), run=60001-60001msec
- Run status group 3 (all jobs):
- READ: bw=50.4MiB/s (52.8MB/s), 50.4MiB/s-50.4MiB/s (52.8MB/s-52.8MB/s), io=1024MiB (1074MB), run=20317-20317msec
- Run status group 4 (all jobs):
- WRITE: bw=1852MiB/s (1942MB/s), 1852MiB/s-1852MiB/s (1942MB/s-1942MB/s), io=1024MiB (1074MB), run=553-553msec
- Run status group 5 (all jobs):
- WRITE: bw=8037KiB/s (8230kB/s), 8037KiB/s-8037KiB/s (8230kB/s-8230kB/s), io=471MiB (494MB), run=60001-60001msec
- Run status group 6 (all jobs):
- READ: bw=49.0MiB/s (52.4MB/s), 49.0MiB/s-49.0MiB/s (52.4MB/s-52.4MB/s), io=511MiB (536MB), run=10219-10219msec
- WRITE: bw=50.2MiB/s (52.7MB/s), 50.2MiB/s-50.2MiB/s (52.7MB/s-52.7MB/s), io=513MiB (538MB), run=10219-10219msec
- Run status group 7 (all jobs):
- READ: bw=6852KiB/s (7017kB/s), 6852KiB/s-6852KiB/s (7017kB/s-7017kB/s), io=402MiB (421MB), run=60001-60001msec
- WRITE: bw=6860KiB/s (7025kB/s), 6860KiB/s-6860KiB/s (7025kB/s-7025kB/s), io=402MiB (421MB), run=60001-60001msec
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement