Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- root@pve ~ # cat bench-4k-IOD4.txt
- seqRead: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
- seqWrite: (g=1): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
- seqWrite/sync: (g=2): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
- randRead: (g=3): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
- randWrite: (g=4): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
- randWrite/sync: (g=5): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
- randReadWrite: (g=6): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
- randReadWrite/sync: (g=7): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
- fio-3.12
- Starting 8 processes
- seqRead: (groupid=0, jobs=1): err= 0: pid=3448: Wed Apr 1 11:36:58 2020
- read: IOPS=515k, BW=2012MiB/s (2110MB/s)(1024MiB/509msec)
- slat (nsec): min=551, max=2723.1k, avg=1481.09, stdev=7857.96
- clat (nsec): min=561, max=555417, avg=6001.59, stdev=10021.93
- lat (nsec): min=1203, max=2763.1k, avg=7523.95, stdev=12691.20
- clat percentiles (nsec):
- | 1.00th=[ 3152], 5.00th=[ 3216], 10.00th=[ 3248], 20.00th=[ 3312],
- | 30.00th=[ 3408], 40.00th=[ 3472], 50.00th=[ 3632], 60.00th=[ 3760],
- | 70.00th=[ 4016], 80.00th=[ 4576], 90.00th=[ 5152], 95.00th=[ 9152],
- | 99.00th=[54016], 99.50th=[58112], 99.90th=[67072], 99.95th=[70144],
- | 99.99th=[88576]
- lat (nsec) : 750=0.01%
- lat (usec) : 2=0.01%, 4=69.98%, 10=25.19%, 20=0.12%, 50=2.94%
- lat (usec) : 100=1.77%, 250=0.01%, 500=0.01%, 750=0.01%
- cpu : usr=20.87%, sys=77.95%, ctx=59, majf=0, minf=13
- IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=262144,0,0,0 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=4
- seqWrite: (groupid=1, jobs=1): err= 0: pid=3456: Wed Apr 1 11:36:58 2020
- write: IOPS=378k, BW=1478MiB/s (1549MB/s)(1024MiB/693msec); 0 zone resets
- slat (nsec): min=1002, max=52558, avg=1566.92, stdev=691.91
- clat (nsec): min=711, max=80971, avg=8170.79, stdev=2059.98
- lat (nsec): min=2013, max=90609, avg=9787.77, stdev=2421.67
- clat percentiles (nsec):
- | 1.00th=[ 6496], 5.00th=[ 6560], 10.00th=[ 6624], 20.00th=[ 6688],
- | 30.00th=[ 6816], 40.00th=[ 7072], 50.00th=[ 7392], 60.00th=[ 7904],
- | 70.00th=[ 9024], 80.00th=[ 9536], 90.00th=[10304], 95.00th=[11072],
- | 99.00th=[14528], 99.50th=[17792], 99.90th=[27264], 99.95th=[33024],
- | 99.99th=[49920]
- bw ( MiB/s): min= 1453, max= 1453, per=98.36%, avg=1453.46, stdev= 0.00, samples=1
- iops : min=372084, max=372084, avg=372084.00, stdev= 0.00, samples=1
- lat (nsec) : 750=0.01%
- lat (usec) : 4=0.01%, 10=86.57%, 20=13.16%, 50=0.25%, 100=0.01%
- cpu : usr=43.64%, sys=55.78%, ctx=70, majf=0, minf=10
- IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=4
- seqWrite/sync: (groupid=2, jobs=1): err= 0: pid=3463: Wed Apr 1 11:36:58 2020
- write: IOPS=909, BW=3638KiB/s (3726kB/s)(213MiB/60002msec); 0 zone resets
- slat (nsec): min=2044, max=57237, avg=8362.27, stdev=3056.59
- clat (usec): min=1052, max=24615, avg=3299.17, stdev=613.15
- lat (usec): min=1061, max=24630, avg=3307.75, stdev=613.38
- clat percentiles (usec):
- | 1.00th=[ 2999], 5.00th=[ 3032], 10.00th=[ 3064], 20.00th=[ 3097],
- | 30.00th=[ 3130], 40.00th=[ 3163], 50.00th=[ 3195], 60.00th=[ 3195],
- | 70.00th=[ 3228], 80.00th=[ 3261], 90.00th=[ 3326], 95.00th=[ 3458],
- | 99.00th=[ 6194], 99.50th=[ 6325], 99.90th=[ 8160], 99.95th=[ 9241],
- | 99.99th=[15795]
- bw ( KiB/s): min= 2638, max= 3444, per=79.99%, avg=2910.03, stdev=114.36, samples=119
- iops : min= 659, max= 861, avg=727.13, stdev=28.63, samples=119
- lat (msec) : 2=0.01%, 4=96.13%, 10=3.82%, 20=0.04%, 50=0.01%
- fsync/fdatasync/sync_file_range:
- sync (nsec): min=20, max=9818, avg=236.40, stdev=134.79
- sync percentiles (nsec):
- | 1.00th=[ 81], 5.00th=[ 120], 10.00th=[ 141], 20.00th=[ 161],
- | 30.00th=[ 181], 40.00th=[ 191], 50.00th=[ 211], 60.00th=[ 231],
- | 70.00th=[ 262], 80.00th=[ 290], 90.00th=[ 342], 95.00th=[ 410],
- | 99.00th=[ 732], 99.50th=[ 892], 99.90th=[ 1208], 99.95th=[ 1400],
- | 99.99th=[ 4048]
- cpu : usr=0.71%, sys=2.72%, ctx=109229, majf=0, minf=11
- IO depths : 1=0.1%, 2=0.1%, 4=200.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=0,54575,0,54573 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=4
- randRead: (groupid=3, jobs=1): err= 0: pid=3717: Wed Apr 1 11:36:58 2020
- read: IOPS=16.9k, BW=66.1MiB/s (69.3MB/s)(1024MiB/15497msec)
- slat (usec): min=19, max=2758, avg=57.70, stdev=22.97
- clat (nsec): min=1593, max=2894.8k, avg=177841.29, stdev=40971.75
- lat (usec): min=69, max=2953, avg=235.77, stdev=48.42
- clat percentiles (usec):
- | 1.00th=[ 161], 5.00th=[ 165], 10.00th=[ 165], 20.00th=[ 169],
- | 30.00th=[ 174], 40.00th=[ 174], 50.00th=[ 176], 60.00th=[ 176],
- | 70.00th=[ 178], 80.00th=[ 182], 90.00th=[ 190], 95.00th=[ 196],
- | 99.00th=[ 210], 99.50th=[ 223], 99.90th=[ 1090], 99.95th=[ 1221],
- | 99.99th=[ 1336]
- bw ( KiB/s): min=63231, max=68228, per=95.36%, avg=64523.50, stdev=912.94, samples=30
- iops : min=15807, max=17057, avg=16130.60, stdev=228.28, samples=30
- lat (usec) : 2=0.01%, 100=0.01%, 250=99.70%, 500=0.11%, 750=0.07%
- lat (usec) : 1000=0.01%
- lat (msec) : 2=0.10%, 4=0.01%
- cpu : usr=3.25%, sys=12.89%, ctx=262149, majf=0, minf=12
- IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=262144,0,0,0 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=4
- randWrite: (groupid=4, jobs=1): err= 0: pid=3757: Wed Apr 1 11:36:58 2020
- write: IOPS=364k, BW=1420MiB/s (1489MB/s)(1024MiB/721msec); 0 zone resets
- slat (nsec): min=1082, max=33342, avg=1655.74, stdev=628.12
- clat (nsec): min=601, max=80190, avg=8479.53, stdev=1926.35
- lat (nsec): min=1994, max=83236, avg=10181.82, stdev=2280.95
- clat percentiles (nsec):
- | 1.00th=[ 6816], 5.00th=[ 6944], 10.00th=[ 7072], 20.00th=[ 7200],
- | 30.00th=[ 7392], 40.00th=[ 7584], 50.00th=[ 7776], 60.00th=[ 8032],
- | 70.00th=[ 8512], 80.00th=[ 9920], 90.00th=[10816], 95.00th=[11840],
- | 99.00th=[14784], 99.50th=[16320], 99.90th=[25472], 99.95th=[30336],
- | 99.99th=[37120]
- bw ( MiB/s): min= 1404, max= 1404, per=98.88%, avg=1404.29, stdev= 0.00, samples=1
- iops : min=359497, max=359497, avg=359497.00, stdev= 0.00, samples=1
- lat (nsec) : 750=0.01%
- lat (usec) : 4=0.01%, 10=81.31%, 20=18.47%, 50=0.22%, 100=0.01%
- cpu : usr=39.86%, sys=59.86%, ctx=72, majf=0, minf=8
- IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=4
- randWrite/sync: (groupid=5, jobs=1): err= 0: pid=3797: Wed Apr 1 11:36:58 2020
- write: IOPS=910, BW=3640KiB/s (3728kB/s)(213MiB/60001msec); 0 zone resets
- slat (nsec): min=2004, max=49493, avg=8457.45, stdev=3103.58
- clat (usec): min=1046, max=16838, avg=3296.99, stdev=669.45
- lat (usec): min=1052, max=16847, avg=3305.67, stdev=669.80
- clat percentiles (usec):
- | 1.00th=[ 2966], 5.00th=[ 3032], 10.00th=[ 3064], 20.00th=[ 3097],
- | 30.00th=[ 3130], 40.00th=[ 3163], 50.00th=[ 3195], 60.00th=[ 3195],
- | 70.00th=[ 3228], 80.00th=[ 3294], 90.00th=[ 3359], 95.00th=[ 3458],
- | 99.00th=[ 6325], 99.50th=[ 8848], 99.90th=[ 9241], 99.95th=[11863],
- | 99.99th=[14615]
- bw ( KiB/s): min= 3314, max= 3787, per=98.38%, avg=3581.10, stdev=111.52, samples=119
- iops : min= 828, max= 946, avg=894.95, stdev=27.92, samples=119
- lat (msec) : 2=0.01%, 4=96.67%, 10=3.25%, 20=0.08%
- fsync/fdatasync/sync_file_range:
- sync (nsec): min=20, max=14657, avg=233.40, stdev=149.54
- sync percentiles (nsec):
- | 1.00th=[ 80], 5.00th=[ 100], 10.00th=[ 121], 20.00th=[ 151],
- | 30.00th=[ 171], 40.00th=[ 191], 50.00th=[ 211], 60.00th=[ 231],
- | 70.00th=[ 262], 80.00th=[ 290], 90.00th=[ 350], 95.00th=[ 430],
- | 99.00th=[ 724], 99.50th=[ 884], 99.90th=[ 1176], 99.95th=[ 1352],
- | 99.99th=[ 3184]
- cpu : usr=0.74%, sys=2.67%, ctx=109291, majf=0, minf=9
- IO depths : 1=0.1%, 2=0.1%, 4=200.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=0,54608,0,54606 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=4
- randReadWrite: (groupid=6, jobs=1): err= 0: pid=4051: Wed Apr 1 11:36:58 2020
- read: IOPS=15.9k, BW=62.2MiB/s (65.2MB/s)(511MiB/8220msec)
- slat (usec): min=49, max=1850, avg=56.95, stdev=20.71
- clat (nsec): min=1343, max=1928.8k, avg=94934.25, stdev=53837.34
- lat (usec): min=58, max=1985, avg=152.09, stdev=58.01
- clat percentiles (usec):
- | 1.00th=[ 11], 5.00th=[ 13], 10.00th=[ 16], 20.00th=[ 64],
- | 30.00th=[ 68], 40.00th=[ 70], 50.00th=[ 109], 60.00th=[ 117],
- | 70.00th=[ 121], 80.00th=[ 126], 90.00th=[ 167], 95.00th=[ 176],
- | 99.00th=[ 188], 99.50th=[ 194], 99.90th=[ 265], 99.95th=[ 668],
- | 99.99th=[ 1287]
- bw ( KiB/s): min=47786, max=63250, per=77.72%, avg=49502.62, stdev=3701.14, samples=16
- iops : min=11946, max=15812, avg=12375.31, stdev=925.25, samples=16
- write: IOPS=15.0k, BW=62.4MiB/s (65.4MB/s)(513MiB/8220msec); 0 zone resets
- slat (nsec): min=1152, max=36588, avg=2833.78, stdev=1125.76
- clat (usec): min=7, max=1933, avg=94.00, stdev=53.29
- lat (usec): min=8, max=1935, avg=96.91, stdev=53.37
- clat percentiles (usec):
- | 1.00th=[ 11], 5.00th=[ 12], 10.00th=[ 15], 20.00th=[ 64],
- | 30.00th=[ 67], 40.00th=[ 69], 50.00th=[ 88], 60.00th=[ 117],
- | 70.00th=[ 121], 80.00th=[ 125], 90.00th=[ 167], 95.00th=[ 176],
- | 99.00th=[ 188], 99.50th=[ 194], 99.90th=[ 260], 99.95th=[ 668],
- | 99.99th=[ 1270]
- bw ( KiB/s): min=47432, max=63292, per=77.73%, avg=49646.44, stdev=3699.67, samples=16
- iops : min=11858, max=15823, avg=12411.31, stdev=924.98, samples=16
- lat (usec) : 2=0.01%, 10=0.60%, 20=11.65%, 50=0.37%, 100=37.39%
- lat (usec) : 250=49.88%, 500=0.03%, 750=0.03%, 1000=0.01%
- lat (msec) : 2=0.04%
- cpu : usr=5.65%, sys=15.78%, ctx=130893, majf=0, minf=10
- IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=130888,131256,0,0 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=4
- randReadWrite/sync: (groupid=7, jobs=1): err= 0: pid=4098: Wed Apr 1 11:36:58 2020
- read: IOPS=795, BW=3183KiB/s (3260kB/s)(187MiB/60001msec)
- slat (usec): min=51, max=1506, avg=67.11, stdev=16.55
- clat (usec): min=291, max=10695, avg=1366.97, stdev=776.98
- lat (usec): min=346, max=10783, avg=1434.44, stdev=778.36
- clat percentiles (usec):
- | 1.00th=[ 330], 5.00th=[ 351], 10.00th=[ 375], 20.00th=[ 420],
- | 30.00th=[ 1237], 40.00th=[ 1287], 50.00th=[ 1319], 60.00th=[ 1352],
- | 70.00th=[ 1418], 80.00th=[ 2180], 90.00th=[ 2278], 95.00th=[ 2343],
- | 99.00th=[ 4178], 99.50th=[ 5014], 99.90th=[ 6194], 99.95th=[ 7963],
- | 99.99th=[ 8586]
- bw ( KiB/s): min= 2632, max= 3696, per=99.99%, avg=3182.80, stdev=224.26, samples=120
- iops : min= 658, max= 924, avg=795.68, stdev=56.07, samples=120
- write: IOPS=803, BW=3212KiB/s (3289kB/s)(188MiB/60001msec); 0 zone resets
- slat (nsec): min=2134, max=96520, avg=7825.41, stdev=2763.40
- clat (usec): min=1002, max=11907, avg=2382.81, stdev=853.54
- lat (usec): min=1012, max=11915, avg=2390.83, stdev=853.70
- clat percentiles (usec):
- | 1.00th=[ 1254], 5.00th=[ 1319], 10.00th=[ 1352], 20.00th=[ 1434],
- | 30.00th=[ 2212], 40.00th=[ 2278], 50.00th=[ 2311], 60.00th=[ 2343],
- | 70.00th=[ 2442], 80.00th=[ 3163], 90.00th=[ 3261], 95.00th=[ 3359],
- | 99.00th=[ 5407], 99.50th=[ 6194], 99.90th=[ 8979], 99.95th=[ 9241],
- | 99.99th=[11076]
- bw ( KiB/s): min= 2936, max= 3424, per=99.99%, avg=3211.78, stdev=103.35, samples=120
- iops : min= 734, max= 856, avg=802.93, stdev=25.84, samples=120
- lat (usec) : 500=12.12%, 750=0.04%, 1000=0.01%
- lat (msec) : 2=36.74%, 4=49.38%, 10=1.69%, 20=0.01%
- fsync/fdatasync/sync_file_range:
- sync (nsec): min=20, max=15339, avg=187.94, stdev=129.92
- sync percentiles (nsec):
- | 1.00th=[ 50], 5.00th=[ 70], 10.00th=[ 90], 20.00th=[ 111],
- | 30.00th=[ 141], 40.00th=[ 161], 50.00th=[ 181], 60.00th=[ 201],
- | 70.00th=[ 221], 80.00th=[ 251], 90.00th=[ 290], 95.00th=[ 322],
- | 99.00th=[ 490], 99.50th=[ 612], 99.90th=[ 964], 99.95th=[ 1128],
- | 99.99th=[ 4768]
- cpu : usr=1.04%, sys=3.93%, ctx=192010, majf=0, minf=11
- IO depths : 1=0.1%, 2=0.1%, 4=200.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=47748,48184,0,95930 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=4
- Run status group 0 (all jobs):
- READ: bw=2012MiB/s (2110MB/s), 2012MiB/s-2012MiB/s (2110MB/s-2110MB/s), io=1024MiB (1074MB), run=509-509msec
- Run status group 1 (all jobs):
- WRITE: bw=1478MiB/s (1549MB/s), 1478MiB/s-1478MiB/s (1549MB/s-1549MB/s), io=1024MiB (1074MB), run=693-693msec
- Run status group 2 (all jobs):
- WRITE: bw=3638KiB/s (3726kB/s), 3638KiB/s-3638KiB/s (3726kB/s-3726kB/s), io=213MiB (224MB), run=60002-60002msec
- Run status group 3 (all jobs):
- READ: bw=66.1MiB/s (69.3MB/s), 66.1MiB/s-66.1MiB/s (69.3MB/s-69.3MB/s), io=1024MiB (1074MB), run=15497-15497msec
- Run status group 4 (all jobs):
- WRITE: bw=1420MiB/s (1489MB/s), 1420MiB/s-1420MiB/s (1489MB/s-1489MB/s), io=1024MiB (1074MB), run=721-721msec
- Run status group 5 (all jobs):
- WRITE: bw=3640KiB/s (3728kB/s), 3640KiB/s-3640KiB/s (3728kB/s-3728kB/s), io=213MiB (224MB), run=60001-60001msec
- Run status group 6 (all jobs):
- READ: bw=62.2MiB/s (65.2MB/s), 62.2MiB/s-62.2MiB/s (65.2MB/s-65.2MB/s), io=511MiB (536MB), run=8220-8220msec
- WRITE: bw=62.4MiB/s (65.4MB/s), 62.4MiB/s-62.4MiB/s (65.4MB/s-65.4MB/s), io=513MiB (538MB), run=8220-8220msec
- Run status group 7 (all jobs):
- READ: bw=3183KiB/s (3260kB/s), 3183KiB/s-3183KiB/s (3260kB/s-3260kB/s), io=187MiB (196MB), run=60001-60001msec
- WRITE: bw=3212KiB/s (3289kB/s), 3212KiB/s-3212KiB/s (3289kB/s-3289kB/s), io=188MiB (197MB), run=60001-60001msec
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement