ubuntu@disk-test:/mnt/vol100$ fio --rw=randread --ioengine=libaio --direct=1 --numjobs=1 --bs=4k --runtime=60 --size=5G --runtime=60 --iodepth=16 --name=/mnt/vol100/fio1.dat --name=/mnt/vol100/fio2.dat --name=/mnt/vol100/fio3.dat --name=/mnt/vol100/fio4.dat
/mnt/vol100/fio1.dat: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
/mnt/vol100/fio2.dat: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
/mnt/vol100/fio3.dat: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
/mnt/vol100/fio4.dat: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio 1.59
Starting 4 processes
/mnt/vol100/fio1.dat: Laying out IO file(s) (1 file(s) / 5120MB)
/mnt/vol100/fio2.dat: Laying out IO file(s) (1 file(s) / 5120MB)
/mnt/vol100/fio3.dat: Laying out IO file(s) (1 file(s) / 5120MB)
/mnt/vol100/fio4.dat: Laying out IO file(s) (1 file(s) / 5120MB)
Jobs: 4 (f=4): [rrrr] [100.0% done] [41966K/0K /s] [10.3K/0  iops] [eta 00m:00s]
/mnt/vol100/fio1.dat: (groupid=0, jobs=1): err= 0: pid=7473
  read : io=698252KB, bw=11637KB/s, iops=2909 , runt= 60004msec
    slat (usec): min=3 , max=1299 , avg=12.65, stdev=23.31
    clat (usec): min=992 , max=92454 , avg=5483.35, stdev=1872.15
     lat (usec): min=999 , max=92467 , avg=5496.64, stdev=1872.43
    bw (KB/s) : min= 9101, max=13577, per=25.05%, avg=11654.01, stdev=789.05
  cpu          : usr=1.41%, sys=4.75%, ctx=34464, majf=0, minf=35
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued r/w/d: total=174563/0/0, short=0/0/0
     lat (usec): 1000=0.01%
     lat (msec): 2=0.39%, 4=16.13%, 10=81.49%, 20=1.96%, 50=0.01%
     lat (msec): 100=0.01%
/mnt/vol100/fio2.dat: (groupid=0, jobs=1): err= 0: pid=7474
  read : io=697868KB, bw=11631KB/s, iops=2907 , runt= 60003msec
    slat (usec): min=3 , max=1651 , avg=12.75, stdev=25.37
    clat (usec): min=929 , max=136972 , avg=5486.18, stdev=1975.32
     lat (usec): min=937 , max=136986 , avg=5499.57, stdev=1975.53
    bw (KB/s) : min= 9112, max=13269, per=25.04%, avg=11649.87, stdev=786.86
  cpu          : usr=1.36%, sys=4.80%, ctx=34556, majf=0, minf=36
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued r/w/d: total=174467/0/0, short=0/0/0
     lat (usec): 1000=0.01%
     lat (msec): 2=0.44%, 4=16.06%, 10=81.39%, 20=2.08%, 50=0.01%
     lat (msec): 100=0.01%, 250=0.01%
/mnt/vol100/fio3.dat: (groupid=0, jobs=1): err= 0: pid=7475
  read : io=698124KB, bw=11635KB/s, iops=2908 , runt= 60002msec
    slat (usec): min=3 , max=2673 , avg=12.68, stdev=24.87
    clat (usec): min=221 , max=108457 , avg=5484.13, stdev=1867.08
     lat (usec): min=985 , max=108462 , avg=5497.45, stdev=1867.21
    bw (KB/s) : min= 9115, max=13272, per=25.05%, avg=11654.82, stdev=776.04
  cpu          : usr=1.47%, sys=4.69%, ctx=34674, majf=0, minf=35
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued r/w/d: total=174531/0/0, short=0/0/0
     lat (usec): 250=0.01%, 500=0.01%, 1000=0.01%
     lat (msec): 2=0.39%, 4=16.49%, 10=80.93%, 20=2.17%, 50=0.01%
     lat (msec): 100=0.01%, 250=0.01%
/mnt/vol100/fio4.dat: (groupid=0, jobs=1): err= 0: pid=7476
  read : io=697052KB, bw=11617KB/s, iops=2904 , runt= 60002msec
    slat (usec): min=3 , max=1869 , avg=12.68, stdev=24.12
    clat (usec): min=970 , max=112539 , avg=5492.62, stdev=1887.71
     lat (usec): min=980 , max=112554 , avg=5505.95, stdev=1887.90
    bw (KB/s) : min= 9216, max=13344, per=25.01%, avg=11636.18, stdev=800.17
  cpu          : usr=1.51%, sys=4.65%, ctx=34382, majf=0, minf=35
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued r/w/d: total=174263/0/0, short=0/0/0
     lat (usec): 1000=0.01%
     lat (msec): 2=0.39%, 4=16.24%, 10=81.15%, 20=2.20%, 50=0.01%
     lat (msec): 100=0.01%, 250=0.01%

Run status group 0 (all jobs):
   READ: io=2725.9MB, aggrb=46518KB/s, minb=11895KB/s, maxb=11916KB/s, mint=60002msec, maxt=60004msec

Disk stats (read/write):
  vdc: ios=697102/4, merge=0/1, ticks=3631916/32, in_queue=3631804, util=99.96%