[root@tendrl14 tendrl14.lab.eng.blr.redhat.com:_vmstore]# fio --name=job1 --rw=read --size=1g  --directory=/var --direct=1
job1: (g=0): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
fio-2.1.7
Starting 1 process
job1: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [R] [12.5% done] [52224KB/0KB/0KB /s] [13.6K/0/0 iops] [eta 00m:21Jobs: 1 (f=1): [R] [17.4% done] [49766KB/0KB/0KB /s] [12.5K/0/0 iops] [eta 00m:19Jobs: 1 (f=1): [R] [21.7% done] [48736KB/0KB/0KB /s] [12.2K/0/0 iops] [eta 00m:18Jobs: 1 (f=1): [R] [27.3% done] [52355KB/0KB/0KB /s] [13.9K/0/0 iops] [eta 00m:16Jobs: 1 (f=1): [R] [31.8% done] [52219KB/0KB/0KB /s] [13.6K/0/0 iops] [eta 00m:15Jobs: 1 (f=1): [R] [38.1% done] [51628KB/0KB/0KB /s] [12.1K/0/0 iops] [eta 00m:13Jobs: 1 (f=1): [R] [42.9% done] [50312KB/0KB/0KB /s] [12.6K/0/0 iops] [eta 00m:12Jobs: 1 (f=1): [R] [47.6% done] [50856KB/0KB/0KB /s] [12.8K/0/0 iops] [eta 00m:11Jobs: 1 (f=1): [R] [52.4% done] [52264KB/0KB/0KB /s] [13.7K/0/0 iops] [eta 00m:10Jobs: 1 (f=1): [R] [57.1% done] [51832KB/0KB/0KB /s] [12.1K/0/0 iops] [eta 00m:09Jobs: 1 (f=1): [R] [61.9% done] [50745KB/0KB/0KB /s] [12.7K/0/0 iops] [eta 00m:08Jobs: 1 (f=1): [R] [66.7% done] [51396KB/0KB/0KB /s] [12.9K/0/0 iops] [eta 00m:07Jobs: 1 (f=1): [R] [71.4% done] [50797KB/0KB/0KB /s] [12.7K/0/0 iops] [eta 00m:06Jobs: 1 (f=1): [R] [76.2% done] [52132KB/0KB/0KB /s] [13.4K/0/0 iops] [eta 00m:05Jobs: 1 (f=1): [R] [81.0% done] [52444KB/0KB/0KB /s] [13.2K/0/0 iops] [eta 00m:04Jobs: 1 (f=1): [R] [85.7% done] [51924KB/0KB/0KB /s] [12.1K/0/0 iops] [eta 00m:03Jobs: 1 (f=1): [R] [90.5% done] [48215KB/0KB/0KB /s] [12.6K/0/0 iops] [eta 00m:02Jobs: 1 (f=1): [R] [95.2% done] [51808KB/0KB/0KB /s] [12.1K/0/0 iops] [eta 00m:01s]
job1: (groupid=0, jobs=1): err= 0: pid=732287: Mon Jun 22 07:31:50 2020
  read : io=1024.0MB, bw=50507KB/s, iops=12626, runt= 20761msec
    clat (usec): min=59, max=54329, avg=77.97, stdev=161.44
     lat (usec): min=59, max=54329, avg=78.10, stdev=161.44
    clat percentiles (usec):
     |  1.00th=[   72],  5.00th=[   73], 10.00th=[   73], 20.00th=[   74],
     | 30.00th=[   74], 40.00th=[   74], 50.00th=[   74], 60.00th=[   75],
     | 70.00th=[   75], 80.00th=[   76], 90.00th=[   81], 95.00th=[   93],
     | 99.00th=[  115], 99.50th=[  120], 99.90th=[  205], 99.95th=[  270],
     | 99.99th=[ 1004]
    bw (KB  /s): min=32904, max=52960, per=99.97%, avg=50492.05, stdev=3549.10
    lat (usec) : 100=96.42%, 250=3.51%, 500=0.06%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%
  cpu          : usr=2.57%, sys=16.61%, ctx=262153, majf=0, minf=7
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=262144/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: io=1024.0MB, aggrb=50507KB/s, minb=50507KB/s, maxb=50507KB/s, mint=20761msec, maxt=20761msec

Disk stats (read/write):
    dm-16: ios=259350/26, merge=0/0, ticks=15906/716, in_queue=16622, util=97.98%, aggrios=262144/58, aggrmerge=0/0, aggrticks=14854/1868, aggrin_queue=16722, aggrutil=97.97%
    dm-3: ios=262144/58, merge=0/0, ticks=14854/1868, in_queue=16722, util=97.97%, aggrios=131072/35, aggrmerge=0/0, aggrticks=7242/1157, aggrin_queue=8400, aggrutil=97.97%
    dm-1: ios=0/12, merge=0/0, ticks=0/447, in_queue=447, util=0.05%, aggrios=262144/47, aggrmerge=0/23, aggrticks=16753/1355, aggrin_queue=1535, aggrutil=97.99%
  sda: ios=262144/47, merge=0/23, ticks=16753/1355, in_queue=1535, util=97.99%
  dm-2: ios=262144/58, merge=0/0, ticks=14485/1868, in_queue=16353, util=97.97%