API tools faq
paste
vheathen

ceph small cluster performance 15.07.2019

vheathen
Jul 15th, 2019
411
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Bash 19.31 KB | None | 0 0
raw download clone embed print report
  1. ###################################
  2. # VM with virtio-scsi BS=8k JOBS=1
  3. ###################################
  4. root@ubuntu:~# fio --name=randrw --rw=randrw --direct=1 --invalidate=1 --ioengine=libaio --bs=8k --numjobs=1 --rwmixread=70 --size=20G --runtime=600 --time_based --group_reporting
  5. randrw: (g=0): rw=randrw, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=libaio, iodepth=1
  6. fio-3.1
  7. Starting 1 process
  8. Jobs: 1 (f=1): [m(1)][100.0%][r=8432KiB/s,w=3531KiB/s][r=1054,w=441 IOPS][eta 00m:00s]
  9. randrw: (groupid=0, jobs=1): err= 0: pid=2206: Mon Jul 15 08:47:41 2019
  10.    read: IOPS=936, BW=7493KiB/s (7672kB/s)(4390MiB/600001msec)
  11.     slat (usec): min=9, max=983, avg=15.81, stdev= 3.31
  12.     clat (usec): min=445, max=48412, avg=973.83, stdev=564.28
  13.      lat (usec): min=462, max=48428, avg=990.55, stdev=564.37
  14.     clat percentiles (usec):
  15.      |  1.00th=[  594],  5.00th=[  701], 10.00th=[  758], 20.00th=[  824],
  16.      | 30.00th=[  873], 40.00th=[  914], 50.00th=[  947], 60.00th=[  988],
  17.      | 70.00th=[ 1037], 80.00th=[ 1090], 90.00th=[ 1172], 95.00th=[ 1237],
  18.      | 99.00th=[ 1369], 99.50th=[ 1450], 99.90th=[ 4490], 99.95th=[14746],
  19.      | 99.99th=[27395]
  20.    bw (  KiB/s): min= 5856, max= 8960, per=100.00%, avg=7491.92, stdev=477.88, samples=1200
  21.    iops        : min=  732, max= 1120, avg=936.48, stdev=59.74, samples=1200
  22.   write: IOPS=402, BW=3219KiB/s (3296kB/s)(1886MiB/600001msec)
  23.     slat (usec): min=12, max=163, avg=17.44, stdev= 3.05
  24.     clat (usec): min=2, max=48358, avg=145.83, stdev=422.67
  25.      lat (usec): min=86, max=48379, avg=164.18, stdev=422.82
  26.     clat percentiles (usec):
  27.      |  1.00th=[   86],  5.00th=[   90], 10.00th=[   93], 20.00th=[   98],
  28.      | 30.00th=[  103], 40.00th=[  110], 50.00th=[  121], 60.00th=[  137],
  29.      | 70.00th=[  153], 80.00th=[  182], 90.00th=[  212], 95.00th=[  243],
  30.      | 99.00th=[  293], 99.50th=[  306], 99.90th=[  611], 99.95th=[ 1090],
  31.      | 99.99th=[26084]
  32.    bw (  KiB/s): min= 2208, max= 4400, per=100.00%, avg=3218.47, stdev=330.56, samples=1200
  33.    iops        : min=  276, max=  550, avg=402.30, stdev=41.32, samples=1200
  34.   lat (usec)   : 4=0.01%, 50=0.01%, 100=7.39%, 250=21.43%, 500=1.24%
  35.   lat (usec)   : 750=6.44%, 1000=37.45%
  36.   lat (msec)   : 2=25.90%, 4=0.05%, 10=0.02%, 20=0.04%, 50=0.02%
  37.   cpu          : usr=1.30%, sys=3.25%, ctx=803653, majf=0, minf=11
  38.   IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  39.      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  40.      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  41.      issued rwt: total=561949,241409,0, short=0,0,0, dropped=0,0,0
  42.      latency   : target=0, window=0, percentile=100.00%, depth=1
  43.  
  44. Run status group 0 (all jobs):
  45.    READ: bw=7493KiB/s (7672kB/s), 7493KiB/s-7493KiB/s (7672kB/s-7672kB/s), io=4390MiB (4603MB), run=600001-600001msec
  46.   WRITE: bw=3219KiB/s (3296kB/s), 3219KiB/s-3219KiB/s (3296kB/s-3296kB/s), io=1886MiB (1978MB), run=600001-600001msec
  47.  
  48. Disk stats (read/write):
  49.   sda: ios=561970/241866, merge=0/501, ticks=8000/5240, in_queue=601880, util=99.68%
  50.  
  51. ###################################
  52. # VM with virtio-scsi BS=8k JOBS=1
  53. ###################################
  54. root@ubuntu:~# fio --name=randrw --rw=randrw --direct=1 --invalidate=1 --ioengine=libaio --bs=8k --numjobs=8 --rwmixread=70 --size=4G --runtime=600 --time_based --group_reporting
  55. randrw: (g=0): rw=randrw, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=libaio, iodepth=1
  56. ...
  57. fio-3.1
  58. Starting 8 processes
  59. Jobs: 8 (f=8): [m(8)][100.0%][r=69.5MiB/s,w=29.6MiB/s][r=8896,w=3790 IOPS][eta 00m:00s]
  60. randrw: (groupid=0, jobs=8): err= 0: pid=2856: Mon Jul 15 12:34:09 2019
  61.    read: IOPS=8641, BW=67.5MiB/s (70.8MB/s)(39.6GiB/600016msec)
  62.     slat (usec): min=7, max=26966, avg=19.69, stdev=14.28
  63.     clat (usec): min=2, max=118335, avg=775.74, stdev=1090.39
  64.      lat (usec): min=131, max=118353, avg=796.72, stdev=1090.68
  65.     clat percentiles (usec):
  66.      |  1.00th=[  529],  5.00th=[  570], 10.00th=[  594], 20.00th=[  635],
  67.      | 30.00th=[  668], 40.00th=[  693], 50.00th=[  725], 60.00th=[  758],
  68.      | 70.00th=[  799], 80.00th=[  848], 90.00th=[  938], 95.00th=[ 1029],
  69.      | 99.00th=[ 1516], 99.50th=[ 1745], 99.90th=[ 3163], 99.95th=[ 5211],
  70.      | 99.99th=[78119]
  71.    bw (  KiB/s): min= 6208, max=10016, per=12.50%, avg=8644.32, stdev=516.61, samples=9600
  72.    iops        : min=  776, max= 1252, avg=1080.51, stdev=64.60, samples=9600
  73.   write: IOPS=3704, BW=28.9MiB/s (30.3MB/s)(16.0GiB/600016msec)
  74.     slat (usec): min=8, max=5541, avg=21.91, stdev= 8.83
  75.     clat (usec): min=2, max=93443, avg=257.12, stdev=640.07
  76.      lat (usec): min=78, max=93464, avg=280.35, stdev=640.11
  77.     clat percentiles (usec):
  78.      |  1.00th=[  100],  5.00th=[  118], 10.00th=[  133], 20.00th=[  159],
  79.      | 30.00th=[  186], 40.00th=[  210], 50.00th=[  235], 60.00th=[  262],
  80.      | 70.00th=[  293], 80.00th=[  334], 90.00th=[  396], 95.00th=[  449],
  81.      | 99.00th=[  553], 99.50th=[  603], 99.90th=[  742], 99.95th=[ 1123],
  82.      | 99.99th=[ 5473]
  83.    bw (  KiB/s): min= 2336, max= 4896, per=12.50%, avg=3705.85, stdev=333.79, samples=9600
  84.    iops        : min=  292, max=  612, avg=463.14, stdev=41.75, samples=9600
  85.   lat (usec)   : 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.32%
  86.   lat (usec)   : 250=16.51%, 500=12.66%, 750=41.26%, 1000=24.89%
  87.   lat (msec)   : 2=4.16%, 4=0.14%, 10=0.02%, 20=0.01%, 50=0.01%
  88.   lat (msec)   : 100=0.01%, 250=0.01%
  89.   cpu          : usr=2.00%, sys=4.85%, ctx=7422501, majf=0, minf=102
  90.   IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  91.      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  92.      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  93.      issued rwt: total=5185322,2222982,0, short=0,0,0, dropped=0,0,0
  94.      latency   : target=0, window=0, percentile=100.00%, depth=1
  95.  
  96. Run status group 0 (all jobs):
  97.    READ: bw=67.5MiB/s (70.8MB/s), 67.5MiB/s-67.5MiB/s (70.8MB/s-70.8MB/s), io=39.6GiB (42.5GB), run=600016-600016msec
  98.   WRITE: bw=28.9MiB/s (30.3MB/s), 28.9MiB/s-28.9MiB/s (30.3MB/s-30.3MB/s), io=16.0GiB (18.2GB), run=600016-600016msec
  99.  
  100. Disk stats (read/write):
  101.   sda: ios=5184958/2223335, merge=0/1258, ticks=91016/23640, in_queue=4729552, util=99.94%
  102.  
  103. ###################################
  104. # baremetal host librbd BS=8k JOBS=1
  105. ###################################
  106. admin@e001n01:/mnt/ceph-block-device$ sudo fio --name=randrw --rw=randrw --direct=1 --invalidate=1 --ioengine=libaio --bs=8k --numjobs=1 --rwmixread=70 --size=20G --runtime=600 --time_based --group_reporting
  107. randrw: (g=0): rw=randrw, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=libaio, iodepth=1
  108. fio-3.1
  109. Starting 1 process
  110. randrw: Laying out IO file (1 file / 20480MiB)
  111. Jobs: 1 (f=1): [m(1)][100.0%][r=4916KiB/s,w=1993KiB/s][r=614,w=249 IOPS][eta 00m:00s]
  112. randrw: (groupid=0, jobs=1): err= 0: pid=1923208: Mon Jul 15 13:36:46 2019
  113.    read: IOPS=599, BW=4794KiB/s (4909kB/s)(2809MiB/600001msec)
  114.     slat (usec): min=7, max=1230, avg=28.52, stdev=22.58
  115.     clat (usec): min=193, max=46537, avg=689.84, stdev=177.88
  116.      lat (usec): min=315, max=46560, avg=718.72, stdev=177.94
  117.     clat percentiles (usec):
  118.      |  1.00th=[  396],  5.00th=[  469], 10.00th=[  515], 20.00th=[  570],
  119.      | 30.00th=[  611], 40.00th=[  644], 50.00th=[  685], 60.00th=[  717],
  120.      | 70.00th=[  750], 80.00th=[  799], 90.00th=[  889], 95.00th=[  947],
  121.      | 99.00th=[ 1029], 99.50th=[ 1057], 99.90th=[ 1352], 99.95th=[ 1745],
  122.      | 99.99th=[ 3785]
  123.    bw (  KiB/s): min= 3168, max= 5904, per=100.00%, avg=4794.00, stdev=323.16, samples=1199
  124.    iops        : min=  396, max=  738, avg=599.25, stdev=40.40, samples=1199
  125.   write: IOPS=257, BW=2061KiB/s (2111kB/s)(1208MiB/600001msec)
  126.     slat (usec): min=8, max=6819, avg=35.51, stdev=41.27
  127.     clat (usec): min=875, max=75536, avg=2160.65, stdev=443.74
  128.      lat (usec): min=935, max=75548, avg=2196.49, stdev=444.57
  129.     clat percentiles (usec):
  130.      |  1.00th=[ 1549],  5.00th=[ 1696], 10.00th=[ 1778], 20.00th=[ 1893],
  131.      | 30.00th=[ 1975], 40.00th=[ 2040], 50.00th=[ 2114], 60.00th=[ 2212],
  132.      | 70.00th=[ 2311], 80.00th=[ 2442], 90.00th=[ 2573], 95.00th=[ 2671],
  133.      | 99.00th=[ 2868], 99.50th=[ 2966], 99.90th=[ 4228], 99.95th=[ 5211],
  134.      | 99.99th=[ 8225]
  135.    bw (  KiB/s): min= 1504, max= 2576, per=100.00%, avg=2061.26, stdev=120.82, samples=1199
  136.    iops        : min=  188, max=  322, avg=257.65, stdev=15.10, samples=1199
  137.   lat (usec)   : 250=0.01%, 500=5.69%, 750=43.08%, 1000=19.94%
  138.   lat (msec)   : 2=11.39%, 4=19.86%, 10=0.04%, 20=0.01%, 50=0.01%
  139.   lat (msec)   : 100=0.01%
  140.   cpu          : usr=0.70%, sys=2.02%, ctx=671879, majf=0, minf=714
  141.   IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  142.      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  143.      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  144.      issued rwt: total=359546,154590,0, short=0,0,0, dropped=0,0,0
  145.      latency   : target=0, window=0, percentile=100.00%, depth=1
  146.  
  147. Run status group 0 (all jobs):
  148.    READ: bw=4794KiB/s (4909kB/s), 4794KiB/s-4794KiB/s (4909kB/s-4909kB/s), io=2809MiB (2945MB), run=600001-600001msec
  149.   WRITE: bw=2061KiB/s (2111kB/s), 2061KiB/s-2061KiB/s (2111kB/s-2111kB/s), io=1208MiB (1266MB), run=600001-600001msec
  150.  
  151. Disk stats (read/write):
  152.   rbd0: ios=359492/154806, merge=0/273, ticks=184/1036, in_queue=589188, util=98.17%
  153.  
  154. ###################################
  155. # baremetal host librbd BS=8k JOBS=8
  156. ###################################
  157. admin@e001n01:/mnt/ceph-block-device$ sudo fio --name=randrw --rw=randrw --direct=1 --invalidate=1 --ioengine=libaio --bs=8k --numjobs=8 --rwmixread=70 --size=4G --runtime=600 --time_based --group_reporting
  158. randrw: (g=0): rw=randrw, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=libaio, iodepth=1
  159. ...
  160. fio-3.1
  161. Starting 8 processes
  162. Jobs: 8 (f=8): [m(8)][100.0%][r=62.7MiB/s,w=25.8MiB/s][r=8023,w=3306 IOPS][eta 00m:00s]
  163. randrw: (groupid=0, jobs=8): err= 0: pid=2026701: Mon Jul 15 17:07:44 2019
  164.    read: IOPS=7855, BW=61.4MiB/s (64.4MB/s)(35.0GiB/600002msec)
  165.     slat (usec): min=5, max=5579, avg=15.42, stdev=20.93
  166.     clat (usec): min=113, max=111218, avg=462.65, stdev=198.10
  167.      lat (usec): min=289, max=111232, avg=478.41, stdev=199.07
  168.     clat percentiles (usec):
  169.      |  1.00th=[  326],  5.00th=[  347], 10.00th=[  363], 20.00th=[  379],
  170.      | 30.00th=[  396], 40.00th=[  408], 50.00th=[  424], 60.00th=[  441],
  171.      | 70.00th=[  465], 80.00th=[  515], 90.00th=[  603], 95.00th=[  668],
  172.      | 99.00th=[ 1139], 99.50th=[ 1352], 99.90th=[ 1876], 99.95th=[ 2507],
  173.      | 99.99th=[ 5211]
  174.    bw (  KiB/s): min= 5296, max= 9600, per=12.51%, avg=7860.04, stdev=409.36, samples=9600
  175.    iops        : min=  662, max= 1200, avg=982.43, stdev=51.20, samples=9600
  176.   write: IOPS=3368, BW=26.3MiB/s (27.6MB/s)(15.4GiB/600002msec)
  177.     slat (usec): min=6, max=5661, avg=18.51, stdev=22.63
  178.     clat (usec): min=712, max=118650, avg=1228.75, stdev=482.03
  179.      lat (usec): min=773, max=118664, avg=1247.59, stdev=482.43
  180.     clat percentiles (usec):
  181.      |  1.00th=[  889],  5.00th=[  938], 10.00th=[  963], 20.00th=[ 1012],
  182.      | 30.00th=[ 1057], 40.00th=[ 1090], 50.00th=[ 1139], 60.00th=[ 1237],
  183.      | 70.00th=[ 1352], 80.00th=[ 1434], 90.00th=[ 1549], 95.00th=[ 1663],
  184.      | 99.00th=[ 1975], 99.50th=[ 2245], 99.90th=[ 3818], 99.95th=[ 4883],
  185.      | 99.99th=[ 9110]
  186.    bw (  KiB/s): min= 2256, max= 4000, per=12.51%, avg=3369.96, stdev=165.83, samples=9600
  187.    iops        : min=  282, max=  500, avg=421.14, stdev=20.72, samples=9600
  188.   lat (usec)   : 250=0.01%, 500=54.71%, 750=13.28%, 1000=6.29%
  189.   lat (msec)   : 2=25.39%, 4=0.29%, 10=0.03%, 20=0.01%, 50=0.01%
  190.   lat (msec)   : 100=0.01%, 250=0.01%
  191.   cpu          : usr=1.04%, sys=2.60%, ctx=7034877, majf=0, minf=920
  192.   IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  193.      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  194.      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  195.      issued rwt: total=4713376,2020869,0, short=0,0,0, dropped=0,0,0
  196.      latency   : target=0, window=0, percentile=100.00%, depth=1
  197.  
  198. Run status group 0 (all jobs):
  199.    READ: bw=61.4MiB/s (64.4MB/s), 61.4MiB/s-61.4MiB/s (64.4MB/s-64.4MB/s), io=35.0GiB (38.6GB), run=600002-600002msec
  200.   WRITE: bw=26.3MiB/s (27.6MB/s), 26.3MiB/s-26.3MiB/s (27.6MB/s-27.6MB/s), io=15.4GiB (16.6GB), run=600002-600002msec
  201.  
  202. Disk stats (read/write):
  203.   rbd0: ios=4713307/2021079, merge=0/240, ticks=5248/11640, in_queue=4683416, util=100.00%
  204.  
  205.  
  206. ###################################
  207. # baremetal host ssd lvm volume BS=8k JOBS=1
  208. ###################################
  209. admin@e001n01:/mnt/ssd-lvm-volume$ sudo fio --name=randrw --rw=randrw --direct=1 --invalidate=1 --ioengine=libaio --bs=8k --numjobs=1 --rwmixread=70 --size=20G --runtime=600 --time_based --group_reporting
  210. randrw: (g=0): rw=randrw, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=libaio, iodepth=1
  211. ...
  212. fio-3.1
  213. Starting 1 process
  214. randrw: Laying out IO file (1 file / 20480MiB)
  215. Jobs: 1 (f=1): [m(1)][100.0%][r=39.0MiB/s,w=16.3MiB/s][r=4996,w=2085 IOPS][eta 00m:00s]
  216. randrw: (groupid=0, jobs=1): err= 0: pid=2080023: Mon Jul 15 18:02:34 2019
  217.    read: IOPS=4827, BW=37.7MiB/s (39.5MB/s)(22.1GiB/600001msec)
  218.     slat (usec): min=5, max=3102, avg=21.57, stdev= 9.14
  219.     clat (nsec): min=1682, max=46102k, avg=144227.30, stdev=247355.13
  220.      lat (usec): min=72, max=46113, avg=166.17, stdev=247.59
  221.     clat percentiles (usec):
  222.      |  1.00th=[  105],  5.00th=[  112], 10.00th=[  114], 20.00th=[  119],
  223.      | 30.00th=[  122], 40.00th=[  123], 50.00th=[  127], 60.00th=[  129],
  224.      | 70.00th=[  133], 80.00th=[  135], 90.00th=[  137], 95.00th=[  141],
  225.      | 99.00th=[  619], 99.50th=[ 1139], 99.90th=[ 4047], 99.95th=[ 5080],
  226.      | 99.99th=[ 8979]
  227.    bw (  KiB/s): min=11664, max=49200, per=100.00%, avg=38652.46, stdev=6309.27, samples=1200
  228.    iops        : min= 1458, max= 6150, avg=4831.51, stdev=788.66, samples=1200
  229.   write: IOPS=2068, BW=16.2MiB/s (16.9MB/s)(9697MiB/600001msec)
  230.     slat (usec): min=6, max=3080, avg=23.33, stdev= 9.56
  231.     clat (nsec): min=1330, max=32418k, avg=58380.49, stdev=221570.11
  232.      lat (usec): min=45, max=32452, avg=82.13, stdev=221.75
  233.     clat percentiles (usec):
  234.      |  1.00th=[   29],  5.00th=[   40], 10.00th=[   41], 20.00th=[   42],
  235.      | 30.00th=[   43], 40.00th=[   44], 50.00th=[   45], 60.00th=[   46],
  236.      | 70.00th=[   48], 80.00th=[   52], 90.00th=[   59], 95.00th=[   64],
  237.      | 99.00th=[  120], 99.50th=[  490], 99.90th=[ 3392], 99.95th=[ 4686],
  238.      | 99.99th=[ 8979]
  239.    bw (  KiB/s): min= 4576, max=22000, per=100.00%, avg=16562.45, stdev=2755.14, samples=1200
  240.    iops        : min=  572, max= 2750, avg=2070.26, stdev=344.39, samples=1200
  241.   lat (usec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=22.72%
  242.   lat (usec)   : 100=6.95%, 250=69.17%, 500=0.23%, 750=0.17%, 1000=0.15%
  243.   lat (msec)   : 2=0.37%, 4=0.14%, 10=0.09%, 20=0.01%, 50=0.01%
  244.   cpu          : usr=6.01%, sys=19.03%, ctx=4138259, majf=0, minf=27
  245.   IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  246.      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  247.      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  248.      issued rwt: total=2896721,1241238,0, short=0,0,0, dropped=0,0,0
  249.      latency   : target=0, window=0, percentile=100.00%, depth=1
  250.  
  251. Run status group 0 (all jobs):
  252.    READ: bw=37.7MiB/s (39.5MB/s), 37.7MiB/s-37.7MiB/s (39.5MB/s-39.5MB/s), io=22.1GiB (23.7GB), run=600001-600001msec
  253.   WRITE: bw=16.2MiB/s (16.9MB/s), 16.2MiB/s-16.2MiB/s (16.9MB/s-16.9MB/s), io=9697MiB (10.2GB), run=600001-600001msec
  254.  
  255. Disk stats (read/write):
  256.     dm-3: ios=2896300/1245215, merge=0/0, ticks=421216/93584, in_queue=514816, util=83.38%, aggrios=3033772/1543796, aggrmerge=882964/1615625, aggrticks=76336/252132, aggrin_queue=1349488, aggrutil=83.30%
  257.   sda: ios=3033772/1543796, merge=882964/1615625, ticks=76336/252132, in_queue=1349488, util=83.30%
  258.  
  259. ###################################
  260. # baremetal host ssd lvm volume BS=8k JOBS=8
  261. ###################################
  262. admin@e001n01:/mnt/ssd-lvm-volume$ sudo fio --name=randrw --rw=randrw --direct=1 --invalidate=1 --ioengine=libaio --bs=8k --numjobs=8 --rwmixread=70 --size=4G --runtime=600 --time_based --group_reporting
  263. randrw: (g=0): rw=randrw, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=libaio, iodepth=1
  264. ...
  265. fio-3.1
  266. Starting 8 processes
  267. Jobs: 8 (f=8): [m(8)][100.0%][r=206MiB/s,w=88.0MiB/s][r=26.3k,w=11.4k IOPS][eta 00m:00s]
  268. randrw: (groupid=0, jobs=8): err= 0: pid=2106041: Mon Jul 15 18:38:37 2019
  269.    read: IOPS=26.1k, BW=204MiB/s (214MB/s)(120GiB/600001msec)
  270.     slat (usec): min=5, max=3340, avg=17.60, stdev=10.55
  271.     clat (nsec): min=1549, max=45558k, avg=230768.39, stdev=156985.42
  272.      lat (usec): min=114, max=45571, avg=248.68, stdev=156.92
  273.     clat percentiles (usec):
  274.      |  1.00th=[  135],  5.00th=[  151], 10.00th=[  161], 20.00th=[  174],
  275.      | 30.00th=[  184], 40.00th=[  194], 50.00th=[  202], 60.00th=[  210],
  276.      | 70.00th=[  223], 80.00th=[  235], 90.00th=[  258], 95.00th=[  322],
  277.      | 99.00th=[ 1123], 99.50th=[ 1237], 99.90th=[ 1401], 99.95th=[ 1467],
  278.      | 99.99th=[ 3392]
  279.    bw (  KiB/s): min=13840, max=28211, per=12.51%, avg=26127.45, stdev=1034.59, samples=9592
  280.    iops        : min= 1730, max= 3526, avg=3265.85, stdev=129.30, samples=9592
  281.   write: IOPS=11.2k, BW=87.4MiB/s (91.7MB/s)(51.2GiB/600001msec)
  282.     slat (usec): min=6, max=3540, avg=19.91, stdev=11.31
  283.     clat (nsec): min=1351, max=45369k, avg=103558.44, stdev=47063.59
  284.      lat (usec): min=46, max=45390, avg=123.77, stdev=46.72
  285.     clat percentiles (usec):
  286.      |  1.00th=[   42],  5.00th=[   55], 10.00th=[   65], 20.00th=[   78],
  287.      | 30.00th=[   88], 40.00th=[   96], 50.00th=[  103], 60.00th=[  112],
  288.      | 70.00th=[  119], 80.00th=[  128], 90.00th=[  139], 95.00th=[  151],
  289.      | 99.00th=[  172], 99.50th=[  182], 99.90th=[  396], 99.95th=[  701],
  290.      | 99.99th=[ 1139]
  291.    bw (  KiB/s): min= 5488, max=12864, per=12.51%, avg=11197.91, stdev=592.08, samples=9592
  292.    iops        : min=  686, max= 1608, avg=1399.63, stdev=74.00, samples=9592
  293.   lat (usec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=1.10%
  294.   lat (usec)   : 100=12.52%, 250=77.57%, 500=5.96%, 750=1.16%, 1000=0.75%
  295.   lat (msec)   : 2=0.92%, 4=0.02%, 10=0.01%, 20=0.01%, 50=0.01%
  296.   cpu          : usr=3.44%, sys=11.11%, ctx=22382195, majf=0, minf=1232
  297.   IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
  298.      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  299.      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  300.      issued rwt: total=15665288,6713993,0, short=0,0,0, dropped=0,0,0
  301.      latency   : target=0, window=0, percentile=100.00%, depth=1
  302.  
  303. Run status group 0 (all jobs):
  304.    READ: bw=204MiB/s (214MB/s), 204MiB/s-204MiB/s (214MB/s-214MB/s), io=120GiB (128GB), run=600001-600001msec
  305.   WRITE: bw=87.4MiB/s (91.7MB/s), 87.4MiB/s-87.4MiB/s (91.7MB/s-91.7MB/s), io=51.2GiB (55.0GB), run=600001-600001msec
  306.  
  307. Disk stats (read/write):
  308.     dm-3: ios=15663176/6713553, merge=0/0, ticks=3625512/692524, in_queue=4340724, util=100.00%, aggrios=15691448/6744064, aggrmerge=90207/16004, aggrticks=2828/1336, aggrin_queue=4381020, aggrutil=100.00%
  309.   sda: ios=15691448/6744064, merge=90207/16004, ticks=2828/1336, in_queue=4381020, util=100.00%
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!