API tools faq
paste
djbon2112

Ceph pool I/O statistics

djbon2112
Jan 23rd, 2016
300
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 13.14 KB | None | 0 0
raw download clone embed print report
  1.  
  2. ~ $ sudo fio --ioengine=libaio --direct=1 --name=test --filename=/var/test --bs=4k --iodepth=64 --size=3G --randrepeat=0 --group_reporting=1 --readwrite=randrw --rwmixread=75
  3. test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
  4. fio-2.1.11
  5. Starting 1 process
  6. Jobs: 1 (f=1): [m(1)] [100.0% done] [38816KB/12800KB/0KB /s] [9704/3200/0 iops] [eta 00m:00s]
  7. test: (groupid=0, jobs=1): err= 0: pid=1215: Sat Jan 23 16:49:53 2016
  8. read : io=2307.1MB, bw=27005KB/s, iops=6751, runt= 87482msec
  9. slat (usec): min=3, max=1213, avg=14.44, stdev= 7.81
  10. clat (usec): min=673, max=837866, avg=7573.85, stdev=24086.00
  11. lat (usec): min=679, max=837904, avg=7588.98, stdev=24086.03
  12. clat percentiles (msec):
  13. | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6],
  14. | 30.00th=[ 6], 40.00th=[ 6], 50.00th=[ 7], 60.00th=[ 7],
  15. | 70.00th=[ 8], 80.00th=[ 8], 90.00th=[ 8], 95.00th=[ 9],
  16. | 99.00th=[ 21], 99.50th=[ 29], 99.90th=[ 594], 99.95th=[ 660],
  17. | 99.99th=[ 717]
  18. bw (KB /s): min= 449, max=42400, per=100.00%, avg=28486.60, stdev=7437.65
  19. write: io=783268KB, bw=8953.5KB/s, iops=2238, runt= 87482msec
  20. slat (usec): min=4, max=3384, avg=17.64, stdev=18.98
  21. clat (usec): min=47, max=618605, avg=5667.85, stdev=19548.10
  22. lat (usec): min=118, max=619125, avg=5686.21, stdev=19552.04
  23. clat percentiles (usec):
  24. | 1.00th=[ 716], 5.00th=[ 3024], 10.00th=[ 3440], 20.00th=[ 3856],
  25. | 30.00th=[ 4256], 40.00th=[ 4640], 50.00th=[ 5152], 60.00th=[ 5472],
  26. | 70.00th=[ 5792], 80.00th=[ 6112], 90.00th=[ 6560], 95.00th=[ 6816],
  27. | 99.00th=[ 7392], 99.50th=[ 7712], 99.90th=[528384], 99.95th=[569344],
  28. | 99.99th=[602112]
  29. bw (KB /s): min= 62, max=14536, per=100.00%, avg=9443.66, stdev=2510.64
  30. lat (usec) : 50=0.01%, 250=0.04%, 500=0.13%, 750=0.09%, 1000=0.10%
  31. lat (msec) : 2=0.67%, 4=6.64%, 10=90.54%, 20=0.97%, 50=0.57%
  32. lat (msec) : 100=0.06%, 250=0.06%, 500=0.01%, 750=0.13%, 1000=0.01%
  33. cpu : usr=6.87%, sys=19.62%, ctx=555837, majf=0, minf=8
  34. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
  35. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  36. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
  37. issued : total=r=590615/w=195817/d=0, short=r=0/w=0/d=0
  38. latency : target=0, window=0, percentile=100.00%, depth=64
  39.  
  40. Run status group 0 (all jobs):
  41. READ: io=2307.1MB, aggrb=27005KB/s, minb=27005KB/s, maxb=27005KB/s, mint=87482msec, maxt=87482msec
  42. WRITE: io=783268KB, aggrb=8953KB/s, minb=8953KB/s, maxb=8953KB/s, mint=87482msec, maxt=87482msec
  43.  
  44. Disk stats (read/write):
  45. vdb: ios=588900/195297, merge=0/0, ticks=4452556/1115244, in_queue=5567756, util=99.98%
  46.  
  47. ~ $ sudo fio --ioengine=libaio --direct=1 --name=test --filename=/var/test --bs=4k --iodepth=64 --size=3G --randrepeat=0 --group_reporting=1 --readwrite=randread
  48. test: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
  49. fio-2.1.11
  50. Starting 1 process
  51. Jobs: 1 (f=1): [r(1)] [100.0% done] [46552KB/0KB/0KB /s] [11.7K/0/0 iops] [eta 00m:00s]
  52. test: (groupid=0, jobs=1): err= 0: pid=1257: Sat Jan 23 16:51:12 2016
  53. read : io=3072.0MB, bw=46362KB/s, iops=11590, runt= 67851msec
  54. slat (usec): min=3, max=1744, avg=14.55, stdev= 6.24
  55. clat (usec): min=833, max=14023, avg=5502.44, stdev=695.81
  56. lat (usec): min=840, max=14040, avg=5517.72, stdev=696.52
  57. clat percentiles (usec):
  58. | 1.00th=[ 4256], 5.00th=[ 4512], 10.00th=[ 4704], 20.00th=[ 4896],
  59. | 30.00th=[ 5088], 40.00th=[ 5280], 50.00th=[ 5472], 60.00th=[ 5664],
  60. | 70.00th=[ 5856], 80.00th=[ 6048], 90.00th=[ 6432], 95.00th=[ 6688],
  61. | 99.00th=[ 7392], 99.50th=[ 7648], 99.90th=[ 8256], 99.95th=[ 8640],
  62. | 99.99th=[10176]
  63. bw (KB /s): min=42616, max=51816, per=100.00%, avg=46373.45, stdev=1877.62
  64. lat (usec) : 1000=0.01%
  65. lat (msec) : 2=0.01%, 4=0.21%, 10=99.76%, 20=0.01%
  66. cpu : usr=8.60%, sys=26.48%, ctx=743213, majf=0, minf=70
  67. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
  68. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  69. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
  70. issued : total=r=786432/w=0/d=0, short=r=0/w=0/d=0
  71. latency : target=0, window=0, percentile=100.00%, depth=64
  72.  
  73. Run status group 0 (all jobs):
  74. READ: io=3072.0MB, aggrb=46362KB/s, minb=46362KB/s, maxb=46362KB/s, mint=67851msec, maxt=67851msec
  75.  
  76. Disk stats (read/write):
  77. vdb: ios=784858/1, merge=0/0, ticks=4311428/0, in_queue=4311864, util=99.94%
  78.  
  79. ~ $ sudo fio --ioengine=libaio --direct=1 --name=test --filename=/var/test --bs=4k --iodepth=64 --size=3G --randrepeat=0 --group_reporting=1 --readwrite=randwrite
  80. test: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
  81. fio-2.1.11
  82. Starting 1 process
  83. Jobs: 1 (f=1): [w(1)] [100.0% done] [0KB/31932KB/0KB /s] [0/7983/0 iops] [eta 00m:00s]
  84. test: (groupid=0, jobs=1): err= 0: pid=1280: Sat Jan 23 16:53:48 2016
  85. write: io=3072.0MB, bw=22391KB/s, iops=5597, runt=140490msec
  86. slat (usec): min=3, max=26771, avg=13.78, stdev=64.17
  87. clat (usec): min=530, max=849382, avg=11415.01, stdev=49024.57
  88. lat (usec): min=553, max=854149, avg=11429.49, stdev=49029.63
  89. clat percentiles (usec):
  90. | 1.00th=[ 1192], 5.00th=[ 1704], 10.00th=[ 2040], 20.00th=[ 2512],
  91. | 30.00th=[ 5536], 40.00th=[ 7136], 50.00th=[ 7968], 60.00th=[ 8640],
  92. | 70.00th=[ 9792], 80.00th=[11584], 90.00th=[14912], 95.00th=[18304],
  93. | 99.00th=[29568], 99.50th=[42752], 99.90th=[790528], 99.95th=[823296],
  94. | 99.99th=[839680]
  95. bw (KB /s): min= 43, max=61128, per=100.00%, avg=24961.89, stdev=14855.27
  96. lat (usec) : 750=0.01%, 1000=0.11%
  97. lat (msec) : 2=9.04%, 4=18.62%, 10=43.91%, 20=24.66%, 50=3.18%
  98. lat (msec) : 100=0.03%, 500=0.02%, 750=0.18%, 1000=0.24%
  99. cpu : usr=3.54%, sys=9.87%, ctx=289361, majf=0, minf=8
  100. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
  101. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  102. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
  103. issued : total=r=0/w=786432/d=0, short=r=0/w=0/d=0
  104. latency : target=0, window=0, percentile=100.00%, depth=64
  105.  
  106. Run status group 0 (all jobs):
  107. WRITE: io=3072.0MB, aggrb=22391KB/s, minb=22391KB/s, maxb=22391KB/s, mint=140490msec, maxt=140490msec
  108.  
  109. Disk stats (read/write):
  110. vdb: ios=0/784681, merge=0/0, ticks=0/8967484, in_queue=8986164, util=100.00%
  111.  
  112. ~ $ sudo fio --ioengine=libaio --direct=1 --name=test --filename=/var/test --bs=4k --iodepth=64 --size=3G --randrepeat=0 --group_reporting=1 --readwrite=rw --rwmixread=75
  113. test: (g=0): rw=rw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
  114. fio-2.1.11
  115. Starting 1 process
  116. Jobs: 1 (f=1): [M(1)] [100.0% done] [27580KB/9164KB/0KB /s] [6895/2291/0 iops] [eta 00m:00s]
  117. test: (groupid=0, jobs=1): err= 0: pid=1322: Sat Jan 23 16:55:32 2016
  118. read : io=2302.8MB, bw=25084KB/s, iops=6270, runt= 94005msec
  119. slat (usec): min=2, max=1013, avg=13.56, stdev= 6.94
  120. clat (usec): min=885, max=639637, avg=9835.35, stdev=19322.83
  121. lat (usec): min=899, max=639652, avg=9849.54, stdev=19322.94
  122. clat percentiles (msec):
  123. | 1.00th=[ 4], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8],
  124. | 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 9], 60.00th=[ 9],
  125. | 70.00th=[ 10], 80.00th=[ 10], 90.00th=[ 11], 95.00th=[ 12],
  126. | 99.00th=[ 18], 99.50th=[ 60], 99.90th=[ 396], 99.95th=[ 449],
  127. | 99.99th=[ 545]
  128. bw (KB /s): min= 534, max=33016, per=100.00%, avg=25521.12, stdev=6760.26
  129. write: io=787732KB, bw=8379.7KB/s, iops=2094, runt= 94005msec
  130. slat (usec): min=3, max=567, avg=16.40, stdev= 7.52
  131. clat (usec): min=34, max=479390, avg=1036.46, stdev=7179.57
  132. lat (usec): min=72, max=479414, avg=1053.52, stdev=7179.64
  133. clat percentiles (usec):
  134. | 1.00th=[ 100], 5.00th=[ 122], 10.00th=[ 143], 20.00th=[ 179],
  135. | 30.00th=[ 213], 40.00th=[ 258], 50.00th=[ 322], 60.00th=[ 450],
  136. | 70.00th=[ 812], 80.00th=[ 1512], 90.00th=[ 2704], 95.00th=[ 3696],
  137. | 99.00th=[ 5216], 99.50th=[ 5600], 99.90th=[ 6688], 99.95th=[ 9024],
  138. | 99.99th=[387072]
  139. bw (KB /s): min= 202, max=10904, per=100.00%, avg=8524.23, stdev=2273.51
  140. lat (usec) : 50=0.01%, 100=0.25%, 250=9.40%, 500=5.93%, 750=1.66%
  141. lat (usec) : 1000=1.03%
  142. lat (msec) : 2=3.22%, 4=3.73%, 10=60.43%, 20=13.64%, 50=0.23%
  143. lat (msec) : 100=0.18%, 250=0.13%, 500=0.14%, 750=0.01%
  144. cpu : usr=5.24%, sys=17.46%, ctx=551837, majf=0, minf=9
  145. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
  146. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  147. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
  148. issued : total=r=589499/w=196933/d=0, short=r=0/w=0/d=0
  149. latency : target=0, window=0, percentile=100.00%, depth=64
  150.  
  151. Run status group 0 (all jobs):
  152. READ: io=2302.8MB, aggrb=25083KB/s, minb=25083KB/s, maxb=25083KB/s, mint=94005msec, maxt=94005msec
  153. WRITE: io=787732KB, aggrb=8379KB/s, minb=8379KB/s, maxb=8379KB/s, mint=94005msec, maxt=94005msec
  154.  
  155. Disk stats (read/write):
  156. vdb: ios=589404/196960, merge=0/0, ticks=5787792/211764, in_queue=5999784, util=99.98%
  157.  
  158. ~ $ sudo fio --ioengine=libaio --direct=1 --name=test --filename=/var/test --bs=4k --iodepth=64 --size=3G --randrepeat=0 --group_reporting=1 --readwrite=read
  159. test: (g=0): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
  160. fio-2.1.11
  161. Starting 1 process
  162. Jobs: 1 (f=1): [R(1)] [100.0% done] [29432KB/0KB/0KB /s] [7358/0/0 iops] [eta 00m:00s]
  163. test: (groupid=0, jobs=1): err= 0: pid=1446: Sat Jan 23 17:01:14 2016
  164. read : io=3072.0MB, bw=29149KB/s, iops=7287, runt=107919msec
  165. slat (usec): min=2, max=1434, avg=14.80, stdev= 7.04
  166. clat (usec): min=926, max=15771, avg=8763.53, stdev=1501.37
  167. lat (usec): min=944, max=15785, avg=8778.99, stdev=1501.74
  168. clat percentiles (usec):
  169. | 1.00th=[ 4576], 5.00th=[ 6432], 10.00th=[ 7136], 20.00th=[ 7712],
  170. | 30.00th=[ 8096], 40.00th=[ 8384], 50.00th=[ 8768], 60.00th=[ 9024],
  171. | 70.00th=[ 9408], 80.00th=[ 9920], 90.00th=[10560], 95.00th=[11200],
  172. | 99.00th=[12480], 99.50th=[12992], 99.90th=[14016], 99.95th=[14272],
  173. | 99.99th=[14912]
  174. bw (KB /s): min=25992, max=34464, per=100.00%, avg=29155.46, stdev=1482.63
  175. lat (usec) : 1000=0.01%
  176. lat (msec) : 2=0.11%, 4=0.57%, 10=81.04%, 20=18.27%
  177. cpu : usr=5.01%, sys=16.51%, ctx=640808, majf=0, minf=71
  178. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
  179. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  180. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
  181. issued : total=r=786432/w=0/d=0, short=r=0/w=0/d=0
  182. latency : target=0, window=0, percentile=100.00%, depth=64
  183.  
  184. Run status group 0 (all jobs):
  185. READ: io=3072.0MB, aggrb=29148KB/s, minb=29148KB/s, maxb=29148KB/s, mint=107919msec, maxt=107919msec
  186.  
  187. Disk stats (read/write):
  188. vdb: ios=785147/0, merge=0/0, ticks=6870936/0, in_queue=6871496, util=99.99%
  189.  
  190. ~ $ sudo fio --ioengine=libaio --direct=1 --name=test --filename=/var/test --bs=4k --iodepth=64 --size=3G --randrepeat=0 --group_reporting=1 --readwrite=write
  191. test: (g=0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
  192. fio-2.1.11
  193. Starting 1 process
  194. Jobs: 1 (f=1): [W(1)] [100.0% done] [0KB/70533KB/0KB /s] [0/17.7K/0 iops] [eta 00m:00s]
  195. test: (groupid=0, jobs=1): err= 0: pid=1488: Sat Jan 23 17:02:46 2016
  196. write: io=3072.0MB, bw=61065KB/s, iops=15266, runt= 51514msec
  197. slat (usec): min=3, max=1051, avg= 7.99, stdev= 7.12
  198. clat (usec): min=392, max=251277, avg=4181.58, stdev=19743.25
  199. lat (usec): min=398, max=251281, avg=4190.05, stdev=19743.00
  200. clat percentiles (usec):
  201. | 1.00th=[ 466], 5.00th=[ 494], 10.00th=[ 516], 20.00th=[ 548],
  202. | 30.00th=[ 572], 40.00th=[ 620], 50.00th=[ 692], 60.00th=[ 764],
  203. | 70.00th=[ 828], 80.00th=[ 940], 90.00th=[ 1240], 95.00th=[ 2352],
  204. | 99.00th=[126464], 99.50th=[154624], 99.90th=[191488], 99.95th=[197632],
  205. | 99.99th=[228352]
  206. bw (KB /s): min=43134, max=89027, per=100.00%, avg=61462.05, stdev=8836.90
  207. lat (usec) : 500=6.72%, 750=51.66%, 1000=25.54%
  208. lat (msec) : 2=10.95%, 4=0.46%, 10=0.44%, 20=0.55%, 50=1.16%
  209. lat (msec) : 100=0.89%, 250=1.61%, 500=0.01%
  210. cpu : usr=4.46%, sys=12.67%, ctx=11015, majf=0, minf=9
  211. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
  212. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  213. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
  214. issued : total=r=0/w=786432/d=0, short=r=0/w=0/d=0
  215. latency : target=0, window=0, percentile=100.00%, depth=64
  216.  
  217. Run status group 0 (all jobs):
  218. WRITE: io=3072.0MB, aggrb=61065KB/s, minb=61065KB/s, maxb=61065KB/s, mint=51514msec, maxt=51514msec
  219.  
  220. Disk stats (read/write):
  221. vdb: ios=0/786461, merge=0/0, ticks=0/2902740, in_queue=2902660, util=99.74%
  222.  
  223. ~ $
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!