API tools faq
paste
Guest User

Untitled

a guest
Apr 15th, 2021
21
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 101.71 KB | None | 0 0
raw download clone embed print report
  1. сервер Dell PowerEdge R740xd - 2x Xeon Gold 6252 @ 2.10GHz
  2. диски Dell Ent NVMe AGN MU AIC 1.6TB (Samsung PM1733)
  3.  
  4. -------------------------------------------------------------------------------------------
  5. root@sus-node-2 ~]# fio --filename=/dev/md0 --direct=1 --rw=randread --bs=4k --ioengine=libaio --iodepth=256 --runtime=600 --numjobs=32 --time_based --group_reporting --name=t --eta-newline=1 --output-format=normal
  6. t: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=256
  7. ...
  8. fio-3.19
  9. Starting 32 processes
  10. Jobs: 32 (f=32): [r(32)][0.5%][r=6849MiB/s][r=1753k IOPS][eta 09m:58s]
  11. Jobs: 32 (f=32): [r(32)][0.7%][r=5190MiB/s][r=1329k IOPS][eta 09m:57s]
  12. Jobs: 32 (f=32): [r(32)][0.8%][r=5162MiB/s][r=1321k IOPS][eta 09m:56s]
  13. Jobs: 32 (f=32): [r(32)][1.0%][r=5177MiB/s][r=1325k IOPS][eta 09m:55s]
  14. Jobs: 32 (f=32): [r(32)][1.2%][r=5188MiB/s][r=1328k IOPS][eta 09m:54s]
  15. Jobs: 32 (f=32): [r(32)][1.3%][r=5170MiB/s][r=1324k IOPS][eta 09m:53s]
  16. Jobs: 32 (f=32): [r(32)][1.5%][r=5170MiB/s][r=1323k IOPS][eta 09m:52s]
  17. Jobs: 32 (f=32): [r(32)][1.7%][r=5219MiB/s][r=1336k IOPS][eta 09m:51s]
  18. Jobs: 32 (f=32): [r(32)][1.8%][r=5206MiB/s][r=1333k IOPS][eta 09m:50s]
  19. Jobs: 32 (f=32): [r(32)][2.0%][r=5202MiB/s][r=1332k IOPS][eta 09m:49s]
  20. Jobs: 32 (f=32): [r(32)][2.2%][r=5201MiB/s][r=1332k IOPS][eta 09m:48s]
  21. Jobs: 32 (f=32): [r(32)][2.3%][r=5218MiB/s][r=1336k IOPS][eta 09m:47s]
  22. Jobs: 32 (f=32): [r(32)][2.5%][r=5190MiB/s][r=1329k IOPS][eta 09m:46s]
  23. Jobs: 32 (f=32): [r(32)][2.7%][r=5192MiB/s][r=1329k IOPS][eta 09m:45s]
  24. Jobs: 32 (f=32): [r(32)][2.8%][r=5198MiB/s][r=1331k IOPS][eta 09m:44s]
  25. Jobs: 32 (f=32): [r(32)][3.0%][r=5237MiB/s][r=1341k IOPS][eta 09m:43s]
  26. Jobs: 32 (f=32): [r(32)][3.2%][r=5197MiB/s][r=1331k IOPS][eta 09m:42s]
  27. Jobs: 32 (f=32): [r(32)][3.3%][r=4789MiB/s][r=1226k IOPS][eta 09m:41s]
  28. Jobs: 32 (f=32): [r(32)][3.5%][r=4365MiB/s][r=1117k IOPS][eta 09m:40s]
  29. Jobs: 32 (f=32): [r(32)][3.7%][r=4375MiB/s][r=1120k IOPS][eta 09m:39s]
  30. Jobs: 32 (f=32): [r(32)][3.8%][r=4362MiB/s][r=1117k IOPS][eta 09m:38s]
  31. Jobs: 32 (f=32): [r(32)][4.0%][r=4359MiB/s][r=1116k IOPS][eta 09m:37s]
  32. Jobs: 32 (f=32): [r(32)][4.2%][r=4347MiB/s][r=1113k IOPS][eta 09m:36s]
  33. Jobs: 32 (f=32): [r(32)][4.3%][r=4310MiB/s][r=1103k IOPS][eta 09m:35s]
  34. Jobs: 32 (f=32): [r(32)][4.5%][r=4224MiB/s][r=1081k IOPS][eta 09m:34s]
  35. Jobs: 32 (f=32): [r(32)][4.7%][r=4072MiB/s][r=1042k IOPS][eta 09m:33s]
  36. Jobs: 32 (f=32): [r(32)][4.8%][r=3912MiB/s][r=1001k IOPS][eta 09m:32s]
  37. Jobs: 32 (f=32): [r(32)][5.0%][r=3896MiB/s][r=997k IOPS][eta 09m:31s]
  38. Jobs: 32 (f=32): [r(32)][5.2%][r=3913MiB/s][r=1002k IOPS][eta 09m:30s]
  39. Jobs: 32 (f=32): [r(32)][5.3%][r=3915MiB/s][r=1002k IOPS][eta 09m:29s]
  40. Jobs: 32 (f=32): [r(32)][5.5%][r=3901MiB/s][r=999k IOPS][eta 09m:28s]
  41. Jobs: 32 (f=32): [r(32)][5.7%][r=3908MiB/s][r=1000k IOPS][eta 09m:27s]
  42. Jobs: 32 (f=32): [r(32)][5.8%][r=3900MiB/s][r=998k IOPS][eta 09m:26s]
  43. Jobs: 32 (f=32): [r(32)][6.0%][r=3904MiB/s][r=999k IOPS][eta 09m:25s]
  44. Jobs: 32 (f=32): [r(32)][6.2%][r=3892MiB/s][r=996k IOPS][eta 09m:24s]
  45. Jobs: 32 (f=32): [r(32)][6.3%][r=3900MiB/s][r=998k IOPS][eta 09m:23s]
  46. Jobs: 32 (f=32): [r(32)][6.5%][r=3904MiB/s][r=999k IOPS][eta 09m:22s]
  47. Jobs: 32 (f=32): [r(32)][6.7%][r=3908MiB/s][r=1000k IOPS][eta 09m:21s]
  48. Jobs: 32 (f=32): [r(32)][6.8%][r=3892MiB/s][r=996k IOPS][eta 09m:20s]
  49. Jobs: 32 (f=32): [r(32)][7.0%][r=3910MiB/s][r=1001k IOPS][eta 09m:19s]
  50. Jobs: 32 (f=32): [r(32)][7.2%][r=3903MiB/s][r=999k IOPS][eta 09m:18s]
  51. Jobs: 32 (f=32): [r(32)][7.3%][r=3901MiB/s][r=999k IOPS][eta 09m:17s]
  52. Jobs: 32 (f=32): [r(32)][7.5%][r=3905MiB/s][r=1000k IOPS][eta 09m:16s]
  53. Jobs: 32 (f=32): [r(32)][7.7%][r=3896MiB/s][r=997k IOPS][eta 09m:15s]
  54. Jobs: 32 (f=32): [r(32)][7.8%][r=3907MiB/s][r=1000k IOPS][eta 09m:14s]
  55. Jobs: 32 (f=32): [r(32)][8.0%][r=3911MiB/s][r=1001k IOPS][eta 09m:13s]
  56. Jobs: 32 (f=32): [r(32)][8.2%][r=3896MiB/s][r=998k IOPS][eta 09m:12s]
  57. Jobs: 32 (f=32): [r(32)][8.3%][r=3895MiB/s][r=997k IOPS][eta 09m:11s]
  58. Jobs: 32 (f=32): [r(32)][8.5%][r=3909MiB/s][r=1001k IOPS][eta 09m:10s]
  59. Jobs: 32 (f=32): [r(32)][8.7%][r=3893MiB/s][r=997k IOPS][eta 09m:09s]
  60. Jobs: 32 (f=32): [r(32)][8.8%][r=3896MiB/s][r=997k IOPS][eta 09m:08s]
  61. Jobs: 32 (f=32): [r(32)][9.0%][r=3900MiB/s][r=998k IOPS][eta 09m:06s]
  62. Jobs: 32 (f=32): [r(32)][9.2%][r=3900MiB/s][r=998k IOPS][eta 09m:05s]
  63. Jobs: 32 (f=32): [r(32)][9.3%][r=3896MiB/s][r=997k IOPS][eta 09m:04s]
  64. Jobs: 32 (f=32): [r(32)][9.5%][r=3894MiB/s][r=997k IOPS][eta 09m:03s]
  65. Jobs: 32 (f=32): [r(32)][9.7%][r=3909MiB/s][r=1001k IOPS][eta 09m:02s]
  66. Jobs: 32 (f=32): [r(32)][9.8%][r=3908MiB/s][r=1000k IOPS][eta 09m:01s]
  67. Jobs: 32 (f=32): [r(32)][10.0%][r=3902MiB/s][r=999k IOPS][eta 09m:00s]
  68. Jobs: 32 (f=32): [r(32)][10.2%][r=3901MiB/s][r=999k IOPS][eta 08m:59s]
  69. Jobs: 32 (f=32): [r(32)][10.3%][r=3904MiB/s][r=999k IOPS][eta 08m:58s]
  70. Jobs: 32 (f=32): [r(32)][10.5%][r=3894MiB/s][r=997k IOPS][eta 08m:57s]
  71. Jobs: 32 (f=32): [r(32)][10.7%][r=3896MiB/s][r=997k IOPS][eta 08m:56s]
  72. Jobs: 32 (f=32): [r(32)][10.8%][r=3896MiB/s][r=997k IOPS][eta 08m:55s]
  73. Jobs: 32 (f=32): [r(32)][11.0%][r=3902MiB/s][r=999k IOPS][eta 08m:54s]
  74. Jobs: 32 (f=32): [r(32)][11.2%][r=3897MiB/s][r=998k IOPS][eta 08m:53s]
  75. Jobs: 32 (f=32): [r(32)][11.3%][r=3903MiB/s][r=999k IOPS][eta 08m:52s]
  76. Jobs: 32 (f=32): [r(32)][11.5%][r=3893MiB/s][r=997k IOPS][eta 08m:51s]
  77. Jobs: 32 (f=32): [r(32)][11.7%][r=3898MiB/s][r=998k IOPS][eta 08m:50s]
  78. Jobs: 32 (f=32): [r(32)][11.8%][r=3893MiB/s][r=997k IOPS][eta 08m:49s]
  79. Jobs: 32 (f=32): [r(32)][12.0%][r=3896MiB/s][r=997k IOPS][eta 08m:48s]
  80. Jobs: 32 (f=32): [r(32)][12.2%][r=3895MiB/s][r=997k IOPS][eta 08m:47s]
  81. Jobs: 32 (f=32): [r(32)][12.3%][r=3899MiB/s][r=998k IOPS][eta 08m:46s]
  82. Jobs: 32 (f=32): [r(32)][12.5%][r=3904MiB/s][r=999k IOPS][eta 08m:45s]
  83. Jobs: 32 (f=32): [r(32)][12.7%][r=3909MiB/s][r=1001k IOPS][eta 08m:44s]
  84. Jobs: 32 (f=32): [r(32)][12.8%][r=3892MiB/s][r=996k IOPS][eta 08m:43s]
  85. Jobs: 32 (f=32): [r(32)][13.0%][r=3888MiB/s][r=995k IOPS][eta 08m:42s]
  86. Jobs: 32 (f=32): [r(32)][13.2%][r=3902MiB/s][r=999k IOPS][eta 08m:41s]
  87. Jobs: 32 (f=32): [r(32)][13.3%][r=3890MiB/s][r=996k IOPS][eta 08m:40s]
  88. Jobs: 32 (f=32): [r(32)][13.5%][r=3907MiB/s][r=1000k IOPS][eta 08m:39s]
  89. Jobs: 32 (f=32): [r(32)][13.7%][r=3898MiB/s][r=998k IOPS][eta 08m:38s]
  90. Jobs: 32 (f=32): [r(32)][13.8%][r=3894MiB/s][r=997k IOPS][eta 08m:37s]
  91. Jobs: 32 (f=32): [r(32)][14.0%][r=3906MiB/s][r=1000k IOPS][eta 08m:36s]
  92. Jobs: 32 (f=32): [r(32)][14.2%][r=3912MiB/s][r=1002k IOPS][eta 08m:35s]
  93. Jobs: 32 (f=32): [r(32)][14.3%][r=3902MiB/s][r=999k IOPS][eta 08m:34s]
  94. Jobs: 32 (f=32): [r(32)][14.5%][r=3908MiB/s][r=1000k IOPS][eta 08m:33s]
  95. Jobs: 32 (f=32): [r(32)][14.7%][r=3902MiB/s][r=999k IOPS][eta 08m:32s]
  96. Jobs: 32 (f=32): [r(32)][14.8%][r=3888MiB/s][r=995k IOPS][eta 08m:31s]
  97. Jobs: 32 (f=32): [r(32)][15.0%][r=3905MiB/s][r=1000k IOPS][eta 08m:30s]
  98. Jobs: 32 (f=32): [r(32)][15.2%][r=3900MiB/s][r=998k IOPS][eta 08m:29s]
  99. Jobs: 32 (f=32): [r(32)][15.3%][r=3893MiB/s][r=997k IOPS][eta 08m:28s]
  100. Jobs: 32 (f=32): [r(32)][15.5%][r=3904MiB/s][r=999k IOPS][eta 08m:27s]
  101. Jobs: 32 (f=32): [r(32)][15.7%][r=3902MiB/s][r=999k IOPS][eta 08m:26s]
  102. Jobs: 32 (f=32): [r(32)][15.8%][r=3889MiB/s][r=995k IOPS][eta 08m:25s]
  103. Jobs: 32 (f=32): [r(32)][16.0%][r=3899MiB/s][r=998k IOPS][eta 08m:24s]
  104. Jobs: 32 (f=32): [r(32)][16.2%][r=3903MiB/s][r=999k IOPS][eta 08m:23s]
  105. Jobs: 32 (f=32): [r(32)][16.3%][r=3885MiB/s][r=995k IOPS][eta 08m:22s]
  106. Jobs: 32 (f=32): [r(32)][16.5%][r=3903MiB/s][r=999k IOPS][eta 08m:21s]
  107. Jobs: 32 (f=32): [r(32)][16.7%][r=3898MiB/s][r=998k IOPS][eta 08m:20s]
  108. Jobs: 32 (f=32): [r(32)][16.8%][r=3899MiB/s][r=998k IOPS][eta 08m:19s]
  109. Jobs: 32 (f=32): [r(32)][17.0%][r=3901MiB/s][r=999k IOPS][eta 08m:18s]
  110. Jobs: 32 (f=32): [r(32)][17.2%][r=3894MiB/s][r=997k IOPS][eta 08m:17s]
  111. Jobs: 32 (f=32): [r(32)][17.3%][r=3905MiB/s][r=1000k IOPS][eta 08m:16s]
  112. Jobs: 32 (f=32): [r(32)][17.5%][r=3906MiB/s][r=1000k IOPS][eta 08m:15s]
  113. Jobs: 32 (f=32): [r(32)][17.7%][r=3893MiB/s][r=997k IOPS][eta 08m:14s]
  114. Jobs: 32 (f=32): [r(32)][17.8%][r=3898MiB/s][r=998k IOPS][eta 08m:13s]
  115. Jobs: 32 (f=32): [r(32)][18.0%][r=3895MiB/s][r=997k IOPS][eta 08m:12s]
  116. Jobs: 32 (f=32): [r(32)][18.2%][r=3893MiB/s][r=997k IOPS][eta 08m:11s]
  117. Jobs: 32 (f=32): [r(32)][18.3%][r=3899MiB/s][r=998k IOPS][eta 08m:10s]
  118. Jobs: 32 (f=32): [r(32)][18.5%][r=3909MiB/s][r=1001k IOPS][eta 08m:09s]
  119. Jobs: 32 (f=32): [r(32)][18.7%][r=3903MiB/s][r=999k IOPS][eta 08m:08s]
  120. Jobs: 32 (f=32): [r(32)][18.8%][r=3891MiB/s][r=996k IOPS][eta 08m:07s]
  121. Jobs: 32 (f=32): [r(32)][19.0%][r=3895MiB/s][r=997k IOPS][eta 08m:06s]
  122. Jobs: 32 (f=32): [r(32)][19.2%][r=3903MiB/s][r=999k IOPS][eta 08m:05s]
  123. Jobs: 32 (f=32): [r(32)][19.3%][r=3899MiB/s][r=998k IOPS][eta 08m:04s]
  124. Jobs: 32 (f=32): [r(32)][19.5%][r=3900MiB/s][r=999k IOPS][eta 08m:03s]
  125. Jobs: 32 (f=32): [r(32)][19.7%][r=3901MiB/s][r=999k IOPS][eta 08m:02s]
  126. Jobs: 32 (f=32): [r(32)][19.8%][r=3899MiB/s][r=998k IOPS][eta 08m:01s]
  127. Jobs: 32 (f=32): [r(32)][20.0%][r=3914MiB/s][r=1002k IOPS][eta 08m:00s]
  128. Jobs: 32 (f=32): [r(32)][20.2%][r=3891MiB/s][r=996k IOPS][eta 07m:59s]
  129. Jobs: 32 (f=32): [r(32)][20.3%][r=3896MiB/s][r=997k IOPS][eta 07m:58s]
  130. Jobs: 32 (f=32): [r(32)][20.5%][r=3909MiB/s][r=1001k IOPS][eta 07m:57s]
  131. Jobs: 32 (f=32): [r(32)][20.7%][r=3899MiB/s][r=998k IOPS][eta 07m:56s]
  132. Jobs: 32 (f=32): [r(32)][20.8%][r=3897MiB/s][r=998k IOPS][eta 07m:55s]
  133. Jobs: 32 (f=32): [r(32)][21.0%][r=3908MiB/s][r=1001k IOPS][eta 07m:54s]
  134. Jobs: 32 (f=32): [r(32)][21.2%][r=3900MiB/s][r=999k IOPS][eta 07m:53s]
  135. Jobs: 32 (f=32): [r(32)][21.3%][r=3889MiB/s][r=996k IOPS][eta 07m:52s]
  136. Jobs: 32 (f=32): [r(32)][21.5%][r=3904MiB/s][r=999k IOPS][eta 07m:51s]
  137. Jobs: 32 (f=32): [r(32)][21.7%][r=3910MiB/s][r=1001k IOPS][eta 07m:50s]
  138. Jobs: 32 (f=32): [r(32)][21.8%][r=3883MiB/s][r=994k IOPS][eta 07m:49s]
  139. Jobs: 32 (f=32): [r(32)][22.0%][r=3904MiB/s][r=999k IOPS][eta 07m:48s]
  140. Jobs: 32 (f=32): [r(32)][22.2%][r=3888MiB/s][r=995k IOPS][eta 07m:47s]
  141. Jobs: 32 (f=32): [r(32)][22.3%][r=3900MiB/s][r=998k IOPS][eta 07m:46s]
  142. Jobs: 32 (f=32): [r(32)][22.5%][r=3897MiB/s][r=998k IOPS][eta 07m:45s]
  143. Jobs: 32 (f=32): [r(32)][22.7%][r=3898MiB/s][r=998k IOPS][eta 07m:44s]
  144. Jobs: 32 (f=32): [r(32)][22.8%][r=3900MiB/s][r=999k IOPS][eta 07m:43s]
  145. Jobs: 32 (f=32): [r(32)][23.0%][r=3901MiB/s][r=999k IOPS][eta 07m:42s]
  146. Jobs: 32 (f=32): [r(32)][23.2%][r=3908MiB/s][r=1000k IOPS][eta 07m:41s]
  147. Jobs: 32 (f=32): [r(32)][23.3%][r=3900MiB/s][r=998k IOPS][eta 07m:40s]
  148. Jobs: 32 (f=32): [r(32)][23.5%][r=3901MiB/s][r=999k IOPS][eta 07m:39s]
  149. Jobs: 32 (f=32): [r(32)][23.7%][r=3910MiB/s][r=1001k IOPS][eta 07m:38s]
  150. Jobs: 32 (f=32): [r(32)][23.8%][r=3905MiB/s][r=1000k IOPS][eta 07m:37s]
  151. Jobs: 32 (f=32): [r(32)][24.0%][r=3919MiB/s][r=1003k IOPS][eta 07m:36s]
  152. Jobs: 32 (f=32): [r(32)][24.2%][r=3908MiB/s][r=1001k IOPS][eta 07m:35s]
  153. Jobs: 32 (f=32): [r(32)][24.3%][r=3895MiB/s][r=997k IOPS][eta 07m:34s]
  154. Jobs: 32 (f=32): [r(32)][24.5%][r=3894MiB/s][r=997k IOPS][eta 07m:33s]
  155. Jobs: 32 (f=32): [r(32)][24.7%][r=3897MiB/s][r=998k IOPS][eta 07m:32s]
  156. Jobs: 32 (f=32): [r(32)][24.8%][r=3907MiB/s][r=1000k IOPS][eta 07m:31s]
  157. Jobs: 32 (f=32): [r(32)][25.0%][r=3906MiB/s][r=1000k IOPS][eta 07m:30s]
  158. Jobs: 32 (f=32): [r(32)][25.2%][r=3919MiB/s][r=1003k IOPS][eta 07m:29s]
  159. Jobs: 32 (f=32): [r(32)][25.3%][r=3923MiB/s][r=1004k IOPS][eta 07m:28s]
  160. Jobs: 32 (f=32): [r(32)][25.5%][r=3914MiB/s][r=1002k IOPS][eta 07m:27s]
  161. Jobs: 32 (f=32): [r(32)][25.7%][r=3890MiB/s][r=996k IOPS][eta 07m:26s]
  162. Jobs: 32 (f=32): [r(32)][25.8%][r=3916MiB/s][r=1003k IOPS][eta 07m:25s]
  163. Jobs: 32 (f=32): [r(32)][26.0%][r=3915MiB/s][r=1002k IOPS][eta 07m:24s]
  164. Jobs: 32 (f=32): [r(32)][26.2%][r=3906MiB/s][r=1000k IOPS][eta 07m:23s]
  165. Jobs: 32 (f=32): [r(32)][26.3%][r=3908MiB/s][r=1000k IOPS][eta 07m:22s]
  166. Jobs: 32 (f=32): [r(32)][26.5%][r=3924MiB/s][r=1004k IOPS][eta 07m:21s]
  167. Jobs: 32 (f=32): [r(32)][26.7%][r=3905MiB/s][r=1000k IOPS][eta 07m:20s]
  168. Jobs: 32 (f=32): [r(32)][26.8%][r=3914MiB/s][r=1002k IOPS][eta 07m:19s]
  169. Jobs: 32 (f=32): [r(32)][27.0%][r=3904MiB/s][r=999k IOPS][eta 07m:18s]
  170. Jobs: 32 (f=32): [r(32)][27.2%][r=3912MiB/s][r=1002k IOPS][eta 07m:17s]
  171. Jobs: 32 (f=32): [r(32)][27.3%][r=3889MiB/s][r=995k IOPS][eta 07m:16s]
  172. Jobs: 32 (f=32): [r(32)][27.5%][r=3905MiB/s][r=1000k IOPS][eta 07m:15s]
  173. Jobs: 32 (f=32): [r(32)][27.7%][r=3911MiB/s][r=1001k IOPS][eta 07m:14s]
  174. Jobs: 32 (f=32): [r(32)][27.8%][r=3914MiB/s][r=1002k IOPS][eta 07m:13s]
  175. Jobs: 32 (f=32): [r(32)][28.0%][r=3897MiB/s][r=998k IOPS][eta 07m:12s]
  176. Jobs: 32 (f=32): [r(32)][28.2%][r=3900MiB/s][r=998k IOPS][eta 07m:11s]
  177. Jobs: 32 (f=32): [r(32)][28.3%][r=3901MiB/s][r=999k IOPS][eta 07m:10s]
  178. Jobs: 32 (f=32): [r(32)][28.5%][r=3898MiB/s][r=998k IOPS][eta 07m:09s]
  179. Jobs: 32 (f=32): [r(32)][28.7%][r=3902MiB/s][r=999k IOPS][eta 07m:08s]
  180. Jobs: 32 (f=32): [r(32)][28.8%][r=3900MiB/s][r=998k IOPS][eta 07m:07s]
  181. Jobs: 32 (f=32): [r(32)][29.0%][r=3902MiB/s][r=999k IOPS][eta 07m:06s]
  182. Jobs: 32 (f=32): [r(32)][29.2%][r=3898MiB/s][r=998k IOPS][eta 07m:05s]
  183. Jobs: 32 (f=32): [r(32)][29.3%][r=3904MiB/s][r=999k IOPS][eta 07m:04s]
  184. Jobs: 32 (f=32): [r(32)][29.5%][r=3909MiB/s][r=1001k IOPS][eta 07m:03s]
  185. Jobs: 32 (f=32): [r(32)][29.7%][r=3892MiB/s][r=996k IOPS][eta 07m:02s]
  186. Jobs: 32 (f=32): [r(32)][29.8%][r=3897MiB/s][r=998k IOPS][eta 07m:01s]
  187. Jobs: 32 (f=32): [r(32)][30.0%][r=3888MiB/s][r=995k IOPS][eta 07m:00s]
  188. Jobs: 32 (f=32): [r(32)][30.2%][r=3918MiB/s][r=1003k IOPS][eta 06m:59s]
  189. Jobs: 32 (f=32): [r(32)][30.3%][r=3912MiB/s][r=1002k IOPS][eta 06m:58s]
  190. Jobs: 32 (f=32): [r(32)][30.5%][r=3905MiB/s][r=1000k IOPS][eta 06m:57s]
  191. Jobs: 32 (f=32): [r(32)][30.7%][r=3904MiB/s][r=1000k IOPS][eta 06m:56s]
  192. Jobs: 32 (f=32): [r(32)][30.8%][r=3901MiB/s][r=999k IOPS][eta 06m:55s]
  193. Jobs: 32 (f=32): [r(32)][31.0%][r=3903MiB/s][r=999k IOPS][eta 06m:54s]
  194. Jobs: 32 (f=32): [r(32)][31.2%][r=3905MiB/s][r=1000k IOPS][eta 06m:53s]
  195. Jobs: 32 (f=32): [r(32)][31.3%][r=3898MiB/s][r=998k IOPS][eta 06m:52s]
  196. Jobs: 32 (f=32): [r(32)][31.5%][r=3907MiB/s][r=1000k IOPS][eta 06m:51s]
  197. Jobs: 32 (f=32): [r(32)][31.7%][r=3904MiB/s][r=1000k IOPS][eta 06m:50s]
  198. Jobs: 32 (f=32): [r(32)][31.8%][r=3896MiB/s][r=997k IOPS][eta 06m:49s]
  199. Jobs: 32 (f=32): [r(32)][32.0%][r=3904MiB/s][r=999k IOPS][eta 06m:48s]
  200. Jobs: 32 (f=32): [r(32)][32.2%][r=3904MiB/s][r=999k IOPS][eta 06m:47s]
  201. Jobs: 32 (f=32): [r(32)][32.3%][r=3899MiB/s][r=998k IOPS][eta 06m:46s]
  202. Jobs: 32 (f=32): [r(32)][32.5%][r=3894MiB/s][r=997k IOPS][eta 06m:45s]
  203. Jobs: 32 (f=32): [r(32)][32.7%][r=3895MiB/s][r=997k IOPS][eta 06m:44s]
  204. Jobs: 32 (f=32): [r(32)][32.8%][r=3902MiB/s][r=999k IOPS][eta 06m:43s]
  205. Jobs: 32 (f=32): [r(32)][33.0%][r=3903MiB/s][r=999k IOPS][eta 06m:42s]
  206. Jobs: 32 (f=32): [r(32)][33.2%][r=3913MiB/s][r=1002k IOPS][eta 06m:41s]
  207. Jobs: 32 (f=32): [r(32)][33.3%][r=3911MiB/s][r=1001k IOPS][eta 06m:40s]
  208. Jobs: 32 (f=32): [r(32)][33.5%][r=3900MiB/s][r=998k IOPS][eta 06m:39s]
  209. Jobs: 32 (f=32): [r(32)][33.7%][r=3904MiB/s][r=999k IOPS][eta 06m:38s]
  210. Jobs: 32 (f=32): [r(32)][33.8%][r=3896MiB/s][r=997k IOPS][eta 06m:37s]
  211. Jobs: 32 (f=32): [r(32)][34.0%][r=3899MiB/s][r=998k IOPS][eta 06m:36s]
  212. Jobs: 32 (f=32): [r(32)][34.2%][r=3886MiB/s][r=995k IOPS][eta 06m:35s]
  213. Jobs: 32 (f=32): [r(32)][34.3%][r=3903MiB/s][r=999k IOPS][eta 06m:34s]
  214. Jobs: 32 (f=32): [r(32)][34.5%][r=3894MiB/s][r=997k IOPS][eta 06m:33s]
  215. Jobs: 32 (f=32): [r(32)][34.7%][r=3909MiB/s][r=1001k IOPS][eta 06m:32s]
  216. Jobs: 32 (f=32): [r(32)][34.8%][r=3898MiB/s][r=998k IOPS][eta 06m:31s]
  217. Jobs: 32 (f=32): [r(32)][35.0%][r=3897MiB/s][r=998k IOPS][eta 06m:30s]
  218. Jobs: 32 (f=32): [r(32)][35.2%][r=3890MiB/s][r=996k IOPS][eta 06m:29s]
  219. Jobs: 32 (f=32): [r(32)][35.3%][r=3900MiB/s][r=998k IOPS][eta 06m:28s]
  220. Jobs: 32 (f=32): [r(32)][35.5%][r=3891MiB/s][r=996k IOPS][eta 06m:27s]
  221. Jobs: 32 (f=32): [r(32)][35.7%][r=3886MiB/s][r=995k IOPS][eta 06m:26s]
  222. Jobs: 32 (f=32): [r(32)][35.8%][r=3895MiB/s][r=997k IOPS][eta 06m:25s]
  223. Jobs: 32 (f=32): [r(32)][36.0%][r=3893MiB/s][r=997k IOPS][eta 06m:24s]
  224. Jobs: 32 (f=32): [r(32)][36.2%][r=3905MiB/s][r=1000k IOPS][eta 06m:23s]
  225. Jobs: 32 (f=32): [r(32)][36.3%][r=3915MiB/s][r=1002k IOPS][eta 06m:22s]
  226. Jobs: 32 (f=32): [r(32)][36.5%][r=3905MiB/s][r=1000k IOPS][eta 06m:21s]
  227. Jobs: 32 (f=32): [r(32)][36.7%][r=3899MiB/s][r=998k IOPS][eta 06m:20s]
  228. Jobs: 32 (f=32): [r(32)][36.8%][r=3896MiB/s][r=997k IOPS][eta 06m:19s]
  229. Jobs: 32 (f=32): [r(32)][37.0%][r=3896MiB/s][r=997k IOPS][eta 06m:18s]
  230. Jobs: 32 (f=32): [r(32)][37.2%][r=3891MiB/s][r=996k IOPS][eta 06m:17s]
  231. Jobs: 32 (f=32): [r(32)][37.3%][r=3908MiB/s][r=1000k IOPS][eta 06m:16s]
  232. Jobs: 32 (f=32): [r(32)][37.5%][r=3899MiB/s][r=998k IOPS][eta 06m:15s]
  233. Jobs: 32 (f=32): [r(32)][37.7%][r=3902MiB/s][r=999k IOPS][eta 06m:14s]
  234. Jobs: 32 (f=32): [r(32)][37.8%][r=3913MiB/s][r=1002k IOPS][eta 06m:13s]
  235. Jobs: 32 (f=32): [r(32)][38.0%][r=3894MiB/s][r=997k IOPS][eta 06m:12s]
  236. Jobs: 32 (f=32): [r(32)][38.2%][r=3898MiB/s][r=998k IOPS][eta 06m:11s]
  237. Jobs: 32 (f=32): [r(32)][38.3%][r=3918MiB/s][r=1003k IOPS][eta 06m:10s]
  238. Jobs: 32 (f=32): [r(32)][38.5%][r=3906MiB/s][r=1000k IOPS][eta 06m:09s]
  239. Jobs: 32 (f=32): [r(32)][38.7%][r=3907MiB/s][r=1000k IOPS][eta 06m:08s]
  240. Jobs: 32 (f=32): [r(32)][38.8%][r=3903MiB/s][r=999k IOPS][eta 06m:07s]
  241. Jobs: 32 (f=32): [r(32)][39.0%][r=3898MiB/s][r=998k IOPS][eta 06m:06s]
  242. Jobs: 32 (f=32): [r(32)][39.2%][r=3913MiB/s][r=1002k IOPS][eta 06m:05s]
  243. Jobs: 32 (f=32): [r(32)][39.3%][r=3903MiB/s][r=999k IOPS][eta 06m:04s]
  244. Jobs: 32 (f=32): [r(32)][39.5%][r=3898MiB/s][r=998k IOPS][eta 06m:03s]
  245. Jobs: 32 (f=32): [r(32)][39.7%][r=3891MiB/s][r=996k IOPS][eta 06m:02s]
  246. Jobs: 32 (f=32): [r(32)][39.8%][r=3900MiB/s][r=998k IOPS][eta 06m:01s]
  247. Jobs: 32 (f=32): [r(32)][40.0%][r=3896MiB/s][r=997k IOPS][eta 06m:00s]
  248. Jobs: 32 (f=32): [r(32)][40.2%][r=3890MiB/s][r=996k IOPS][eta 05m:59s]
  249. Jobs: 32 (f=32): [r(32)][40.3%][r=3900MiB/s][r=999k IOPS][eta 05m:58s]
  250. Jobs: 32 (f=32): [r(32)][40.5%][r=3895MiB/s][r=997k IOPS][eta 05m:57s]
  251. Jobs: 32 (f=32): [r(32)][40.8%][r=3902MiB/s][r=999k IOPS][eta 05m:56s]
  252. Jobs: 32 (f=32): [r(32)][40.9%][r=3896MiB/s][r=997k IOPS][eta 05m:55s]
  253. Jobs: 32 (f=32): [r(32)][41.1%][r=3917MiB/s][r=1003k IOPS][eta 05m:54s]
  254. Jobs: 32 (f=32): [r(32)][41.3%][r=3902MiB/s][r=999k IOPS][eta 05m:53s]
  255. Jobs: 32 (f=32): [r(32)][41.4%][r=3907MiB/s][r=1000k IOPS][eta 05m:52s]
  256. Jobs: 32 (f=32): [r(32)][41.6%][r=3890MiB/s][r=996k IOPS][eta 05m:51s]
  257. Jobs: 32 (f=32): [r(32)][41.8%][r=3895MiB/s][r=997k IOPS][eta 05m:50s]
  258. Jobs: 32 (f=32): [r(32)][41.9%][r=3890MiB/s][r=996k IOPS][eta 05m:49s]
  259. Jobs: 32 (f=32): [r(32)][42.1%][r=3890MiB/s][r=996k IOPS][eta 05m:48s]
  260. Jobs: 32 (f=32): [r(32)][42.3%][r=3897MiB/s][r=998k IOPS][eta 05m:47s]
  261. Jobs: 32 (f=32): [r(32)][42.4%][r=3892MiB/s][r=996k IOPS][eta 05m:46s]
  262. Jobs: 32 (f=32): [r(32)][42.6%][r=3902MiB/s][r=999k IOPS][eta 05m:45s]
  263. Jobs: 32 (f=32): [r(32)][42.8%][r=3894MiB/s][r=997k IOPS][eta 05m:44s]
  264. Jobs: 32 (f=32): [r(32)][42.9%][r=3889MiB/s][r=996k IOPS][eta 05m:43s]
  265. Jobs: 32 (f=32): [r(32)][43.1%][r=3890MiB/s][r=996k IOPS][eta 05m:42s]
  266. Jobs: 32 (f=32): [r(32)][43.3%][r=3898MiB/s][r=998k IOPS][eta 05m:41s]
  267. Jobs: 32 (f=32): [r(32)][43.4%][r=3895MiB/s][r=997k IOPS][eta 05m:40s]
  268. Jobs: 32 (f=32): [r(32)][43.6%][r=3897MiB/s][r=998k IOPS][eta 05m:39s]
  269. Jobs: 32 (f=32): [r(32)][43.8%][r=3897MiB/s][r=998k IOPS][eta 05m:38s]
  270. Jobs: 32 (f=32): [r(32)][43.9%][r=3910MiB/s][r=1001k IOPS][eta 05m:37s]
  271. Jobs: 32 (f=32): [r(32)][44.1%][r=3883MiB/s][r=994k IOPS][eta 05m:36s]
  272. Jobs: 32 (f=32): [r(32)][44.3%][r=3893MiB/s][r=997k IOPS][eta 05m:35s]
  273. Jobs: 32 (f=32): [r(32)][44.4%][r=3899MiB/s][r=998k IOPS][eta 05m:34s]
  274. Jobs: 32 (f=32): [r(32)][44.6%][r=3895MiB/s][r=997k IOPS][eta 05m:33s]
  275. Jobs: 32 (f=32): [r(32)][44.8%][r=3909MiB/s][r=1001k IOPS][eta 05m:32s]
  276. Jobs: 32 (f=32): [r(32)][44.9%][r=3898MiB/s][r=998k IOPS][eta 05m:31s]
  277. Jobs: 32 (f=32): [r(32)][45.1%][r=3899MiB/s][r=998k IOPS][eta 05m:30s]
  278. Jobs: 32 (f=32): [r(32)][45.3%][r=3910MiB/s][r=1001k IOPS][eta 05m:29s]
  279. Jobs: 32 (f=32): [r(32)][45.4%][r=3908MiB/s][r=1000k IOPS][eta 05m:28s]
  280. Jobs: 32 (f=32): [r(32)][45.6%][r=3896MiB/s][r=997k IOPS][eta 05m:27s]
  281. Jobs: 32 (f=32): [r(32)][45.8%][r=3911MiB/s][r=1001k IOPS][eta 05m:26s]
  282. Jobs: 32 (f=32): [r(32)][45.9%][r=3901MiB/s][r=999k IOPS][eta 05m:25s]
  283. Jobs: 32 (f=32): [r(32)][46.1%][r=3901MiB/s][r=999k IOPS][eta 05m:24s]
  284. Jobs: 32 (f=32): [r(32)][46.3%][r=3885MiB/s][r=995k IOPS][eta 05m:23s]
  285. ^Cbs: 32 (f=32): [r(32)][46.4%][r=3893MiB/s][r=997k IOPS][eta 05m:22s]
  286. fio: terminating on signal 2
  287.  
  288. t: (groupid=0, jobs=32): err= 0: pid=230702: Thu Apr 15 13:35:34 2021
  289. read: IOPS=1027k, BW=4010MiB/s (4205MB/s)(1089GiB/278011msec)
  290. slat (nsec): min=1759, max=4859.6k, avg=7579.72, stdev=13430.87
  291. clat (usec): min=30, max=34765, avg=7970.44, stdev=4812.87
  292. lat (usec): min=35, max=34773, avg=7978.16, stdev=4812.46
  293. clat percentiles (usec):
  294. | 1.00th=[ 502], 5.00th=[ 1254], 10.00th=[ 2114], 20.00th=[ 3818],
  295. | 30.00th=[ 5342], 40.00th=[ 6194], 50.00th=[ 7046], 60.00th=[ 8455],
  296. | 70.00th=[10159], 80.00th=[11731], 90.00th=[13698], 95.00th=[17957],
  297. | 99.00th=[22152], 99.50th=[22938], 99.90th=[23987], 99.95th=[24249],
  298. | 99.99th=[25035]
  299. bw ( MiB/s): min= 3806, max= 7812, per=100.00%, avg=4012.30, stdev=12.86, samples=17696
  300. iops : min=974539, max=2000104, avg=1027136.13, stdev=3292.78, samples=17696
  301. lat (usec) : 50=0.01%, 100=0.01%, 250=0.13%, 500=0.85%, 750=1.21%
  302. lat (usec) : 1000=1.34%
  303. lat (msec) : 2=5.72%, 4=11.90%, 10=48.23%, 20=27.75%, 50=2.86%
  304. cpu : usr=6.45%, sys=17.22%, ctx=103189823, majf=0, minf=81904
  305. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
  306. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  307. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
  308. issued rwts: total=285399759,0,0,0 short=0,0,0,0 dropped=0,0,0,0
  309. latency : target=0, window=0, percentile=100.00%, depth=256
  310.  
  311. Run status group 0 (all jobs):
  312. READ: bw=4010MiB/s (4205MB/s), 4010MiB/s-4010MiB/s (4205MB/s-4205MB/s), io=1089GiB (1169GB), run=278011-278011msec
  313.  
  314. Disk stats (read/write):
  315. md0: ios=285386838/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=142699879/0, aggrmerge=0/0, aggrticks=9223372035776288860/0, aggrin_queue=1001416471, aggrutil=100.00%
  316. nvme0n1: ios=142697930/0, merge=0/0, ticks=18446744070483650647/0, in_queue=1001283939, util=100.00%
  317. nvme1n1: ios=142701829/0, merge=0/0, ticks=1068927073/0, in_queue=1001549003, util=100.00%
  318.  
  319. -------------------------------------------------------------------------------------------
  320.  
  321. [root@sus-node-2 ~]# fio --filename=/dev/md0 --direct=1 --rw=randwrite --bs=4k --ioengine=libaio --iodepth=256 --runtime=600 --numjobs=32 --time_based --group_reporting --name=t --eta-newline=1 --output-format=normal
  322. t: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=256
  323. ...
  324. fio-3.19
  325. Starting 32 processes
  326. Jobs: 32 (f=32): [w(32)][0.5%][w=2260MiB/s][w=579k IOPS][eta 09m:58s]
  327. Jobs: 32 (f=32): [w(32)][0.7%][w=2277MiB/s][w=583k IOPS][eta 09m:57s]
  328. Jobs: 32 (f=32): [w(32)][0.8%][w=2278MiB/s][w=583k IOPS][eta 09m:56s]
  329. Jobs: 32 (f=32): [w(32)][1.0%][w=2274MiB/s][w=582k IOPS][eta 09m:55s]
  330. Jobs: 32 (f=32): [w(32)][1.2%][w=2274MiB/s][w=582k IOPS][eta 09m:54s]
  331. Jobs: 32 (f=32): [w(32)][1.3%][w=2287MiB/s][w=585k IOPS][eta 09m:53s]
  332. Jobs: 32 (f=32): [w(32)][1.5%][w=2263MiB/s][w=579k IOPS][eta 09m:52s]
  333. Jobs: 32 (f=32): [w(32)][1.7%][w=2259MiB/s][w=578k IOPS][eta 09m:51s]
  334. Jobs: 32 (f=32): [w(32)][1.8%][w=2272MiB/s][w=582k IOPS][eta 09m:50s]
  335. Jobs: 32 (f=32): [w(32)][2.0%][w=2275MiB/s][w=582k IOPS][eta 09m:49s]
  336. Jobs: 32 (f=32): [w(32)][2.2%][w=2273MiB/s][w=582k IOPS][eta 09m:48s]
  337. Jobs: 32 (f=32): [w(32)][2.3%][w=2247MiB/s][w=575k IOPS][eta 09m:47s]
  338. Jobs: 32 (f=32): [w(32)][2.5%][w=2267MiB/s][w=580k IOPS][eta 09m:46s]
  339. Jobs: 32 (f=32): [w(32)][2.7%][w=2287MiB/s][w=585k IOPS][eta 09m:45s]
  340. Jobs: 32 (f=32): [w(32)][2.8%][w=2281MiB/s][w=584k IOPS][eta 09m:44s]
  341. Jobs: 32 (f=32): [w(32)][3.0%][w=2288MiB/s][w=586k IOPS][eta 09m:43s]
  342. Jobs: 32 (f=32): [w(32)][3.2%][w=2276MiB/s][w=583k IOPS][eta 09m:42s]
  343. Jobs: 32 (f=32): [w(32)][3.3%][w=2274MiB/s][w=582k IOPS][eta 09m:41s]
  344. Jobs: 32 (f=32): [w(32)][3.5%][w=2269MiB/s][w=581k IOPS][eta 09m:40s]
  345. Jobs: 32 (f=32): [w(32)][3.7%][w=2273MiB/s][w=582k IOPS][eta 09m:39s]
  346. Jobs: 32 (f=32): [w(32)][3.8%][w=2274MiB/s][w=582k IOPS][eta 09m:38s]
  347. Jobs: 32 (f=32): [w(32)][4.0%][w=2244MiB/s][w=575k IOPS][eta 09m:37s]
  348. Jobs: 32 (f=32): [w(32)][4.2%][w=2272MiB/s][w=582k IOPS][eta 09m:36s]
  349. Jobs: 32 (f=32): [w(32)][4.3%][w=2263MiB/s][w=579k IOPS][eta 09m:35s]
  350. Jobs: 32 (f=32): [w(32)][4.5%][w=2265MiB/s][w=580k IOPS][eta 09m:34s]
  351. Jobs: 32 (f=32): [w(32)][4.7%][w=2279MiB/s][w=583k IOPS][eta 09m:33s]
  352. Jobs: 32 (f=32): [w(32)][4.8%][w=2269MiB/s][w=581k IOPS][eta 09m:32s]
  353. Jobs: 32 (f=32): [w(32)][5.0%][w=2266MiB/s][w=580k IOPS][eta 09m:31s]
  354. Jobs: 32 (f=32): [w(32)][5.2%][w=2271MiB/s][w=581k IOPS][eta 09m:30s]
  355. Jobs: 32 (f=32): [w(32)][5.3%][w=2284MiB/s][w=585k IOPS][eta 09m:29s]
  356. Jobs: 32 (f=32): [w(32)][5.5%][w=2277MiB/s][w=583k IOPS][eta 09m:28s]
  357. Jobs: 32 (f=32): [w(32)][5.7%][w=2256MiB/s][w=578k IOPS][eta 09m:27s]
  358. Jobs: 32 (f=32): [w(32)][5.8%][w=2253MiB/s][w=577k IOPS][eta 09m:26s]
  359. Jobs: 32 (f=32): [w(32)][6.0%][w=2262MiB/s][w=579k IOPS][eta 09m:25s]
  360. Jobs: 32 (f=32): [w(32)][6.2%][w=2270MiB/s][w=581k IOPS][eta 09m:24s]
  361. Jobs: 32 (f=32): [w(32)][6.3%][w=2258MiB/s][w=578k IOPS][eta 09m:23s]
  362. Jobs: 32 (f=32): [w(32)][6.5%][w=2266MiB/s][w=580k IOPS][eta 09m:22s]
  363. Jobs: 32 (f=32): [w(32)][6.7%][w=2269MiB/s][w=581k IOPS][eta 09m:21s]
  364. Jobs: 32 (f=32): [w(32)][6.8%][w=2268MiB/s][w=581k IOPS][eta 09m:20s]
  365. Jobs: 32 (f=32): [w(32)][7.0%][w=2277MiB/s][w=583k IOPS][eta 09m:19s]
  366. Jobs: 32 (f=32): [w(32)][7.2%][w=2275MiB/s][w=582k IOPS][eta 09m:18s]
  367. Jobs: 32 (f=32): [w(32)][7.3%][w=2281MiB/s][w=584k IOPS][eta 09m:17s]
  368. Jobs: 32 (f=32): [w(32)][7.5%][w=2251MiB/s][w=576k IOPS][eta 09m:16s]
  369. Jobs: 32 (f=32): [w(32)][7.7%][w=2278MiB/s][w=583k IOPS][eta 09m:15s]
  370. Jobs: 32 (f=32): [w(32)][7.8%][w=2271MiB/s][w=581k IOPS][eta 09m:14s]
  371. Jobs: 32 (f=32): [w(32)][8.0%][w=2270MiB/s][w=581k IOPS][eta 09m:13s]
  372. Jobs: 32 (f=32): [w(32)][8.2%][w=2281MiB/s][w=584k IOPS][eta 09m:12s]
  373. Jobs: 32 (f=32): [w(32)][8.3%][w=2274MiB/s][w=582k IOPS][eta 09m:11s]
  374. Jobs: 32 (f=32): [w(32)][8.5%][w=2275MiB/s][w=582k IOPS][eta 09m:10s]
  375. Jobs: 32 (f=32): [w(32)][8.7%][w=2273MiB/s][w=582k IOPS][eta 09m:09s]
  376. Jobs: 32 (f=32): [w(32)][8.8%][w=2267MiB/s][w=580k IOPS][eta 09m:08s]
  377. Jobs: 32 (f=32): [w(32)][9.0%][w=2274MiB/s][w=582k IOPS][eta 09m:07s]
  378. Jobs: 32 (f=32): [w(32)][9.2%][w=2253MiB/s][w=577k IOPS][eta 09m:06s]
  379. Jobs: 32 (f=32): [w(32)][9.3%][w=2279MiB/s][w=584k IOPS][eta 09m:05s]
  380. Jobs: 32 (f=32): [w(32)][9.5%][w=2280MiB/s][w=584k IOPS][eta 09m:04s]
  381. Jobs: 32 (f=32): [w(32)][9.7%][w=2290MiB/s][w=586k IOPS][eta 09m:03s]
  382. Jobs: 32 (f=32): [w(32)][9.8%][w=2276MiB/s][w=583k IOPS][eta 09m:02s]
  383. Jobs: 32 (f=32): [w(32)][10.0%][w=2276MiB/s][w=583k IOPS][eta 09m:01s]
  384. Jobs: 32 (f=32): [w(32)][10.1%][w=2271MiB/s][w=581k IOPS][eta 09m:00s]
  385. Jobs: 32 (f=32): [w(32)][10.3%][w=2277MiB/s][w=583k IOPS][eta 08m:59s]
  386. Jobs: 32 (f=32): [w(32)][10.5%][w=2282MiB/s][w=584k IOPS][eta 08m:58s]
  387. Jobs: 32 (f=32): [w(32)][10.6%][w=2278MiB/s][w=583k IOPS][eta 08m:57s]
  388. Jobs: 32 (f=32): [w(32)][10.8%][w=2252MiB/s][w=577k IOPS][eta 08m:56s]
  389. Jobs: 32 (f=32): [w(32)][11.0%][w=2271MiB/s][w=581k IOPS][eta 08m:55s]
  390. Jobs: 32 (f=32): [w(32)][11.1%][w=2281MiB/s][w=584k IOPS][eta 08m:54s]
  391. Jobs: 32 (f=32): [w(32)][11.3%][w=2283MiB/s][w=585k IOPS][eta 08m:53s]
  392. Jobs: 32 (f=32): [w(32)][11.5%][w=2277MiB/s][w=583k IOPS][eta 08m:52s]
  393. Jobs: 32 (f=32): [w(32)][11.6%][w=2281MiB/s][w=584k IOPS][eta 08m:51s]
  394. Jobs: 32 (f=32): [w(32)][11.8%][w=2284MiB/s][w=585k IOPS][eta 08m:50s]
  395. Jobs: 32 (f=32): [w(32)][12.0%][w=2284MiB/s][w=585k IOPS][eta 08m:49s]
  396. Jobs: 32 (f=32): [w(32)][12.1%][w=2263MiB/s][w=579k IOPS][eta 08m:48s]
  397. Jobs: 32 (f=32): [w(32)][12.3%][w=2262MiB/s][w=579k IOPS][eta 08m:47s]
  398. Jobs: 32 (f=32): [w(32)][12.5%][w=2264MiB/s][w=580k IOPS][eta 08m:46s]
  399. Jobs: 32 (f=32): [w(32)][12.6%][w=2275MiB/s][w=582k IOPS][eta 08m:45s]
  400. Jobs: 32 (f=32): [w(32)][12.8%][w=2271MiB/s][w=581k IOPS][eta 08m:44s]
  401. Jobs: 32 (f=32): [w(32)][13.0%][w=2267MiB/s][w=580k IOPS][eta 08m:43s]
  402. Jobs: 32 (f=32): [w(32)][13.1%][w=2273MiB/s][w=582k IOPS][eta 08m:42s]
  403. Jobs: 32 (f=32): [w(32)][13.3%][w=2275MiB/s][w=582k IOPS][eta 08m:41s]
  404. Jobs: 32 (f=32): [w(32)][13.5%][w=2271MiB/s][w=581k IOPS][eta 08m:40s]
  405. Jobs: 32 (f=32): [w(32)][13.6%][w=2271MiB/s][w=581k IOPS][eta 08m:39s]
  406. Jobs: 32 (f=32): [w(32)][13.8%][w=2293MiB/s][w=587k IOPS][eta 08m:38s]
  407. Jobs: 32 (f=32): [w(32)][14.0%][w=2273MiB/s][w=582k IOPS][eta 08m:37s]
  408. Jobs: 32 (f=32): [w(32)][14.1%][w=2258MiB/s][w=578k IOPS][eta 08m:36s]
  409. Jobs: 32 (f=32): [w(32)][14.3%][w=2265MiB/s][w=580k IOPS][eta 08m:35s]
  410. Jobs: 32 (f=32): [w(32)][14.5%][w=2272MiB/s][w=582k IOPS][eta 08m:34s]
  411. Jobs: 32 (f=32): [w(32)][14.6%][w=2270MiB/s][w=581k IOPS][eta 08m:33s]
  412. Jobs: 32 (f=32): [w(32)][14.8%][w=2264MiB/s][w=580k IOPS][eta 08m:31s]
  413. Jobs: 32 (f=32): [w(32)][15.0%][w=2269MiB/s][w=581k IOPS][eta 08m:30s]
  414. Jobs: 32 (f=32): [w(32)][15.2%][w=2261MiB/s][w=579k IOPS][eta 08m:29s]
  415. Jobs: 32 (f=32): [w(32)][15.3%][w=2278MiB/s][w=583k IOPS][eta 08m:28s]
  416. Jobs: 32 (f=32): [w(32)][15.5%][w=2265MiB/s][w=580k IOPS][eta 08m:27s]
  417. Jobs: 32 (f=32): [w(32)][15.7%][w=2284MiB/s][w=585k IOPS][eta 08m:26s]
  418. Jobs: 32 (f=32): [w(32)][15.8%][w=2240MiB/s][w=573k IOPS][eta 08m:25s]
  419. Jobs: 32 (f=32): [w(32)][16.0%][w=2281MiB/s][w=584k IOPS][eta 08m:24s]
  420. Jobs: 32 (f=32): [w(32)][16.2%][w=2277MiB/s][w=583k IOPS][eta 08m:23s]
  421. Jobs: 32 (f=32): [w(32)][16.3%][w=2283MiB/s][w=585k IOPS][eta 08m:22s]
  422. Jobs: 32 (f=32): [w(32)][16.5%][w=2272MiB/s][w=582k IOPS][eta 08m:21s]
  423. Jobs: 32 (f=32): [w(32)][16.7%][w=2271MiB/s][w=581k IOPS][eta 08m:20s]
  424. Jobs: 32 (f=32): [w(32)][16.8%][w=2257MiB/s][w=578k IOPS][eta 08m:19s]
  425. Jobs: 32 (f=32): [w(32)][17.0%][w=2268MiB/s][w=581k IOPS][eta 08m:18s]
  426. Jobs: 32 (f=32): [w(32)][17.2%][w=2259MiB/s][w=578k IOPS][eta 08m:17s]
  427. Jobs: 32 (f=32): [w(32)][17.3%][w=2267MiB/s][w=580k IOPS][eta 08m:16s]
  428. Jobs: 32 (f=32): [w(32)][17.5%][w=2227MiB/s][w=570k IOPS][eta 08m:15s]
  429. Jobs: 32 (f=32): [w(32)][17.7%][w=2275MiB/s][w=583k IOPS][eta 08m:14s]
  430. Jobs: 32 (f=32): [w(32)][17.8%][w=2282MiB/s][w=584k IOPS][eta 08m:13s]
  431. Jobs: 32 (f=32): [w(32)][18.0%][w=2275MiB/s][w=582k IOPS][eta 08m:12s]
  432. Jobs: 32 (f=32): [w(32)][18.2%][w=2281MiB/s][w=584k IOPS][eta 08m:11s]
  433. Jobs: 32 (f=32): [w(32)][18.3%][w=2291MiB/s][w=587k IOPS][eta 08m:10s]
  434. Jobs: 32 (f=32): [w(32)][18.5%][w=2270MiB/s][w=581k IOPS][eta 08m:09s]
  435. Jobs: 32 (f=32): [w(32)][18.7%][w=2274MiB/s][w=582k IOPS][eta 08m:08s]
  436. Jobs: 32 (f=32): [w(32)][18.8%][w=2275MiB/s][w=582k IOPS][eta 08m:07s]
  437. Jobs: 32 (f=32): [w(32)][19.0%][w=2278MiB/s][w=583k IOPS][eta 08m:06s]
  438. Jobs: 32 (f=32): [w(32)][19.2%][w=2250MiB/s][w=576k IOPS][eta 08m:05s]
  439. Jobs: 32 (f=32): [w(32)][19.3%][w=2274MiB/s][w=582k IOPS][eta 08m:04s]
  440. Jobs: 32 (f=32): [w(32)][19.5%][w=2277MiB/s][w=583k IOPS][eta 08m:03s]
  441. Jobs: 32 (f=32): [w(32)][19.7%][w=2270MiB/s][w=581k IOPS][eta 08m:02s]
  442. Jobs: 32 (f=32): [w(32)][19.8%][w=2283MiB/s][w=584k IOPS][eta 08m:01s]
  443. Jobs: 32 (f=32): [w(32)][20.0%][w=2276MiB/s][w=583k IOPS][eta 08m:00s]
  444. Jobs: 32 (f=32): [w(32)][20.2%][w=2271MiB/s][w=581k IOPS][eta 07m:59s]
  445. Jobs: 32 (f=32): [w(32)][20.3%][w=2282MiB/s][w=584k IOPS][eta 07m:58s]
  446. Jobs: 32 (f=32): [w(32)][20.5%][w=2286MiB/s][w=585k IOPS][eta 07m:57s]
  447. Jobs: 32 (f=32): [w(32)][20.7%][w=2279MiB/s][w=583k IOPS][eta 07m:56s]
  448. Jobs: 32 (f=32): [w(32)][20.8%][w=2253MiB/s][w=577k IOPS][eta 07m:55s]
  449. Jobs: 32 (f=32): [w(32)][21.0%][w=2272MiB/s][w=582k IOPS][eta 07m:54s]
  450. Jobs: 32 (f=32): [w(32)][21.2%][w=2270MiB/s][w=581k IOPS][eta 07m:53s]
  451. Jobs: 32 (f=32): [w(32)][21.3%][w=2283MiB/s][w=584k IOPS][eta 07m:52s]
  452. Jobs: 32 (f=32): [w(32)][21.5%][w=2268MiB/s][w=581k IOPS][eta 07m:51s]
  453. Jobs: 32 (f=32): [w(32)][21.7%][w=2268MiB/s][w=581k IOPS][eta 07m:50s]
  454. Jobs: 32 (f=32): [w(32)][21.8%][w=2271MiB/s][w=581k IOPS][eta 07m:49s]
  455. Jobs: 32 (f=32): [w(32)][22.0%][w=2274MiB/s][w=582k IOPS][eta 07m:48s]
  456. Jobs: 32 (f=32): [w(32)][22.2%][w=2284MiB/s][w=585k IOPS][eta 07m:47s]
  457. Jobs: 32 (f=32): [w(32)][22.3%][w=2281MiB/s][w=584k IOPS][eta 07m:46s]
  458. Jobs: 32 (f=32): [w(32)][22.5%][w=2237MiB/s][w=573k IOPS][eta 07m:45s]
  459. Jobs: 32 (f=32): [w(32)][22.7%][w=2297MiB/s][w=588k IOPS][eta 07m:44s]
  460. Jobs: 32 (f=32): [w(32)][22.8%][w=2273MiB/s][w=582k IOPS][eta 07m:43s]
  461. Jobs: 32 (f=32): [w(32)][23.0%][w=2256MiB/s][w=578k IOPS][eta 07m:42s]
  462. Jobs: 32 (f=32): [w(32)][23.2%][w=2267MiB/s][w=580k IOPS][eta 07m:41s]
  463. Jobs: 32 (f=32): [w(32)][23.3%][w=2271MiB/s][w=581k IOPS][eta 07m:40s]
  464. Jobs: 32 (f=32): [w(32)][23.5%][w=2270MiB/s][w=581k IOPS][eta 07m:39s]
  465. Jobs: 32 (f=32): [w(32)][23.7%][w=2269MiB/s][w=581k IOPS][eta 07m:38s]
  466. Jobs: 32 (f=32): [w(32)][23.8%][w=2270MiB/s][w=581k IOPS][eta 07m:37s]
  467. Jobs: 32 (f=32): [w(32)][24.0%][w=2240MiB/s][w=573k IOPS][eta 07m:36s]
  468. Jobs: 32 (f=32): [w(32)][24.2%][w=2242MiB/s][w=574k IOPS][eta 07m:35s]
  469. Jobs: 32 (f=32): [w(32)][24.3%][w=2281MiB/s][w=584k IOPS][eta 07m:34s]
  470. Jobs: 32 (f=32): [w(32)][24.5%][w=2274MiB/s][w=582k IOPS][eta 07m:33s]
  471. Jobs: 32 (f=32): [w(32)][24.7%][w=2285MiB/s][w=585k IOPS][eta 07m:32s]
  472. Jobs: 32 (f=32): [w(32)][24.8%][w=2269MiB/s][w=581k IOPS][eta 07m:31s]
  473. Jobs: 32 (f=32): [w(32)][25.0%][w=2271MiB/s][w=581k IOPS][eta 07m:30s]
  474. Jobs: 32 (f=32): [w(32)][25.2%][w=2270MiB/s][w=581k IOPS][eta 07m:29s]
  475. Jobs: 32 (f=32): [w(32)][25.3%][w=2271MiB/s][w=581k IOPS][eta 07m:28s]
  476. Jobs: 32 (f=32): [w(32)][25.5%][w=2260MiB/s][w=579k IOPS][eta 07m:27s]
  477. Jobs: 32 (f=32): [w(32)][25.7%][w=2266MiB/s][w=580k IOPS][eta 07m:26s]
  478. Jobs: 32 (f=32): [w(32)][25.8%][w=2270MiB/s][w=581k IOPS][eta 07m:25s]
  479. Jobs: 32 (f=32): [w(32)][26.0%][w=2278MiB/s][w=583k IOPS][eta 07m:24s]
  480. Jobs: 32 (f=32): [w(32)][26.2%][w=2272MiB/s][w=582k IOPS][eta 07m:23s]
  481. Jobs: 32 (f=32): [w(32)][26.3%][w=2285MiB/s][w=585k IOPS][eta 07m:22s]
  482. Jobs: 32 (f=32): [w(32)][26.5%][w=2285MiB/s][w=585k IOPS][eta 07m:21s]
  483. Jobs: 32 (f=32): [w(32)][26.7%][w=2279MiB/s][w=583k IOPS][eta 07m:20s]
  484. Jobs: 32 (f=32): [w(32)][26.8%][w=2290MiB/s][w=586k IOPS][eta 07m:19s]
  485. Jobs: 32 (f=32): [w(32)][27.0%][w=2282MiB/s][w=584k IOPS][eta 07m:18s]
  486. Jobs: 32 (f=32): [w(32)][27.2%][w=2277MiB/s][w=583k IOPS][eta 07m:17s]
  487. Jobs: 32 (f=32): [w(32)][27.3%][w=2272MiB/s][w=582k IOPS][eta 07m:16s]
  488. Jobs: 32 (f=32): [w(32)][27.5%][w=2246MiB/s][w=575k IOPS][eta 07m:15s]
  489. Jobs: 32 (f=32): [w(32)][27.7%][w=2281MiB/s][w=584k IOPS][eta 07m:14s]
  490. Jobs: 32 (f=32): [w(32)][27.8%][w=2275MiB/s][w=582k IOPS][eta 07m:13s]
  491. Jobs: 32 (f=32): [w(32)][28.0%][w=2279MiB/s][w=583k IOPS][eta 07m:12s]
  492. Jobs: 32 (f=32): [w(32)][28.2%][w=2273MiB/s][w=582k IOPS][eta 07m:11s]
  493. Jobs: 32 (f=32): [w(32)][28.3%][w=2260MiB/s][w=578k IOPS][eta 07m:10s]
  494. Jobs: 32 (f=32): [w(32)][28.5%][w=2284MiB/s][w=585k IOPS][eta 07m:09s]
  495. Jobs: 32 (f=32): [w(32)][28.7%][w=2280MiB/s][w=584k IOPS][eta 07m:08s]
  496. Jobs: 32 (f=32): [w(32)][28.8%][w=2287MiB/s][w=586k IOPS][eta 07m:07s]
  497. Jobs: 32 (f=32): [w(32)][29.0%][w=2285MiB/s][w=585k IOPS][eta 07m:06s]
  498. Jobs: 32 (f=32): [w(32)][29.2%][w=2260MiB/s][w=579k IOPS][eta 07m:05s]
  499. Jobs: 32 (f=32): [w(32)][29.3%][w=2281MiB/s][w=584k IOPS][eta 07m:04s]
  500. Jobs: 32 (f=32): [w(32)][29.5%][w=2275MiB/s][w=582k IOPS][eta 07m:03s]
  501. Jobs: 32 (f=32): [w(32)][29.7%][w=2279MiB/s][w=583k IOPS][eta 07m:02s]
  502. Jobs: 32 (f=32): [w(32)][29.8%][w=2269MiB/s][w=581k IOPS][eta 07m:01s]
  503. Jobs: 32 (f=32): [w(32)][30.0%][w=2270MiB/s][w=581k IOPS][eta 07m:00s]
  504. Jobs: 32 (f=32): [w(32)][30.2%][w=2272MiB/s][w=582k IOPS][eta 06m:59s]
  505. Jobs: 32 (f=32): [w(32)][30.3%][w=2272MiB/s][w=582k IOPS][eta 06m:58s]
  506. Jobs: 32 (f=32): [w(32)][30.5%][w=2250MiB/s][w=576k IOPS][eta 06m:57s]
  507. Jobs: 32 (f=32): [w(32)][30.7%][w=2238MiB/s][w=573k IOPS][eta 06m:56s]
  508. Jobs: 32 (f=32): [w(32)][30.8%][w=2239MiB/s][w=573k IOPS][eta 06m:55s]
  509. Jobs: 32 (f=32): [w(32)][31.0%][w=2284MiB/s][w=585k IOPS][eta 06m:54s]
  510. Jobs: 32 (f=32): [w(32)][31.2%][w=2271MiB/s][w=581k IOPS][eta 06m:53s]
  511. Jobs: 32 (f=32): [w(32)][31.3%][w=2277MiB/s][w=583k IOPS][eta 06m:52s]
  512. Jobs: 32 (f=32): [w(32)][31.5%][w=2273MiB/s][w=582k IOPS][eta 06m:51s]
  513. Jobs: 32 (f=32): [w(32)][31.7%][w=2262MiB/s][w=579k IOPS][eta 06m:50s]
  514. Jobs: 32 (f=32): [w(32)][31.8%][w=2227MiB/s][w=570k IOPS][eta 06m:49s]
  515. Jobs: 32 (f=32): [w(32)][32.0%][w=2262MiB/s][w=579k IOPS][eta 06m:48s]
  516. Jobs: 32 (f=32): [w(32)][32.2%][w=2277MiB/s][w=583k IOPS][eta 06m:47s]
  517. Jobs: 32 (f=32): [w(32)][32.3%][w=2251MiB/s][w=576k IOPS][eta 06m:46s]
  518. Jobs: 32 (f=32): [w(32)][32.5%][w=2264MiB/s][w=580k IOPS][eta 06m:45s]
  519. Jobs: 32 (f=32): [w(32)][32.7%][w=2243MiB/s][w=574k IOPS][eta 06m:44s]
  520. Jobs: 32 (f=32): [w(32)][32.8%][w=2268MiB/s][w=581k IOPS][eta 06m:43s]
  521. Jobs: 32 (f=32): [w(32)][33.0%][w=2278MiB/s][w=583k IOPS][eta 06m:42s]
  522. Jobs: 32 (f=32): [w(32)][33.2%][w=2279MiB/s][w=583k IOPS][eta 06m:41s]
  523. Jobs: 32 (f=32): [w(32)][33.3%][w=2281MiB/s][w=584k IOPS][eta 06m:40s]
  524. Jobs: 32 (f=32): [w(32)][33.5%][w=2286MiB/s][w=585k IOPS][eta 06m:39s]
  525. Jobs: 32 (f=32): [w(32)][33.7%][w=2284MiB/s][w=585k IOPS][eta 06m:38s]
  526. Jobs: 32 (f=32): [w(32)][33.8%][w=2269MiB/s][w=581k IOPS][eta 06m:37s]
  527. Jobs: 32 (f=32): [w(32)][34.0%][w=2274MiB/s][w=582k IOPS][eta 06m:36s]
  528. Jobs: 32 (f=32): [w(32)][34.2%][w=2267MiB/s][w=580k IOPS][eta 06m:35s]
  529. Jobs: 32 (f=32): [w(32)][34.3%][w=2255MiB/s][w=577k IOPS][eta 06m:34s]
  530. Jobs: 32 (f=32): [w(32)][34.5%][w=2277MiB/s][w=583k IOPS][eta 06m:33s]
  531. Jobs: 32 (f=32): [w(32)][34.7%][w=2246MiB/s][w=575k IOPS][eta 06m:32s]
  532. Jobs: 32 (f=32): [w(32)][34.8%][w=2247MiB/s][w=575k IOPS][eta 06m:31s]
  533. Jobs: 32 (f=32): [w(32)][35.0%][w=2269MiB/s][w=581k IOPS][eta 06m:30s]
  534. Jobs: 32 (f=32): [w(32)][35.2%][w=2277MiB/s][w=583k IOPS][eta 06m:29s]
  535. Jobs: 32 (f=32): [w(32)][35.3%][w=2291MiB/s][w=586k IOPS][eta 06m:28s]
  536. Jobs: 32 (f=32): [w(32)][35.5%][w=2277MiB/s][w=583k IOPS][eta 06m:27s]
  537. Jobs: 32 (f=32): [w(32)][35.7%][w=2279MiB/s][w=584k IOPS][eta 06m:26s]
  538. Jobs: 32 (f=32): [w(32)][35.8%][w=2280MiB/s][w=584k IOPS][eta 06m:25s]
  539. Jobs: 32 (f=32): [w(32)][36.0%][w=2255MiB/s][w=577k IOPS][eta 06m:24s]
  540. Jobs: 32 (f=32): [w(32)][36.2%][w=2269MiB/s][w=581k IOPS][eta 06m:23s]
  541. Jobs: 32 (f=32): [w(32)][36.3%][w=2267MiB/s][w=580k IOPS][eta 06m:22s]
  542. Jobs: 32 (f=32): [w(32)][36.5%][w=2268MiB/s][w=581k IOPS][eta 06m:21s]
  543. Jobs: 32 (f=32): [w(32)][36.7%][w=2257MiB/s][w=578k IOPS][eta 06m:20s]
  544. Jobs: 32 (f=32): [w(32)][36.8%][w=2277MiB/s][w=583k IOPS][eta 06m:19s]
  545. Jobs: 32 (f=32): [w(32)][37.0%][w=2274MiB/s][w=582k IOPS][eta 06m:18s]
  546. Jobs: 32 (f=32): [w(32)][37.2%][w=2270MiB/s][w=581k IOPS][eta 06m:17s]
  547. Jobs: 32 (f=32): [w(32)][37.3%][w=2281MiB/s][w=584k IOPS][eta 06m:16s]
  548. Jobs: 32 (f=32): [w(32)][37.5%][w=2276MiB/s][w=583k IOPS][eta 06m:15s]
  549. Jobs: 32 (f=32): [w(32)][37.7%][w=2263MiB/s][w=579k IOPS][eta 06m:14s]
  550. Jobs: 32 (f=32): [w(32)][37.8%][w=2278MiB/s][w=583k IOPS][eta 06m:13s]
  551. Jobs: 32 (f=32): [w(32)][38.0%][w=2273MiB/s][w=582k IOPS][eta 06m:12s]
  552. Jobs: 32 (f=32): [w(32)][38.2%][w=2267MiB/s][w=580k IOPS][eta 06m:11s]
  553. Jobs: 32 (f=32): [w(32)][38.3%][w=2275MiB/s][w=582k IOPS][eta 06m:10s]
  554. Jobs: 32 (f=32): [w(32)][38.5%][w=2274MiB/s][w=582k IOPS][eta 06m:09s]
  555. Jobs: 32 (f=32): [w(32)][38.7%][w=2266MiB/s][w=580k IOPS][eta 06m:08s]
  556. Jobs: 32 (f=32): [w(32)][38.8%][w=2261MiB/s][w=579k IOPS][eta 06m:07s]
  557. Jobs: 32 (f=32): [w(32)][39.0%][w=2285MiB/s][w=585k IOPS][eta 06m:06s]
  558. Jobs: 32 (f=32): [w(32)][39.2%][w=2260MiB/s][w=579k IOPS][eta 06m:05s]
  559. Jobs: 32 (f=32): [w(32)][39.3%][w=2253MiB/s][w=577k IOPS][eta 06m:04s]
  560. Jobs: 32 (f=32): [w(32)][39.5%][w=2264MiB/s][w=579k IOPS][eta 06m:03s]
  561. Jobs: 32 (f=32): [w(32)][39.7%][w=2285MiB/s][w=585k IOPS][eta 06m:02s]
  562. Jobs: 32 (f=32): [w(32)][39.8%][w=2286MiB/s][w=585k IOPS][eta 06m:01s]
  563. Jobs: 32 (f=32): [w(32)][40.0%][w=2271MiB/s][w=581k IOPS][eta 06m:00s]
  564. Jobs: 32 (f=32): [w(32)][40.2%][w=2275MiB/s][w=582k IOPS][eta 05m:59s]
  565. Jobs: 32 (f=32): [w(32)][40.3%][w=2272MiB/s][w=582k IOPS][eta 05m:58s]
  566. Jobs: 32 (f=32): [w(32)][40.5%][w=2284MiB/s][w=585k IOPS][eta 05m:57s]
  567. Jobs: 32 (f=32): [w(32)][40.7%][w=2272MiB/s][w=582k IOPS][eta 05m:56s]
  568. Jobs: 32 (f=32): [w(32)][40.8%][w=2252MiB/s][w=577k IOPS][eta 05m:55s]
  569. Jobs: 32 (f=32): [w(32)][41.0%][w=2239MiB/s][w=573k IOPS][eta 05m:54s]
  570. Jobs: 32 (f=32): [w(32)][41.2%][w=2275MiB/s][w=582k IOPS][eta 05m:53s]
  571. Jobs: 32 (f=32): [w(32)][41.3%][w=2269MiB/s][w=581k IOPS][eta 05m:52s]
  572. Jobs: 32 (f=32): [w(32)][41.5%][w=2277MiB/s][w=583k IOPS][eta 05m:51s]
  573. Jobs: 32 (f=32): [w(32)][41.7%][w=2269MiB/s][w=581k IOPS][eta 05m:50s]
  574. Jobs: 32 (f=32): [w(32)][41.8%][w=2277MiB/s][w=583k IOPS][eta 05m:49s]
  575. Jobs: 32 (f=32): [w(32)][42.0%][w=2288MiB/s][w=586k IOPS][eta 05m:48s]
  576. Jobs: 32 (f=32): [w(32)][42.2%][w=2285MiB/s][w=585k IOPS][eta 05m:47s]
  577. Jobs: 32 (f=32): [w(32)][42.3%][w=2281MiB/s][w=584k IOPS][eta 05m:46s]
  578. Jobs: 32 (f=32): [w(32)][42.5%][w=2268MiB/s][w=581k IOPS][eta 05m:45s]
  579. Jobs: 32 (f=32): [w(32)][42.7%][w=2256MiB/s][w=577k IOPS][eta 05m:44s]
  580. Jobs: 32 (f=32): [w(32)][42.8%][w=2266MiB/s][w=580k IOPS][eta 05m:43s]
  581. Jobs: 32 (f=32): [w(32)][43.0%][w=2277MiB/s][w=583k IOPS][eta 05m:42s]
  582. Jobs: 32 (f=32): [w(32)][43.2%][w=2272MiB/s][w=582k IOPS][eta 05m:41s]
  583. Jobs: 32 (f=32): [w(32)][43.3%][w=2263MiB/s][w=579k IOPS][eta 05m:40s]
  584. Jobs: 32 (f=32): [w(32)][43.5%][w=2270MiB/s][w=581k IOPS][eta 05m:39s]
  585. Jobs: 32 (f=32): [w(32)][43.7%][w=2273MiB/s][w=582k IOPS][eta 05m:38s]
  586. Jobs: 32 (f=32): [w(32)][43.8%][w=2272MiB/s][w=582k IOPS][eta 05m:37s]
  587. Jobs: 32 (f=32): [w(32)][44.0%][w=2284MiB/s][w=585k IOPS][eta 05m:36s]
  588. Jobs: 32 (f=32): [w(32)][44.2%][w=2265MiB/s][w=580k IOPS][eta 05m:35s]
  589. Jobs: 32 (f=32): [w(32)][44.3%][w=2273MiB/s][w=582k IOPS][eta 05m:34s]
  590. Jobs: 32 (f=32): [w(32)][44.5%][w=2266MiB/s][w=580k IOPS][eta 05m:33s]
  591. Jobs: 32 (f=32): [w(32)][44.7%][w=2283MiB/s][w=584k IOPS][eta 05m:32s]
  592. Jobs: 32 (f=32): [w(32)][44.8%][w=2266MiB/s][w=580k IOPS][eta 05m:31s]
  593. Jobs: 32 (f=32): [w(32)][45.0%][w=2262MiB/s][w=579k IOPS][eta 05m:30s]
  594. Jobs: 32 (f=32): [w(32)][45.2%][w=2270MiB/s][w=581k IOPS][eta 05m:29s]
  595. Jobs: 32 (f=32): [w(32)][45.3%][w=2257MiB/s][w=578k IOPS][eta 05m:28s]
  596. Jobs: 32 (f=32): [w(32)][45.5%][w=2257MiB/s][w=578k IOPS][eta 05m:27s]
  597. Jobs: 32 (f=32): [w(32)][45.7%][w=2272MiB/s][w=582k IOPS][eta 05m:26s]
  598. Jobs: 32 (f=32): [w(32)][45.8%][w=2276MiB/s][w=583k IOPS][eta 05m:25s]
  599. Jobs: 32 (f=32): [w(32)][46.0%][w=2281MiB/s][w=584k IOPS][eta 05m:24s]
  600. Jobs: 32 (f=32): [w(32)][46.2%][w=2247MiB/s][w=575k IOPS][eta 05m:23s]
  601. Jobs: 32 (f=32): [w(32)][46.3%][w=2285MiB/s][w=585k IOPS][eta 05m:22s]
  602. Jobs: 32 (f=32): [w(32)][46.5%][w=2279MiB/s][w=583k IOPS][eta 05m:21s]
  603. Jobs: 32 (f=32): [w(32)][46.7%][w=2287MiB/s][w=586k IOPS][eta 05m:20s]
  604. Jobs: 32 (f=32): [w(32)][46.8%][w=2274MiB/s][w=582k IOPS][eta 05m:19s]
  605. Jobs: 32 (f=32): [w(32)][47.0%][w=2265MiB/s][w=580k IOPS][eta 05m:18s]
  606. Jobs: 32 (f=32): [w(32)][47.2%][w=2274MiB/s][w=582k IOPS][eta 05m:17s]
  607. Jobs: 32 (f=32): [w(32)][47.3%][w=2276MiB/s][w=583k IOPS][eta 05m:16s]
  608. Jobs: 32 (f=32): [w(32)][47.5%][w=2258MiB/s][w=578k IOPS][eta 05m:15s]
  609. Jobs: 32 (f=32): [w(32)][47.7%][w=2266MiB/s][w=580k IOPS][eta 05m:14s]
  610. Jobs: 32 (f=32): [w(32)][47.8%][w=2252MiB/s][w=576k IOPS][eta 05m:13s]
  611. Jobs: 32 (f=32): [w(32)][48.0%][w=2277MiB/s][w=583k IOPS][eta 05m:12s]
  612. Jobs: 32 (f=32): [w(32)][48.2%][w=2272MiB/s][w=582k IOPS][eta 05m:11s]
  613. Jobs: 32 (f=32): [w(32)][48.3%][w=2268MiB/s][w=581k IOPS][eta 05m:10s]
  614. Jobs: 32 (f=32): [w(32)][48.5%][w=2282MiB/s][w=584k IOPS][eta 05m:09s]
  615. Jobs: 32 (f=32): [w(32)][48.7%][w=2282MiB/s][w=584k IOPS][eta 05m:08s]
  616. Jobs: 32 (f=32): [w(32)][48.8%][w=2279MiB/s][w=583k IOPS][eta 05m:07s]
  617. Jobs: 32 (f=32): [w(32)][49.0%][w=2279MiB/s][w=583k IOPS][eta 05m:06s]
  618. Jobs: 32 (f=32): [w(32)][49.2%][w=2262MiB/s][w=579k IOPS][eta 05m:05s]
  619. Jobs: 32 (f=32): [w(32)][49.3%][w=2277MiB/s][w=583k IOPS][eta 05m:04s]
  620. Jobs: 32 (f=32): [w(32)][49.5%][w=2252MiB/s][w=576k IOPS][eta 05m:03s]
  621. Jobs: 32 (f=32): [w(32)][49.7%][w=2280MiB/s][w=584k IOPS][eta 05m:02s]
  622. ^Cbs: 32 (f=32): [w(32)][49.8%][w=2278MiB/s][w=583k IOPS][eta 05m:01s]
  623. fio: terminating on signal 2
  624.  
  625. t: (groupid=0, jobs=32): err= 0: pid=231552: Thu Apr 15 13:40:44 2021
  626. write: IOPS=581k, BW=2269MiB/s (2380MB/s)(662GiB/298662msec); 0 zone resets
  627. slat (usec): min=3, max=6937, avg=53.49, stdev=45.74
  628. clat (usec): min=51, max=31334, avg=14044.52, stdev=1104.47
  629. lat (usec): min=94, max=31380, avg=14098.12, stdev=1107.24
  630. clat percentiles (usec):
  631. | 1.00th=[13173], 5.00th=[13304], 10.00th=[13435], 20.00th=[13566],
  632. | 30.00th=[13566], 40.00th=[13698], 50.00th=[13829], 60.00th=[13960],
  633. | 70.00th=[14091], 80.00th=[14222], 90.00th=[14484], 95.00th=[15008],
  634. | 99.00th=[21365], 99.50th=[22152], 99.90th=[23462], 99.95th=[23987],
  635. | 99.99th=[25035]
  636. bw ( MiB/s): min= 1926, max= 2350, per=100.00%, avg=2271.23, stdev= 0.98, samples=19040
  637. iops : min=493174, max=601802, avg=581425.12, stdev=251.81, samples=19040
  638. lat (usec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
  639. lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=98.60%, 50=1.39%
  640. cpu : usr=2.64%, sys=63.40%, ctx=578206, majf=0, minf=186122
  641. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
  642. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  643. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
  644. issued rwts: total=0,173513510,0,0 short=0,0,0,0 dropped=0,0,0,0
  645. latency : target=0, window=0, percentile=100.00%, depth=256
  646.  
  647. Run status group 0 (all jobs):
  648. WRITE: bw=2269MiB/s (2380MB/s), 2269MiB/s-2269MiB/s (2380MB/s-2380MB/s), io=662GiB (711GB), run=298662-298662msec
  649.  
  650. Disk stats (read/write):
  651. md0: ios=38/173425980, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=47/173513511, aggrmerge=0/0, aggrticks=2/34250069, aggrin_queue=2648672, aggrutil=100.00%
  652. nvme0n1: ios=87/173513511, merge=0/0, ticks=5/30891088, in_queue=1511356, util=100.00%
  653. nvme1n1: ios=7/173513511, merge=0/0, ticks=0/37609051, in_queue=3785988, util=100.00%
  654.  
  655. -------------------------------------------------------------------------------------------
  656.  
  657. [root@sus-node-2 ~]# fio --filename=/dev/md0 --direct=1 --rw=randrw --bs=4k --ioengine=libaio --iodepth=256 --runtime=600 --numjobs=32 --time_based --group_reporting --name=t --eta-newline=1 --output-format=normal
  658. t: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=256
  659. ...
  660. fio-3.19
  661. Starting 32 processes
  662. Jobs: 32 (f=32): [m(32)][0.5%][r=1453MiB/s,w=1450MiB/s][r=372k,w=371k IOPS][eta 09m:58s]
  663. Jobs: 32 (f=32): [m(32)][0.7%][r=1434MiB/s,w=1433MiB/s][r=367k,w=367k IOPS][eta 09m:57s]
  664. Jobs: 32 (f=32): [m(32)][0.8%][r=1442MiB/s,w=1433MiB/s][r=369k,w=367k IOPS][eta 09m:56s]
  665. Jobs: 32 (f=32): [m(32)][1.0%][r=1449MiB/s,w=1454MiB/s][r=371k,w=372k IOPS][eta 09m:55s]
  666. Jobs: 32 (f=32): [m(32)][1.2%][r=1434MiB/s,w=1436MiB/s][r=367k,w=368k IOPS][eta 09m:54s]
  667. Jobs: 32 (f=32): [m(32)][1.3%][r=1429MiB/s,w=1432MiB/s][r=366k,w=367k IOPS][eta 09m:53s]
  668. Jobs: 32 (f=32): [m(32)][1.5%][r=1445MiB/s,w=1443MiB/s][r=370k,w=369k IOPS][eta 09m:52s]
  669. Jobs: 32 (f=32): [m(32)][1.7%][r=1437MiB/s,w=1437MiB/s][r=368k,w=368k IOPS][eta 09m:51s]
  670. Jobs: 32 (f=32): [m(32)][1.8%][r=1435MiB/s,w=1434MiB/s][r=367k,w=367k IOPS][eta 09m:50s]
  671. Jobs: 32 (f=32): [m(32)][2.0%][r=1431MiB/s,w=1437MiB/s][r=366k,w=368k IOPS][eta 09m:49s]
  672. Jobs: 32 (f=32): [m(32)][2.2%][r=1432MiB/s,w=1435MiB/s][r=366k,w=367k IOPS][eta 09m:48s]
  673. Jobs: 32 (f=32): [m(32)][2.3%][r=1445MiB/s,w=1446MiB/s][r=370k,w=370k IOPS][eta 09m:47s]
  674. Jobs: 32 (f=32): [m(32)][2.5%][r=1428MiB/s,w=1428MiB/s][r=366k,w=366k IOPS][eta 09m:46s]
  675. Jobs: 32 (f=32): [m(32)][2.7%][r=1443MiB/s,w=1448MiB/s][r=369k,w=371k IOPS][eta 09m:45s]
  676. Jobs: 32 (f=32): [m(32)][2.8%][r=1438MiB/s,w=1432MiB/s][r=368k,w=367k IOPS][eta 09m:44s]
  677. Jobs: 32 (f=32): [m(32)][3.0%][r=1424MiB/s,w=1427MiB/s][r=364k,w=365k IOPS][eta 09m:43s]
  678. Jobs: 32 (f=32): [m(32)][3.2%][r=1443MiB/s,w=1442MiB/s][r=370k,w=369k IOPS][eta 09m:42s]
  679. Jobs: 32 (f=32): [m(32)][3.3%][r=1435MiB/s,w=1433MiB/s][r=367k,w=367k IOPS][eta 09m:41s]
  680. Jobs: 32 (f=32): [m(32)][3.5%][r=1425MiB/s,w=1428MiB/s][r=365k,w=366k IOPS][eta 09m:40s]
  681. Jobs: 32 (f=32): [m(32)][3.7%][r=1438MiB/s,w=1440MiB/s][r=368k,w=369k IOPS][eta 09m:39s]
  682. Jobs: 32 (f=32): [m(32)][3.8%][r=1431MiB/s,w=1432MiB/s][r=366k,w=367k IOPS][eta 09m:38s]
  683. Jobs: 32 (f=32): [m(32)][4.0%][r=1424MiB/s,w=1425MiB/s][r=364k,w=365k IOPS][eta 09m:37s]
  684. Jobs: 32 (f=32): [m(32)][4.2%][r=1444MiB/s,w=1441MiB/s][r=370k,w=369k IOPS][eta 09m:36s]
  685. Jobs: 32 (f=32): [m(32)][4.3%][r=1446MiB/s,w=1443MiB/s][r=370k,w=369k IOPS][eta 09m:35s]
  686. Jobs: 32 (f=32): [m(32)][4.5%][r=1428MiB/s,w=1434MiB/s][r=366k,w=367k IOPS][eta 09m:34s]
  687. Jobs: 32 (f=32): [m(32)][4.7%][r=1426MiB/s,w=1426MiB/s][r=365k,w=365k IOPS][eta 09m:33s]
  688. Jobs: 32 (f=32): [m(32)][4.8%][r=1451MiB/s,w=1450MiB/s][r=371k,w=371k IOPS][eta 09m:32s]
  689. Jobs: 32 (f=32): [m(32)][5.0%][r=1439MiB/s,w=1436MiB/s][r=368k,w=368k IOPS][eta 09m:31s]
  690. Jobs: 32 (f=32): [m(32)][5.2%][r=1432MiB/s,w=1429MiB/s][r=367k,w=366k IOPS][eta 09m:30s]
  691. Jobs: 32 (f=32): [m(32)][5.3%][r=1443MiB/s,w=1444MiB/s][r=369k,w=370k IOPS][eta 09m:29s]
  692. Jobs: 32 (f=32): [m(32)][5.5%][r=1422MiB/s,w=1428MiB/s][r=364k,w=366k IOPS][eta 09m:28s]
  693. Jobs: 32 (f=32): [m(32)][5.7%][r=1435MiB/s,w=1430MiB/s][r=367k,w=366k IOPS][eta 09m:27s]
  694. Jobs: 32 (f=32): [m(32)][5.8%][r=1443MiB/s,w=1443MiB/s][r=370k,w=369k IOPS][eta 09m:26s]
  695. Jobs: 32 (f=32): [m(32)][6.0%][r=1419MiB/s,w=1422MiB/s][r=363k,w=364k IOPS][eta 09m:25s]
  696. Jobs: 32 (f=32): [m(32)][6.2%][r=1434MiB/s,w=1435MiB/s][r=367k,w=367k IOPS][eta 09m:24s]
  697. Jobs: 32 (f=32): [m(32)][6.3%][r=1433MiB/s,w=1434MiB/s][r=367k,w=367k IOPS][eta 09m:23s]
  698. Jobs: 32 (f=32): [m(32)][6.5%][r=1442MiB/s,w=1438MiB/s][r=369k,w=368k IOPS][eta 09m:22s]
  699. Jobs: 32 (f=32): [m(32)][6.7%][r=1440MiB/s,w=1441MiB/s][r=369k,w=369k IOPS][eta 09m:21s]
  700. Jobs: 32 (f=32): [m(32)][6.8%][r=1432MiB/s,w=1434MiB/s][r=367k,w=367k IOPS][eta 09m:20s]
  701. Jobs: 32 (f=32): [m(32)][7.0%][r=1450MiB/s,w=1445MiB/s][r=371k,w=370k IOPS][eta 09m:19s]
  702. Jobs: 32 (f=32): [m(32)][7.2%][r=1439MiB/s,w=1437MiB/s][r=368k,w=368k IOPS][eta 09m:18s]
  703. Jobs: 32 (f=32): [m(32)][7.3%][r=1431MiB/s,w=1426MiB/s][r=366k,w=365k IOPS][eta 09m:17s]
  704. Jobs: 32 (f=32): [m(32)][7.5%][r=1448MiB/s,w=1445MiB/s][r=371k,w=370k IOPS][eta 09m:16s]
  705. Jobs: 32 (f=32): [m(32)][7.7%][r=1431MiB/s,w=1433MiB/s][r=366k,w=367k IOPS][eta 09m:15s]
  706. Jobs: 32 (f=32): [m(32)][7.8%][r=1418MiB/s,w=1423MiB/s][r=363k,w=364k IOPS][eta 09m:14s]
  707. Jobs: 32 (f=32): [m(32)][8.0%][r=1443MiB/s,w=1445MiB/s][r=369k,w=370k IOPS][eta 09m:13s]
  708. Jobs: 32 (f=32): [m(32)][8.2%][r=1433MiB/s,w=1436MiB/s][r=367k,w=368k IOPS][eta 09m:12s]
  709. Jobs: 32 (f=32): [m(32)][8.3%][r=1441MiB/s,w=1441MiB/s][r=369k,w=369k IOPS][eta 09m:11s]
  710. Jobs: 32 (f=32): [m(32)][8.5%][r=1429MiB/s,w=1423MiB/s][r=366k,w=364k IOPS][eta 09m:10s]
  711. Jobs: 32 (f=32): [m(32)][8.7%][r=1453MiB/s,w=1451MiB/s][r=372k,w=372k IOPS][eta 09m:09s]
  712. Jobs: 32 (f=32): [m(32)][8.8%][r=1436MiB/s,w=1438MiB/s][r=368k,w=368k IOPS][eta 09m:08s]
  713. Jobs: 32 (f=32): [m(32)][9.0%][r=1426MiB/s,w=1429MiB/s][r=365k,w=366k IOPS][eta 09m:07s]
  714. Jobs: 32 (f=32): [m(32)][9.2%][r=1457MiB/s,w=1452MiB/s][r=373k,w=372k IOPS][eta 09m:06s]
  715. Jobs: 32 (f=32): [m(32)][9.3%][r=1428MiB/s,w=1432MiB/s][r=366k,w=366k IOPS][eta 09m:05s]
  716. Jobs: 32 (f=32): [m(32)][9.5%][r=1419MiB/s,w=1423MiB/s][r=363k,w=364k IOPS][eta 09m:04s]
  717. Jobs: 32 (f=32): [m(32)][9.7%][r=1455MiB/s,w=1458MiB/s][r=372k,w=373k IOPS][eta 09m:03s]
  718. Jobs: 32 (f=32): [m(32)][9.8%][r=1440MiB/s,w=1437MiB/s][r=369k,w=368k IOPS][eta 09m:02s]
  719. Jobs: 32 (f=32): [m(32)][10.0%][r=1441MiB/s,w=1441MiB/s][r=369k,w=369k IOPS][eta 09m:01s]
  720. Jobs: 32 (f=32): [m(32)][10.1%][r=1435MiB/s,w=1431MiB/s][r=367k,w=366k IOPS][eta 09m:00s]
  721. Jobs: 32 (f=32): [m(32)][10.3%][r=1429MiB/s,w=1434MiB/s][r=366k,w=367k IOPS][eta 08m:59s]
  722. Jobs: 32 (f=32): [m(32)][10.5%][r=1428MiB/s,w=1430MiB/s][r=366k,w=366k IOPS][eta 08m:58s]
  723. Jobs: 32 (f=32): [m(32)][10.6%][r=1423MiB/s,w=1422MiB/s][r=364k,w=364k IOPS][eta 08m:57s]
  724. Jobs: 32 (f=32): [m(32)][10.8%][r=1438MiB/s,w=1438MiB/s][r=368k,w=368k IOPS][eta 08m:56s]
  725. Jobs: 32 (f=32): [m(32)][11.0%][r=1425MiB/s,w=1423MiB/s][r=365k,w=364k IOPS][eta 08m:55s]
  726. Jobs: 32 (f=32): [m(32)][11.1%][r=1431MiB/s,w=1427MiB/s][r=366k,w=365k IOPS][eta 08m:54s]
  727. Jobs: 32 (f=32): [m(32)][11.3%][r=1445MiB/s,w=1450MiB/s][r=370k,w=371k IOPS][eta 08m:53s]
  728. Jobs: 32 (f=32): [m(32)][11.5%][r=1441MiB/s,w=1438MiB/s][r=369k,w=368k IOPS][eta 08m:52s]
  729. Jobs: 32 (f=32): [m(32)][11.6%][r=1437MiB/s,w=1436MiB/s][r=368k,w=367k IOPS][eta 08m:51s]
  730. Jobs: 32 (f=32): [m(32)][11.8%][r=1449MiB/s,w=1446MiB/s][r=371k,w=370k IOPS][eta 08m:50s]
  731. Jobs: 32 (f=32): [m(32)][12.0%][r=1433MiB/s,w=1433MiB/s][r=367k,w=367k IOPS][eta 08m:49s]
  732. Jobs: 32 (f=32): [m(32)][12.1%][r=1431MiB/s,w=1432MiB/s][r=366k,w=367k IOPS][eta 08m:48s]
  733. Jobs: 32 (f=32): [m(32)][12.3%][r=1436MiB/s,w=1431MiB/s][r=368k,w=366k IOPS][eta 08m:47s]
  734. Jobs: 32 (f=32): [m(32)][12.5%][r=1445MiB/s,w=1444MiB/s][r=370k,w=370k IOPS][eta 08m:46s]
  735. Jobs: 32 (f=32): [m(32)][12.6%][r=1434MiB/s,w=1433MiB/s][r=367k,w=367k IOPS][eta 08m:45s]
  736. Jobs: 32 (f=32): [m(32)][12.8%][r=1427MiB/s,w=1427MiB/s][r=365k,w=365k IOPS][eta 08m:44s]
  737. Jobs: 32 (f=32): [m(32)][13.0%][r=1436MiB/s,w=1436MiB/s][r=368k,w=368k IOPS][eta 08m:43s]
  738. Jobs: 32 (f=32): [m(32)][13.1%][r=1430MiB/s,w=1435MiB/s][r=366k,w=367k IOPS][eta 08m:42s]
  739. Jobs: 32 (f=32): [m(32)][13.3%][r=1420MiB/s,w=1420MiB/s][r=363k,w=364k IOPS][eta 08m:41s]
  740. Jobs: 32 (f=32): [m(32)][13.5%][r=1439MiB/s,w=1441MiB/s][r=368k,w=369k IOPS][eta 08m:40s]
  741. Jobs: 32 (f=32): [m(32)][13.7%][r=1428MiB/s,w=1425MiB/s][r=366k,w=365k IOPS][eta 08m:38s]
  742. Jobs: 32 (f=32): [m(32)][13.8%][r=1422MiB/s,w=1425MiB/s][r=364k,w=365k IOPS][eta 08m:37s]
  743. Jobs: 32 (f=32): [m(32)][14.0%][r=1437MiB/s,w=1433MiB/s][r=368k,w=367k IOPS][eta 08m:36s]
  744. Jobs: 32 (f=32): [m(32)][14.2%][r=1435MiB/s,w=1437MiB/s][r=367k,w=368k IOPS][eta 08m:35s]
  745. Jobs: 32 (f=32): [m(32)][14.3%][r=1434MiB/s,w=1432MiB/s][r=367k,w=367k IOPS][eta 08m:34s]
  746. Jobs: 32 (f=32): [m(32)][14.5%][r=1415MiB/s,w=1418MiB/s][r=362k,w=363k IOPS][eta 08m:33s]
  747. Jobs: 32 (f=32): [m(32)][14.7%][r=1444MiB/s,w=1442MiB/s][r=370k,w=369k IOPS][eta 08m:32s]
  748. Jobs: 32 (f=32): [m(32)][14.8%][r=1430MiB/s,w=1428MiB/s][r=366k,w=366k IOPS][eta 08m:31s]
  749. Jobs: 32 (f=32): [m(32)][15.0%][r=1427MiB/s,w=1421MiB/s][r=365k,w=364k IOPS][eta 08m:30s]
  750. Jobs: 32 (f=32): [m(32)][15.2%][r=1445MiB/s,w=1444MiB/s][r=370k,w=370k IOPS][eta 08m:29s]
  751. Jobs: 32 (f=32): [m(32)][15.3%][r=1442MiB/s,w=1441MiB/s][r=369k,w=369k IOPS][eta 08m:28s]
  752. Jobs: 32 (f=32): [m(32)][15.5%][r=1425MiB/s,w=1421MiB/s][r=365k,w=364k IOPS][eta 08m:27s]
  753. Jobs: 32 (f=32): [m(32)][15.7%][r=1452MiB/s,w=1453MiB/s][r=372k,w=372k IOPS][eta 08m:26s]
  754. Jobs: 32 (f=32): [m(32)][15.8%][r=1439MiB/s,w=1442MiB/s][r=368k,w=369k IOPS][eta 08m:25s]
  755. Jobs: 32 (f=32): [m(32)][16.0%][r=1439MiB/s,w=1436MiB/s][r=368k,w=368k IOPS][eta 08m:24s]
  756. Jobs: 32 (f=32): [m(32)][16.2%][r=1441MiB/s,w=1444MiB/s][r=369k,w=370k IOPS][eta 08m:23s]
  757. Jobs: 32 (f=32): [m(32)][16.3%][r=1439MiB/s,w=1442MiB/s][r=368k,w=369k IOPS][eta 08m:22s]
  758. Jobs: 32 (f=32): [m(32)][16.5%][r=1433MiB/s,w=1435MiB/s][r=367k,w=367k IOPS][eta 08m:21s]
  759. Jobs: 32 (f=32): [m(32)][16.7%][r=1431MiB/s,w=1428MiB/s][r=366k,w=366k IOPS][eta 08m:20s]
  760. Jobs: 32 (f=32): [m(32)][16.8%][r=1450MiB/s,w=1450MiB/s][r=371k,w=371k IOPS][eta 08m:19s]
  761. Jobs: 32 (f=32): [m(32)][17.0%][r=1439MiB/s,w=1435MiB/s][r=368k,w=367k IOPS][eta 08m:18s]
  762. Jobs: 32 (f=32): [m(32)][17.2%][r=1421MiB/s,w=1423MiB/s][r=364k,w=364k IOPS][eta 08m:17s]
  763. Jobs: 32 (f=32): [m(32)][17.3%][r=1454MiB/s,w=1454MiB/s][r=372k,w=372k IOPS][eta 08m:16s]
  764. Jobs: 32 (f=32): [m(32)][17.5%][r=1436MiB/s,w=1437MiB/s][r=368k,w=368k IOPS][eta 08m:15s]
  765. Jobs: 32 (f=32): [m(32)][17.7%][r=1428MiB/s,w=1431MiB/s][r=366k,w=366k IOPS][eta 08m:14s]
  766. Jobs: 32 (f=32): [m(32)][17.8%][r=1441MiB/s,w=1442MiB/s][r=369k,w=369k IOPS][eta 08m:13s]
  767. Jobs: 32 (f=32): [m(32)][18.0%][r=1438MiB/s,w=1441MiB/s][r=368k,w=369k IOPS][eta 08m:12s]
  768. Jobs: 32 (f=32): [m(32)][18.2%][r=1442MiB/s,w=1440MiB/s][r=369k,w=369k IOPS][eta 08m:11s]
  769. Jobs: 32 (f=32): [m(32)][18.3%][r=1427MiB/s,w=1427MiB/s][r=365k,w=365k IOPS][eta 08m:10s]
  770. Jobs: 32 (f=32): [m(32)][18.5%][r=1447MiB/s,w=1448MiB/s][r=370k,w=371k IOPS][eta 08m:09s]
  771. Jobs: 32 (f=32): [m(32)][18.7%][r=1445MiB/s,w=1444MiB/s][r=370k,w=370k IOPS][eta 08m:08s]
  772. Jobs: 32 (f=32): [m(32)][18.8%][r=1439MiB/s,w=1429MiB/s][r=368k,w=366k IOPS][eta 08m:07s]
  773. Jobs: 32 (f=32): [m(32)][19.0%][r=1452MiB/s,w=1449MiB/s][r=372k,w=371k IOPS][eta 08m:06s]
  774. Jobs: 32 (f=32): [m(32)][19.2%][r=1439MiB/s,w=1436MiB/s][r=368k,w=368k IOPS][eta 08m:05s]
  775. Jobs: 32 (f=32): [m(32)][19.3%][r=1434MiB/s,w=1427MiB/s][r=367k,w=365k IOPS][eta 08m:04s]
  776. Jobs: 32 (f=32): [m(32)][19.5%][r=1448MiB/s,w=1445MiB/s][r=371k,w=370k IOPS][eta 08m:03s]
  777. Jobs: 32 (f=32): [m(32)][19.7%][r=1436MiB/s,w=1440MiB/s][r=368k,w=369k IOPS][eta 08m:02s]
  778. Jobs: 32 (f=32): [m(32)][19.8%][r=1430MiB/s,w=1434MiB/s][r=366k,w=367k IOPS][eta 08m:01s]
  779. Jobs: 32 (f=32): [m(32)][20.0%][r=1430MiB/s,w=1428MiB/s][r=366k,w=366k IOPS][eta 08m:00s]
  780. Jobs: 32 (f=32): [m(32)][20.2%][r=1445MiB/s,w=1447MiB/s][r=370k,w=370k IOPS][eta 07m:59s]
  781. Jobs: 32 (f=32): [m(32)][20.3%][r=1442MiB/s,w=1441MiB/s][r=369k,w=369k IOPS][eta 07m:58s]
  782. Jobs: 32 (f=32): [m(32)][20.5%][r=1418MiB/s,w=1417MiB/s][r=363k,w=363k IOPS][eta 07m:57s]
  783. Jobs: 32 (f=32): [m(32)][20.7%][r=1446MiB/s,w=1445MiB/s][r=370k,w=370k IOPS][eta 07m:56s]
  784. Jobs: 32 (f=32): [m(32)][20.8%][r=1432MiB/s,w=1434MiB/s][r=367k,w=367k IOPS][eta 07m:55s]
  785. Jobs: 32 (f=32): [m(32)][21.0%][r=1428MiB/s,w=1424MiB/s][r=366k,w=365k IOPS][eta 07m:54s]
  786. Jobs: 32 (f=32): [m(32)][21.2%][r=1459MiB/s,w=1456MiB/s][r=373k,w=373k IOPS][eta 07m:53s]
  787. Jobs: 32 (f=32): [m(32)][21.3%][r=1435MiB/s,w=1435MiB/s][r=367k,w=367k IOPS][eta 07m:52s]
  788. Jobs: 32 (f=32): [m(32)][21.5%][r=1434MiB/s,w=1432MiB/s][r=367k,w=367k IOPS][eta 07m:51s]
  789. Jobs: 32 (f=32): [m(32)][21.7%][r=1435MiB/s,w=1437MiB/s][r=367k,w=368k IOPS][eta 07m:50s]
  790. Jobs: 32 (f=32): [m(32)][21.8%][r=1437MiB/s,w=1438MiB/s][r=368k,w=368k IOPS][eta 07m:49s]
  791. Jobs: 32 (f=32): [m(32)][22.0%][r=1445MiB/s,w=1435MiB/s][r=370k,w=367k IOPS][eta 07m:48s]
  792. Jobs: 32 (f=32): [m(32)][22.2%][r=1438MiB/s,w=1438MiB/s][r=368k,w=368k IOPS][eta 07m:47s]
  793. Jobs: 32 (f=32): [m(32)][22.3%][r=1444MiB/s,w=1445MiB/s][r=370k,w=370k IOPS][eta 07m:46s]
  794. Jobs: 32 (f=32): [m(32)][22.5%][r=1426MiB/s,w=1426MiB/s][r=365k,w=365k IOPS][eta 07m:45s]
  795. Jobs: 32 (f=32): [m(32)][22.7%][r=1421MiB/s,w=1422MiB/s][r=364k,w=364k IOPS][eta 07m:44s]
  796. Jobs: 32 (f=32): [m(32)][22.8%][r=1452MiB/s,w=1450MiB/s][r=372k,w=371k IOPS][eta 07m:43s]
  797. Jobs: 32 (f=32): [m(32)][23.0%][r=1436MiB/s,w=1436MiB/s][r=368k,w=368k IOPS][eta 07m:42s]
  798. Jobs: 32 (f=32): [m(32)][23.2%][r=1431MiB/s,w=1424MiB/s][r=366k,w=365k IOPS][eta 07m:41s]
  799. Jobs: 32 (f=32): [m(32)][23.3%][r=1454MiB/s,w=1449MiB/s][r=372k,w=371k IOPS][eta 07m:40s]
  800. Jobs: 32 (f=32): [m(32)][23.5%][r=1433MiB/s,w=1435MiB/s][r=367k,w=367k IOPS][eta 07m:39s]
  801. Jobs: 32 (f=32): [m(32)][23.7%][r=1435MiB/s,w=1429MiB/s][r=367k,w=366k IOPS][eta 07m:38s]
  802. Jobs: 32 (f=32): [m(32)][23.8%][r=1427MiB/s,w=1428MiB/s][r=365k,w=365k IOPS][eta 07m:37s]
  803. Jobs: 32 (f=32): [m(32)][24.0%][r=1437MiB/s,w=1434MiB/s][r=368k,w=367k IOPS][eta 07m:36s]
  804. Jobs: 32 (f=32): [m(32)][24.2%][r=1438MiB/s,w=1434MiB/s][r=368k,w=367k IOPS][eta 07m:35s]
  805. Jobs: 32 (f=32): [m(32)][24.3%][r=1434MiB/s,w=1436MiB/s][r=367k,w=368k IOPS][eta 07m:34s]
  806. Jobs: 32 (f=32): [m(32)][24.5%][r=1456MiB/s,w=1451MiB/s][r=373k,w=371k IOPS][eta 07m:33s]
  807. Jobs: 32 (f=32): [m(32)][24.7%][r=1436MiB/s,w=1434MiB/s][r=368k,w=367k IOPS][eta 07m:32s]
  808. Jobs: 32 (f=32): [m(32)][24.8%][r=1428MiB/s,w=1431MiB/s][r=366k,w=366k IOPS][eta 07m:31s]
  809. Jobs: 32 (f=32): [m(32)][25.0%][r=1447MiB/s,w=1447MiB/s][r=370k,w=370k IOPS][eta 07m:30s]
  810. Jobs: 32 (f=32): [m(32)][25.2%][r=1443MiB/s,w=1440MiB/s][r=369k,w=369k IOPS][eta 07m:29s]
  811. Jobs: 32 (f=32): [m(32)][25.3%][r=1426MiB/s,w=1429MiB/s][r=365k,w=366k IOPS][eta 07m:28s]
  812. Jobs: 32 (f=32): [m(32)][25.5%][r=1442MiB/s,w=1440MiB/s][r=369k,w=369k IOPS][eta 07m:27s]
  813. Jobs: 32 (f=32): [m(32)][25.7%][r=1437MiB/s,w=1434MiB/s][r=368k,w=367k IOPS][eta 07m:26s]
  814. Jobs: 32 (f=32): [m(32)][25.8%][r=1450MiB/s,w=1448MiB/s][r=371k,w=371k IOPS][eta 07m:25s]
  815. Jobs: 32 (f=32): [m(32)][26.0%][r=1423MiB/s,w=1428MiB/s][r=364k,w=366k IOPS][eta 07m:24s]
  816. Jobs: 32 (f=32): [m(32)][26.2%][r=1454MiB/s,w=1447MiB/s][r=372k,w=370k IOPS][eta 07m:23s]
  817. Jobs: 32 (f=32): [m(32)][26.3%][r=1430MiB/s,w=1432MiB/s][r=366k,w=367k IOPS][eta 07m:22s]
  818. Jobs: 32 (f=32): [m(32)][26.5%][r=1423MiB/s,w=1427MiB/s][r=364k,w=365k IOPS][eta 07m:21s]
  819. Jobs: 32 (f=32): [m(32)][26.7%][r=1449MiB/s,w=1447MiB/s][r=371k,w=370k IOPS][eta 07m:20s]
  820. Jobs: 32 (f=32): [m(32)][26.8%][r=1439MiB/s,w=1438MiB/s][r=368k,w=368k IOPS][eta 07m:19s]
  821. Jobs: 32 (f=32): [m(32)][27.0%][r=1424MiB/s,w=1428MiB/s][r=364k,w=366k IOPS][eta 07m:18s]
  822. Jobs: 32 (f=32): [m(32)][27.2%][r=1450MiB/s,w=1449MiB/s][r=371k,w=371k IOPS][eta 07m:17s]
  823. Jobs: 32 (f=32): [m(32)][27.3%][r=1429MiB/s,w=1433MiB/s][r=366k,w=367k IOPS][eta 07m:16s]
  824. Jobs: 32 (f=32): [m(32)][27.5%][r=1436MiB/s,w=1439MiB/s][r=368k,w=368k IOPS][eta 07m:15s]
  825. Jobs: 32 (f=32): [m(32)][27.7%][r=1431MiB/s,w=1431MiB/s][r=366k,w=366k IOPS][eta 07m:14s]
  826. Jobs: 32 (f=32): [m(32)][27.8%][r=1458MiB/s,w=1452MiB/s][r=373k,w=372k IOPS][eta 07m:13s]
  827. Jobs: 32 (f=32): [m(32)][28.0%][r=1427MiB/s,w=1429MiB/s][r=365k,w=366k IOPS][eta 07m:12s]
  828. Jobs: 32 (f=32): [m(32)][28.2%][r=1430MiB/s,w=1430MiB/s][r=366k,w=366k IOPS][eta 07m:11s]
  829. Jobs: 32 (f=32): [m(32)][28.3%][r=1449MiB/s,w=1451MiB/s][r=371k,w=372k IOPS][eta 07m:10s]
  830. Jobs: 32 (f=32): [m(32)][28.5%][r=1438MiB/s,w=1432MiB/s][r=368k,w=367k IOPS][eta 07m:09s]
  831. Jobs: 32 (f=32): [m(32)][28.7%][r=1434MiB/s,w=1438MiB/s][r=367k,w=368k IOPS][eta 07m:08s]
  832. Jobs: 32 (f=32): [m(32)][28.8%][r=1448MiB/s,w=1450MiB/s][r=371k,w=371k IOPS][eta 07m:07s]
  833. Jobs: 32 (f=32): [m(32)][29.0%][r=1434MiB/s,w=1440MiB/s][r=367k,w=369k IOPS][eta 07m:06s]
  834. Jobs: 32 (f=32): [m(32)][29.2%][r=1436MiB/s,w=1438MiB/s][r=368k,w=368k IOPS][eta 07m:05s]
  835. Jobs: 32 (f=32): [m(32)][29.3%][r=1446MiB/s,w=1443MiB/s][r=370k,w=369k IOPS][eta 07m:04s]
  836. Jobs: 32 (f=32): [m(32)][29.5%][r=1441MiB/s,w=1444MiB/s][r=369k,w=370k IOPS][eta 07m:03s]
  837. Jobs: 32 (f=32): [m(32)][29.7%][r=1442MiB/s,w=1442MiB/s][r=369k,w=369k IOPS][eta 07m:02s]
  838. Jobs: 32 (f=32): [m(32)][29.8%][r=1429MiB/s,w=1432MiB/s][r=366k,w=367k IOPS][eta 07m:01s]
  839. Jobs: 32 (f=32): [m(32)][30.0%][r=1452MiB/s,w=1450MiB/s][r=372k,w=371k IOPS][eta 07m:00s]
  840. Jobs: 32 (f=32): [m(32)][30.2%][r=1436MiB/s,w=1436MiB/s][r=368k,w=368k IOPS][eta 06m:59s]
  841. Jobs: 32 (f=32): [m(32)][30.3%][r=1425MiB/s,w=1425MiB/s][r=365k,w=365k IOPS][eta 06m:58s]
  842. Jobs: 32 (f=32): [m(32)][30.5%][r=1452MiB/s,w=1448MiB/s][r=372k,w=371k IOPS][eta 06m:57s]
  843. Jobs: 32 (f=32): [m(32)][30.7%][r=1432MiB/s,w=1440MiB/s][r=367k,w=369k IOPS][eta 06m:56s]
  844. Jobs: 32 (f=32): [m(32)][30.8%][r=1434MiB/s,w=1436MiB/s][r=367k,w=368k IOPS][eta 06m:55s]
  845. Jobs: 32 (f=32): [m(32)][31.0%][r=1437MiB/s,w=1439MiB/s][r=368k,w=368k IOPS][eta 06m:54s]
  846. Jobs: 32 (f=32): [m(32)][31.2%][r=1439MiB/s,w=1438MiB/s][r=368k,w=368k IOPS][eta 06m:53s]
  847. Jobs: 32 (f=32): [m(32)][31.3%][r=1441MiB/s,w=1438MiB/s][r=369k,w=368k IOPS][eta 06m:52s]
  848. Jobs: 32 (f=32): [m(32)][31.5%][r=1431MiB/s,w=1434MiB/s][r=366k,w=367k IOPS][eta 06m:51s]
  849. Jobs: 32 (f=32): [m(32)][31.7%][r=1451MiB/s,w=1452MiB/s][r=372k,w=372k IOPS][eta 06m:50s]
  850. Jobs: 32 (f=32): [m(32)][31.8%][r=1437MiB/s,w=1440MiB/s][r=368k,w=369k IOPS][eta 06m:49s]
  851. Jobs: 32 (f=32): [m(32)][32.0%][r=1426MiB/s,w=1430MiB/s][r=365k,w=366k IOPS][eta 06m:48s]
  852. Jobs: 32 (f=32): [m(32)][32.2%][r=1455MiB/s,w=1459MiB/s][r=373k,w=373k IOPS][eta 06m:47s]
  853. Jobs: 32 (f=32): [m(32)][32.3%][r=1426MiB/s,w=1428MiB/s][r=365k,w=366k IOPS][eta 06m:46s]
  854. Jobs: 32 (f=32): [m(32)][32.5%][r=1435MiB/s,w=1435MiB/s][r=367k,w=367k IOPS][eta 06m:45s]
  855. Jobs: 32 (f=32): [m(32)][32.7%][r=1454MiB/s,w=1451MiB/s][r=372k,w=371k IOPS][eta 06m:44s]
  856. Jobs: 32 (f=32): [m(32)][32.8%][r=1437MiB/s,w=1442MiB/s][r=368k,w=369k IOPS][eta 06m:43s]
  857. Jobs: 32 (f=32): [m(32)][33.0%][r=1425MiB/s,w=1434MiB/s][r=365k,w=367k IOPS][eta 06m:42s]
  858. Jobs: 32 (f=32): [m(32)][33.2%][r=1442MiB/s,w=1443MiB/s][r=369k,w=369k IOPS][eta 06m:41s]
  859. Jobs: 32 (f=32): [m(32)][33.3%][r=1440MiB/s,w=1435MiB/s][r=369k,w=367k IOPS][eta 06m:40s]
  860. Jobs: 32 (f=32): [m(32)][33.5%][r=1443MiB/s,w=1443MiB/s][r=369k,w=369k IOPS][eta 06m:39s]
  861. Jobs: 32 (f=32): [m(32)][33.7%][r=1437MiB/s,w=1434MiB/s][r=368k,w=367k IOPS][eta 06m:38s]
  862. Jobs: 32 (f=32): [m(32)][33.8%][r=1451MiB/s,w=1450MiB/s][r=371k,w=371k IOPS][eta 06m:37s]
  863. Jobs: 32 (f=32): [m(32)][34.0%][r=1442MiB/s,w=1442MiB/s][r=369k,w=369k IOPS][eta 06m:36s]
  864. Jobs: 32 (f=32): [m(32)][34.2%][r=1434MiB/s,w=1435MiB/s][r=367k,w=367k IOPS][eta 06m:35s]
  865. Jobs: 32 (f=32): [m(32)][34.3%][r=1450MiB/s,w=1449MiB/s][r=371k,w=371k IOPS][eta 06m:34s]
  866. Jobs: 32 (f=32): [m(32)][34.5%][r=1435MiB/s,w=1438MiB/s][r=367k,w=368k IOPS][eta 06m:33s]
  867. Jobs: 32 (f=32): [m(32)][34.7%][r=1434MiB/s,w=1432MiB/s][r=367k,w=366k IOPS][eta 06m:32s]
  868. Jobs: 32 (f=32): [m(32)][34.8%][r=1441MiB/s,w=1442MiB/s][r=369k,w=369k IOPS][eta 06m:31s]
  869. Jobs: 32 (f=32): [m(32)][35.0%][r=1453MiB/s,w=1452MiB/s][r=372k,w=372k IOPS][eta 06m:30s]
  870. Jobs: 32 (f=32): [m(32)][35.2%][r=1434MiB/s,w=1431MiB/s][r=367k,w=366k IOPS][eta 06m:29s]
  871. Jobs: 32 (f=32): [m(32)][35.3%][r=1435MiB/s,w=1435MiB/s][r=367k,w=367k IOPS][eta 06m:28s]
  872. Jobs: 32 (f=32): [m(32)][35.5%][r=1459MiB/s,w=1461MiB/s][r=374k,w=374k IOPS][eta 06m:27s]
  873. Jobs: 32 (f=32): [m(32)][35.7%][r=1439MiB/s,w=1440MiB/s][r=368k,w=369k IOPS][eta 06m:26s]
  874. Jobs: 32 (f=32): [m(32)][35.8%][r=1435MiB/s,w=1435MiB/s][r=367k,w=367k IOPS][eta 06m:25s]
  875. Jobs: 32 (f=32): [m(32)][36.0%][r=1451MiB/s,w=1456MiB/s][r=371k,w=373k IOPS][eta 06m:24s]
  876. Jobs: 32 (f=32): [m(32)][36.2%][r=1438MiB/s,w=1439MiB/s][r=368k,w=368k IOPS][eta 06m:23s]
  877. Jobs: 32 (f=32): [m(32)][36.3%][r=1430MiB/s,w=1430MiB/s][r=366k,w=366k IOPS][eta 06m:22s]
  878. Jobs: 32 (f=32): [m(32)][36.5%][r=1446MiB/s,w=1449MiB/s][r=370k,w=371k IOPS][eta 06m:21s]
  879. Jobs: 32 (f=32): [m(32)][36.7%][r=1444MiB/s,w=1442MiB/s][r=370k,w=369k IOPS][eta 06m:20s]
  880. Jobs: 32 (f=32): [m(32)][36.8%][r=1436MiB/s,w=1429MiB/s][r=368k,w=366k IOPS][eta 06m:19s]
  881. Jobs: 32 (f=32): [m(32)][37.0%][r=1433MiB/s,w=1436MiB/s][r=367k,w=368k IOPS][eta 06m:18s]
  882. Jobs: 32 (f=32): [m(32)][37.2%][r=1457MiB/s,w=1454MiB/s][r=373k,w=372k IOPS][eta 06m:17s]
  883. Jobs: 32 (f=32): [m(32)][37.3%][r=1433MiB/s,w=1434MiB/s][r=367k,w=367k IOPS][eta 06m:16s]
  884. Jobs: 32 (f=32): [m(32)][37.5%][r=1438MiB/s,w=1438MiB/s][r=368k,w=368k IOPS][eta 06m:15s]
  885. Jobs: 32 (f=32): [m(32)][37.7%][r=1457MiB/s,w=1457MiB/s][r=373k,w=373k IOPS][eta 06m:14s]
  886. Jobs: 32 (f=32): [m(32)][37.8%][r=1438MiB/s,w=1441MiB/s][r=368k,w=369k IOPS][eta 06m:13s]
  887. Jobs: 32 (f=32): [m(32)][38.0%][r=1464MiB/s,w=1464MiB/s][r=375k,w=375k IOPS][eta 06m:12s]
  888. Jobs: 32 (f=32): [m(32)][38.2%][r=1490MiB/s,w=1487MiB/s][r=381k,w=381k IOPS][eta 06m:11s]
  889. Jobs: 32 (f=32): [m(32)][38.3%][r=1476MiB/s,w=1481MiB/s][r=378k,w=379k IOPS][eta 06m:10s]
  890. Jobs: 32 (f=32): [m(32)][38.5%][r=1484MiB/s,w=1488MiB/s][r=380k,w=381k IOPS][eta 06m:09s]
  891. Jobs: 32 (f=32): [m(32)][38.7%][r=1479MiB/s,w=1481MiB/s][r=379k,w=379k IOPS][eta 06m:08s]
  892. Jobs: 32 (f=32): [m(32)][38.8%][r=1503MiB/s,w=1501MiB/s][r=385k,w=384k IOPS][eta 06m:07s]
  893. Jobs: 32 (f=32): [m(32)][39.0%][r=1483MiB/s,w=1482MiB/s][r=380k,w=379k IOPS][eta 06m:06s]
  894. Jobs: 32 (f=32): [m(32)][39.2%][r=1487MiB/s,w=1484MiB/s][r=381k,w=380k IOPS][eta 06m:05s]
  895. Jobs: 32 (f=32): [m(32)][39.3%][r=1496MiB/s,w=1497MiB/s][r=383k,w=383k IOPS][eta 06m:04s]
  896. Jobs: 32 (f=32): [m(32)][39.5%][r=1535MiB/s,w=1532MiB/s][r=393k,w=392k IOPS][eta 06m:03s]
  897. Jobs: 32 (f=32): [m(32)][39.7%][r=1543MiB/s,w=1544MiB/s][r=395k,w=395k IOPS][eta 06m:02s]
  898. Jobs: 32 (f=32): [m(32)][39.8%][r=1538MiB/s,w=1538MiB/s][r=394k,w=394k IOPS][eta 06m:01s]
  899. Jobs: 32 (f=32): [m(32)][40.0%][r=1553MiB/s,w=1561MiB/s][r=398k,w=400k IOPS][eta 06m:00s]
  900. Jobs: 32 (f=32): [m(32)][40.2%][r=1553MiB/s,w=1550MiB/s][r=398k,w=397k IOPS][eta 05m:59s]
  901. Jobs: 32 (f=32): [m(32)][40.3%][r=1549MiB/s,w=1551MiB/s][r=397k,w=397k IOPS][eta 05m:58s]
  902. Jobs: 32 (f=32): [m(32)][40.5%][r=1542MiB/s,w=1537MiB/s][r=395k,w=394k IOPS][eta 05m:57s]
  903. Jobs: 32 (f=32): [m(32)][40.7%][r=1561MiB/s,w=1567MiB/s][r=400k,w=401k IOPS][eta 05m:56s]
  904. Jobs: 32 (f=32): [m(32)][40.8%][r=1551MiB/s,w=1555MiB/s][r=397k,w=398k IOPS][eta 05m:55s]
  905. Jobs: 32 (f=32): [m(32)][41.0%][r=1552MiB/s,w=1559MiB/s][r=397k,w=399k IOPS][eta 05m:54s]
  906. Jobs: 32 (f=32): [m(32)][41.2%][r=1565MiB/s,w=1564MiB/s][r=401k,w=400k IOPS][eta 05m:53s]
  907. Jobs: 32 (f=32): [m(32)][41.3%][r=1568MiB/s,w=1577MiB/s][r=401k,w=404k IOPS][eta 05m:52s]
  908. Jobs: 32 (f=32): [m(32)][41.5%][r=1584MiB/s,w=1588MiB/s][r=405k,w=406k IOPS][eta 05m:51s]
  909. Jobs: 32 (f=32): [m(32)][41.7%][r=1566MiB/s,w=1568MiB/s][r=401k,w=401k IOPS][eta 05m:50s]
  910. Jobs: 32 (f=32): [m(32)][41.8%][r=1570MiB/s,w=1567MiB/s][r=402k,w=401k IOPS][eta 05m:49s]
  911. Jobs: 32 (f=32): [m(32)][42.0%][r=1564MiB/s,w=1568MiB/s][r=400k,w=401k IOPS][eta 05m:48s]
  912. Jobs: 32 (f=32): [m(32)][42.2%][r=1597MiB/s,w=1593MiB/s][r=409k,w=408k IOPS][eta 05m:47s]
  913. Jobs: 32 (f=32): [m(32)][42.3%][r=1567MiB/s,w=1568MiB/s][r=401k,w=401k IOPS][eta 05m:46s]
  914. Jobs: 32 (f=32): [m(32)][42.5%][r=1559MiB/s,w=1558MiB/s][r=399k,w=399k IOPS][eta 05m:45s]
  915. Jobs: 32 (f=32): [m(32)][42.7%][r=1562MiB/s,w=1570MiB/s][r=400k,w=402k IOPS][eta 05m:44s]
  916. Jobs: 32 (f=32): [m(32)][42.8%][r=1583MiB/s,w=1589MiB/s][r=405k,w=407k IOPS][eta 05m:43s]
  917. Jobs: 32 (f=32): [m(32)][43.0%][r=1548MiB/s,w=1552MiB/s][r=396k,w=397k IOPS][eta 05m:42s]
  918. Jobs: 32 (f=32): [m(32)][43.2%][r=1561MiB/s,w=1556MiB/s][r=400k,w=398k IOPS][eta 05m:41s]
  919. Jobs: 32 (f=32): [m(32)][43.3%][r=1564MiB/s,w=1563MiB/s][r=400k,w=400k IOPS][eta 05m:40s]
  920. Jobs: 32 (f=32): [m(32)][43.5%][r=1578MiB/s,w=1578MiB/s][r=404k,w=404k IOPS][eta 05m:39s]
  921. Jobs: 32 (f=32): [m(32)][43.7%][r=1574MiB/s,w=1569MiB/s][r=403k,w=402k IOPS][eta 05m:38s]
  922. Jobs: 32 (f=32): [m(32)][43.8%][r=1566MiB/s,w=1567MiB/s][r=401k,w=401k IOPS][eta 05m:37s]
  923. Jobs: 32 (f=32): [m(32)][44.0%][r=1561MiB/s,w=1566MiB/s][r=400k,w=401k IOPS][eta 05m:36s]
  924. Jobs: 32 (f=32): [m(32)][44.2%][r=1584MiB/s,w=1581MiB/s][r=405k,w=405k IOPS][eta 05m:35s]
  925. Jobs: 32 (f=32): [m(32)][44.3%][r=1555MiB/s,w=1556MiB/s][r=398k,w=398k IOPS][eta 05m:34s]
  926. Jobs: 32 (f=32): [m(32)][44.5%][r=1562MiB/s,w=1561MiB/s][r=400k,w=400k IOPS][eta 05m:33s]
  927. Jobs: 32 (f=32): [m(32)][44.7%][r=1558MiB/s,w=1555MiB/s][r=399k,w=398k IOPS][eta 05m:32s]
  928. Jobs: 32 (f=32): [m(32)][44.8%][r=1589MiB/s,w=1587MiB/s][r=407k,w=406k IOPS][eta 05m:31s]
  929. Jobs: 32 (f=32): [m(32)][45.0%][r=1566MiB/s,w=1567MiB/s][r=401k,w=401k IOPS][eta 05m:30s]
  930. Jobs: 32 (f=32): [m(32)][45.2%][r=1568MiB/s,w=1565MiB/s][r=401k,w=401k IOPS][eta 05m:29s]
  931. Jobs: 32 (f=32): [m(32)][45.3%][r=1564MiB/s,w=1563MiB/s][r=400k,w=400k IOPS][eta 05m:28s]
  932. Jobs: 32 (f=32): [m(32)][45.5%][r=1575MiB/s,w=1571MiB/s][r=403k,w=402k IOPS][eta 05m:27s]
  933. Jobs: 32 (f=32): [m(32)][45.7%][r=1584MiB/s,w=1586MiB/s][r=405k,w=406k IOPS][eta 05m:26s]
  934. Jobs: 32 (f=32): [m(32)][45.8%][r=1563MiB/s,w=1562MiB/s][r=400k,w=400k IOPS][eta 05m:25s]
  935. Jobs: 32 (f=32): [m(32)][46.0%][r=1558MiB/s,w=1562MiB/s][r=399k,w=400k IOPS][eta 05m:24s]
  936. Jobs: 32 (f=32): [m(32)][46.2%][r=1564MiB/s,w=1568MiB/s][r=400k,w=401k IOPS][eta 05m:23s]
  937. Jobs: 32 (f=32): [m(32)][46.3%][r=1586MiB/s,w=1583MiB/s][r=406k,w=405k IOPS][eta 05m:22s]
  938. Jobs: 32 (f=32): [m(32)][46.5%][r=1578MiB/s,w=1575MiB/s][r=404k,w=403k IOPS][eta 05m:21s]
  939. Jobs: 32 (f=32): [m(32)][46.7%][r=1562MiB/s,w=1560MiB/s][r=400k,w=399k IOPS][eta 05m:20s]
  940. Jobs: 32 (f=32): [m(32)][46.8%][r=1570MiB/s,w=1567MiB/s][r=402k,w=401k IOPS][eta 05m:19s]
  941. Jobs: 32 (f=32): [m(32)][47.0%][r=1592MiB/s,w=1591MiB/s][r=408k,w=407k IOPS][eta 05m:18s]
  942. Jobs: 32 (f=32): [m(32)][47.2%][r=1564MiB/s,w=1564MiB/s][r=400k,w=400k IOPS][eta 05m:17s]
  943. Jobs: 32 (f=32): [m(32)][47.3%][r=1559MiB/s,w=1558MiB/s][r=399k,w=399k IOPS][eta 05m:16s]
  944. Jobs: 32 (f=32): [m(32)][47.5%][r=1561MiB/s,w=1557MiB/s][r=400k,w=399k IOPS][eta 05m:15s]
  945. Jobs: 32 (f=32): [m(32)][47.7%][r=1582MiB/s,w=1587MiB/s][r=405k,w=406k IOPS][eta 05m:14s]
  946. Jobs: 32 (f=32): [m(32)][47.8%][r=1566MiB/s,w=1558MiB/s][r=401k,w=399k IOPS][eta 05m:13s]
  947. Jobs: 32 (f=32): [m(32)][48.0%][r=1560MiB/s,w=1563MiB/s][r=399k,w=400k IOPS][eta 05m:12s]
  948. Jobs: 32 (f=32): [m(32)][48.2%][r=1559MiB/s,w=1566MiB/s][r=399k,w=401k IOPS][eta 05m:11s]
  949. Jobs: 32 (f=32): [m(32)][48.3%][r=1596MiB/s,w=1591MiB/s][r=409k,w=407k IOPS][eta 05m:10s]
  950. Jobs: 32 (f=32): [m(32)][48.5%][r=1563MiB/s,w=1564MiB/s][r=400k,w=400k IOPS][eta 05m:09s]
  951. Jobs: 32 (f=32): [m(32)][48.7%][r=1556MiB/s,w=1559MiB/s][r=398k,w=399k IOPS][eta 05m:08s]
  952. Jobs: 32 (f=32): [m(32)][48.8%][r=1561MiB/s,w=1560MiB/s][r=400k,w=399k IOPS][eta 05m:07s]
  953. Jobs: 32 (f=32): [m(32)][49.0%][r=1584MiB/s,w=1584MiB/s][r=406k,w=406k IOPS][eta 05m:06s]
  954. Jobs: 32 (f=32): [m(32)][49.2%][r=1579MiB/s,w=1578MiB/s][r=404k,w=404k IOPS][eta 05m:05s]
  955. Jobs: 32 (f=32): [m(32)][49.3%][r=1565MiB/s,w=1558MiB/s][r=401k,w=399k IOPS][eta 05m:04s]
  956. Jobs: 32 (f=32): [m(32)][49.5%][r=1572MiB/s,w=1576MiB/s][r=402k,w=403k IOPS][eta 05m:03s]
  957. Jobs: 32 (f=32): [m(32)][49.7%][r=1578MiB/s,w=1573MiB/s][r=404k,w=403k IOPS][eta 05m:02s]
  958. Jobs: 32 (f=32): [m(32)][49.8%][r=1595MiB/s,w=1596MiB/s][r=408k,w=409k IOPS][eta 05m:01s]
  959. Jobs: 32 (f=32): [m(32)][50.0%][r=1565MiB/s,w=1568MiB/s][r=401k,w=402k IOPS][eta 05m:00s]
  960. Jobs: 32 (f=32): [m(32)][50.2%][r=1567MiB/s,w=1559MiB/s][r=401k,w=399k IOPS][eta 04m:59s]
  961. Jobs: 32 (f=32): [m(32)][50.3%][r=1569MiB/s,w=1566MiB/s][r=402k,w=401k IOPS][eta 04m:58s]
  962. Jobs: 32 (f=32): [m(32)][50.5%][r=1585MiB/s,w=1588MiB/s][r=406k,w=407k IOPS][eta 04m:57s]
  963. Jobs: 32 (f=32): [m(32)][50.7%][r=1568MiB/s,w=1567MiB/s][r=401k,w=401k IOPS][eta 04m:56s]
  964. Jobs: 32 (f=32): [m(32)][50.8%][r=1563MiB/s,w=1564MiB/s][r=400k,w=401k IOPS][eta 04m:55s]
  965. Jobs: 32 (f=32): [m(32)][51.0%][r=1559MiB/s,w=1561MiB/s][r=399k,w=400k IOPS][eta 04m:54s]
  966. Jobs: 32 (f=32): [m(32)][51.2%][r=1592MiB/s,w=1589MiB/s][r=408k,w=407k IOPS][eta 04m:53s]
  967. Jobs: 32 (f=32): [m(32)][51.3%][r=1465MiB/s,w=1465MiB/s][r=375k,w=375k IOPS][eta 04m:52s]
  968. Jobs: 32 (f=32): [m(32)][51.5%][r=1499MiB/s,w=1497MiB/s][r=384k,w=383k IOPS][eta 04m:51s]
  969. Jobs: 32 (f=32): [m(32)][51.7%][r=1508MiB/s,w=1501MiB/s][r=386k,w=384k IOPS][eta 04m:50s]
  970. Jobs: 32 (f=32): [m(32)][51.8%][r=1550MiB/s,w=1549MiB/s][r=397k,w=397k IOPS][eta 04m:49s]
  971. Jobs: 32 (f=32): [m(32)][52.0%][r=1573MiB/s,w=1577MiB/s][r=403k,w=404k IOPS][eta 04m:48s]
  972. Jobs: 32 (f=32): [m(32)][52.2%][r=1456MiB/s,w=1451MiB/s][r=373k,w=371k IOPS][eta 04m:47s]
  973. Jobs: 32 (f=32): [m(32)][52.3%][r=1578MiB/s,w=1581MiB/s][r=404k,w=405k IOPS][eta 04m:46s]
  974. Jobs: 32 (f=32): [m(32)][52.5%][r=1569MiB/s,w=1568MiB/s][r=402k,w=401k IOPS][eta 04m:45s]
  975. Jobs: 32 (f=32): [m(32)][52.7%][r=1437MiB/s,w=1440MiB/s][r=368k,w=369k IOPS][eta 04m:44s]
  976. Jobs: 32 (f=32): [m(32)][52.8%][r=1444MiB/s,w=1443MiB/s][r=370k,w=369k IOPS][eta 04m:43s]
  977. Jobs: 32 (f=32): [m(32)][53.0%][r=1575MiB/s,w=1577MiB/s][r=403k,w=404k IOPS][eta 04m:42s]
  978. Jobs: 32 (f=32): [m(32)][53.2%][r=1401MiB/s,w=1405MiB/s][r=359k,w=360k IOPS][eta 04m:41s]
  979. Jobs: 32 (f=32): [m(32)][53.3%][r=1511MiB/s,w=1504MiB/s][r=387k,w=385k IOPS][eta 04m:40s]
  980. Jobs: 32 (f=32): [m(32)][53.5%][r=1445MiB/s,w=1443MiB/s][r=370k,w=369k IOPS][eta 04m:39s]
  981. Jobs: 32 (f=32): [m(32)][53.7%][r=1536MiB/s,w=1541MiB/s][r=393k,w=394k IOPS][eta 04m:38s]
  982. Jobs: 32 (f=32): [m(32)][53.8%][r=1524MiB/s,w=1526MiB/s][r=390k,w=391k IOPS][eta 04m:37s]
  983. Jobs: 32 (f=32): [m(32)][54.0%][r=1405MiB/s,w=1401MiB/s][r=360k,w=359k IOPS][eta 04m:36s]
  984. Jobs: 32 (f=32): [m(32)][54.2%][r=1587MiB/s,w=1582MiB/s][r=406k,w=405k IOPS][eta 04m:35s]
  985. Jobs: 32 (f=32): [m(32)][54.3%][r=1502MiB/s,w=1508MiB/s][r=384k,w=386k IOPS][eta 04m:34s]
  986. Jobs: 32 (f=32): [m(32)][54.5%][r=1402MiB/s,w=1403MiB/s][r=359k,w=359k IOPS][eta 04m:33s]
  987. Jobs: 32 (f=32): [m(32)][54.7%][r=1519MiB/s,w=1519MiB/s][r=389k,w=389k IOPS][eta 04m:32s]
  988. Jobs: 32 (f=32): [m(32)][54.8%][r=1599MiB/s,w=1598MiB/s][r=409k,w=409k IOPS][eta 04m:31s]
  989. Jobs: 32 (f=32): [m(32)][55.0%][r=1411MiB/s,w=1415MiB/s][r=361k,w=362k IOPS][eta 04m:30s]
  990. Jobs: 32 (f=32): [m(32)][55.2%][r=1521MiB/s,w=1519MiB/s][r=389k,w=389k IOPS][eta 04m:29s]
  991. Jobs: 32 (f=32): [m(32)][55.3%][r=1493MiB/s,w=1493MiB/s][r=382k,w=382k IOPS][eta 04m:28s]
  992. Jobs: 32 (f=32): [m(32)][55.5%][r=1508MiB/s,w=1509MiB/s][r=386k,w=386k IOPS][eta 04m:27s]
  993. Jobs: 32 (f=32): [m(32)][55.7%][r=1513MiB/s,w=1512MiB/s][r=387k,w=387k IOPS][eta 04m:26s]
  994. Jobs: 32 (f=32): [m(32)][55.8%][r=1401MiB/s,w=1400MiB/s][r=359k,w=359k IOPS][eta 04m:25s]
  995. Jobs: 32 (f=32): [m(32)][56.0%][r=1580MiB/s,w=1577MiB/s][r=404k,w=404k IOPS][eta 04m:24s]
  996. Jobs: 32 (f=32): [m(32)][56.2%][r=1521MiB/s,w=1519MiB/s][r=389k,w=389k IOPS][eta 04m:23s]
  997. Jobs: 32 (f=32): [m(32)][56.3%][r=1399MiB/s,w=1404MiB/s][r=358k,w=359k IOPS][eta 04m:22s]
  998. Jobs: 32 (f=32): [m(32)][56.5%][r=1503MiB/s,w=1508MiB/s][r=385k,w=386k IOPS][eta 04m:21s]
  999. Jobs: 32 (f=32): [m(32)][56.7%][r=1582MiB/s,w=1580MiB/s][r=405k,w=404k IOPS][eta 04m:20s]
  1000. Jobs: 32 (f=32): [m(32)][56.8%][r=1416MiB/s,w=1416MiB/s][r=363k,w=362k IOPS][eta 04m:19s]
  1001. Jobs: 32 (f=32): [m(32)][57.0%][r=1503MiB/s,w=1505MiB/s][r=385k,w=385k IOPS][eta 04m:18s]
  1002. Jobs: 32 (f=32): [m(32)][57.2%][r=1543MiB/s,w=1551MiB/s][r=395k,w=397k IOPS][eta 04m:17s]
  1003. Jobs: 32 (f=32): [m(32)][57.3%][r=1496MiB/s,w=1490MiB/s][r=383k,w=381k IOPS][eta 04m:16s]
  1004. Jobs: 32 (f=32): [m(32)][57.5%][r=1501MiB/s,w=1504MiB/s][r=384k,w=385k IOPS][eta 04m:15s]
  1005. Jobs: 32 (f=32): [m(32)][57.7%][r=1444MiB/s,w=1448MiB/s][r=370k,w=371k IOPS][eta 04m:14s]
  1006. Jobs: 32 (f=32): [m(32)][57.8%][r=1574MiB/s,w=1582MiB/s][r=403k,w=405k IOPS][eta 04m:13s]
  1007. Jobs: 32 (f=32): [m(32)][58.0%][r=1514MiB/s,w=1512MiB/s][r=388k,w=387k IOPS][eta 04m:12s]
  1008. Jobs: 32 (f=32): [m(32)][58.2%][r=1392MiB/s,w=1397MiB/s][r=356k,w=358k IOPS][eta 04m:11s]
  1009. Jobs: 32 (f=32): [m(32)][58.3%][r=1501MiB/s,w=1501MiB/s][r=384k,w=384k IOPS][eta 04m:10s]
  1010. Jobs: 32 (f=32): [m(32)][58.5%][r=1580MiB/s,w=1587MiB/s][r=404k,w=406k IOPS][eta 04m:09s]
  1011. Jobs: 32 (f=32): [m(32)][58.7%][r=1404MiB/s,w=1403MiB/s][r=359k,w=359k IOPS][eta 04m:08s]
  1012. Jobs: 32 (f=32): [m(32)][58.8%][r=1506MiB/s,w=1503MiB/s][r=385k,w=385k IOPS][eta 04m:07s]
  1013. Jobs: 32 (f=32): [m(32)][59.0%][r=1586MiB/s,w=1581MiB/s][r=406k,w=405k IOPS][eta 04m:06s]
  1014. Jobs: 32 (f=32): [m(32)][59.2%][r=1410MiB/s,w=1410MiB/s][r=361k,w=361k IOPS][eta 04m:05s]
  1015. Jobs: 32 (f=32): [m(32)][59.3%][r=1518MiB/s,w=1521MiB/s][r=389k,w=389k IOPS][eta 04m:04s]
  1016. Jobs: 32 (f=32): [m(32)][59.5%][r=1404MiB/s,w=1396MiB/s][r=359k,w=357k IOPS][eta 04m:03s]
  1017. Jobs: 32 (f=32): [m(32)][59.7%][r=1588MiB/s,w=1590MiB/s][r=407k,w=407k IOPS][eta 04m:02s]
  1018. Jobs: 32 (f=32): [m(32)][59.8%][r=1511MiB/s,w=1510MiB/s][r=387k,w=386k IOPS][eta 04m:01s]
  1019. Jobs: 32 (f=32): [m(32)][60.0%][r=1404MiB/s,w=1402MiB/s][r=359k,w=359k IOPS][eta 04m:00s]
  1020. Jobs: 32 (f=32): [m(32)][60.2%][r=1544MiB/s,w=1540MiB/s][r=395k,w=394k IOPS][eta 03m:59s]
  1021. Jobs: 32 (f=32): [m(32)][60.3%][r=1560MiB/s,w=1558MiB/s][r=399k,w=399k IOPS][eta 03m:58s]
  1022. Jobs: 32 (f=32): [m(32)][60.5%][r=1398MiB/s,w=1396MiB/s][r=358k,w=357k IOPS][eta 03m:57s]
  1023. Jobs: 32 (f=32): [m(32)][60.7%][r=1508MiB/s,w=1507MiB/s][r=386k,w=386k IOPS][eta 03m:56s]
  1024. Jobs: 32 (f=32): [m(32)][60.8%][r=1573MiB/s,w=1576MiB/s][r=403k,w=404k IOPS][eta 03m:55s]
  1025. Jobs: 32 (f=32): [m(32)][61.0%][r=1407MiB/s,w=1408MiB/s][r=360k,w=360k IOPS][eta 03m:54s]
  1026. Jobs: 32 (f=32): [m(32)][61.2%][r=1507MiB/s,w=1506MiB/s][r=386k,w=385k IOPS][eta 03m:53s]
  1027. Jobs: 32 (f=32): [m(32)][61.3%][r=1433MiB/s,w=1436MiB/s][r=367k,w=368k IOPS][eta 03m:52s]
  1028. Jobs: 32 (f=32): [m(32)][61.5%][r=1564MiB/s,w=1557MiB/s][r=400k,w=399k IOPS][eta 03m:51s]
  1029. Jobs: 32 (f=32): [m(32)][61.7%][r=1510MiB/s,w=1509MiB/s][r=387k,w=386k IOPS][eta 03m:50s]
  1030. Jobs: 32 (f=32): [m(32)][61.8%][r=1455MiB/s,w=1457MiB/s][r=373k,w=373k IOPS][eta 03m:49s]
  1031. Jobs: 32 (f=32): [m(32)][62.0%][r=1584MiB/s,w=1580MiB/s][r=405k,w=405k IOPS][eta 03m:48s]
  1032. Jobs: 32 (f=32): [m(32)][62.2%][r=1514MiB/s,w=1519MiB/s][r=388k,w=389k IOPS][eta 03m:47s]
  1033. Jobs: 32 (f=32): [m(32)][62.3%][r=1400MiB/s,w=1407MiB/s][r=358k,w=360k IOPS][eta 03m:46s]
  1034. Jobs: 32 (f=32): [m(32)][62.5%][r=1497MiB/s,w=1502MiB/s][r=383k,w=384k IOPS][eta 03m:45s]
  1035. Jobs: 32 (f=32): [m(32)][62.7%][r=1584MiB/s,w=1582MiB/s][r=406k,w=405k IOPS][eta 03m:44s]
  1036. Jobs: 32 (f=32): [m(32)][62.8%][r=1410MiB/s,w=1406MiB/s][r=361k,w=360k IOPS][eta 03m:43s]
  1037. Jobs: 32 (f=32): [m(32)][63.0%][r=1495MiB/s,w=1498MiB/s][r=383k,w=383k IOPS][eta 03m:42s]
  1038. Jobs: 32 (f=32): [m(32)][63.2%][r=1475MiB/s,w=1469MiB/s][r=378k,w=376k IOPS][eta 03m:41s]
  1039. Jobs: 32 (f=32): [m(32)][63.3%][r=1475MiB/s,w=1477MiB/s][r=378k,w=378k IOPS][eta 03m:40s]
  1040. Jobs: 32 (f=32): [m(32)][63.5%][r=1499MiB/s,w=1505MiB/s][r=384k,w=385k IOPS][eta 03m:39s]
  1041. Jobs: 32 (f=32): [m(32)][63.7%][r=1371MiB/s,w=1370MiB/s][r=351k,w=351k IOPS][eta 03m:38s]
  1042. Jobs: 32 (f=32): [m(32)][63.8%][r=1582MiB/s,w=1572MiB/s][r=405k,w=402k IOPS][eta 03m:37s]
  1043. Jobs: 32 (f=32): [m(32)][64.0%][r=1511MiB/s,w=1503MiB/s][r=387k,w=385k IOPS][eta 03m:36s]
  1044. Jobs: 32 (f=32): [m(32)][64.2%][r=1374MiB/s,w=1379MiB/s][r=352k,w=353k IOPS][eta 03m:35s]
  1045. Jobs: 32 (f=32): [m(32)][64.3%][r=1519MiB/s,w=1515MiB/s][r=389k,w=388k IOPS][eta 03m:34s]
  1046. Jobs: 32 (f=32): [m(32)][64.5%][r=1573MiB/s,w=1576MiB/s][r=403k,w=403k IOPS][eta 03m:33s]
  1047. Jobs: 32 (f=32): [m(32)][64.7%][r=1449MiB/s,w=1448MiB/s][r=371k,w=371k IOPS][eta 03m:32s]
  1048. Jobs: 32 (f=32): [m(32)][64.8%][r=1501MiB/s,w=1506MiB/s][r=384k,w=385k IOPS][eta 03m:31s]
  1049. Jobs: 32 (f=32): [m(32)][65.0%][r=1573MiB/s,w=1574MiB/s][r=403k,w=403k IOPS][eta 03m:30s]
  1050. Jobs: 32 (f=32): [m(32)][65.2%][r=1414MiB/s,w=1416MiB/s][r=362k,w=363k IOPS][eta 03m:29s]
  1051. Jobs: 32 (f=32): [m(32)][65.3%][r=1514MiB/s,w=1517MiB/s][r=388k,w=388k IOPS][eta 03m:28s]
  1052. Jobs: 32 (f=32): [m(32)][65.5%][r=1405MiB/s,w=1403MiB/s][r=360k,w=359k IOPS][eta 03m:27s]
  1053. Jobs: 32 (f=32): [m(32)][65.7%][r=1576MiB/s,w=1576MiB/s][r=403k,w=404k IOPS][eta 03m:26s]
  1054. Jobs: 32 (f=32): [m(32)][65.8%][r=1468MiB/s,w=1467MiB/s][r=376k,w=376k IOPS][eta 03m:25s]
  1055. Jobs: 32 (f=32): [m(32)][66.0%][r=1449MiB/s,w=1456MiB/s][r=371k,w=373k IOPS][eta 03m:24s]
  1056. Jobs: 32 (f=32): [m(32)][66.2%][r=1571MiB/s,w=1567MiB/s][r=402k,w=401k IOPS][eta 03m:23s]
  1057. Jobs: 32 (f=32): [m(32)][66.3%][r=1469MiB/s,w=1474MiB/s][r=376k,w=377k IOPS][eta 03m:22s]
  1058. Jobs: 32 (f=32): [m(32)][66.5%][r=1463MiB/s,w=1462MiB/s][r=375k,w=374k IOPS][eta 03m:21s]
  1059. Jobs: 32 (f=32): [m(32)][66.7%][r=1455MiB/s,w=1454MiB/s][r=372k,w=372k IOPS][eta 03m:20s]
  1060. Jobs: 32 (f=32): [m(32)][66.9%][r=1579MiB/s,w=1580MiB/s][r=404k,w=405k IOPS][eta 03m:19s]
  1061. Jobs: 32 (f=32): [m(32)][67.1%][r=1403MiB/s,w=1404MiB/s][r=359k,w=359k IOPS][eta 03m:18s]
  1062. Jobs: 32 (f=32): [m(32)][67.2%][r=1502MiB/s,w=1508MiB/s][r=385k,w=386k IOPS][eta 03m:17s]
  1063. Jobs: 32 (f=32): [m(32)][67.4%][r=1472MiB/s,w=1476MiB/s][r=377k,w=378k IOPS][eta 03m:16s]
  1064. Jobs: 32 (f=32): [m(32)][67.6%][r=1569MiB/s,w=1570MiB/s][r=402k,w=402k IOPS][eta 03m:15s]
  1065. Jobs: 32 (f=32): [m(32)][67.7%][r=1503MiB/s,w=1499MiB/s][r=385k,w=384k IOPS][eta 03m:14s]
  1066. Jobs: 32 (f=32): [m(32)][67.9%][r=1463MiB/s,w=1465MiB/s][r=375k,w=375k IOPS][eta 03m:13s]
  1067. Jobs: 32 (f=32): [m(32)][68.1%][r=1574MiB/s,w=1574MiB/s][r=403k,w=403k IOPS][eta 03m:12s]
  1068. Jobs: 32 (f=32): [m(32)][68.2%][r=1473MiB/s,w=1472MiB/s][r=377k,w=377k IOPS][eta 03m:11s]
  1069. Jobs: 32 (f=32): [m(32)][68.4%][r=1465MiB/s,w=1462MiB/s][r=375k,w=374k IOPS][eta 03m:10s]
  1070. Jobs: 32 (f=32): [m(32)][68.6%][r=1457MiB/s,w=1452MiB/s][r=373k,w=372k IOPS][eta 03m:09s]
  1071. Jobs: 32 (f=32): [m(32)][68.7%][r=1577MiB/s,w=1571MiB/s][r=404k,w=402k IOPS][eta 03m:08s]
  1072. Jobs: 32 (f=32): [m(32)][68.9%][r=1455MiB/s,w=1452MiB/s][r=373k,w=372k IOPS][eta 03m:07s]
  1073. Jobs: 32 (f=32): [m(32)][69.1%][r=1451MiB/s,w=1455MiB/s][r=371k,w=372k IOPS][eta 03m:06s]
  1074. Jobs: 32 (f=32): [m(32)][69.2%][r=1553MiB/s,w=1553MiB/s][r=398k,w=398k IOPS][eta 03m:05s]
  1075. Jobs: 32 (f=32): [m(32)][69.4%][r=1514MiB/s,w=1515MiB/s][r=387k,w=388k IOPS][eta 03m:04s]
  1076. Jobs: 32 (f=32): [m(32)][69.6%][r=1442MiB/s,w=1441MiB/s][r=369k,w=369k IOPS][eta 03m:03s]
  1077. Jobs: 32 (f=32): [m(32)][69.7%][r=1486MiB/s,w=1492MiB/s][r=380k,w=382k IOPS][eta 03m:02s]
  1078. Jobs: 32 (f=32): [m(32)][69.9%][r=1578MiB/s,w=1580MiB/s][r=404k,w=404k IOPS][eta 03m:01s]
  1079. Jobs: 32 (f=32): [m(32)][70.0%][r=1447MiB/s,w=1451MiB/s][r=371k,w=372k IOPS][eta 03m:00s]
  1080. Jobs: 32 (f=32): [m(32)][70.2%][r=1457MiB/s,w=1460MiB/s][r=373k,w=374k IOPS][eta 02m:59s]
  1081. Jobs: 32 (f=32): [m(32)][70.4%][r=1447MiB/s,w=1442MiB/s][r=371k,w=369k IOPS][eta 02m:58s]
  1082. Jobs: 32 (f=32): [m(32)][70.5%][r=1578MiB/s,w=1578MiB/s][r=404k,w=404k IOPS][eta 02m:57s]
  1083. Jobs: 32 (f=32): [m(32)][70.7%][r=1451MiB/s,w=1454MiB/s][r=371k,w=372k IOPS][eta 02m:56s]
  1084. Jobs: 32 (f=32): [m(32)][70.9%][r=1450MiB/s,w=1451MiB/s][r=371k,w=372k IOPS][eta 02m:55s]
  1085. Jobs: 32 (f=32): [m(32)][71.0%][r=1568MiB/s,w=1568MiB/s][r=401k,w=401k IOPS][eta 02m:54s]
  1086. Jobs: 32 (f=32): [m(32)][71.2%][r=1452MiB/s,w=1455MiB/s][r=372k,w=373k IOPS][eta 02m:53s]
  1087. Jobs: 32 (f=32): [m(32)][71.4%][r=1448MiB/s,w=1451MiB/s][r=371k,w=371k IOPS][eta 02m:52s]
  1088. Jobs: 32 (f=32): [m(32)][71.5%][r=1397MiB/s,w=1398MiB/s][r=358k,w=358k IOPS][eta 02m:51s]
  1089. Jobs: 32 (f=32): [m(32)][71.7%][r=1576MiB/s,w=1574MiB/s][r=404k,w=403k IOPS][eta 02m:50s]
  1090. Jobs: 32 (f=32): [m(32)][71.9%][r=1442MiB/s,w=1449MiB/s][r=369k,w=371k IOPS][eta 02m:49s]
  1091. Jobs: 32 (f=32): [m(32)][72.0%][r=1416MiB/s,w=1411MiB/s][r=363k,w=361k IOPS][eta 02m:48s]
  1092. Jobs: 32 (f=32): [m(32)][72.2%][r=1578MiB/s,w=1572MiB/s][r=404k,w=403k IOPS][eta 02m:47s]
  1093. Jobs: 32 (f=32): [m(32)][72.4%][r=1449MiB/s,w=1454MiB/s][r=371k,w=372k IOPS][eta 02m:46s]
  1094. Jobs: 32 (f=32): [m(32)][72.5%][r=1459MiB/s,w=1462MiB/s][r=374k,w=374k IOPS][eta 02m:45s]
  1095. Jobs: 32 (f=32): [m(32)][72.7%][r=1448MiB/s,w=1446MiB/s][r=371k,w=370k IOPS][eta 02m:44s]
  1096. Jobs: 32 (f=32): [m(32)][72.9%][r=1574MiB/s,w=1577MiB/s][r=403k,w=404k IOPS][eta 02m:43s]
  1097. Jobs: 32 (f=32): [m(32)][73.0%][r=1436MiB/s,w=1441MiB/s][r=368k,w=369k IOPS][eta 02m:42s]
  1098. Jobs: 32 (f=32): [m(32)][73.2%][r=1447MiB/s,w=1448MiB/s][r=370k,w=371k IOPS][eta 02m:41s]
  1099. Jobs: 32 (f=32): [m(32)][73.4%][r=1563MiB/s,w=1566MiB/s][r=400k,w=401k IOPS][eta 02m:40s]
  1100. Jobs: 32 (f=32): [m(32)][73.5%][r=1460MiB/s,w=1462MiB/s][r=374k,w=374k IOPS][eta 02m:39s]
  1101. Jobs: 32 (f=32): [m(32)][73.7%][r=1445MiB/s,w=1451MiB/s][r=370k,w=371k IOPS][eta 02m:38s]
  1102. Jobs: 32 (f=32): [m(32)][73.9%][r=1440MiB/s,w=1444MiB/s][r=369k,w=370k IOPS][eta 02m:37s]
  1103. Jobs: 32 (f=32): [m(32)][74.0%][r=1582MiB/s,w=1576MiB/s][r=405k,w=404k IOPS][eta 02m:36s]
  1104. Jobs: 32 (f=32): [m(32)][74.2%][r=1496MiB/s,w=1504MiB/s][r=383k,w=385k IOPS][eta 02m:35s]
  1105. Jobs: 32 (f=32): [m(32)][74.4%][r=1452MiB/s,w=1450MiB/s][r=372k,w=371k IOPS][eta 02m:34s]
  1106. Jobs: 32 (f=32): [m(32)][74.5%][r=1507MiB/s,w=1506MiB/s][r=386k,w=385k IOPS][eta 02m:33s]
  1107. Jobs: 32 (f=32): [m(32)][74.7%][r=1569MiB/s,w=1576MiB/s][r=402k,w=403k IOPS][eta 02m:32s]
  1108. Jobs: 32 (f=32): [m(32)][74.9%][r=1452MiB/s,w=1452MiB/s][r=372k,w=372k IOPS][eta 02m:31s]
  1109. Jobs: 32 (f=32): [m(32)][75.0%][r=1499MiB/s,w=1503MiB/s][r=384k,w=385k IOPS][eta 02m:30s]
  1110. Jobs: 32 (f=32): [m(32)][75.2%][r=1580MiB/s,w=1579MiB/s][r=405k,w=404k IOPS][eta 02m:29s]
  1111. Jobs: 32 (f=32): [m(32)][75.4%][r=1455MiB/s,w=1459MiB/s][r=372k,w=374k IOPS][eta 02m:28s]
  1112. Jobs: 32 (f=32): [m(32)][75.5%][r=1504MiB/s,w=1505MiB/s][r=385k,w=385k IOPS][eta 02m:27s]
  1113. Jobs: 32 (f=32): [m(32)][75.7%][r=1449MiB/s,w=1439MiB/s][r=371k,w=368k IOPS][eta 02m:26s]
  1114. Jobs: 32 (f=32): [m(32)][75.9%][r=1561MiB/s,w=1566MiB/s][r=400k,w=401k IOPS][eta 02m:25s]
  1115. Jobs: 32 (f=32): [m(32)][76.0%][r=1453MiB/s,w=1449MiB/s][r=372k,w=371k IOPS][eta 02m:24s]
  1116. Jobs: 32 (f=32): [m(32)][76.2%][r=1431MiB/s,w=1435MiB/s][r=366k,w=367k IOPS][eta 02m:23s]
  1117. Jobs: 32 (f=32): [m(32)][76.4%][r=1523MiB/s,w=1525MiB/s][r=390k,w=390k IOPS][eta 02m:22s]
  1118. Jobs: 32 (f=32): [m(32)][76.5%][r=1536MiB/s,w=1538MiB/s][r=393k,w=394k IOPS][eta 02m:21s]
  1119. Jobs: 32 (f=32): [m(32)][76.7%][r=1453MiB/s,w=1452MiB/s][r=372k,w=372k IOPS][eta 02m:20s]
  1120. Jobs: 32 (f=32): [m(32)][76.9%][r=1457MiB/s,w=1458MiB/s][r=373k,w=373k IOPS][eta 02m:19s]
  1121. Jobs: 32 (f=32): [m(32)][77.0%][r=1569MiB/s,w=1566MiB/s][r=402k,w=401k IOPS][eta 02m:18s]
  1122. Jobs: 32 (f=32): [m(32)][77.2%][r=1453MiB/s,w=1453MiB/s][r=372k,w=372k IOPS][eta 02m:17s]
  1123. Jobs: 32 (f=32): [m(32)][77.4%][r=1448MiB/s,w=1447MiB/s][r=371k,w=370k IOPS][eta 02m:16s]
  1124. Jobs: 32 (f=32): [m(32)][77.5%][r=1433MiB/s,w=1431MiB/s][r=367k,w=366k IOPS][eta 02m:15s]
  1125. Jobs: 32 (f=32): [m(32)][77.7%][r=1551MiB/s,w=1547MiB/s][r=397k,w=396k IOPS][eta 02m:14s]
  1126. Jobs: 32 (f=32): [m(32)][77.9%][r=1444MiB/s,w=1447MiB/s][r=370k,w=370k IOPS][eta 02m:13s]
  1127. Jobs: 32 (f=32): [m(32)][78.0%][r=1412MiB/s,w=1410MiB/s][r=361k,w=361k IOPS][eta 02m:12s]
  1128. Jobs: 32 (f=32): [m(32)][78.2%][r=1575MiB/s,w=1578MiB/s][r=403k,w=404k IOPS][eta 02m:11s]
  1129. Jobs: 32 (f=32): [m(32)][78.4%][r=1428MiB/s,w=1429MiB/s][r=366k,w=366k IOPS][eta 02m:10s]
  1130. Jobs: 32 (f=32): [m(32)][78.5%][r=1450MiB/s,w=1455MiB/s][r=371k,w=373k IOPS][eta 02m:09s]
  1131. Jobs: 32 (f=32): [m(32)][78.7%][r=1477MiB/s,w=1478MiB/s][r=378k,w=378k IOPS][eta 02m:08s]
  1132. Jobs: 32 (f=32): [m(32)][78.9%][r=1504MiB/s,w=1511MiB/s][r=385k,w=387k IOPS][eta 02m:07s]
  1133. Jobs: 32 (f=32): [m(32)][79.0%][r=1441MiB/s,w=1441MiB/s][r=369k,w=369k IOPS][eta 02m:06s]
  1134. Jobs: 32 (f=32): [m(32)][79.3%][r=1287MiB/s,w=1288MiB/s][r=329k,w=330k IOPS][eta 02m:04s]
  1135. Jobs: 32 (f=32): [m(32)][79.5%][r=1510MiB/s,w=1510MiB/s][r=387k,w=387k IOPS][eta 02m:03s]
  1136. Jobs: 32 (f=32): [m(32)][79.7%][r=1412MiB/s,w=1407MiB/s][r=362k,w=360k IOPS][eta 02m:02s]
  1137. Jobs: 32 (f=32): [m(32)][79.8%][r=1329MiB/s,w=1329MiB/s][r=340k,w=340k IOPS][eta 02m:01s]
  1138. Jobs: 32 (f=32): [m(32)][80.0%][r=1469MiB/s,w=1468MiB/s][r=376k,w=376k IOPS][eta 02m:00s]
  1139. Jobs: 32 (f=32): [m(32)][80.2%][r=1337MiB/s,w=1335MiB/s][r=342k,w=342k IOPS][eta 01m:59s]
  1140. Jobs: 32 (f=32): [m(32)][80.3%][r=1322MiB/s,w=1323MiB/s][r=338k,w=339k IOPS][eta 01m:58s]
  1141. Jobs: 32 (f=32): [m(32)][80.5%][r=1512MiB/s,w=1510MiB/s][r=387k,w=386k IOPS][eta 01m:57s]
  1142. Jobs: 32 (f=32): [m(32)][80.7%][r=1393MiB/s,w=1388MiB/s][r=357k,w=355k IOPS][eta 01m:56s]
  1143. Jobs: 32 (f=32): [m(32)][80.8%][r=1349MiB/s,w=1345MiB/s][r=345k,w=344k IOPS][eta 01m:55s]
  1144. Jobs: 32 (f=32): [m(32)][81.0%][r=1401MiB/s,w=1407MiB/s][r=359k,w=360k IOPS][eta 01m:54s]
  1145. Jobs: 32 (f=32): [m(32)][81.2%][r=1361MiB/s,w=1362MiB/s][r=348k,w=349k IOPS][eta 01m:53s]
  1146. Jobs: 32 (f=32): [m(32)][81.3%][r=1370MiB/s,w=1374MiB/s][r=351k,w=352k IOPS][eta 01m:52s]
  1147. Jobs: 32 (f=32): [m(32)][81.5%][r=1489MiB/s,w=1491MiB/s][r=381k,w=382k IOPS][eta 01m:51s]
  1148. Jobs: 32 (f=32): [m(32)][81.7%][r=1408MiB/s,w=1406MiB/s][r=360k,w=360k IOPS][eta 01m:50s]
  1149. Jobs: 32 (f=32): [m(32)][81.8%][r=1373MiB/s,w=1369MiB/s][r=351k,w=350k IOPS][eta 01m:49s]
  1150. Jobs: 32 (f=32): [m(32)][82.0%][r=1452MiB/s,w=1461MiB/s][r=372k,w=374k IOPS][eta 01m:48s]
  1151. Jobs: 32 (f=32): [m(32)][82.2%][r=1402MiB/s,w=1405MiB/s][r=359k,w=360k IOPS][eta 01m:47s]
  1152. Jobs: 32 (f=32): [m(32)][82.3%][r=1355MiB/s,w=1359MiB/s][r=347k,w=348k IOPS][eta 01m:46s]
  1153. Jobs: 32 (f=32): [m(32)][82.5%][r=1449MiB/s,w=1445MiB/s][r=371k,w=370k IOPS][eta 01m:45s]
  1154. Jobs: 32 (f=32): [m(32)][82.7%][r=1383MiB/s,w=1382MiB/s][r=354k,w=354k IOPS][eta 01m:44s]
  1155. Jobs: 32 (f=32): [m(32)][82.8%][r=1317MiB/s,w=1321MiB/s][r=337k,w=338k IOPS][eta 01m:43s]
  1156. Jobs: 32 (f=32): [m(32)][83.0%][r=1453MiB/s,w=1448MiB/s][r=372k,w=371k IOPS][eta 01m:42s]
  1157. Jobs: 32 (f=32): [m(32)][83.2%][r=1449MiB/s,w=1452MiB/s][r=371k,w=372k IOPS][eta 01m:41s]
  1158. Jobs: 32 (f=32): [m(32)][83.3%][r=1319MiB/s,w=1316MiB/s][r=338k,w=337k IOPS][eta 01m:40s]
  1159. Jobs: 32 (f=32): [m(32)][83.5%][r=1373MiB/s,w=1372MiB/s][r=352k,w=351k IOPS][eta 01m:39s]
  1160. Jobs: 32 (f=32): [m(32)][83.7%][r=1373MiB/s,w=1381MiB/s][r=352k,w=354k IOPS][eta 01m:38s]
  1161. Jobs: 32 (f=32): [m(32)][83.8%][r=1314MiB/s,w=1317MiB/s][r=336k,w=337k IOPS][eta 01m:37s]
  1162. Jobs: 32 (f=32): [m(32)][84.0%][r=1412MiB/s,w=1408MiB/s][r=361k,w=361k IOPS][eta 01m:36s]
  1163. Jobs: 32 (f=32): [m(32)][84.2%][r=1362MiB/s,w=1361MiB/s][r=349k,w=349k IOPS][eta 01m:35s]
  1164. Jobs: 32 (f=32): [m(32)][84.3%][r=1279MiB/s,w=1277MiB/s][r=327k,w=327k IOPS][eta 01m:34s]
  1165. Jobs: 32 (f=32): [m(32)][84.5%][r=1371MiB/s,w=1375MiB/s][r=351k,w=352k IOPS][eta 01m:33s]
  1166. Jobs: 32 (f=32): [m(32)][84.7%][r=1367MiB/s,w=1367MiB/s][r=350k,w=350k IOPS][eta 01m:32s]
  1167. Jobs: 32 (f=32): [m(32)][84.8%][r=1308MiB/s,w=1307MiB/s][r=335k,w=335k IOPS][eta 01m:31s]
  1168. Jobs: 32 (f=32): [m(32)][85.0%][r=1401MiB/s,w=1397MiB/s][r=359k,w=358k IOPS][eta 01m:30s]
  1169. Jobs: 32 (f=32): [m(32)][85.2%][r=1357MiB/s,w=1355MiB/s][r=347k,w=347k IOPS][eta 01m:29s]
  1170. Jobs: 32 (f=32): [m(32)][85.3%][r=1326MiB/s,w=1319MiB/s][r=339k,w=338k IOPS][eta 01m:28s]
  1171. Jobs: 32 (f=32): [m(32)][85.5%][r=1449MiB/s,w=1452MiB/s][r=371k,w=372k IOPS][eta 01m:27s]
  1172. Jobs: 32 (f=32): [m(32)][85.7%][r=1408MiB/s,w=1409MiB/s][r=361k,w=361k IOPS][eta 01m:26s]
  1173. Jobs: 32 (f=32): [m(32)][85.8%][r=1312MiB/s,w=1309MiB/s][r=336k,w=335k IOPS][eta 01m:25s]
  1174. Jobs: 32 (f=32): [m(32)][86.0%][r=1426MiB/s,w=1423MiB/s][r=365k,w=364k IOPS][eta 01m:24s]
  1175. Jobs: 32 (f=32): [m(32)][86.2%][r=1316MiB/s,w=1317MiB/s][r=337k,w=337k IOPS][eta 01m:23s]
  1176. Jobs: 32 (f=32): [m(32)][86.3%][r=1309MiB/s,w=1311MiB/s][r=335k,w=336k IOPS][eta 01m:22s]
  1177. Jobs: 32 (f=32): [m(32)][86.5%][r=1407MiB/s,w=1415MiB/s][r=360k,w=362k IOPS][eta 01m:21s]
  1178. Jobs: 32 (f=32): [m(32)][86.7%][r=1276MiB/s,w=1274MiB/s][r=327k,w=326k IOPS][eta 01m:20s]
  1179. Jobs: 32 (f=32): [m(32)][86.8%][r=1295MiB/s,w=1293MiB/s][r=332k,w=331k IOPS][eta 01m:19s]
  1180. Jobs: 32 (f=32): [m(32)][87.0%][r=1424MiB/s,w=1428MiB/s][r=365k,w=365k IOPS][eta 01m:18s]
  1181. Jobs: 32 (f=32): [m(32)][87.2%][r=1313MiB/s,w=1313MiB/s][r=336k,w=336k IOPS][eta 01m:17s]
  1182. Jobs: 32 (f=32): [m(32)][87.3%][r=1297MiB/s,w=1292MiB/s][r=332k,w=331k IOPS][eta 01m:16s]
  1183. Jobs: 32 (f=32): [m(32)][87.5%][r=1390MiB/s,w=1394MiB/s][r=356k,w=357k IOPS][eta 01m:15s]
  1184. Jobs: 32 (f=32): [m(32)][87.7%][r=1314MiB/s,w=1317MiB/s][r=336k,w=337k IOPS][eta 01m:14s]
  1185. Jobs: 32 (f=32): [m(32)][87.8%][r=1364MiB/s,w=1360MiB/s][r=349k,w=348k IOPS][eta 01m:13s]
  1186. Jobs: 32 (f=32): [m(32)][88.0%][r=1386MiB/s,w=1385MiB/s][r=355k,w=354k IOPS][eta 01m:12s]
  1187. Jobs: 32 (f=32): [m(32)][88.2%][r=1305MiB/s,w=1307MiB/s][r=334k,w=335k IOPS][eta 01m:11s]
  1188. Jobs: 32 (f=32): [m(32)][88.3%][r=1376MiB/s,w=1376MiB/s][r=352k,w=352k IOPS][eta 01m:10s]
  1189. Jobs: 32 (f=32): [m(32)][88.5%][r=1374MiB/s,w=1379MiB/s][r=352k,w=353k IOPS][eta 01m:09s]
  1190. Jobs: 32 (f=32): [m(32)][88.7%][r=1306MiB/s,w=1304MiB/s][r=334k,w=334k IOPS][eta 01m:08s]
  1191. Jobs: 32 (f=32): [m(32)][88.8%][r=1385MiB/s,w=1387MiB/s][r=355k,w=355k IOPS][eta 01m:07s]
  1192. Jobs: 32 (f=32): [m(32)][89.0%][r=1264MiB/s,w=1264MiB/s][r=324k,w=323k IOPS][eta 01m:06s]
  1193. Jobs: 32 (f=32): [m(32)][89.2%][r=1311MiB/s,w=1311MiB/s][r=336k,w=336k IOPS][eta 01m:05s]
  1194. Jobs: 32 (f=32): [m(32)][89.3%][r=1399MiB/s,w=1398MiB/s][r=358k,w=358k IOPS][eta 01m:04s]
  1195. Jobs: 32 (f=32): [m(32)][89.5%][r=1262MiB/s,w=1262MiB/s][r=323k,w=323k IOPS][eta 01m:03s]
  1196. Jobs: 32 (f=32): [m(32)][89.7%][r=1284MiB/s,w=1283MiB/s][r=329k,w=329k IOPS][eta 01m:02s]
  1197. Jobs: 32 (f=32): [m(32)][89.8%][r=1272MiB/s,w=1270MiB/s][r=326k,w=325k IOPS][eta 01m:01s]
  1198. Jobs: 32 (f=32): [m(32)][90.0%][r=1213MiB/s,w=1216MiB/s][r=311k,w=311k IOPS][eta 01m:00s]
  1199. Jobs: 32 (f=32): [m(32)][90.2%][r=1289MiB/s,w=1287MiB/s][r=330k,w=330k IOPS][eta 00m:59s]
  1200. Jobs: 32 (f=32): [m(32)][90.3%][r=1225MiB/s,w=1222MiB/s][r=313k,w=313k IOPS][eta 00m:58s]
  1201. Jobs: 32 (f=32): [m(32)][90.5%][r=1193MiB/s,w=1196MiB/s][r=305k,w=306k IOPS][eta 00m:57s]
  1202. Jobs: 32 (f=32): [m(32)][90.7%][r=1280MiB/s,w=1283MiB/s][r=328k,w=328k IOPS][eta 00m:56s]
  1203. Jobs: 32 (f=32): [m(32)][90.8%][r=1225MiB/s,w=1219MiB/s][r=314k,w=312k IOPS][eta 00m:55s]
  1204. Jobs: 32 (f=32): [m(32)][91.0%][r=1290MiB/s,w=1290MiB/s][r=330k,w=330k IOPS][eta 00m:54s]
  1205. Jobs: 32 (f=32): [m(32)][91.2%][r=1147MiB/s,w=1143MiB/s][r=294k,w=293k IOPS][eta 00m:53s]
  1206. Jobs: 32 (f=32): [m(32)][91.3%][r=1226MiB/s,w=1225MiB/s][r=314k,w=314k IOPS][eta 00m:52s]
  1207. Jobs: 32 (f=32): [m(32)][91.5%][r=1274MiB/s,w=1272MiB/s][r=326k,w=326k IOPS][eta 00m:51s]
  1208. Jobs: 32 (f=32): [m(32)][91.7%][r=1142MiB/s,w=1144MiB/s][r=292k,w=293k IOPS][eta 00m:50s]
  1209. Jobs: 32 (f=32): [m(32)][91.8%][r=1249MiB/s,w=1253MiB/s][r=320k,w=321k IOPS][eta 00m:49s]
  1210. Jobs: 32 (f=32): [m(32)][92.0%][r=1171MiB/s,w=1171MiB/s][r=300k,w=300k IOPS][eta 00m:48s]
  1211. Jobs: 32 (f=32): [m(32)][92.2%][r=1173MiB/s,w=1176MiB/s][r=300k,w=301k IOPS][eta 00m:47s]
  1212. Jobs: 32 (f=32): [m(32)][92.3%][r=1233MiB/s,w=1235MiB/s][r=316k,w=316k IOPS][eta 00m:46s]
  1213. Jobs: 32 (f=32): [m(32)][92.5%][r=1089MiB/s,w=1090MiB/s][r=279k,w=279k IOPS][eta 00m:45s]
  1214. Jobs: 32 (f=32): [m(32)][92.7%][r=1193MiB/s,w=1199MiB/s][r=305k,w=307k IOPS][eta 00m:44s]
  1215. Jobs: 32 (f=32): [m(32)][92.8%][r=1117MiB/s,w=1117MiB/s][r=286k,w=286k IOPS][eta 00m:43s]
  1216. Jobs: 32 (f=32): [m(32)][93.0%][r=1196MiB/s,w=1197MiB/s][r=306k,w=307k IOPS][eta 00m:42s]
  1217. Jobs: 32 (f=32): [m(32)][93.2%][r=1233MiB/s,w=1229MiB/s][r=316k,w=315k IOPS][eta 00m:41s]
  1218. Jobs: 32 (f=32): [m(32)][93.3%][r=1207MiB/s,w=1207MiB/s][r=309k,w=309k IOPS][eta 00m:40s]
  1219. Jobs: 32 (f=32): [m(32)][93.5%][r=1182MiB/s,w=1179MiB/s][r=303k,w=302k IOPS][eta 00m:39s]
  1220. Jobs: 32 (f=32): [m(32)][93.7%][r=1142MiB/s,w=1140MiB/s][r=292k,w=292k IOPS][eta 00m:38s]
  1221. Jobs: 32 (f=32): [m(32)][93.8%][r=1178MiB/s,w=1180MiB/s][r=302k,w=302k IOPS][eta 00m:37s]
  1222. Jobs: 32 (f=32): [m(32)][94.0%][r=1090MiB/s,w=1096MiB/s][r=279k,w=281k IOPS][eta 00m:36s]
  1223. Jobs: 32 (f=32): [m(32)][94.2%][r=1118MiB/s,w=1118MiB/s][r=286k,w=286k IOPS][eta 00m:35s]
  1224. Jobs: 32 (f=32): [m(32)][94.3%][r=1116MiB/s,w=1119MiB/s][r=286k,w=286k IOPS][eta 00m:34s]
  1225. Jobs: 32 (f=32): [m(32)][94.5%][r=1117MiB/s,w=1120MiB/s][r=286k,w=287k IOPS][eta 00m:33s]
  1226. Jobs: 32 (f=32): [m(32)][94.7%][r=1086MiB/s,w=1090MiB/s][r=278k,w=279k IOPS][eta 00m:32s]
  1227. Jobs: 32 (f=32): [m(32)][94.8%][r=1139MiB/s,w=1142MiB/s][r=292k,w=292k IOPS][eta 00m:31s]
  1228. Jobs: 32 (f=32): [m(32)][95.0%][r=1087MiB/s,w=1090MiB/s][r=278k,w=279k IOPS][eta 00m:30s]
  1229. Jobs: 32 (f=32): [m(32)][95.2%][r=1106MiB/s,w=1111MiB/s][r=283k,w=284k IOPS][eta 00m:29s]
  1230. Jobs: 32 (f=32): [m(32)][95.3%][r=1139MiB/s,w=1135MiB/s][r=292k,w=291k IOPS][eta 00m:28s]
  1231. Jobs: 32 (f=32): [m(32)][95.5%][r=1130MiB/s,w=1130MiB/s][r=289k,w=289k IOPS][eta 00m:27s]
  1232. Jobs: 32 (f=32): [m(32)][95.7%][r=1066MiB/s,w=1063MiB/s][r=273k,w=272k IOPS][eta 00m:26s]
  1233. Jobs: 32 (f=32): [m(32)][95.8%][r=1086MiB/s,w=1085MiB/s][r=278k,w=278k IOPS][eta 00m:25s]
  1234. Jobs: 32 (f=32): [m(32)][96.0%][r=1124MiB/s,w=1122MiB/s][r=288k,w=287k IOPS][eta 00m:24s]
  1235. Jobs: 32 (f=32): [m(32)][96.2%][r=1109MiB/s,w=1111MiB/s][r=284k,w=284k IOPS][eta 00m:23s]
  1236. Jobs: 32 (f=32): [m(32)][96.3%][r=1105MiB/s,w=1101MiB/s][r=283k,w=282k IOPS][eta 00m:22s]
  1237. Jobs: 32 (f=32): [m(32)][96.5%][r=1072MiB/s,w=1067MiB/s][r=274k,w=273k IOPS][eta 00m:21s]
  1238. Jobs: 32 (f=32): [m(32)][96.7%][r=1088MiB/s,w=1089MiB/s][r=278k,w=279k IOPS][eta 00m:20s]
  1239. Jobs: 32 (f=32): [m(32)][96.8%][r=1074MiB/s,w=1074MiB/s][r=275k,w=275k IOPS][eta 00m:19s]
  1240. Jobs: 32 (f=32): [m(32)][97.0%][r=1039MiB/s,w=1038MiB/s][r=266k,w=266k IOPS][eta 00m:18s]
  1241. Jobs: 32 (f=32): [m(32)][97.2%][r=1027MiB/s,w=1026MiB/s][r=263k,w=263k IOPS][eta 00m:17s]
  1242. Jobs: 32 (f=32): [m(32)][97.3%][r=1063MiB/s,w=1062MiB/s][r=272k,w=272k IOPS][eta 00m:16s]
  1243. Jobs: 32 (f=32): [m(32)][97.5%][r=1028MiB/s,w=1026MiB/s][r=263k,w=263k IOPS][eta 00m:15s]
  1244. Jobs: 32 (f=32): [m(32)][97.7%][r=1029MiB/s,w=1023MiB/s][r=263k,w=262k IOPS][eta 00m:14s]
  1245. Jobs: 32 (f=32): [m(32)][97.8%][r=1035MiB/s,w=1034MiB/s][r=265k,w=265k IOPS][eta 00m:13s]
  1246. Jobs: 32 (f=32): [m(32)][98.0%][r=1013MiB/s,w=1013MiB/s][r=259k,w=259k IOPS][eta 00m:12s]
  1247. Jobs: 32 (f=32): [m(32)][98.2%][r=995MiB/s,w=1000MiB/s][r=255k,w=256k IOPS][eta 00m:11s]
  1248. Jobs: 32 (f=32): [m(32)][98.3%][r=1046MiB/s,w=1045MiB/s][r=268k,w=268k IOPS][eta 00m:10s]
  1249. Jobs: 32 (f=32): [m(32)][98.5%][r=1035MiB/s,w=1034MiB/s][r=265k,w=265k IOPS][eta 00m:09s]
  1250. Jobs: 32 (f=32): [m(32)][98.7%][r=1018MiB/s,w=1022MiB/s][r=261k,w=262k IOPS][eta 00m:08s]
  1251. Jobs: 32 (f=32): [m(32)][98.8%][r=1017MiB/s,w=1020MiB/s][r=260k,w=261k IOPS][eta 00m:07s]
  1252. Jobs: 32 (f=32): [m(32)][99.0%][r=1051MiB/s,w=1058MiB/s][r=269k,w=271k IOPS][eta 00m:06s]
  1253. Jobs: 32 (f=32): [m(32)][99.2%][r=1024MiB/s,w=1025MiB/s][r=262k,w=262k IOPS][eta 00m:05s]
  1254. Jobs: 32 (f=32): [m(32)][99.3%][r=1030MiB/s,w=1029MiB/s][r=264k,w=264k IOPS][eta 00m:04s]
  1255. Jobs: 32 (f=32): [m(32)][99.5%][r=1049MiB/s,w=1046MiB/s][r=269k,w=268k IOPS][eta 00m:03s]
  1256. Jobs: 32 (f=32): [m(32)][99.7%][r=1011MiB/s,w=1012MiB/s][r=259k,w=259k IOPS][eta 00m:02s]
  1257. Jobs: 32 (f=32): [m(32)][99.8%][r=1029MiB/s,w=1028MiB/s][r=263k,w=263k IOPS][eta 00m:01s]
  1258. Jobs: 32 (f=32): [m(32)][100.0%][r=1060MiB/s,w=1063MiB/s][r=271k,w=272k IOPS][eta 00m:00s]
  1259. t: (groupid=0, jobs=32): err= 0: pid=232504: Thu Apr 15 13:51:09 2021
  1260. read: IOPS=365k, BW=1427MiB/s (1497MB/s)(836GiB/600005msec)
  1261. slat (nsec): min=1594, max=5522.8k, avg=8656.42, stdev=17207.96
  1262. clat (usec): min=245, max=91281, avg=10954.73, stdev=3116.86
  1263. lat (usec): min=259, max=91287, avg=10963.52, stdev=3117.07
  1264. clat percentiles (usec):
  1265. | 1.00th=[ 9110], 5.00th=[ 9372], 10.00th=[ 9634], 20.00th=[ 9765],
  1266. | 30.00th=[10028], 40.00th=[10159], 50.00th=[10290], 60.00th=[10552],
  1267. | 70.00th=[10814], 80.00th=[11207], 90.00th=[12256], 95.00th=[14091],
  1268. | 99.00th=[22676], 99.50th=[40109], 99.90th=[46400], 99.95th=[50070],
  1269. | 99.99th=[57934]
  1270. bw ( MiB/s): min= 930, max= 1648, per=100.00%, avg=1428.25, stdev= 4.44, samples=38272
  1271. iops : min=238271, max=422037, avg=365619.74, stdev=1136.65, samples=38272
  1272. write: IOPS=365k, BW=1427MiB/s (1497MB/s)(836GiB/600005msec); 0 zone resets
  1273. slat (usec): min=3, max=36899, avg=75.07, stdev=248.01
  1274. clat (usec): min=93, max=85130, avg=11379.40, stdev=3267.76
  1275. lat (usec): min=194, max=85179, avg=11454.60, stdev=3283.05
  1276. clat percentiles (usec):
  1277. | 1.00th=[ 9241], 5.00th=[ 9503], 10.00th=[ 9634], 20.00th=[ 9896],
  1278. | 30.00th=[10159], 40.00th=[10421], 50.00th=[10683], 60.00th=[11076],
  1279. | 70.00th=[11469], 80.00th=[11994], 90.00th=[12911], 95.00th=[14877],
  1280. | 99.00th=[24249], 99.50th=[41157], 99.90th=[46924], 99.95th=[48497],
  1281. | 99.99th=[57410]
  1282. bw ( MiB/s): min= 934, max= 1627, per=100.00%, avg=1428.38, stdev= 4.41, samples=38272
  1283. iops : min=239264, max=416618, avg=365652.90, stdev=1127.96, samples=38272
  1284. lat (usec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
  1285. lat (msec) : 2=0.01%, 4=0.01%, 10=27.09%, 20=71.42%, 50=1.44%
  1286. lat (msec) : 100=0.04%
  1287. cpu : usr=3.37%, sys=47.07%, ctx=2470252, majf=0, minf=700915
  1288. IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
  1289. submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  1290. complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
  1291. issued rwts: total=219218478,219238054,0,0 short=0,0,0,0 dropped=0,0,0,0
  1292. latency : target=0, window=0, percentile=100.00%, depth=256
  1293.  
  1294. Run status group 0 (all jobs):
  1295. READ: bw=1427MiB/s (1497MB/s), 1427MiB/s-1427MiB/s (1497MB/s-1497MB/s), io=836GiB (898GB), run=600005-600005msec
  1296. WRITE: bw=1427MiB/s (1497MB/s), 1427MiB/s-1427MiB/s (1497MB/s-1497MB/s), io=836GiB (898GB), run=600005-600005msec
  1297.  
  1298. Disk stats (read/write):
  1299. md0: ios=219169590/219189401, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=109609336/219238055, aggrmerge=0/0, aggrticks=30990358/120008048, aggrin_queue=60001016, aggrutil=98.09%
  1300. nvme0n1: ios=103977539/219238055, merge=0/0, ticks=30014032/127600283, in_queue=65819155, util=98.09%
  1301. nvme1n1: ios=115241133/219238055, merge=0/0, ticks=31966684/112415814, in_queue=54182877, util=97.70%
  1302.  
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!