API tools faq
paste
Advertisement
arcagamer

IA tests GGUF

arcagamer
Sep 17th, 2023 (edited)
601
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
HTML 8.24 KB | None | 0 0
raw download clone embed print report
  1. Testes com valor das camadas usadas pela GPU e CPU entre ():
  2.  
  3. tulpar 7b(35 camadas) testing benchmark:
  4.  
  5. #Testes com contexto curto(21 tokens)
  6.  
  7. CPU(35) GPU(0):
  8. ```
  9. llama_print_timings:        load time =  2776.49 ms
  10. llama_print_timings:      sample time =    28.02 ms /   100 runs   (    0.28 ms per token,  3569.52 tokens per second)
  11. llama_print_timings: prompt eval time =  1225.50 ms /    11 tokens (  111.41 ms per token,     8.98 tokens per second)
  12. llama_print_timings:        eval time = 19845.56 ms /    99 runs   (  200.46 ms per token,     4.99 tokens per second)
  13. llama_print_timings:       total time = 21449.92 ms
  14. Output generated in 21.81 seconds (4.59 tokens/s, 100 tokens, context 21, seed 80807575)
  15. ```
  16. Velocidade: 4.59 tokens por segundo
  17.  
  18. CPU(30) GPU(5):
  19. ```
  20. llama_print_timings:        load time =  2380.79 ms
  21. llama_print_timings:      sample time =    28.52 ms /   100 runs   (    0.29 ms per token,  3505.94 tokens per second)
  22. llama_print_timings: prompt eval time =     0.00 ms /     1 tokens (    0.00 ms per token,      inf tokens per second)
  23. llama_print_timings:        eval time = 17758.01 ms /   100 runs   (  177.58 ms per token,     5.63 tokens per second)
  24. llama_print_timings:       total time = 18136.91 ms
  25. Output generated in 18.49 seconds (5.41 tokens/s, 100 tokens, context 21, seed 80807575)
  26. ```
  27. Velocidade: 5.41 tokens por segundo
  28. CPU(25) GPU(10):
  29. ```
  30. llama_print_timings:        load time =  2139.83 ms
  31. llama_print_timings:      sample time =    28.17 ms /   100 runs   (    0.28 ms per token,  3550.13 tokens per second)
  32. llama_print_timings: prompt eval time =     0.00 ms /     1 tokens (    0.00 ms per token,      inf tokens per second)
  33. llama_print_timings:        eval time = 16001.52 ms /   100 runs   (  160.02 ms per token,     6.25 tokens per second)
  34. llama_print_timings:       total time = 16382.37 ms
  35. Output generated in 16.73 seconds (5.98 tokens/s, 100 tokens, context 21, seed 80807575)
  36. ```
  37. Velocidade: 5.98 tokens por segundo
  38. CPU(15) GPU(20) Low vram:
  39. ```
  40. llama_print_timings:        load time =  1455.39 ms
  41. llama_print_timings:      sample time =    28.39 ms /   100 runs   (    0.28 ms per token,  3522.74 tokens per second)
  42. llama_print_timings: prompt eval time =     0.00 ms /     1 tokens (    0.00 ms per token,      inf tokens per second)
  43. llama_print_timings:        eval time = 12303.83 ms /   100 runs   (  123.04 ms per token,     8.13 tokens per second)
  44. llama_print_timings:       total time = 12699.78 ms
  45. Output generated in 13.07 seconds (7.65 tokens/s, 100 tokens, context 21, seed 80807575)
  46. ```
  47. Velocidade: 7.65 tokens por segundo
  48.  
  49. #Testes com contexto longo(1179)
  50.  
  51. CPU(35) GPU(0):
  52. ```
  53. llama_print_timings:        load time =  6006.68 ms
  54. llama_print_timings:      sample time =    28.43 ms /   100 runs   (    0.28 ms per token,  3517.66 tokens per second)
  55. llama_print_timings: prompt eval time =     0.00 ms /     1 tokens (    0.00 ms per token,      inf tokens per second)
  56. llama_print_timings:        eval time = 23526.07 ms /   100 runs   (  235.26 ms per token,     4.25 tokens per second)
  57. llama_print_timings:       total time = 23910.16 ms
  58. Output generated in 24.28 seconds (4.12 tokens/s, 100 tokens, context 1179, seed 80807575)
  59. ```
  60. Velocidade: 4.12 tokens por segundo
  61. CPU(30) GPU(5):
  62. ```
  63. llama_print_timings:        load time =  5556.94 ms
  64. llama_print_timings:      sample time =    25.83 ms /    92 runs   (    0.28 ms per token,  3561.75 tokens per second)
  65. llama_print_timings: prompt eval time =     0.00 ms /     1 tokens (    0.00 ms per token,      inf tokens per second)
  66. llama_print_timings:        eval time = 19757.52 ms /    92 runs   (  214.76 ms per token,     4.66 tokens per second)
  67. llama_print_timings:       total time = 20113.70 ms
  68. Output generated in 20.48 seconds (4.44 tokens/s, 91 tokens, context 1179, seed 80807575)
  69. ```
  70. Velocidade: 4.44 tokens por segundo
  71. CPU(25) GPU(10):
  72. ```
  73. llama_print_timings:        load time =  5161.97 ms
  74. llama_print_timings:      sample time =    25.74 ms /    92 runs   (    0.28 ms per token,  3574.20 tokens per second)
  75. llama_print_timings: prompt eval time =     0.00 ms /     1 tokens (    0.00 ms per token,      inf tokens per second)
  76. llama_print_timings:        eval time = 18213.97 ms /    92 runs   (  197.98 ms per token,     5.05 tokens per second)
  77. llama_print_timings:       total time = 18574.51 ms
  78. Output generated in 18.93 seconds (4.81 tokens/s, 91 tokens, context 1179, seed 80807575)
  79. ```
  80. Velocidade: 4.81 tokens por segundo
  81. CPU(15) GPU(20) Low vram:
  82. ```
  83. llama_print_timings:        load time =  4627.16 ms
  84. llama_print_timings:      sample time =    11.01 ms /    39 runs   (    0.28 ms per token,  3542.23 tokens per second)
  85. llama_print_timings: prompt eval time =     0.00 ms /     1 tokens (    0.00 ms per token,      inf tokens per second)
  86. llama_print_timings:        eval time =  6192.65 ms /    39 runs   (  158.79 ms per token,     6.30 tokens per second)
  87. llama_print_timings:       total time =  6343.34 ms
  88. Output generated in 6.72 seconds (5.66 tokens/s, 38 tokens, context 1179, seed 80807575)
  89. ```
  90. Velocidade: 5.66 tokens por segundo
  91.  
  92. Mlewdchat L2 13b q4_K_S(43 camadas) testing benchmark:
  93.  
  94. #Testes com contexto curto(21 tokens)
  95.  
  96. CPU(43) GPU(0):
  97. ```
  98. llama_print_timings:        load time = 33123.68 ms
  99. llama_print_timings:      sample time =    28.14 ms /   100 runs   (    0.28 ms per token,  3554.29 tokens per second)
  100. llama_print_timings: prompt eval time =     0.00 ms /     1 tokens (    0.00 ms per token,      inf tokens per second)
  101. llama_print_timings:        eval time = 36035.70 ms /   100 runs   (  360.36 ms per token,     2.78 tokens per second)
  102. llama_print_timings:       total time = 36417.16 ms
  103. Output generated in 36.79 seconds (2.72 tokens/s, 100 tokens, context 21, seed 80807575)
  104. ```
  105. Velocidade: 2.72 tokens por segundo
  106. CPU(38) GPU(5):
  107. ```
  108. llama_print_timings:        load time =  4640.32 ms
  109. llama_print_timings:      sample time =    28.01 ms /   100 runs   (    0.28 ms per token,  3570.66 tokens per second)
  110. llama_print_timings: prompt eval time =     0.00 ms /     1 tokens (    0.00 ms per token,      inf tokens per second)
  111. llama_print_timings:        eval time = 31952.95 ms /   100 runs   (  319.53 ms per token,     3.13 tokens per second)
  112. llama_print_timings:       total time = 32338.01 ms
  113. Output generated in 32.70 seconds (3.06 tokens/s, 100 tokens, context 21, seed 80807575)
  114. ```
  115. Velocidade: 3.06 tokens por segundo
  116. CPU(33) GPU(10):
  117. ```
  118. llama_print_timings:        load time =  4213.47 ms
  119. llama_print_timings:      sample time =    28.12 ms /   100 runs   (    0.28 ms per token,  3556.19 tokens per second)
  120. llama_print_timings: prompt eval time =     0.00 ms /     1 tokens (    0.00 ms per token,      inf tokens per second)
  121. llama_print_timings:        eval time = 29386.01 ms /   100 runs   (  293.86 ms per token,     3.40 tokens per second)
  122. llama_print_timings:       total time = 29800.18 ms
  123. Output generated in 30.16 seconds (3.32 tokens/s, 100 tokens, context 21, seed 80807575)
  124. ```
  125. Velocidade: 3.32 tokens por segundo
  126. CPU(23) GPU(20) Low vram:
  127. ```
  128. llama_print_timings:        load time =  3102.83 ms
  129. llama_print_timings:      sample time =    28.72 ms /   100 runs   (    0.29 ms per token,  3481.53 tokens per second)
  130. llama_print_timings: prompt eval time =     0.00 ms /     1 tokens (    0.00 ms per token,      inf tokens per second)
  131. llama_print_timings:        eval time = 23273.40 ms /   100 runs   (  232.73 ms per token,     4.30 tokens per second)
  132. llama_print_timings:       total time = 23662.53 ms
  133. Output generated in 24.04 seconds (4.16 tokens/s, 100 tokens, context 21, seed 80807575)
  134. ```
  135. Velocidade: 4.16 tokens por segundo
  136.  
  137. #Testes com contexto longo(1179)
  138.  
  139. CPU(23) GPU(20) Low vram:
  140. ```
  141. llama_print_timings:        load time =  3102.83 ms
  142. llama_print_timings:      sample time =    29.02 ms /   100 runs   (    0.29 ms per token,  3446.26 tokens per second)
  143. llama_print_timings: prompt eval time =     0.00 ms /     1 tokens (    0.00 ms per token,      inf tokens per second)
  144. llama_print_timings:        eval time = 28811.66 ms /   100 runs   (  288.12 ms per token,     3.47 tokens per second)
  145. llama_print_timings:       total time = 29202.11 ms
  146. Output generated in 29.57 seconds (3.38 tokens/s, 100 tokens, context 1179, seed 80807575)
  147. ```
  148. Velocidade: 3.38 tokens por segundo
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!