API tools faq
paste
Guest User

Untitled

a guest
Oct 4th, 2025
42
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 35.03 KB | None | 0 0
raw download clone embed print report
  1. Starting LLM server with model: c:\Users\MiszS\Desktop\machine_learning\stock prediction\LLM\Llama-3-13B-Instruct-v0.1.Q8_0.gguf
  2. Accessible at: http://<your-ip>:8000/v1/completions
  3. llama_model_loader: loaded meta data with 22 key-value pairs and 507 tensors from c:\Users\MiszS\Desktop\machine_learning\stock prediction\LLM\Llama-3-13B-Instruct-v0.1.Q8_0.gguf (version GGUF V3 (latest))
  4. llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
  5. llama_model_loader: - kv 0: general.architecture str = llama
  6. llama_model_loader: - kv 1: general.name str = hub
  7. llama_model_loader: - kv 2: llama.vocab_size u32 = 128256
  8. llama_model_loader: - kv 3: llama.context_length u32 = 8192
  9. llama_model_loader: - kv 4: llama.embedding_length u32 = 4096
  10. llama_model_loader: - kv 5: llama.block_count u32 = 56
  11. llama_model_loader: - kv 6: llama.feed_forward_length u32 = 14336
  12. llama_model_loader: - kv 7: llama.rope.dimension_count u32 = 128
  13. llama_model_loader: - kv 8: llama.attention.head_count u32 = 32
  14. llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 8
  15. llama_model_loader: - kv 10: llama.attention.layer_norm_rms_epsilon f32 = 0.000010
  16. llama_model_loader: - kv 11: llama.rope.freq_base f32 = 500000.000000
  17. llama_model_loader: - kv 12: general.file_type u32 = 7
  18. llama_model_loader: - kv 13: tokenizer.ggml.model str = gpt2
  19. llama_model_loader: - kv 14: tokenizer.ggml.tokens arr[str,128256] = ["!", "\"", "#", "$", "%", "&", "'", ...
  20. llama_model_loader: - kv 15: tokenizer.ggml.scores arr[f32,128256] = [0.000000, 0.000000, 0.000000, 0.0000...
  21. llama_model_loader: - kv 16: tokenizer.ggml.token_type arr[i32,128256] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
  22. llama_model_loader: - kv 17: tokenizer.ggml.merges arr[str,280147] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
  23. llama_model_loader: - kv 18: tokenizer.ggml.bos_token_id u32 = 128000
  24. llama_model_loader: - kv 19: tokenizer.ggml.eos_token_id u32 = 128001
  25. llama_model_loader: - kv 20: tokenizer.chat_template str = {% set loop_messages = messages %}{% ...
  26. llama_model_loader: - kv 21: general.quantization_version u32 = 2
  27. llama_model_loader: - type f32: 113 tensors
  28. llama_model_loader: - type q8_0: 394 tensors
  29. print_info: file format = GGUF V3 (latest)
  30. print_info: file type = Q8_0
  31. print_info: file size = 13.13 GiB (8.50 BPW)
  32. load: missing pre-tokenizer type, using: 'default'
  33. load:
  34. load: ************************************
  35. load: GENERATION QUALITY WILL BE DEGRADED!
  36. load: CONSIDER REGENERATING THE MODEL
  37. load: ************************************
  38. load:
  39. init_tokenizer: initializing tokenizer for type 2
  40. load: control token: 128095 '<|reserved_special_token_90|>' is not marked as EOG
  41. load: control token: 128188 '<|reserved_special_token_183|>' is not marked as EOG
  42. load: control token: 128127 '<|reserved_special_token_122|>' is not marked as EOG
  43. load: control token: 128116 '<|reserved_special_token_111|>' is not marked as EOG
  44. load: control token: 128133 '<|reserved_special_token_128|>' is not marked as EOG
  45. load: control token: 128152 '<|reserved_special_token_147|>' is not marked as EOG
  46. load: control token: 128193 '<|reserved_special_token_188|>' is not marked as EOG
  47. load: control token: 128098 '<|reserved_special_token_93|>' is not marked as EOG
  48. load: control token: 128135 '<|reserved_special_token_130|>' is not marked as EOG
  49. load: control token: 128178 '<|reserved_special_token_173|>' is not marked as EOG
  50. load: control token: 128031 '<|reserved_special_token_26|>' is not marked as EOG
  51. load: control token: 128206 '<|reserved_special_token_201|>' is not marked as EOG
  52. load: control token: 128028 '<|reserved_special_token_23|>' is not marked as EOG
  53. load: control token: 128047 '<|reserved_special_token_42|>' is not marked as EOG
  54. load: control token: 128241 '<|reserved_special_token_236|>' is not marked as EOG
  55. load: control token: 128145 '<|reserved_special_token_140|>' is not marked as EOG
  56. load: control token: 128195 '<|reserved_special_token_190|>' is not marked as EOG
  57. load: control token: 128226 '<|reserved_special_token_221|>' is not marked as EOG
  58. load: control token: 128162 '<|reserved_special_token_157|>' is not marked as EOG
  59. load: control token: 128243 '<|reserved_special_token_238|>' is not marked as EOG
  60. load: control token: 128014 '<|reserved_special_token_9|>' is not marked as EOG
  61. load: control token: 128213 '<|reserved_special_token_208|>' is not marked as EOG
  62. load: control token: 128158 '<|reserved_special_token_153|>' is not marked as EOG
  63. load: control token: 128221 '<|reserved_special_token_216|>' is not marked as EOG
  64. load: control token: 128079 '<|reserved_special_token_74|>' is not marked as EOG
  65. load: control token: 128246 '<|reserved_special_token_241|>' is not marked as EOG
  66. load: control token: 128104 '<|reserved_special_token_99|>' is not marked as EOG
  67. load: control token: 128076 '<|reserved_special_token_71|>' is not marked as EOG
  68. load: control token: 128222 '<|reserved_special_token_217|>' is not marked as EOG
  69. load: control token: 128172 '<|reserved_special_token_167|>' is not marked as EOG
  70. load: control token: 128220 '<|reserved_special_token_215|>' is not marked as EOG
  71. load: control token: 128179 '<|reserved_special_token_174|>' is not marked as EOG
  72. load: control token: 128065 '<|reserved_special_token_60|>' is not marked as EOG
  73. load: control token: 128249 '<|reserved_special_token_244|>' is not marked as EOG
  74. load: control token: 128175 '<|reserved_special_token_170|>' is not marked as EOG
  75. load: control token: 128218 '<|reserved_special_token_213|>' is not marked as EOG
  76. load: control token: 128049 '<|reserved_special_token_44|>' is not marked as EOG
  77. load: control token: 128119 '<|reserved_special_token_114|>' is not marked as EOG
  78. load: control token: 128148 '<|reserved_special_token_143|>' is not marked as EOG
  79. load: control token: 128118 '<|reserved_special_token_113|>' is not marked as EOG
  80. load: control token: 128155 '<|reserved_special_token_150|>' is not marked as EOG
  81. load: control token: 128093 '<|reserved_special_token_88|>' is not marked as EOG
  82. load: control token: 128087 '<|reserved_special_token_82|>' is not marked as EOG
  83. load: control token: 128235 '<|reserved_special_token_230|>' is not marked as EOG
  84. load: control token: 128136 '<|reserved_special_token_131|>' is not marked as EOG
  85. load: control token: 128173 '<|reserved_special_token_168|>' is not marked as EOG
  86. load: control token: 128074 '<|reserved_special_token_69|>' is not marked as EOG
  87. load: control token: 128211 '<|reserved_special_token_206|>' is not marked as EOG
  88. load: control token: 128168 '<|reserved_special_token_163|>' is not marked as EOG
  89. load: control token: 128109 '<|reserved_special_token_104|>' is not marked as EOG
  90. load: control token: 128177 '<|reserved_special_token_172|>' is not marked as EOG
  91. load: control token: 128057 '<|reserved_special_token_52|>' is not marked as EOG
  92. load: control token: 128000 '<|begin_of_text|>' is not marked as EOG
  93. load: control token: 128149 '<|reserved_special_token_144|>' is not marked as EOG
  94. load: control token: 128113 '<|reserved_special_token_108|>' is not marked as EOG
  95. load: control token: 128069 '<|reserved_special_token_64|>' is not marked as EOG
  96. load: control token: 128056 '<|reserved_special_token_51|>' is not marked as EOG
  97. load: control token: 128091 '<|reserved_special_token_86|>' is not marked as EOG
  98. load: control token: 128184 '<|reserved_special_token_179|>' is not marked as EOG
  99. load: control token: 128100 '<|reserved_special_token_95|>' is not marked as EOG
  100. load: control token: 128124 '<|reserved_special_token_119|>' is not marked as EOG
  101. load: control token: 128020 '<|reserved_special_token_15|>' is not marked as EOG
  102. load: control token: 128034 '<|reserved_special_token_29|>' is not marked as EOG
  103. load: control token: 128225 '<|reserved_special_token_220|>' is not marked as EOG
  104. load: control token: 128002 '<|reserved_special_token_0|>' is not marked as EOG
  105. load: control token: 128006 '<|start_header_id|>' is not marked as EOG
  106. load: control token: 128088 '<|reserved_special_token_83|>' is not marked as EOG
  107. load: control token: 128041 '<|reserved_special_token_36|>' is not marked as EOG
  108. load: control token: 128215 '<|reserved_special_token_210|>' is not marked as EOG
  109. load: control token: 128208 '<|reserved_special_token_203|>' is not marked as EOG
  110. load: control token: 128070 '<|reserved_special_token_65|>' is not marked as EOG
  111. load: control token: 128165 '<|reserved_special_token_160|>' is not marked as EOG
  112. load: control token: 128180 '<|reserved_special_token_175|>' is not marked as EOG
  113. load: control token: 128231 '<|reserved_special_token_226|>' is not marked as EOG
  114. load: control token: 128232 '<|reserved_special_token_227|>' is not marked as EOG
  115. load: control token: 128064 '<|reserved_special_token_59|>' is not marked as EOG
  116. load: control token: 128036 '<|reserved_special_token_31|>' is not marked as EOG
  117. load: control token: 128103 '<|reserved_special_token_98|>' is not marked as EOG
  118. load: control token: 128247 '<|reserved_special_token_242|>' is not marked as EOG
  119. load: control token: 128170 '<|reserved_special_token_165|>' is not marked as EOG
  120. load: control token: 128123 '<|reserved_special_token_118|>' is not marked as EOG
  121. load: control token: 128044 '<|reserved_special_token_39|>' is not marked as EOG
  122. load: control token: 128237 '<|reserved_special_token_232|>' is not marked as EOG
  123. load: control token: 128042 '<|reserved_special_token_37|>' is not marked as EOG
  124. load: control token: 128192 '<|reserved_special_token_187|>' is not marked as EOG
  125. load: control token: 128075 '<|reserved_special_token_70|>' is not marked as EOG
  126. load: control token: 128134 '<|reserved_special_token_129|>' is not marked as EOG
  127. load: control token: 128183 '<|reserved_special_token_178|>' is not marked as EOG
  128. load: control token: 128045 '<|reserved_special_token_40|>' is not marked as EOG
  129. load: control token: 128073 '<|reserved_special_token_68|>' is not marked as EOG
  130. load: control token: 128026 '<|reserved_special_token_21|>' is not marked as EOG
  131. load: control token: 128010 '<|reserved_special_token_5|>' is not marked as EOG
  132. load: control token: 128194 '<|reserved_special_token_189|>' is not marked as EOG
  133. load: control token: 128053 '<|reserved_special_token_48|>' is not marked as EOG
  134. load: control token: 128120 '<|reserved_special_token_115|>' is not marked as EOG
  135. load: control token: 128092 '<|reserved_special_token_87|>' is not marked as EOG
  136. load: control token: 128086 '<|reserved_special_token_81|>' is not marked as EOG
  137. load: control token: 128054 '<|reserved_special_token_49|>' is not marked as EOG
  138. load: control token: 128160 '<|reserved_special_token_155|>' is not marked as EOG
  139. load: control token: 128005 '<|reserved_special_token_3|>' is not marked as EOG
  140. load: control token: 128050 '<|reserved_special_token_45|>' is not marked as EOG
  141. load: control token: 128157 '<|reserved_special_token_152|>' is not marked as EOG
  142. load: control token: 128219 '<|reserved_special_token_214|>' is not marked as EOG
  143. load: control token: 128032 '<|reserved_special_token_27|>' is not marked as EOG
  144. load: control token: 128159 '<|reserved_special_token_154|>' is not marked as EOG
  145. load: control token: 128202 '<|reserved_special_token_197|>' is not marked as EOG
  146. load: control token: 128106 '<|reserved_special_token_101|>' is not marked as EOG
  147. load: control token: 128182 '<|reserved_special_token_177|>' is not marked as EOG
  148. load: control token: 128111 '<|reserved_special_token_106|>' is not marked as EOG
  149. load: control token: 128156 '<|reserved_special_token_151|>' is not marked as EOG
  150. load: control token: 128176 '<|reserved_special_token_171|>' is not marked as EOG
  151. load: control token: 128112 '<|reserved_special_token_107|>' is not marked as EOG
  152. load: control token: 128084 '<|reserved_special_token_79|>' is not marked as EOG
  153. load: control token: 128110 '<|reserved_special_token_105|>' is not marked as EOG
  154. load: control token: 128051 '<|reserved_special_token_46|>' is not marked as EOG
  155. load: control token: 128027 '<|reserved_special_token_22|>' is not marked as EOG
  156. load: control token: 128167 '<|reserved_special_token_162|>' is not marked as EOG
  157. load: control token: 128008 '<|reserved_special_token_4|>' is not marked as EOG
  158. load: control token: 128061 '<|reserved_special_token_56|>' is not marked as EOG
  159. load: control token: 128115 '<|reserved_special_token_110|>' is not marked as EOG
  160. load: control token: 128203 '<|reserved_special_token_198|>' is not marked as EOG
  161. load: control token: 128096 '<|reserved_special_token_91|>' is not marked as EOG
  162. load: control token: 128130 '<|reserved_special_token_125|>' is not marked as EOG
  163. load: control token: 128187 '<|reserved_special_token_182|>' is not marked as EOG
  164. load: control token: 128094 '<|reserved_special_token_89|>' is not marked as EOG
  165. load: control token: 128083 '<|reserved_special_token_78|>' is not marked as EOG
  166. load: control token: 128117 '<|reserved_special_token_112|>' is not marked as EOG
  167. load: control token: 128190 '<|reserved_special_token_185|>' is not marked as EOG
  168. load: control token: 128046 '<|reserved_special_token_41|>' is not marked as EOG
  169. load: control token: 128239 '<|reserved_special_token_234|>' is not marked as EOG
  170. load: control token: 128139 '<|reserved_special_token_134|>' is not marked as EOG
  171. load: control token: 128185 '<|reserved_special_token_180|>' is not marked as EOG
  172. load: control token: 128141 '<|reserved_special_token_136|>' is not marked as EOG
  173. load: control token: 128244 '<|reserved_special_token_239|>' is not marked as EOG
  174. load: control token: 128062 '<|reserved_special_token_57|>' is not marked as EOG
  175. load: control token: 128114 '<|reserved_special_token_109|>' is not marked as EOG
  176. load: control token: 128030 '<|reserved_special_token_25|>' is not marked as EOG
  177. load: control token: 128181 '<|reserved_special_token_176|>' is not marked as EOG
  178. load: control token: 128037 '<|reserved_special_token_32|>' is not marked as EOG
  179. load: control token: 128201 '<|reserved_special_token_196|>' is not marked as EOG
  180. load: control token: 128207 '<|reserved_special_token_202|>' is not marked as EOG
  181. load: control token: 128242 '<|reserved_special_token_237|>' is not marked as EOG
  182. load: control token: 128132 '<|reserved_special_token_127|>' is not marked as EOG
  183. load: control token: 128068 '<|reserved_special_token_63|>' is not marked as EOG
  184. load: control token: 128150 '<|reserved_special_token_145|>' is not marked as EOG
  185. load: control token: 128191 '<|reserved_special_token_186|>' is not marked as EOG
  186. load: control token: 128174 '<|reserved_special_token_169|>' is not marked as EOG
  187. load: control token: 128233 '<|reserved_special_token_228|>' is not marked as EOG
  188. load: control token: 128245 '<|reserved_special_token_240|>' is not marked as EOG
  189. load: control token: 128238 '<|reserved_special_token_233|>' is not marked as EOG
  190. load: control token: 128209 '<|reserved_special_token_204|>' is not marked as EOG
  191. load: control token: 128204 '<|reserved_special_token_199|>' is not marked as EOG
  192. load: control token: 128003 '<|reserved_special_token_1|>' is not marked as EOG
  193. load: control token: 128004 '<|reserved_special_token_2|>' is not marked as EOG
  194. load: control token: 128007 '<|end_header_id|>' is not marked as EOG
  195. load: control token: 128011 '<|reserved_special_token_6|>' is not marked as EOG
  196. load: control token: 128012 '<|reserved_special_token_7|>' is not marked as EOG
  197. load: control token: 128013 '<|reserved_special_token_8|>' is not marked as EOG
  198. load: control token: 128015 '<|reserved_special_token_10|>' is not marked as EOG
  199. load: control token: 128016 '<|reserved_special_token_11|>' is not marked as EOG
  200. load: control token: 128017 '<|reserved_special_token_12|>' is not marked as EOG
  201. load: control token: 128018 '<|reserved_special_token_13|>' is not marked as EOG
  202. load: control token: 128019 '<|reserved_special_token_14|>' is not marked as EOG
  203. load: control token: 128021 '<|reserved_special_token_16|>' is not marked as EOG
  204. load: control token: 128022 '<|reserved_special_token_17|>' is not marked as EOG
  205. load: control token: 128023 '<|reserved_special_token_18|>' is not marked as EOG
  206. load: control token: 128024 '<|reserved_special_token_19|>' is not marked as EOG
  207. load: control token: 128025 '<|reserved_special_token_20|>' is not marked as EOG
  208. load: control token: 128029 '<|reserved_special_token_24|>' is not marked as EOG
  209. load: control token: 128033 '<|reserved_special_token_28|>' is not marked as EOG
  210. load: control token: 128035 '<|reserved_special_token_30|>' is not marked as EOG
  211. load: control token: 128038 '<|reserved_special_token_33|>' is not marked as EOG
  212. load: control token: 128039 '<|reserved_special_token_34|>' is not marked as EOG
  213. load: control token: 128040 '<|reserved_special_token_35|>' is not marked as EOG
  214. load: control token: 128043 '<|reserved_special_token_38|>' is not marked as EOG
  215. load: control token: 128048 '<|reserved_special_token_43|>' is not marked as EOG
  216. load: control token: 128052 '<|reserved_special_token_47|>' is not marked as EOG
  217. load: control token: 128055 '<|reserved_special_token_50|>' is not marked as EOG
  218. load: control token: 128058 '<|reserved_special_token_53|>' is not marked as EOG
  219. load: control token: 128059 '<|reserved_special_token_54|>' is not marked as EOG
  220. load: control token: 128060 '<|reserved_special_token_55|>' is not marked as EOG
  221. load: control token: 128063 '<|reserved_special_token_58|>' is not marked as EOG
  222. load: control token: 128066 '<|reserved_special_token_61|>' is not marked as EOG
  223. load: control token: 128067 '<|reserved_special_token_62|>' is not marked as EOG
  224. load: control token: 128071 '<|reserved_special_token_66|>' is not marked as EOG
  225. load: control token: 128072 '<|reserved_special_token_67|>' is not marked as EOG
  226. load: control token: 128077 '<|reserved_special_token_72|>' is not marked as EOG
  227. load: control token: 128078 '<|reserved_special_token_73|>' is not marked as EOG
  228. load: control token: 128080 '<|reserved_special_token_75|>' is not marked as EOG
  229. load: control token: 128081 '<|reserved_special_token_76|>' is not marked as EOG
  230. load: control token: 128082 '<|reserved_special_token_77|>' is not marked as EOG
  231. load: control token: 128085 '<|reserved_special_token_80|>' is not marked as EOG
  232. load: control token: 128089 '<|reserved_special_token_84|>' is not marked as EOG
  233. load: control token: 128090 '<|reserved_special_token_85|>' is not marked as EOG
  234. load: control token: 128097 '<|reserved_special_token_92|>' is not marked as EOG
  235. load: control token: 128099 '<|reserved_special_token_94|>' is not marked as EOG
  236. load: control token: 128101 '<|reserved_special_token_96|>' is not marked as EOG
  237. load: control token: 128102 '<|reserved_special_token_97|>' is not marked as EOG
  238. load: control token: 128105 '<|reserved_special_token_100|>' is not marked as EOG
  239. load: control token: 128107 '<|reserved_special_token_102|>' is not marked as EOG
  240. load: control token: 128108 '<|reserved_special_token_103|>' is not marked as EOG
  241. load: control token: 128121 '<|reserved_special_token_116|>' is not marked as EOG
  242. load: control token: 128122 '<|reserved_special_token_117|>' is not marked as EOG
  243. load: control token: 128125 '<|reserved_special_token_120|>' is not marked as EOG
  244. load: control token: 128126 '<|reserved_special_token_121|>' is not marked as EOG
  245. load: control token: 128128 '<|reserved_special_token_123|>' is not marked as EOG
  246. load: control token: 128129 '<|reserved_special_token_124|>' is not marked as EOG
  247. load: control token: 128131 '<|reserved_special_token_126|>' is not marked as EOG
  248. load: control token: 128137 '<|reserved_special_token_132|>' is not marked as EOG
  249. load: control token: 128138 '<|reserved_special_token_133|>' is not marked as EOG
  250. load: control token: 128140 '<|reserved_special_token_135|>' is not marked as EOG
  251. load: control token: 128142 '<|reserved_special_token_137|>' is not marked as EOG
  252. load: control token: 128143 '<|reserved_special_token_138|>' is not marked as EOG
  253. load: control token: 128144 '<|reserved_special_token_139|>' is not marked as EOG
  254. load: control token: 128146 '<|reserved_special_token_141|>' is not marked as EOG
  255. load: control token: 128147 '<|reserved_special_token_142|>' is not marked as EOG
  256. load: control token: 128151 '<|reserved_special_token_146|>' is not marked as EOG
  257. load: control token: 128153 '<|reserved_special_token_148|>' is not marked as EOG
  258. load: control token: 128154 '<|reserved_special_token_149|>' is not marked as EOG
  259. load: control token: 128161 '<|reserved_special_token_156|>' is not marked as EOG
  260. load: control token: 128163 '<|reserved_special_token_158|>' is not marked as EOG
  261. load: control token: 128164 '<|reserved_special_token_159|>' is not marked as EOG
  262. load: control token: 128166 '<|reserved_special_token_161|>' is not marked as EOG
  263. load: control token: 128169 '<|reserved_special_token_164|>' is not marked as EOG
  264. load: control token: 128171 '<|reserved_special_token_166|>' is not marked as EOG
  265. load: control token: 128186 '<|reserved_special_token_181|>' is not marked as EOG
  266. load: control token: 128189 '<|reserved_special_token_184|>' is not marked as EOG
  267. load: control token: 128196 '<|reserved_special_token_191|>' is not marked as EOG
  268. load: control token: 128197 '<|reserved_special_token_192|>' is not marked as EOG
  269. load: control token: 128198 '<|reserved_special_token_193|>' is not marked as EOG
  270. load: control token: 128199 '<|reserved_special_token_194|>' is not marked as EOG
  271. load: control token: 128200 '<|reserved_special_token_195|>' is not marked as EOG
  272. load: control token: 128205 '<|reserved_special_token_200|>' is not marked as EOG
  273. load: control token: 128210 '<|reserved_special_token_205|>' is not marked as EOG
  274. load: control token: 128212 '<|reserved_special_token_207|>' is not marked as EOG
  275. load: control token: 128214 '<|reserved_special_token_209|>' is not marked as EOG
  276. load: control token: 128216 '<|reserved_special_token_211|>' is not marked as EOG
  277. load: control token: 128217 '<|reserved_special_token_212|>' is not marked as EOG
  278. load: control token: 128223 '<|reserved_special_token_218|>' is not marked as EOG
  279. load: control token: 128224 '<|reserved_special_token_219|>' is not marked as EOG
  280. load: control token: 128227 '<|reserved_special_token_222|>' is not marked as EOG
  281. load: control token: 128228 '<|reserved_special_token_223|>' is not marked as EOG
  282. load: control token: 128229 '<|reserved_special_token_224|>' is not marked as EOG
  283. load: control token: 128230 '<|reserved_special_token_225|>' is not marked as EOG
  284. load: control token: 128234 '<|reserved_special_token_229|>' is not marked as EOG
  285. load: control token: 128236 '<|reserved_special_token_231|>' is not marked as EOG
  286. load: control token: 128240 '<|reserved_special_token_235|>' is not marked as EOG
  287. load: control token: 128248 '<|reserved_special_token_243|>' is not marked as EOG
  288. load: control token: 128250 '<|reserved_special_token_245|>' is not marked as EOG
  289. load: control token: 128251 '<|reserved_special_token_246|>' is not marked as EOG
  290. load: control token: 128252 '<|reserved_special_token_247|>' is not marked as EOG
  291. load: control token: 128253 '<|reserved_special_token_248|>' is not marked as EOG
  292. load: control token: 128254 '<|reserved_special_token_249|>' is not marked as EOG
  293. load: control token: 128255 '<|reserved_special_token_250|>' is not marked as EOG
  294. load: printing all EOG tokens:
  295. load: - 128001 ('<|end_of_text|>')
  296. load: - 128009 ('<|eot_id|>')
  297. load: special tokens cache size = 256
  298. load: token to piece cache size = 0.8000 MB
  299. print_info: arch = llama
  300. print_info: vocab_only = 0
  301. print_info: n_ctx_train = 8192
  302. print_info: n_embd = 4096
  303. print_info: n_layer = 56
  304. print_info: n_head = 32
  305. print_info: n_head_kv = 8
  306. print_info: n_rot = 128
  307. print_info: n_swa = 0
  308. print_info: is_swa_any = 0
  309. print_info: n_embd_head_k = 128
  310. print_info: n_embd_head_v = 128
  311. print_info: n_gqa = 4
  312. print_info: n_embd_k_gqa = 1024
  313. print_info: n_embd_v_gqa = 1024
  314. print_info: f_norm_eps = 0.0e+00
  315. print_info: f_norm_rms_eps = 1.0e-05
  316. print_info: f_clamp_kqv = 0.0e+00
  317. print_info: f_max_alibi_bias = 0.0e+00
  318. print_info: f_logit_scale = 0.0e+00
  319. print_info: f_attn_scale = 0.0e+00
  320. print_info: n_ff = 14336
  321. print_info: n_expert = 0
  322. print_info: n_expert_used = 0
  323. print_info: causal attn = 1
  324. print_info: pooling type = 0
  325. print_info: rope type = 0
  326. print_info: rope scaling = linear
  327. print_info: freq_base_train = 500000.0
  328. print_info: freq_scale_train = 1
  329. print_info: n_ctx_orig_yarn = 8192
  330. print_info: rope_finetuned = unknown
  331. print_info: model type = ?B
  332. print_info: model params = 13.26 B
  333. print_info: general.name = hub
  334. print_info: vocab type = BPE
  335. print_info: n_vocab = 128256
  336. print_info: n_merges = 280147
  337. print_info: BOS token = 128000 '<|begin_of_text|>'
  338. print_info: EOS token = 128001 '<|end_of_text|>'
  339. print_info: EOT token = 128009 '<|eot_id|>'
  340. print_info: LF token = 198 'Ċ'
  341. print_info: EOG token = 128001 '<|end_of_text|>'
  342. print_info: EOG token = 128009 '<|eot_id|>'
  343. print_info: max token length = 256
  344. load_tensors: loading model tensors, this can take a while... (mmap = true)
  345. load_tensors: layer 0 assigned to device CPU, is_swa = 0
  346. load_tensors: layer 1 assigned to device CPU, is_swa = 0
  347. load_tensors: layer 2 assigned to device CPU, is_swa = 0
  348. load_tensors: layer 3 assigned to device CPU, is_swa = 0
  349. load_tensors: layer 4 assigned to device CPU, is_swa = 0
  350. load_tensors: layer 5 assigned to device CPU, is_swa = 0
  351. load_tensors: layer 6 assigned to device CPU, is_swa = 0
  352. load_tensors: layer 7 assigned to device CPU, is_swa = 0
  353. load_tensors: layer 8 assigned to device CPU, is_swa = 0
  354. load_tensors: layer 9 assigned to device CPU, is_swa = 0
  355. load_tensors: layer 10 assigned to device CPU, is_swa = 0
  356. load_tensors: layer 11 assigned to device CPU, is_swa = 0
  357. load_tensors: layer 12 assigned to device CPU, is_swa = 0
  358. load_tensors: layer 13 assigned to device CPU, is_swa = 0
  359. load_tensors: layer 14 assigned to device CPU, is_swa = 0
  360. load_tensors: layer 15 assigned to device CPU, is_swa = 0
  361. load_tensors: layer 16 assigned to device CPU, is_swa = 0
  362. load_tensors: layer 17 assigned to device CPU, is_swa = 0
  363. load_tensors: layer 18 assigned to device CPU, is_swa = 0
  364. load_tensors: layer 19 assigned to device CPU, is_swa = 0
  365. load_tensors: layer 20 assigned to device CPU, is_swa = 0
  366. load_tensors: layer 21 assigned to device CPU, is_swa = 0
  367. load_tensors: layer 22 assigned to device CPU, is_swa = 0
  368. load_tensors: layer 23 assigned to device CPU, is_swa = 0
  369. load_tensors: layer 24 assigned to device CPU, is_swa = 0
  370. load_tensors: layer 25 assigned to device CPU, is_swa = 0
  371. load_tensors: layer 26 assigned to device CPU, is_swa = 0
  372. load_tensors: layer 27 assigned to device CPU, is_swa = 0
  373. load_tensors: layer 28 assigned to device CPU, is_swa = 0
  374. load_tensors: layer 29 assigned to device CPU, is_swa = 0
  375. load_tensors: layer 30 assigned to device CPU, is_swa = 0
  376. load_tensors: layer 31 assigned to device CPU, is_swa = 0
  377. load_tensors: layer 32 assigned to device CPU, is_swa = 0
  378. load_tensors: layer 33 assigned to device CPU, is_swa = 0
  379. load_tensors: layer 34 assigned to device CPU, is_swa = 0
  380. load_tensors: layer 35 assigned to device CPU, is_swa = 0
  381. load_tensors: layer 36 assigned to device CPU, is_swa = 0
  382. load_tensors: layer 37 assigned to device CPU, is_swa = 0
  383. load_tensors: layer 38 assigned to device CPU, is_swa = 0
  384. load_tensors: layer 39 assigned to device CPU, is_swa = 0
  385. load_tensors: layer 40 assigned to device CPU, is_swa = 0
  386. load_tensors: layer 41 assigned to device CPU, is_swa = 0
  387. load_tensors: layer 42 assigned to device CPU, is_swa = 0
  388. load_tensors: layer 43 assigned to device CPU, is_swa = 0
  389. load_tensors: layer 44 assigned to device CPU, is_swa = 0
  390. load_tensors: layer 45 assigned to device CPU, is_swa = 0
  391. load_tensors: layer 46 assigned to device CPU, is_swa = 0
  392. load_tensors: layer 47 assigned to device CPU, is_swa = 0
  393. load_tensors: layer 48 assigned to device CPU, is_swa = 0
  394. load_tensors: layer 49 assigned to device CPU, is_swa = 0
  395. load_tensors: layer 50 assigned to device CPU, is_swa = 0
  396. load_tensors: layer 51 assigned to device CPU, is_swa = 0
  397. load_tensors: layer 52 assigned to device CPU, is_swa = 0
  398. load_tensors: layer 53 assigned to device CPU, is_swa = 0
  399. load_tensors: layer 54 assigned to device CPU, is_swa = 0
  400. load_tensors: layer 55 assigned to device CPU, is_swa = 0
  401. load_tensors: layer 56 assigned to device CPU, is_swa = 0
  402. load_tensors: tensor 'token_embd.weight' (q8_0) (and 506 others) cannot be used with preferred buffer type CPU_REPACK, using CPU instead
  403. load_tensors: CPU_Mapped model buffer size = 13442.39 MiB
  404. ...............................................................................................
  405. llama_context: constructing llama_context
  406. llama_context: n_seq_max = 1
  407. llama_context: n_ctx = 2048
  408. llama_context: n_ctx_per_seq = 2048
  409. llama_context: n_batch = 512
  410. llama_context: n_ubatch = 512
  411. llama_context: causal_attn = 1
  412. llama_context: flash_attn = 0
  413. llama_context: kv_unified = false
  414. llama_context: freq_base = 500000.0
  415. llama_context: freq_scale = 1
  416. llama_context: n_ctx_per_seq (2048) < n_ctx_train (8192) -- the full capacity of the model will not be utilized
  417. set_abort_callback: call
  418. llama_context: CPU output buffer size = 0.49 MiB
  419. create_memory: n_ctx = 2048 (padded)
  420. llama_kv_cache_unified: layer 0: dev = CPU
  421. llama_kv_cache_unified: layer 1: dev = CPU
  422. llama_kv_cache_unified: layer 2: dev = CPU
  423. llama_kv_cache_unified: layer 3: dev = CPU
  424. llama_kv_cache_unified: layer 4: dev = CPU
  425. llama_kv_cache_unified: layer 5: dev = CPU
  426. llama_kv_cache_unified: layer 6: dev = CPU
  427. llama_kv_cache_unified: layer 7: dev = CPU
  428. llama_kv_cache_unified: layer 8: dev = CPU
  429. llama_kv_cache_unified: layer 9: dev = CPU
  430. llama_kv_cache_unified: layer 10: dev = CPU
  431. llama_kv_cache_unified: layer 11: dev = CPU
  432. llama_kv_cache_unified: layer 12: dev = CPU
  433. llama_kv_cache_unified: layer 13: dev = CPU
  434. llama_kv_cache_unified: layer 14: dev = CPU
  435. llama_kv_cache_unified: layer 15: dev = CPU
  436. llama_kv_cache_unified: layer 16: dev = CPU
  437. llama_kv_cache_unified: layer 17: dev = CPU
  438. llama_kv_cache_unified: layer 18: dev = CPU
  439. llama_kv_cache_unified: layer 19: dev = CPU
  440. llama_kv_cache_unified: layer 20: dev = CPU
  441. llama_kv_cache_unified: layer 21: dev = CPU
  442. llama_kv_cache_unified: layer 22: dev = CPU
  443. llama_kv_cache_unified: layer 23: dev = CPU
  444. llama_kv_cache_unified: layer 24: dev = CPU
  445. llama_kv_cache_unified: layer 25: dev = CPU
  446. llama_kv_cache_unified: layer 26: dev = CPU
  447. llama_kv_cache_unified: layer 27: dev = CPU
  448. llama_kv_cache_unified: layer 28: dev = CPU
  449. llama_kv_cache_unified: layer 29: dev = CPU
  450. llama_kv_cache_unified: layer 30: dev = CPU
  451. llama_kv_cache_unified: layer 31: dev = CPU
  452. llama_kv_cache_unified: layer 32: dev = CPU
  453. llama_kv_cache_unified: layer 33: dev = CPU
  454. llama_kv_cache_unified: layer 34: dev = CPU
  455. llama_kv_cache_unified: layer 35: dev = CPU
  456. llama_kv_cache_unified: layer 36: dev = CPU
  457. llama_kv_cache_unified: layer 37: dev = CPU
  458. llama_kv_cache_unified: layer 38: dev = CPU
  459. llama_kv_cache_unified: layer 39: dev = CPU
  460. llama_kv_cache_unified: layer 40: dev = CPU
  461. llama_kv_cache_unified: layer 41: dev = CPU
  462. llama_kv_cache_unified: layer 42: dev = CPU
  463. llama_kv_cache_unified: layer 43: dev = CPU
  464. llama_kv_cache_unified: layer 44: dev = CPU
  465. llama_kv_cache_unified: layer 45: dev = CPU
  466. llama_kv_cache_unified: layer 46: dev = CPU
  467. llama_kv_cache_unified: layer 47: dev = CPU
  468. llama_kv_cache_unified: layer 48: dev = CPU
  469. llama_kv_cache_unified: layer 49: dev = CPU
  470. llama_kv_cache_unified: layer 50: dev = CPU
  471. llama_kv_cache_unified: layer 51: dev = CPU
  472. llama_kv_cache_unified: layer 52: dev = CPU
  473. llama_kv_cache_unified: layer 53: dev = CPU
  474. llama_kv_cache_unified: layer 54: dev = CPU
  475. llama_kv_cache_unified: layer 55: dev = CPU
  476. llama_kv_cache_unified: CPU KV buffer size = 448.00 MiB
  477. llama_kv_cache_unified: size = 448.00 MiB ( 2048 cells, 56 layers, 1/1 seqs), K (f16): 224.00 MiB, V (f16): 224.00 MiB
  478. llama_context: enumerating backends
  479. llama_context: backend_ptrs.size() = 1
  480. llama_context: max_nodes = 4056
  481. llama_context: worst-case: n_tokens = 512, n_seqs = 1, n_outputs = 0
  482. graph_reserve: reserving a graph for ubatch with n_tokens = 512, n_seqs = 1, n_outputs = 512
  483. graph_reserve: reserving a graph for ubatch with n_tokens = 1, n_seqs = 1, n_outputs = 1
  484. graph_reserve: reserving a graph for ubatch with n_tokens = 512, n_seqs = 1, n_outputs = 512
  485. llama_context: CPU compute buffer size = 258.50 MiB
  486. llama_context: graph nodes = 1966
  487. llama_context: graph splits = 1
  488. CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | AVX512 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
  489. Model metadata: {'general.name': 'hub', 'general.architecture': 'llama', 'llama.block_count': '56', 'llama.vocab_size': '128256', 'llama.context_length': '8192', 'llama.rope.dimension_count': '128', 'llama.embedding_length': '4096', 'llama.feed_forward_length': '14336', 'llama.attention.head_count': '32', 'tokenizer.ggml.eos_token_id': '128001', 'general.file_type': '7', 'llama.attention.head_count_kv': '8', 'llama.attention.layer_norm_rms_epsilon': '0.000010', 'llama.rope.freq_base': '500000.000000', 'tokenizer.ggml.model': 'gpt2', 'general.quantization_version': '2', 'tokenizer.ggml.bos_token_id': '128000', 'tokenizer.chat_template': "{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}"}
  490. Available chat formats from metadata: chat_template.default
  491. Using gguf chat template: {% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>
  492.  
  493. '+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{{ '<|start_header_id|>assistant<|end_header_id|>
  494.  
  495. ' }}
  496. Using chat eos_token: <|end_of_text|>
  497. Using chat bos_token: <|begin_of_text|>
  498. INFO: Started server process [7412]
  499. INFO: Waiting for application startup.
  500. INFO: Application startup complete.
  501. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!