API tools faq
paste
Guest User

Untitled

a guest
Apr 7th, 2025
28
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 10.47 KB | None | 0 0
raw download clone embed print report
  1. ggml_vulkan: Found 1 Vulkan devices:
  2. ggml_vulkan: 0 = AMD Radeon RX 570 Series (RADV POLARIS10) (radv) | uma: 0 | fp16: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix cores: none
  3. build: 1 (6bf28f0) with cc (Debian 12.2.0-14) 12.2.0 for x86_64-linux-gnu
  4. system info: n_threads = 2, n_threads_batch = 2, total_threads = 4
  5.  
  6. system_info: n_threads = 2 (n_threads_batch = 2) / 4 | CPU : SSE3 = 1 | SSSE3 = 1 | LLAMAFILE = 1 | OPENMP = 1 | AARCH64_REPACK = 1 |
  7.  
  8. main: binding port with default address family
  9. main: HTTP server is listening, hostname: 0.0.0.0, port: 8080, http threads: 5
  10. main: loading model
  11. srv load_model: loading model 'L3-8B-Lunaris-v1-Q4_K_M.gguf'
  12. llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon RX 570 Series (RADV POLARIS10)) - 7936 MiB free
  13. llama_model_loader: loaded meta data with 26 key-value pairs and 291 tensors from L3-8B-Lunaris-v1-Q4_K_M.gguf (version GGUF V3 (latest))
  14. llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
  15. llama_model_loader: - kv 0: general.architecture str = llama
  16. llama_model_loader: - kv 1: general.name str = L3-8B-Lunaris-v1
  17. llama_model_loader: - kv 2: llama.block_count u32 = 32
  18. llama_model_loader: - kv 3: llama.context_length u32 = 8192
  19. llama_model_loader: - kv 4: llama.embedding_length u32 = 4096
  20. llama_model_loader: - kv 5: llama.feed_forward_length u32 = 14336
  21. llama_model_loader: - kv 6: llama.attention.head_count u32 = 32
  22. llama_model_loader: - kv 7: llama.attention.head_count_kv u32 = 8
  23. llama_model_loader: - kv 8: llama.rope.freq_base f32 = 500000.000000
  24. llama_model_loader: - kv 9: llama.attention.layer_norm_rms_epsilon f32 = 0.000010
  25. llama_model_loader: - kv 10: general.file_type u32 = 15
  26. llama_model_loader: - kv 11: llama.vocab_size u32 = 128256
  27. llama_model_loader: - kv 12: llama.rope.dimension_count u32 = 128
  28. llama_model_loader: - kv 13: tokenizer.ggml.model str = gpt2
  29. llama_model_loader: - kv 14: tokenizer.ggml.pre str = llama-bpe
  30. llama_model_loader: - kv 15: tokenizer.ggml.tokens arr[str,128256] = ["!", "\"", "#", "$", "%", "&", "'", ...
  31. llama_model_loader: - kv 16: tokenizer.ggml.token_type arr[i32,128256] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
  32. llama_model_loader: - kv 17: tokenizer.ggml.merges arr[str,280147] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
  33. llama_model_loader: - kv 18: tokenizer.ggml.bos_token_id u32 = 128000
  34. llama_model_loader: - kv 19: tokenizer.ggml.eos_token_id u32 = 128009
  35. llama_model_loader: - kv 20: tokenizer.chat_template str = {% set loop_messages = messages %}{% ...
  36. llama_model_loader: - kv 21: general.quantization_version u32 = 2
  37. llama_model_loader: - kv 22: quantize.imatrix.file str = /models/L3-8B-Lunaris-v1-GGUF/L3-8B-L...
  38. llama_model_loader: - kv 23: quantize.imatrix.dataset str = /training_data/calibration_datav3.txt
  39. llama_model_loader: - kv 24: quantize.imatrix.entries_count i32 = 224
  40. llama_model_loader: - kv 25: quantize.imatrix.chunks_count i32 = 125
  41. llama_model_loader: - type f32: 65 tensors
  42. llama_model_loader: - type q4_K: 193 tensors
  43. llama_model_loader: - type q6_K: 33 tensors
  44. print_info: file format = GGUF V3 (latest)
  45. print_info: file type = Q4_K - Medium
  46. print_info: file size = 4.58 GiB (4.89 BPW)
  47. load: special tokens cache size = 256
  48. load: token to piece cache size = 0.8000 MB
  49. print_info: arch = llama
  50. print_info: vocab_only = 0
  51. print_info: n_ctx_train = 8192
  52. print_info: n_embd = 4096
  53. print_info: n_layer = 32
  54. print_info: n_head = 32
  55. print_info: n_head_kv = 8
  56. print_info: n_rot = 128
  57. print_info: n_swa = 0
  58. print_info: n_swa_pattern = 1
  59. print_info: n_embd_head_k = 128
  60. print_info: n_embd_head_v = 128
  61. print_info: n_gqa = 4
  62. print_info: n_embd_k_gqa = 1024
  63. print_info: n_embd_v_gqa = 1024
  64. print_info: f_norm_eps = 0.0e+00
  65. print_info: f_norm_rms_eps = 1.0e-05
  66. print_info: f_clamp_kqv = 0.0e+00
  67. print_info: f_max_alibi_bias = 0.0e+00
  68. print_info: f_logit_scale = 0.0e+00
  69. print_info: f_attn_scale = 0.0e+00
  70. print_info: n_ff = 14336
  71. print_info: n_expert = 0
  72. print_info: n_expert_used = 0
  73. print_info: causal attn = 1
  74. print_info: pooling type = 0
  75. print_info: rope type = 0
  76. print_info: rope scaling = linear
  77. print_info: freq_base_train = 500000.0
  78. print_info: freq_scale_train = 1
  79. print_info: n_ctx_orig_yarn = 8192
  80. print_info: rope_finetuned = unknown
  81. print_info: ssm_d_conv = 0
  82. print_info: ssm_d_inner = 0
  83. print_info: ssm_d_state = 0
  84. print_info: ssm_dt_rank = 0
  85. print_info: ssm_dt_b_c_rms = 0
  86. print_info: model type = 8B
  87. print_info: model params = 8.03 B
  88. print_info: general.name = L3-8B-Lunaris-v1
  89. print_info: vocab type = BPE
  90. print_info: n_vocab = 128256
  91. print_info: n_merges = 280147
  92. print_info: BOS token = 128000 '<|begin_of_text|>'
  93. print_info: EOS token = 128009 '<|eot_id|>'
  94. print_info: EOT token = 128009 '<|eot_id|>'
  95. print_info: LF token = 198 'Ċ'
  96. print_info: EOG token = 128009 '<|eot_id|>'
  97. print_info: max token length = 256
  98. load_tensors: loading model tensors, this can take a while... (mmap = true)
  99. load_tensors: offloading 32 repeating layers to GPU
  100. load_tensors: offloading output layer to GPU
  101. load_tensors: offloaded 33/33 layers to GPU
  102. load_tensors: Vulkan0 model buffer size = 4403.49 MiB
  103. load_tensors: CPU_Mapped model buffer size = 281.81 MiB
  104. .......................................................................................
  105. llama_context: constructing llama_context
  106. llama_context: n_seq_max = 3
  107. llama_context: n_ctx = 8192
  108. llama_context: n_ctx_per_seq = 2730
  109. llama_context: n_batch = 2048
  110. llama_context: n_ubatch = 512
  111. llama_context: causal_attn = 1
  112. llama_context: flash_attn = 0
  113. llama_context: freq_base = 500000.0
  114. llama_context: freq_scale = 1
  115. llama_context: n_ctx_per_seq (2730) < n_ctx_train (8192) -- the full capacity of the model will not be utilized
  116. llama_context: Vulkan_Host output buffer size = 1.47 MiB
  117. init: kv_size = 8192, offload = 1, type_k = 'f16', type_v = 'f16', n_layer = 32, can_shift = 1
  118. init: Vulkan0 KV buffer size = 1024.00 MiB
  119. llama_context: KV self size = 1024.00 MiB, K (f16): 512.00 MiB, V (f16): 512.00 MiB
  120. llama_context: Vulkan0 compute buffer size = 560.00 MiB
  121. llama_context: Vulkan_Host compute buffer size = 24.01 MiB
  122. llama_context: graph nodes = 1094
  123. llama_context: graph splits = 2
  124. common_init_from_params: setting dry_penalty_last_n to ctx_size = 8192
  125. common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
  126. srv init: initializing slots, n_slots = 3
  127. slot init: id 0 | task -1 | new slot n_ctx_slot = 2730
  128. slot init: id 1 | task -1 | new slot n_ctx_slot = 2730
  129. slot init: id 2 | task -1 | new slot n_ctx_slot = 2730
  130. main: model loaded
  131. main: chat template, chat_template: {% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>
  132.  
  133. '+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>
  134.  
  135. ' }}{% endif %}, example_format: '<|start_header_id|>system<|end_header_id|>
  136.  
  137. You are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>
  138.  
  139. Hello<|eot_id|><|start_header_id|>assistant<|end_header_id|>
  140.  
  141. Hi there<|eot_id|><|start_header_id|>user<|end_header_id|>
  142.  
  143. How are you?<|eot_id|><|start_header_id|>assistant<|end_header_id|>
  144.  
  145. '
  146. main: server is listening on http://0.0.0.0:8080 - starting the main loop
  147. srv update_slots: all slots are idle
  148. srv log_server_r: request: GET / 192.168.230.116 200
  149. srv log_server_r: request: GET /favicon.ico 192.168.230.116 404
  150. srv params_from_: Chat format: Content-only
  151. slot launch_slot_: id 0 | task 0 | processing task
  152. slot update_slots: id 0 | task 0 | new prompt, n_ctx_slot = 2730, n_keep = 0, n_prompt_tokens = 30
  153. slot update_slots: id 0 | task 0 | kv cache rm [0, end)
  154. slot update_slots: id 0 | task 0 | prompt processing progress, n_past = 30, n_tokens = 30, progress = 1.000000
  155. slot update_slots: id 0 | task 0 | prompt done, n_past = 30, n_tokens = 30
  156. slot release: id 0 | task 0 | stop processing: n_past = 201, truncated = 0
  157. slot print_timing: id 0 | task 0 |
  158. prompt eval time = 8501.27 ms / 30 tokens ( 283.38 ms per token, 3.53 tokens per second)
  159. eval time = 7197.72 ms / 172 tokens ( 41.85 ms per token, 23.90 tokens per second)
  160. total time = 15698.99 ms / 202 tokens
  161. srv update_slots: all slots are idle
  162. srv log_server_r: request: POST /v1/chat/completions 192.168.230.116 200
  163. srv params_from_: Chat format: Content-only
  164. slot launch_slot_: id 0 | task 173 | processing task
  165. slot update_slots: id 0 | task 173 | new prompt, n_ctx_slot = 2730, n_keep = 0, n_prompt_tokens = 214
  166. slot update_slots: id 0 | task 173 | kv cache rm [201, end)
  167. slot update_slots: id 0 | task 173 | prompt processing progress, n_past = 214, n_tokens = 13, progress = 0.060748
  168. slot update_slots: id 0 | task 173 | prompt done, n_past = 214, n_tokens = 13
  169. slot release: id 0 | task 173 | stop processing: n_past = 476, truncated = 0
  170. slot print_timing: id 0 | task 173 |
  171. prompt eval time = 7094.93 ms / 13 tokens ( 545.76 ms per token, 1.83 tokens per second)
  172. eval time = 11213.61 ms / 263 tokens ( 42.64 ms per token, 23.45 tokens per second)
  173. total time = 18308.54 ms / 276 tokens
  174. srv update_slots: all slots are idle
  175.  
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!