API tools faq
paste
Lissanro

ik_llala.cpp generated bad output with long context when using Llama 4

Lissanro
Apr 20th, 2025
68
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 22.71 KB | None | 0 0
raw download clone embed print report
  1. > ~/pkgs/ik_llama.cpp/build/bin/llama-server --model /home/lissanro/neuro/Llama-4-Maverick-17B-128E-Instruct-GGUF-UD-Q4_K_XL-1048576seq/Llama-4-Maverick-17B-128E-Instruct-UD-Q4_K_XL-00001-of-00005.gguf --ctx-size 524288 --n-gpu-layers 49 --tensor-split 25,25,25,25 -fa -ctk q8_0 -ctv q8_0 -rtr --override-tensor "exps=CPU" --threads 64 --host 0.0.0.0 --port 5000
  2. ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
  3. ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
  4. ggml_cuda_init: found 4 CUDA devices:
  5. Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes
  6. Device 1: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes
  7. Device 2: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes
  8. Device 3: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes
  9. INFO [ main] build info | tid="124290453823488" timestamp=1745155819 build=3639 commit="3bb64d93"
  10. INFO [ main] system info | tid="124290453823488" timestamp=1745155819 n_threads=64 n_threads_batch=-1 total_threads=128 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | "
  11. llama_model_loader: additional 4 GGUFs metadata loaded.
  12. llama_model_loader: loaded meta data with 49 key-value pairs and 531 tensors from /home/lissanro/neuro/Llama-4-Maverick-17B-128E-Instruct-GGUF-UD-Q4_K_XL-1048576seq/Llama-4-Maverick-17B-128E-Instruct-UD-Q4_K_XL-00001-of-00005.gguf (version GGUF V3 (latest))
  13. llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
  14. llama_model_loader: - kv 0: general.architecture str = llama4
  15. llama_model_loader: - kv 1: general.type str = model
  16. llama_model_loader: - kv 2: general.name str = Llama-4-Maverick-17B-128E-Instruct
  17. llama_model_loader: - kv 3: general.quantized_by str = Unsloth
  18. llama_model_loader: - kv 4: general.size_label str = 128x17B
  19. llama_model_loader: - kv 5: general.license str = other
  20. llama_model_loader: - kv 6: general.license.name str = llama4
  21. llama_model_loader: - kv 7: general.repo_url str = https://huggingface.co/unsloth
  22. llama_model_loader: - kv 8: general.base_model.count u32 = 1
  23. llama_model_loader: - kv 9: general.base_model.0.name str = Llama 4 Maverick 17B 128E
  24. llama_model_loader: - kv 10: general.base_model.0.organization str = Meta Llama
  25. llama_model_loader: - kv 11: general.base_model.0.repo_url str = https://huggingface.co/meta-llama/Lla...
  26. llama_model_loader: - kv 12: general.tags arr[str,5] = ["facebook", "meta", "pytorch", "llam...
  27. llama_model_loader: - kv 13: general.languages arr[str,12] = ["ar", "de", "en", "es", "fr", "hi", ...
  28. llama_model_loader: - kv 14: llama4.block_count u32 = 48
  29. llama_model_loader: - kv 15: llama4.context_length u32 = 1048576
  30. llama_model_loader: - kv 16: llama4.embedding_length u32 = 5120
  31. llama_model_loader: - kv 17: llama4.feed_forward_length u32 = 16384
  32. llama_model_loader: - kv 18: llama4.attention.head_count u32 = 40
  33. llama_model_loader: - kv 19: llama4.attention.head_count_kv u32 = 8
  34. llama_model_loader: - kv 20: llama4.rope.freq_base f32 = 500000.000000
  35. llama_model_loader: - kv 21: llama4.attention.layer_norm_rms_epsilon f32 = 0.000010
  36. llama_model_loader: - kv 22: llama4.expert_count u32 = 128
  37. llama_model_loader: - kv 23: llama4.expert_used_count u32 = 1
  38. llama_model_loader: - kv 24: llama4.attention.key_length u32 = 128
  39. llama_model_loader: - kv 25: llama4.attention.value_length u32 = 128
  40. llama_model_loader: - kv 26: llama4.vocab_size u32 = 202048
  41. llama_model_loader: - kv 27: llama4.rope.dimension_count u32 = 128
  42. llama_model_loader: - kv 28: llama4.interleave_moe_layer_step u32 = 2
  43. llama_model_loader: - kv 29: llama4.expert_feed_forward_length u32 = 8192
  44. llama_model_loader: - kv 30: tokenizer.ggml.model str = gpt2
  45. llama_model_loader: - kv 31: tokenizer.ggml.pre str = llama4
  46. llama_model_loader: - kv 32: tokenizer.ggml.tokens arr[str,202048] = ["À", "Á", "õ", "ö", "÷", "ø", ...
  47. llama_model_loader: - kv 33: tokenizer.ggml.token_type arr[i32,202048] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
  48. llama_model_loader: - kv 34: tokenizer.ggml.merges arr[str,439802] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
  49. llama_model_loader: - kv 35: tokenizer.ggml.bos_token_id u32 = 200000
  50. llama_model_loader: - kv 36: tokenizer.ggml.eos_token_id u32 = 200008
  51. llama_model_loader: - kv 37: tokenizer.ggml.padding_token_id u32 = 201134
  52. llama_model_loader: - kv 38: tokenizer.chat_template str = {{- bos_token }}\n{%- if custom_tools ...
  53. llama_model_loader: - kv 39: tokenizer.ggml.add_bos_token bool = true
  54. llama_model_loader: - kv 40: general.quantization_version u32 = 2
  55. llama_model_loader: - kv 41: general.file_type u32 = 15
  56. llama_model_loader: - kv 42: quantize.imatrix.file str = /mnt/disks/unslothai/Llama-4-Maverick...
  57. llama_model_loader: - kv 43: quantize.imatrix.dataset str = unsloth_calibration_data_generic_long...
  58. llama_model_loader: - kv 44: quantize.imatrix.entries_count i32 = 423
  59. llama_model_loader: - kv 45: quantize.imatrix.chunks_count i32 = 1510
  60. llama_model_loader: - kv 46: split.no u16 = 0
  61. llama_model_loader: - kv 47: split.tensors.count i32 = 531
  62. llama_model_loader: - kv 48: split.count u16 = 5
  63. llama_model_loader: - type f32: 121 tensors
  64. llama_model_loader: - type q4_K: 325 tensors
  65. llama_model_loader: - type q6_K: 85 tensors
  66. llm_load_vocab: special tokens cache size = 1135
  67. llm_load_vocab: token to piece cache size = 1.3873 MB
  68. llm_load_print_meta: format = GGUF V3 (latest)
  69. llm_load_print_meta: arch = llama4
  70. llm_load_print_meta: vocab type = BPE
  71. llm_load_print_meta: n_vocab = 202048
  72. llm_load_print_meta: n_merges = 439802
  73. llm_load_print_meta: vocab_only = 0
  74. llm_load_print_meta: n_ctx_train = 1048576
  75. llm_load_print_meta: n_embd = 5120
  76. llm_load_print_meta: n_layer = 48
  77. llm_load_print_meta: n_head = 40
  78. llm_load_print_meta: n_head_kv = 8
  79. llm_load_print_meta: n_rot = 128
  80. llm_load_print_meta: n_swa = 1
  81. llm_load_print_meta: n_embd_head_k = 128
  82. llm_load_print_meta: n_embd_head_v = 128
  83. llm_load_print_meta: n_gqa = 5
  84. llm_load_print_meta: n_embd_k_gqa = 1024
  85. llm_load_print_meta: n_embd_v_gqa = 1024
  86. llm_load_print_meta: f_norm_eps = 0.0e+00
  87. llm_load_print_meta: f_norm_rms_eps = 1.0e-05
  88. llm_load_print_meta: f_clamp_kqv = 0.0e+00
  89. llm_load_print_meta: f_max_alibi_bias = 0.0e+00
  90. llm_load_print_meta: f_logit_scale = 0.0e+00
  91. llm_load_print_meta: n_ff = 16384
  92. llm_load_print_meta: n_expert = 128
  93. llm_load_print_meta: n_expert_used = 1
  94. llm_load_print_meta: causal attn = 1
  95. llm_load_print_meta: pooling type = 0
  96. llm_load_print_meta: rope type = 0
  97. llm_load_print_meta: rope scaling = linear
  98. llm_load_print_meta: freq_base_train = 500000.0
  99. llm_load_print_meta: freq_scale_train = 1
  100. llm_load_print_meta: n_ctx_orig_yarn = 1048576
  101. llm_load_print_meta: rope_finetuned = unknown
  102. llm_load_print_meta: ssm_d_conv = 0
  103. llm_load_print_meta: ssm_d_inner = 0
  104. llm_load_print_meta: ssm_d_state = 0
  105. llm_load_print_meta: ssm_dt_rank = 0
  106. llm_load_print_meta: model type = 17Bx128E (Maverick)
  107. llm_load_print_meta: model ftype = Q4_K - Medium
  108. llm_load_print_meta: model params = 400.712 B
  109. llm_load_print_meta: model size = 226.445 GiB (4.854 BPW)
  110. llm_load_print_meta: repeating layers = 225.113 GiB (4.851 BPW, 398.643 B parameters)
  111. llm_load_print_meta: general.name = Llama-4-Maverick-17B-128E-Instruct
  112. llm_load_print_meta: BOS token = 200000 '<|begin_of_text|>'
  113. llm_load_print_meta: EOS token = 200008 '<|eot|>'
  114. llm_load_print_meta: PAD token = 201134 '<|finetune_right_pad_id|>'
  115. llm_load_print_meta: LF token = 139 'Ä'
  116. llm_load_print_meta: max token length = 192
  117. llm_load_tensors: ggml ctx size = 1.19 MiB
  118. Tensor blk.1.ffn_gate_exps.weight buffer type overriden to CPU
  119. Tensor blk.1.ffn_down_exps.weight buffer type overriden to CPU
  120. Tensor blk.1.ffn_up_exps.weight buffer type overriden to CPU
  121. Tensor blk.3.ffn_gate_exps.weight buffer type overriden to CPU
  122. Tensor blk.3.ffn_down_exps.weight buffer type overriden to CPU
  123. Tensor blk.3.ffn_up_exps.weight buffer type overriden to CPU
  124. Tensor blk.5.ffn_gate_exps.weight buffer type overriden to CPU
  125. Tensor blk.5.ffn_down_exps.weight buffer type overriden to CPU
  126. Tensor blk.5.ffn_up_exps.weight buffer type overriden to CPU
  127. Tensor blk.7.ffn_gate_exps.weight buffer type overriden to CPU
  128. Tensor blk.7.ffn_down_exps.weight buffer type overriden to CPU
  129. Tensor blk.7.ffn_up_exps.weight buffer type overriden to CPU
  130. Tensor blk.9.ffn_gate_exps.weight buffer type overriden to CPU
  131. Tensor blk.9.ffn_down_exps.weight buffer type overriden to CPU
  132. Tensor blk.9.ffn_up_exps.weight buffer type overriden to CPU
  133. Tensor blk.11.ffn_gate_exps.weight buffer type overriden to CPU
  134. Tensor blk.11.ffn_down_exps.weight buffer type overriden to CPU
  135. Tensor blk.11.ffn_up_exps.weight buffer type overriden to CPU
  136. Tensor blk.13.ffn_gate_exps.weight buffer type overriden to CPU
  137. Tensor blk.13.ffn_down_exps.weight buffer type overriden to CPU
  138. Tensor blk.13.ffn_up_exps.weight buffer type overriden to CPU
  139. Tensor blk.15.ffn_gate_exps.weight buffer type overriden to CPU
  140. Tensor blk.15.ffn_down_exps.weight buffer type overriden to CPU
  141. Tensor blk.15.ffn_up_exps.weight buffer type overriden to CPU
  142. Tensor blk.17.ffn_gate_exps.weight buffer type overriden to CPU
  143. Tensor blk.17.ffn_down_exps.weight buffer type overriden to CPU
  144. Tensor blk.17.ffn_up_exps.weight buffer type overriden to CPU
  145. Tensor blk.19.ffn_gate_exps.weight buffer type overriden to CPU
  146. Tensor blk.19.ffn_down_exps.weight buffer type overriden to CPU
  147. Tensor blk.19.ffn_up_exps.weight buffer type overriden to CPU
  148. Tensor blk.21.ffn_gate_exps.weight buffer type overriden to CPU
  149. Tensor blk.21.ffn_down_exps.weight buffer type overriden to CPU
  150. Tensor blk.21.ffn_up_exps.weight buffer type overriden to CPU
  151. Tensor blk.23.ffn_gate_exps.weight buffer type overriden to CPU
  152. Tensor blk.23.ffn_down_exps.weight buffer type overriden to CPU
  153. Tensor blk.23.ffn_up_exps.weight buffer type overriden to CPU
  154. Tensor blk.25.ffn_gate_exps.weight buffer type overriden to CPU
  155. Tensor blk.25.ffn_down_exps.weight buffer type overriden to CPU
  156. Tensor blk.25.ffn_up_exps.weight buffer type overriden to CPU
  157. Tensor blk.27.ffn_gate_exps.weight buffer type overriden to CPU
  158. Tensor blk.27.ffn_down_exps.weight buffer type overriden to CPU
  159. Tensor blk.27.ffn_up_exps.weight buffer type overriden to CPU
  160. Tensor blk.29.ffn_gate_exps.weight buffer type overriden to CPU
  161. Tensor blk.29.ffn_down_exps.weight buffer type overriden to CPU
  162. Tensor blk.29.ffn_up_exps.weight buffer type overriden to CPU
  163. Tensor blk.31.ffn_gate_exps.weight buffer type overriden to CPU
  164. Tensor blk.31.ffn_down_exps.weight buffer type overriden to CPU
  165. Tensor blk.31.ffn_up_exps.weight buffer type overriden to CPU
  166. Tensor blk.33.ffn_gate_exps.weight buffer type overriden to CPU
  167. Tensor blk.33.ffn_down_exps.weight buffer type overriden to CPU
  168. Tensor blk.33.ffn_up_exps.weight buffer type overriden to CPU
  169. Tensor blk.35.ffn_gate_exps.weight buffer type overriden to CPU
  170. Tensor blk.35.ffn_down_exps.weight buffer type overriden to CPU
  171. Tensor blk.35.ffn_up_exps.weight buffer type overriden to CPU
  172. Tensor blk.37.ffn_gate_exps.weight buffer type overriden to CPU
  173. Tensor blk.37.ffn_down_exps.weight buffer type overriden to CPU
  174. Tensor blk.37.ffn_up_exps.weight buffer type overriden to CPU
  175. Tensor blk.39.ffn_gate_exps.weight buffer type overriden to CPU
  176. Tensor blk.39.ffn_down_exps.weight buffer type overriden to CPU
  177. Tensor blk.39.ffn_up_exps.weight buffer type overriden to CPU
  178. Tensor blk.41.ffn_gate_exps.weight buffer type overriden to CPU
  179. Tensor blk.41.ffn_down_exps.weight buffer type overriden to CPU
  180. Tensor blk.41.ffn_up_exps.weight buffer type overriden to CPU
  181. Tensor blk.43.ffn_gate_exps.weight buffer type overriden to CPU
  182. Tensor blk.43.ffn_down_exps.weight buffer type overriden to CPU
  183. Tensor blk.43.ffn_up_exps.weight buffer type overriden to CPU
  184. Tensor blk.45.ffn_gate_exps.weight buffer type overriden to CPU
  185. Tensor blk.45.ffn_down_exps.weight buffer type overriden to CPU
  186. Tensor blk.45.ffn_up_exps.weight buffer type overriden to CPU
  187. Tensor blk.47.ffn_gate_exps.weight buffer type overriden to CPU
  188. Tensor blk.47.ffn_down_exps.weight buffer type overriden to CPU
  189. Tensor blk.47.ffn_up_exps.weight buffer type overriden to CPU
  190. llm_load_tensors: offloading 48 repeating layers to GPU
  191. llm_load_tensors: offloading non-repeating layers to GPU
  192. llm_load_tensors: offloaded 49/49 layers to GPU
  193. llm_load_tensors: CPU buffer size = 223200.00 MiB
  194. llm_load_tensors: CUDA_Host buffer size = 554.94 MiB
  195. llm_load_tensors: CUDA0 buffer size = 2020.82 MiB
  196. llm_load_tensors: CUDA1 buffer size = 1826.25 MiB
  197. llm_load_tensors: CUDA2 buffer size = 1826.25 MiB
  198. llm_load_tensors: CUDA3 buffer size = 2451.30 MiB
  199. ............................................................................
  200. ============ Repacked 72 tensors
  201. llama_new_context_with_model: n_ctx = 524288
  202. llama_new_context_with_model: n_batch = 2048
  203. llama_new_context_with_model: n_ubatch = 512
  204. llama_new_context_with_model: flash_attn = 1
  205. llama_new_context_with_model: mla_attn = 0
  206. llama_new_context_with_model: attn_max_b = 0
  207. llama_new_context_with_model: fused_moe = 0
  208. llama_new_context_with_model: ser = -1, 0
  209. llama_new_context_with_model: freq_base = 500000.0
  210. llama_new_context_with_model: freq_scale = 1
  211. llama_kv_cache_init: CUDA0 KV buffer size = 14144.01 MiB
  212. llama_kv_cache_init: CUDA1 KV buffer size = 13056.01 MiB
  213. llama_kv_cache_init: CUDA2 KV buffer size = 13056.01 MiB
  214. llama_kv_cache_init: CUDA3 KV buffer size = 11968.01 MiB
  215. llama_new_context_with_model: KV self size = 52224.00 MiB, K (q8_0): 26112.00 MiB, V (q8_0): 26112.00 MiB
  216. llama_new_context_with_model: CUDA_Host output buffer size = 1.54 MiB
  217. llama_new_context_with_model: pipeline parallelism enabled (n_copies=4)
  218. llama_new_context_with_model: CUDA0 compute buffer size = 5014.02 MiB
  219. llama_new_context_with_model: CUDA1 compute buffer size = 2444.02 MiB
  220. llama_new_context_with_model: CUDA2 compute buffer size = 2444.02 MiB
  221. llama_new_context_with_model: CUDA3 compute buffer size = 2732.65 MiB
  222. llama_new_context_with_model: CUDA_Host compute buffer size = 5127.99 MiB
  223. llama_new_context_with_model: graph nodes = 1566
  224. llama_new_context_with_model: graph splits = 53
  225. INFO [ init] initializing slots | tid="124290453823488" timestamp=1745155956 n_slots=1
  226. INFO [ init] new slot | tid="124290453823488" timestamp=1745155956 id_slot=0 n_ctx_slot=524288
  227. INFO [ main] model loaded | tid="124290453823488" timestamp=1745155956
  228. INFO [ main] chat template | tid="124290453823488" timestamp=1745155956 chat_example="<|header_start|>system<|header_end|>\n\nYou are a helpful assistant<|eot|><|header_start|>user<|header_end|>\n\nHello<|eot|><|header_start|>assistant<|header_end|>\n\nHi there<|eot|><|header_start|>user<|header_end|>\n\nHow are you?<|eot|><|header_start|>assistant<|header_end|>\n\n" built_in=true
  229. INFO [ main] HTTP server listening | tid="124290453823488" timestamp=1745155956 n_threads_http="127" port="5000" hostname="0.0.0.0"
  230. INFO [ update_slots] all slots are idle | tid="124290453823488" timestamp=1745155956
  231. INFO [ log_server_request] request | tid="124286211710976" timestamp=1745155969 remote_addr="127.0.0.1" remote_port=44858 status=200 method="GET" path="/v1/models" params={}
  232. INFO [ log_server_request] request | tid="124286211710976" timestamp=1745155969 remote_addr="127.0.0.1" remote_port=44858 status=200 method="GET" path="/props" params={}
  233. INFO [ launch_slot_with_task] slot is processing task | tid="124290453823488" timestamp=1745155975 id_slot=0 id_task=0
  234. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745155975 id_slot=0 id_task=0 p0=0
  235. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745155980 id_slot=0 id_task=0 p0=2048
  236. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745155985 id_slot=0 id_task=0 p0=4096
  237. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745155990 id_slot=0 id_task=0 p0=6144
  238. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745155995 id_slot=0 id_task=0 p0=8192
  239. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156000 id_slot=0 id_task=0 p0=10240
  240. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156005 id_slot=0 id_task=0 p0=12288
  241. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156009 id_slot=0 id_task=0 p0=14336
  242. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156013 id_slot=0 id_task=0 p0=16384
  243. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156018 id_slot=0 id_task=0 p0=18432
  244. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156023 id_slot=0 id_task=0 p0=20480
  245. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156027 id_slot=0 id_task=0 p0=22528
  246. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156032 id_slot=0 id_task=0 p0=24576
  247. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156037 id_slot=0 id_task=0 p0=26624
  248. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156042 id_slot=0 id_task=0 p0=28672
  249. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156047 id_slot=0 id_task=0 p0=30720
  250. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156052 id_slot=0 id_task=0 p0=32768
  251. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156057 id_slot=0 id_task=0 p0=34816
  252. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156062 id_slot=0 id_task=0 p0=36864
  253. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156067 id_slot=0 id_task=0 p0=38912
  254. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156073 id_slot=0 id_task=0 p0=40960
  255. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156078 id_slot=0 id_task=0 p0=43008
  256. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156084 id_slot=0 id_task=0 p0=45056
  257. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156090 id_slot=0 id_task=0 p0=47104
  258. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156096 id_slot=0 id_task=0 p0=49152
  259. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156102 id_slot=0 id_task=0 p0=51200
  260. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156108 id_slot=0 id_task=0 p0=53248
  261. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156114 id_slot=0 id_task=0 p0=55296
  262. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156120 id_slot=0 id_task=0 p0=57344
  263. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156127 id_slot=0 id_task=0 p0=59392
  264. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156133 id_slot=0 id_task=0 p0=61440
  265. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156139 id_slot=0 id_task=0 p0=63488
  266. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156146 id_slot=0 id_task=0 p0=65536
  267. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156153 id_slot=0 id_task=0 p0=67584
  268. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156160 id_slot=0 id_task=0 p0=69632
  269. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745156167 id_slot=0 id_task=0 p0=71680
  270. INFO [ log_server_request] request | tid="124286203318272" timestamp=1745157360 remote_addr="127.0.0.1" remote_port=44870 status=200 method="POST" path="/completion" params={}
  271. INFO [ update_slots] slot released | tid="124290453823488" timestamp=1745157360 id_slot=0 id_task=0 n_ctx=524288 n_past=85484 n_system_tokens=0 n_cache_tokens=85484 truncated=false
  272. INFO [ update_slots] all slots are idle | tid="124290453823488" timestamp=1745157360
  273. INFO [ launch_slot_with_task] slot is processing task | tid="124290453823488" timestamp=1745157360 id_slot=0 id_task=12285
  274. INFO [ update_slots] kv cache rm [p0, end) | tid="124290453823488" timestamp=1745157361 id_slot=0 id_task=12285 p0=73234
  275. ^C^CReceived second interrupt, terminating immediately.
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!