API tools faq
paste
Lissanro

Kimi K2 crashes in ik_llama.cpp after the "Tool calls support from mainline" patch

Lissanro
Sep 4th, 2025
92
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 14.00 KB | None | 0 0
raw download clone embed print report
  1. ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
  2. ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
  3. ggml_cuda_init: found 4 CUDA devices:
  4. Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes
  5. Device 1: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes
  6. Device 2: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes
  7. Device 3: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes
  8. INFO [ main] build info | tid="131404752666624" timestamp=1756967001 build=3874 commit="3433c7b5"
  9. INFO [ main] system info | tid="131404752666624" timestamp=1756967001 n_threads=64 n_threads_batch=-1 total_threads=128 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | "
  10. llama_model_loader: loaded meta data with 48 key-value pairs and 1157 tensors from /mnt/neuro/models/Kimi-K2-Instruct/Kimi-K2-Instruct-IQ4_XS.gguf (version GGUF V3 (latest))
  11. llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
  12. llama_model_loader: - kv 0: general.architecture str = deepseek2
  13. llama_model_loader: - kv 1: general.type str = model
  14. llama_model_loader: - kv 2: general.name str = Kimi K2 Instruct BF16
  15. llama_model_loader: - kv 3: general.finetune str = Instruct
  16. llama_model_loader: - kv 4: general.basename str = Kimi-K2
  17. llama_model_loader: - kv 5: general.size_label str = 384x15B
  18. llama_model_loader: - kv 6: general.license str = other
  19. llama_model_loader: - kv 7: general.license.name str = modified-mit
  20. llama_model_loader: - kv 8: deepseek2.block_count u32 = 61
  21. llama_model_loader: - kv 9: deepseek2.context_length u32 = 131072
  22. llama_model_loader: - kv 10: deepseek2.embedding_length u32 = 7168
  23. llama_model_loader: - kv 11: deepseek2.feed_forward_length u32 = 18432
  24. llama_model_loader: - kv 12: deepseek2.attention.head_count u32 = 64
  25. llama_model_loader: - kv 13: deepseek2.attention.head_count_kv u32 = 64
  26. llama_model_loader: - kv 14: deepseek2.rope.freq_base f32 = 50000.000000
  27. llama_model_loader: - kv 15: deepseek2.attention.layer_norm_rms_epsilon f32 = 0.000001
  28. llama_model_loader: - kv 16: deepseek2.expert_used_count u32 = 8
  29. llama_model_loader: - kv 17: general.file_type u32 = 30
  30. llama_model_loader: - kv 18: deepseek2.leading_dense_block_count u32 = 1
  31. llama_model_loader: - kv 19: deepseek2.vocab_size u32 = 163840
  32. llama_model_loader: - kv 20: deepseek2.attention.q_lora_rank u32 = 1536
  33. llama_model_loader: - kv 21: deepseek2.attention.kv_lora_rank u32 = 512
  34. llama_model_loader: - kv 22: deepseek2.attention.key_length u32 = 192
  35. llama_model_loader: - kv 23: deepseek2.attention.value_length u32 = 128
  36. llama_model_loader: - kv 24: deepseek2.expert_feed_forward_length u32 = 2048
  37. llama_model_loader: - kv 25: deepseek2.expert_count u32 = 384
  38. llama_model_loader: - kv 26: deepseek2.expert_shared_count u32 = 1
  39. llama_model_loader: - kv 27: deepseek2.expert_weights_scale f32 = 2.827000
  40. llama_model_loader: - kv 28: deepseek2.expert_weights_norm bool = true
  41. llama_model_loader: - kv 29: deepseek2.expert_gating_func u32 = 2
  42. llama_model_loader: - kv 30: deepseek2.rope.dimension_count u32 = 64
  43. llama_model_loader: - kv 31: deepseek2.rope.scaling.type str = yarn
  44. llama_model_loader: - kv 32: deepseek2.rope.scaling.factor f32 = 32.000000
  45. llama_model_loader: - kv 33: deepseek2.rope.scaling.original_context_length u32 = 4096
  46. llama_model_loader: - kv 34: deepseek2.rope.scaling.yarn_log_multiplier f32 = 0.100000
  47. llama_model_loader: - kv 35: tokenizer.ggml.model str = gpt2
  48. llama_model_loader: - kv 36: tokenizer.ggml.pre str = kimi-k2
  49. llama_model_loader: - kv 37: tokenizer.ggml.tokens arr[str,163840] = ["!", "\"", "#", "$", "%", "&", "'", ...
  50. llama_model_loader: - kv 38: tokenizer.ggml.token_type arr[i32,163840] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
  51. llama_model_loader: - kv 39: tokenizer.ggml.merges arr[str,163328] = ["Ġ Ġ", "ĠĠ ĠĠ", "Ġ t", "i n",...
  52. llama_model_loader: - kv 40: tokenizer.ggml.bos_token_id u32 = 163584
  53. llama_model_loader: - kv 41: tokenizer.ggml.eos_token_id u32 = 163585
  54. llama_model_loader: - kv 42: tokenizer.chat_template str = {%- if tools -%}\n <|im_system|>tool_...
  55. llama_model_loader: - kv 43: general.quantization_version u32 = 2
  56. llama_model_loader: - kv 44: quantize.imatrix.file str = imatrix.dat
  57. llama_model_loader: - kv 45: quantize.imatrix.dataset str = /home/lissanro/pkgs/imatrix/compact-i...
  58. llama_model_loader: - kv 46: quantize.imatrix.entries_count i32 = 728
  59. llama_model_loader: - kv 47: quantize.imatrix.chunks_count i32 = 1390
  60. llama_model_loader: - type f32: 365 tensors
  61. llama_model_loader: - type q8_0: 183 tensors
  62. llama_model_loader: - type q5_K: 61 tensors
  63. llama_model_loader: - type q6_K: 1 tensors
  64. llama_model_loader: - type iq4_xs: 547 tensors
  65. init_tokenizer: initializing tokenizer for type 2
  66. load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
  67. load: printing all EOG tokens:
  68. load: - 163585 ('[EOS]')
  69. load: - 163586 ('<|im_end|>')
  70. load: special tokens cache size = 256
  71. load: token to piece cache size = 1.0607 MB
  72. llm_load_print_meta: format = GGUF V3 (latest)
  73. llm_load_print_meta: arch = deepseek2
  74. llm_load_print_meta: n_ctx_train = 131072
  75. llm_load_print_meta: n_embd = 7168
  76. llm_load_print_meta: n_layer = 61
  77. llm_load_print_meta: n_head = 64
  78. llm_load_print_meta: n_head_kv = 64
  79. llm_load_print_meta: n_rot = 64
  80. llm_load_print_meta: n_swa = 0
  81. llm_load_print_meta: n_swa_pattern = 1
  82. llm_load_print_meta: n_embd_head_k = 192
  83. llm_load_print_meta: n_embd_head_v = 128
  84. llm_load_print_meta: n_gqa = 1
  85. llm_load_print_meta: n_embd_k_gqa = 12288
  86. llm_load_print_meta: n_embd_v_gqa = 8192
  87. llm_load_print_meta: f_norm_eps = 0.0e+00
  88. llm_load_print_meta: f_norm_rms_eps = 1.0e-06
  89. llm_load_print_meta: f_clamp_kqv = 0.0e+00
  90. llm_load_print_meta: f_max_alibi_bias = 0.0e+00
  91. llm_load_print_meta: f_logit_scale = 0.0e+00
  92. llm_load_print_meta: n_ff = 18432
  93. llm_load_print_meta: n_expert = 384
  94. llm_load_print_meta: n_expert_used = 8
  95. llm_load_print_meta: causal attn = 1
  96. llm_load_print_meta: pooling type = 0
  97. llm_load_print_meta: rope type = 0
  98. llm_load_print_meta: rope scaling = yarn
  99. llm_load_print_meta: freq_base_train = 50000.0
  100. llm_load_print_meta: freq_scale_train = 0.03125
  101. llm_load_print_meta: n_ctx_orig_yarn = 4096
  102. llm_load_print_meta: rope_finetuned = unknown
  103. llm_load_print_meta: ssm_d_conv = 0
  104. llm_load_print_meta: ssm_d_inner = 0
  105. llm_load_print_meta: ssm_d_state = 0
  106. llm_load_print_meta: ssm_dt_rank = 0
  107. llm_load_print_meta: model type = 671B
  108. llm_load_print_meta: model ftype = IQ4_XS - 4.25 bpw
  109. llm_load_print_meta: model params = 1.027 T
  110. llm_load_print_meta: model size = 509.963 GiB (4.266 BPW)
  111. llm_load_print_meta: repeating layers = 508.485 GiB (4.263 BPW, 1024.571 B parameters)
  112. llm_load_print_meta: general.name = Kimi K2 Instruct BF16
  113. llm_load_print_meta: n_layer_dense_lead = 1
  114. llm_load_print_meta: n_lora_q = 1536
  115. llm_load_print_meta: n_lora_kv = 512
  116. llm_load_print_meta: n_ff_exp = 2048
  117. llm_load_print_meta: n_expert_shared = 1
  118. llm_load_print_meta: expert_weights_scale = 2.8
  119. llm_load_print_meta: expert_weights_norm = 1
  120. llm_load_print_meta: expert_gating_func = sigmoid
  121. llm_load_print_meta: rope_yarn_log_mul = 0.1000
  122. print_info: vocab type = BPE
  123. print_info: n_vocab = 163840
  124. print_info: n_merges = 163328
  125. print_info: BOS token = 163584 '[BOS]'
  126. print_info: EOS token = 163585 '[EOS]'
  127. print_info: EOT token = 163586 '<|im_end|>'
  128. print_info: LF token = 198 'Ċ'
  129. print_info: EOG token = 163585 '[EOS]'
  130. print_info: EOG token = 163586 '<|im_end|>'
  131. print_info: max token length = 512
  132. llm_load_tensors: ggml ctx size = 2.35 MiB
  133. Tensor blk.1.ffn_down_exps.weight buffer type overriden to CPU
  134. Tensor blk.2.ffn_down_exps.weight buffer type overriden to CPU
  135. Tensor blk.3.ffn_down_exps.weight buffer type overriden to CPU
  136. Tensor blk.3.ffn_up_exps.weight buffer type overriden to CUDA0
  137. Tensor blk.4.ffn_down_exps.weight buffer type overriden to CPU
  138. Tensor blk.4.ffn_up_exps.weight buffer type overriden to CUDA1
  139. Tensor blk.5.ffn_down_exps.weight buffer type overriden to CPU
  140. Tensor blk.5.ffn_up_exps.weight buffer type overriden to CUDA2
  141. Tensor blk.6.ffn_down_exps.weight buffer type overriden to CPU
  142. Tensor blk.6.ffn_up_exps.weight buffer type overriden to CUDA3
  143. Tensor blk.7.ffn_down_exps.weight buffer type overriden to CPU
  144. Tensor blk.8.ffn_down_exps.weight buffer type overriden to CPU
  145. Tensor blk.9.ffn_down_exps.weight buffer type overriden to CPU
  146. Tensor blk.10.ffn_down_exps.weight buffer type overriden to CPU
  147. Tensor blk.11.ffn_down_exps.weight buffer type overriden to CPU
  148. Tensor blk.12.ffn_down_exps.weight buffer type overriden to CPU
  149. Tensor blk.13.ffn_down_exps.weight buffer type overriden to CPU
  150. Tensor blk.14.ffn_down_exps.weight buffer type overriden to CPU
  151. Tensor blk.15.ffn_down_exps.weight buffer type overriden to CPU
  152. Tensor blk.16.ffn_down_exps.weight buffer type overriden to CPU
  153. Tensor blk.17.ffn_down_exps.weight buffer type overriden to CPU
  154. Tensor blk.18.ffn_down_exps.weight buffer type overriden to CPU
  155. Tensor blk.19.ffn_down_exps.weight buffer type overriden to CPU
  156. Tensor blk.20.ffn_down_exps.weight buffer type overriden to CPU
  157. Tensor blk.21.ffn_down_exps.weight buffer type overriden to CPU
  158. Tensor blk.22.ffn_down_exps.weight buffer type overriden to CPU
  159. Tensor blk.23.ffn_down_exps.weight buffer type overriden to CPU
  160. Tensor blk.24.ffn_down_exps.weight buffer type overriden to CPU
  161. Tensor blk.25.ffn_down_exps.weight buffer type overriden to CPU
  162. Tensor blk.26.ffn_down_exps.weight buffer type overriden to CPU
  163. Tensor blk.27.ffn_down_exps.weight buffer type overriden to CPU
  164. Tensor blk.28.ffn_down_exps.weight buffer type overriden to CPU
  165. Tensor blk.29.ffn_down_exps.weight buffer type overriden to CPU
  166. Tensor blk.30.ffn_down_exps.weight buffer type overriden to CPU
  167. Tensor blk.31.ffn_down_exps.weight buffer type overriden to CPU
  168. Tensor blk.32.ffn_down_exps.weight buffer type overriden to CPU
  169. Tensor blk.33.ffn_down_exps.weight buffer type overriden to CPU
  170. Tensor blk.34.ffn_down_exps.weight buffer type overriden to CPU
  171. Tensor blk.35.ffn_down_exps.weight buffer type overriden to CPU
  172. Tensor blk.36.ffn_down_exps.weight buffer type overriden to CPU
  173. Tensor blk.37.ffn_down_exps.weight buffer type overriden to CPU
  174. Tensor blk.38.ffn_down_exps.weight buffer type overriden to CPU
  175. Tensor blk.39.ffn_down_exps.weight buffer type overriden to CPU
  176. Tensor blk.40.ffn_down_exps.weight buffer type overriden to CPU
  177. Tensor blk.41.ffn_down_exps.weight buffer type overriden to CPU
  178. Tensor blk.42.ffn_down_exps.weight buffer type overriden to CPU
  179. Tensor blk.43.ffn_down_exps.weight buffer type overriden to CPU
  180. Tensor blk.44.ffn_down_exps.weight buffer type overriden to CPU
  181. Tensor blk.45.ffn_down_exps.weight buffer type overriden to CPU
  182. Tensor blk.46.ffn_down_exps.weight buffer type overriden to CPU
  183. Tensor blk.47.ffn_down_exps.weight buffer type overriden to CPU
  184. Tensor blk.48.ffn_down_exps.weight buffer type overriden to CPU
  185. Tensor blk.49.ffn_down_exps.weight buffer type overriden to CPU
  186. Tensor blk.50.ffn_down_exps.weight buffer type overriden to CPU
  187. Tensor blk.51.ffn_down_exps.weight buffer type overriden to CPU
  188. Tensor blk.52.ffn_down_exps.weight buffer type overriden to CPU
  189. Tensor blk.53.ffn_down_exps.weight buffer type overriden to CPU
  190. Tensor blk.54.ffn_down_exps.weight buffer type overriden to CPU
  191. Tensor blk.55.ffn_down_exps.weight buffer type overriden to CPU
  192. Tensor blk.56.ffn_down_exps.weight buffer type overriden to CPU
  193. Tensor blk.57.ffn_down_exps.weight buffer type overriden to CPU
  194. Tensor blk.58.ffn_down_exps.weight buffer type overriden to CPU
  195. Tensor blk.59.ffn_down_exps.weight buffer type overriden to CPU
  196. Tensor blk.60.ffn_down_exps.weight buffer type overriden to CPU
  197. ggml_backend_cuda_buffer_type_alloc_buffer: allocating 44063.76 MiB on device 0: cudaMalloc failed: out of memory
  198. llama_model_load: error loading model: unable to allocate backend buffer
  199. llama_load_model_from_file: failed to load model
  200. llama_init_from_gpt_params: error: failed to load model '/mnt/neuro/models/Kimi-K2-Instruct/Kimi-K2-Instruct-IQ4_XS.gguf'
  201. ERR [ load_model] unable to load model | tid="131404752666624" timestamp=1756967078 model="/mnt/neuro/models/Kimi-K2-Instruct/Kimi-K2-Instruct-IQ4_XS.gguf"
  202. free(): invalid pointer
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!