API tools faq
paste
Lissanro

Failure to convert Kimi K2 FP8 to BF16

Lissanro
Jul 25th, 2025 (edited)
60
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 119.01 KB | None | 0 0
raw download clone embed print report
  1. > python3 ~/pkgs/llama.cpp-fp8-to-bf16/llama.cpp/convert_hf_to_gguf.py --outtype bf16 --outfile /mnt/neuro/models/Kimi-K2-Instruct-BF16.gguf /mnt/neuro/models/Kimi-K2-Instruct/
  2. INFO:hf-to-gguf:Loading model: Kimi-K2-Instruct
  3. INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only
  4. INFO:hf-to-gguf:Exporting model...
  5. INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json'
  6. INFO:hf-to-gguf:gguf: loading model part 'model-1-of-61.safetensors'
  7. INFO:hf-to-gguf:gguf: loading model part 'model-10-of-61.safetensors'
  8. INFO:hf-to-gguf:gguf: loading model part 'model-11-of-61.safetensors'
  9. INFO:hf-to-gguf:gguf: loading model part 'model-12-of-61.safetensors'
  10. INFO:hf-to-gguf:gguf: loading model part 'model-13-of-61.safetensors'
  11. INFO:hf-to-gguf:gguf: loading model part 'model-14-of-61.safetensors'
  12. INFO:hf-to-gguf:gguf: loading model part 'model-15-of-61.safetensors'
  13. INFO:hf-to-gguf:gguf: loading model part 'model-16-of-61.safetensors'
  14. INFO:hf-to-gguf:gguf: loading model part 'model-17-of-61.safetensors'
  15. INFO:hf-to-gguf:gguf: loading model part 'model-18-of-61.safetensors'
  16. INFO:hf-to-gguf:gguf: loading model part 'model-19-of-61.safetensors'
  17. INFO:hf-to-gguf:gguf: loading model part 'model-2-of-61.safetensors'
  18. INFO:hf-to-gguf:gguf: loading model part 'model-20-of-61.safetensors'
  19. INFO:hf-to-gguf:gguf: loading model part 'model-21-of-61.safetensors'
  20. INFO:hf-to-gguf:gguf: loading model part 'model-22-of-61.safetensors'
  21. INFO:hf-to-gguf:gguf: loading model part 'model-23-of-61.safetensors'
  22. INFO:hf-to-gguf:gguf: loading model part 'model-24-of-61.safetensors'
  23. INFO:hf-to-gguf:gguf: loading model part 'model-25-of-61.safetensors'
  24. INFO:hf-to-gguf:gguf: loading model part 'model-26-of-61.safetensors'
  25. INFO:hf-to-gguf:gguf: loading model part 'model-27-of-61.safetensors'
  26. INFO:hf-to-gguf:gguf: loading model part 'model-28-of-61.safetensors'
  27. INFO:hf-to-gguf:gguf: loading model part 'model-29-of-61.safetensors'
  28. INFO:hf-to-gguf:gguf: loading model part 'model-3-of-61.safetensors'
  29. INFO:hf-to-gguf:gguf: loading model part 'model-30-of-61.safetensors'
  30. INFO:hf-to-gguf:gguf: loading model part 'model-31-of-61.safetensors'
  31. INFO:hf-to-gguf:gguf: loading model part 'model-32-of-61.safetensors'
  32. INFO:hf-to-gguf:gguf: loading model part 'model-33-of-61.safetensors'
  33. INFO:hf-to-gguf:gguf: loading model part 'model-34-of-61.safetensors'
  34. INFO:hf-to-gguf:gguf: loading model part 'model-35-of-61.safetensors'
  35. INFO:hf-to-gguf:gguf: loading model part 'model-36-of-61.safetensors'
  36. INFO:hf-to-gguf:gguf: loading model part 'model-37-of-61.safetensors'
  37. INFO:hf-to-gguf:gguf: loading model part 'model-38-of-61.safetensors'
  38. INFO:hf-to-gguf:gguf: loading model part 'model-39-of-61.safetensors'
  39. INFO:hf-to-gguf:gguf: loading model part 'model-4-of-61.safetensors'
  40. INFO:hf-to-gguf:gguf: loading model part 'model-40-of-61.safetensors'
  41. INFO:hf-to-gguf:gguf: loading model part 'model-41-of-61.safetensors'
  42. INFO:hf-to-gguf:gguf: loading model part 'model-42-of-61.safetensors'
  43. INFO:hf-to-gguf:gguf: loading model part 'model-43-of-61.safetensors'
  44. INFO:hf-to-gguf:gguf: loading model part 'model-44-of-61.safetensors'
  45. INFO:hf-to-gguf:gguf: loading model part 'model-45-of-61.safetensors'
  46. INFO:hf-to-gguf:gguf: loading model part 'model-46-of-61.safetensors'
  47. INFO:hf-to-gguf:gguf: loading model part 'model-47-of-61.safetensors'
  48. INFO:hf-to-gguf:gguf: loading model part 'model-48-of-61.safetensors'
  49. INFO:hf-to-gguf:gguf: loading model part 'model-49-of-61.safetensors'
  50. INFO:hf-to-gguf:gguf: loading model part 'model-5-of-61.safetensors'
  51. INFO:hf-to-gguf:gguf: loading model part 'model-50-of-61.safetensors'
  52. INFO:hf-to-gguf:gguf: loading model part 'model-51-of-61.safetensors'
  53. INFO:hf-to-gguf:gguf: loading model part 'model-52-of-61.safetensors'
  54. INFO:hf-to-gguf:gguf: loading model part 'model-53-of-61.safetensors'
  55. INFO:hf-to-gguf:gguf: loading model part 'model-54-of-61.safetensors'
  56. INFO:hf-to-gguf:gguf: loading model part 'model-55-of-61.safetensors'
  57. INFO:hf-to-gguf:gguf: loading model part 'model-56-of-61.safetensors'
  58. INFO:hf-to-gguf:gguf: loading model part 'model-57-of-61.safetensors'
  59. INFO:hf-to-gguf:gguf: loading model part 'model-58-of-61.safetensors'
  60. INFO:hf-to-gguf:gguf: loading model part 'model-59-of-61.safetensors'
  61. INFO:hf-to-gguf:gguf: loading model part 'model-6-of-61.safetensors'
  62. INFO:hf-to-gguf:gguf: loading model part 'model-60-of-61.safetensors'
  63. INFO:hf-to-gguf:gguf: loading model part 'model-61-of-61.safetensors'
  64. INFO:hf-to-gguf:gguf: loading model part 'model-7-of-61.safetensors'
  65. INFO:hf-to-gguf:gguf: loading model part 'model-8-of-61.safetensors'
  66. INFO:hf-to-gguf:gguf: loading model part 'model-9-of-61.safetensors'
  67. INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json'
  68. INFO:hf-to-gguf:gguf: loading model part 'model-1-of-61.safetensors'
  69. INFO:hf-to-gguf:token_embd.weight, torch.bfloat16 --> BF16, shape = {7168, 163840}
  70. INFO:hf-to-gguf:blk.0.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  71. INFO:hf-to-gguf:blk.0.ffn_down.weight, torch.float8_e4m3fn --> BF16, shape = {18432, 7168}
  72. INFO:hf-to-gguf:blk.0.ffn_gate.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 18432}
  73. INFO:hf-to-gguf:blk.0.ffn_up.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 18432}
  74. INFO:hf-to-gguf:blk.0.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  75. INFO:hf-to-gguf:blk.0.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  76. INFO:hf-to-gguf:blk.0.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  77. INFO:hf-to-gguf:blk.0.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  78. INFO:hf-to-gguf:blk.0.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  79. INFO:hf-to-gguf:blk.0.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  80. INFO:hf-to-gguf:blk.0.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  81. INFO:hf-to-gguf:blk.0.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  82. INFO:hf-to-gguf:blk.0.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  83. INFO:hf-to-gguf:blk.0.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  84. INFO:hf-to-gguf:gguf: loading model part 'model-10-of-61.safetensors'
  85. INFO:hf-to-gguf:blk.9.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  86. INFO:hf-to-gguf:blk.9.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  87. INFO:hf-to-gguf:blk.9.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  88. INFO:hf-to-gguf:blk.9.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  89. INFO:hf-to-gguf:blk.9.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  90. INFO:hf-to-gguf:blk.9.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  91. INFO:hf-to-gguf:blk.9.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  92. INFO:hf-to-gguf:blk.9.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  93. INFO:hf-to-gguf:blk.9.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  94. INFO:hf-to-gguf:blk.9.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  95. INFO:hf-to-gguf:blk.9.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  96. INFO:hf-to-gguf:blk.9.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  97. INFO:hf-to-gguf:blk.9.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  98. INFO:hf-to-gguf:blk.9.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  99. INFO:hf-to-gguf:blk.9.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  100. INFO:hf-to-gguf:blk.9.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  101. INFO:hf-to-gguf:blk.9.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  102. INFO:hf-to-gguf:blk.9.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  103. INFO:hf-to-gguf:blk.9.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  104. INFO:hf-to-gguf:gguf: loading model part 'model-11-of-61.safetensors'
  105. INFO:hf-to-gguf:blk.10.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  106. INFO:hf-to-gguf:blk.10.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  107. INFO:hf-to-gguf:blk.10.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  108. INFO:hf-to-gguf:blk.10.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  109. INFO:hf-to-gguf:blk.10.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  110. INFO:hf-to-gguf:blk.10.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  111. INFO:hf-to-gguf:blk.10.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  112. INFO:hf-to-gguf:blk.10.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  113. INFO:hf-to-gguf:blk.10.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  114. INFO:hf-to-gguf:blk.10.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  115. INFO:hf-to-gguf:blk.10.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  116. INFO:hf-to-gguf:blk.10.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  117. INFO:hf-to-gguf:blk.10.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  118. INFO:hf-to-gguf:blk.10.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  119. INFO:hf-to-gguf:blk.10.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  120. INFO:hf-to-gguf:blk.10.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  121. INFO:hf-to-gguf:blk.10.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  122. INFO:hf-to-gguf:blk.10.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  123. INFO:hf-to-gguf:blk.10.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  124. INFO:hf-to-gguf:gguf: loading model part 'model-12-of-61.safetensors'
  125. INFO:hf-to-gguf:blk.11.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  126. INFO:hf-to-gguf:blk.11.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  127. INFO:hf-to-gguf:blk.11.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  128. INFO:hf-to-gguf:blk.11.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  129. INFO:hf-to-gguf:blk.11.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  130. INFO:hf-to-gguf:blk.11.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  131. INFO:hf-to-gguf:blk.11.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  132. INFO:hf-to-gguf:blk.11.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  133. INFO:hf-to-gguf:blk.11.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  134. INFO:hf-to-gguf:blk.11.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  135. INFO:hf-to-gguf:blk.11.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  136. INFO:hf-to-gguf:blk.11.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  137. INFO:hf-to-gguf:blk.11.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  138. INFO:hf-to-gguf:blk.11.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  139. INFO:hf-to-gguf:blk.11.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  140. INFO:hf-to-gguf:blk.11.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  141. INFO:hf-to-gguf:blk.11.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  142. INFO:hf-to-gguf:blk.11.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  143. INFO:hf-to-gguf:blk.11.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  144. INFO:hf-to-gguf:gguf: loading model part 'model-13-of-61.safetensors'
  145. INFO:hf-to-gguf:blk.12.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  146. INFO:hf-to-gguf:blk.12.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  147. INFO:hf-to-gguf:blk.12.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  148. INFO:hf-to-gguf:blk.12.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  149. INFO:hf-to-gguf:blk.12.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  150. INFO:hf-to-gguf:blk.12.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  151. INFO:hf-to-gguf:blk.12.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  152. INFO:hf-to-gguf:blk.12.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  153. INFO:hf-to-gguf:blk.12.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  154. INFO:hf-to-gguf:blk.12.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  155. INFO:hf-to-gguf:blk.12.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  156. INFO:hf-to-gguf:blk.12.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  157. INFO:hf-to-gguf:blk.12.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  158. INFO:hf-to-gguf:blk.12.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  159. INFO:hf-to-gguf:blk.12.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  160. INFO:hf-to-gguf:blk.12.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  161. INFO:hf-to-gguf:blk.12.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  162. INFO:hf-to-gguf:blk.12.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  163. INFO:hf-to-gguf:blk.12.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  164. INFO:hf-to-gguf:gguf: loading model part 'model-14-of-61.safetensors'
  165. INFO:hf-to-gguf:blk.13.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  166. INFO:hf-to-gguf:blk.13.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  167. INFO:hf-to-gguf:blk.13.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  168. INFO:hf-to-gguf:blk.13.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  169. INFO:hf-to-gguf:blk.13.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  170. INFO:hf-to-gguf:blk.13.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  171. INFO:hf-to-gguf:blk.13.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  172. INFO:hf-to-gguf:blk.13.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  173. INFO:hf-to-gguf:blk.13.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  174. INFO:hf-to-gguf:blk.13.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  175. INFO:hf-to-gguf:blk.13.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  176. INFO:hf-to-gguf:blk.13.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  177. INFO:hf-to-gguf:blk.13.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  178. INFO:hf-to-gguf:blk.13.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  179. INFO:hf-to-gguf:blk.13.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  180. INFO:hf-to-gguf:blk.13.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  181. INFO:hf-to-gguf:blk.13.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  182. INFO:hf-to-gguf:blk.13.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  183. INFO:hf-to-gguf:blk.13.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  184. INFO:hf-to-gguf:gguf: loading model part 'model-15-of-61.safetensors'
  185. INFO:hf-to-gguf:blk.14.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  186. INFO:hf-to-gguf:blk.14.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  187. INFO:hf-to-gguf:blk.14.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  188. INFO:hf-to-gguf:blk.14.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  189. INFO:hf-to-gguf:blk.14.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  190. INFO:hf-to-gguf:blk.14.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  191. INFO:hf-to-gguf:blk.14.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  192. INFO:hf-to-gguf:blk.14.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  193. INFO:hf-to-gguf:blk.14.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  194. INFO:hf-to-gguf:blk.14.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  195. INFO:hf-to-gguf:blk.14.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  196. INFO:hf-to-gguf:blk.14.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  197. INFO:hf-to-gguf:blk.14.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  198. INFO:hf-to-gguf:blk.14.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  199. INFO:hf-to-gguf:blk.14.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  200. INFO:hf-to-gguf:blk.14.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  201. INFO:hf-to-gguf:blk.14.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  202. INFO:hf-to-gguf:blk.14.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  203. INFO:hf-to-gguf:blk.14.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  204. INFO:hf-to-gguf:gguf: loading model part 'model-16-of-61.safetensors'
  205. INFO:hf-to-gguf:blk.15.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  206. INFO:hf-to-gguf:blk.15.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  207. INFO:hf-to-gguf:blk.15.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  208. INFO:hf-to-gguf:blk.15.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  209. INFO:hf-to-gguf:blk.15.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  210. INFO:hf-to-gguf:blk.15.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  211. INFO:hf-to-gguf:blk.15.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  212. INFO:hf-to-gguf:blk.15.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  213. INFO:hf-to-gguf:blk.15.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  214. INFO:hf-to-gguf:blk.15.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  215. INFO:hf-to-gguf:blk.15.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  216. INFO:hf-to-gguf:blk.15.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  217. INFO:hf-to-gguf:blk.15.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  218. INFO:hf-to-gguf:blk.15.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  219. INFO:hf-to-gguf:blk.15.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  220. INFO:hf-to-gguf:blk.15.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  221. INFO:hf-to-gguf:blk.15.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  222. INFO:hf-to-gguf:blk.15.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  223. INFO:hf-to-gguf:blk.15.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  224. INFO:hf-to-gguf:gguf: loading model part 'model-17-of-61.safetensors'
  225. INFO:hf-to-gguf:blk.16.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  226. INFO:hf-to-gguf:blk.16.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  227. INFO:hf-to-gguf:blk.16.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  228. INFO:hf-to-gguf:blk.16.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  229. INFO:hf-to-gguf:blk.16.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  230. INFO:hf-to-gguf:blk.16.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  231. INFO:hf-to-gguf:blk.16.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  232. INFO:hf-to-gguf:blk.16.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  233. INFO:hf-to-gguf:blk.16.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  234. INFO:hf-to-gguf:blk.16.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  235. INFO:hf-to-gguf:blk.16.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  236. INFO:hf-to-gguf:blk.16.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  237. INFO:hf-to-gguf:blk.16.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  238. INFO:hf-to-gguf:blk.16.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  239. INFO:hf-to-gguf:blk.16.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  240. INFO:hf-to-gguf:blk.16.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  241. INFO:hf-to-gguf:blk.16.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  242. INFO:hf-to-gguf:blk.16.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  243. INFO:hf-to-gguf:blk.16.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  244. INFO:hf-to-gguf:gguf: loading model part 'model-18-of-61.safetensors'
  245. INFO:hf-to-gguf:blk.17.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  246. INFO:hf-to-gguf:blk.17.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  247. INFO:hf-to-gguf:blk.17.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  248. INFO:hf-to-gguf:blk.17.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  249. INFO:hf-to-gguf:blk.17.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  250. INFO:hf-to-gguf:blk.17.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  251. INFO:hf-to-gguf:blk.17.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  252. INFO:hf-to-gguf:blk.17.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  253. INFO:hf-to-gguf:blk.17.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  254. INFO:hf-to-gguf:blk.17.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  255. INFO:hf-to-gguf:blk.17.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  256. INFO:hf-to-gguf:blk.17.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  257. INFO:hf-to-gguf:blk.17.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  258. INFO:hf-to-gguf:blk.17.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  259. INFO:hf-to-gguf:blk.17.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  260. INFO:hf-to-gguf:blk.17.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  261. INFO:hf-to-gguf:blk.17.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  262. INFO:hf-to-gguf:blk.17.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  263. INFO:hf-to-gguf:blk.17.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  264. INFO:hf-to-gguf:gguf: loading model part 'model-19-of-61.safetensors'
  265. INFO:hf-to-gguf:blk.18.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  266. INFO:hf-to-gguf:blk.18.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  267. INFO:hf-to-gguf:blk.18.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  268. INFO:hf-to-gguf:blk.18.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  269. INFO:hf-to-gguf:blk.18.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  270. INFO:hf-to-gguf:blk.18.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  271. INFO:hf-to-gguf:blk.18.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  272. INFO:hf-to-gguf:blk.18.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  273. INFO:hf-to-gguf:blk.18.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  274. INFO:hf-to-gguf:blk.18.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  275. INFO:hf-to-gguf:blk.18.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  276. INFO:hf-to-gguf:blk.18.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  277. INFO:hf-to-gguf:blk.18.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  278. INFO:hf-to-gguf:blk.18.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  279. INFO:hf-to-gguf:blk.18.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  280. INFO:hf-to-gguf:blk.18.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  281. INFO:hf-to-gguf:blk.18.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  282. INFO:hf-to-gguf:blk.18.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  283. INFO:hf-to-gguf:blk.18.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  284. INFO:hf-to-gguf:gguf: loading model part 'model-2-of-61.safetensors'
  285. INFO:hf-to-gguf:blk.1.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  286. INFO:hf-to-gguf:blk.1.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  287. INFO:hf-to-gguf:blk.1.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  288. INFO:hf-to-gguf:blk.1.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  289. INFO:hf-to-gguf:blk.1.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  290. INFO:hf-to-gguf:blk.1.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  291. INFO:hf-to-gguf:blk.1.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  292. INFO:hf-to-gguf:blk.1.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  293. INFO:hf-to-gguf:blk.1.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  294. INFO:hf-to-gguf:blk.1.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  295. INFO:hf-to-gguf:blk.1.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  296. INFO:hf-to-gguf:blk.1.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  297. INFO:hf-to-gguf:blk.1.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  298. INFO:hf-to-gguf:blk.1.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  299. INFO:hf-to-gguf:blk.1.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  300. INFO:hf-to-gguf:blk.1.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  301. INFO:hf-to-gguf:blk.1.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  302. INFO:hf-to-gguf:blk.1.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  303. INFO:hf-to-gguf:blk.1.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  304. INFO:hf-to-gguf:gguf: loading model part 'model-20-of-61.safetensors'
  305. INFO:hf-to-gguf:blk.19.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  306. INFO:hf-to-gguf:blk.19.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  307. INFO:hf-to-gguf:blk.19.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  308. INFO:hf-to-gguf:blk.19.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  309. INFO:hf-to-gguf:blk.19.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  310. INFO:hf-to-gguf:blk.19.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  311. INFO:hf-to-gguf:blk.19.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  312. INFO:hf-to-gguf:blk.19.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  313. INFO:hf-to-gguf:blk.19.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  314. INFO:hf-to-gguf:blk.19.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  315. INFO:hf-to-gguf:blk.19.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  316. INFO:hf-to-gguf:blk.19.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  317. INFO:hf-to-gguf:blk.19.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  318. INFO:hf-to-gguf:blk.19.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  319. INFO:hf-to-gguf:blk.19.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  320. INFO:hf-to-gguf:blk.19.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  321. INFO:hf-to-gguf:blk.19.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  322. INFO:hf-to-gguf:blk.19.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  323. INFO:hf-to-gguf:blk.19.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  324. INFO:hf-to-gguf:gguf: loading model part 'model-21-of-61.safetensors'
  325. INFO:hf-to-gguf:blk.20.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  326. INFO:hf-to-gguf:blk.20.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  327. INFO:hf-to-gguf:blk.20.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  328. INFO:hf-to-gguf:blk.20.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  329. INFO:hf-to-gguf:blk.20.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  330. INFO:hf-to-gguf:blk.20.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  331. INFO:hf-to-gguf:blk.20.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  332. INFO:hf-to-gguf:blk.20.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  333. INFO:hf-to-gguf:blk.20.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  334. INFO:hf-to-gguf:blk.20.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  335. INFO:hf-to-gguf:blk.20.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  336. INFO:hf-to-gguf:blk.20.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  337. INFO:hf-to-gguf:blk.20.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  338. INFO:hf-to-gguf:blk.20.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  339. INFO:hf-to-gguf:blk.20.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  340. INFO:hf-to-gguf:blk.20.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  341. INFO:hf-to-gguf:blk.20.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  342. INFO:hf-to-gguf:blk.20.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  343. INFO:hf-to-gguf:blk.20.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  344. INFO:hf-to-gguf:gguf: loading model part 'model-22-of-61.safetensors'
  345. INFO:hf-to-gguf:blk.21.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  346. INFO:hf-to-gguf:blk.21.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  347. INFO:hf-to-gguf:blk.21.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  348. INFO:hf-to-gguf:blk.21.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  349. INFO:hf-to-gguf:blk.21.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  350. INFO:hf-to-gguf:blk.21.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  351. INFO:hf-to-gguf:blk.21.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  352. INFO:hf-to-gguf:blk.21.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  353. INFO:hf-to-gguf:blk.21.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  354. INFO:hf-to-gguf:blk.21.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  355. INFO:hf-to-gguf:blk.21.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  356. INFO:hf-to-gguf:blk.21.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  357. INFO:hf-to-gguf:blk.21.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  358. INFO:hf-to-gguf:blk.21.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  359. INFO:hf-to-gguf:blk.21.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  360. INFO:hf-to-gguf:blk.21.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  361. INFO:hf-to-gguf:blk.21.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  362. INFO:hf-to-gguf:blk.21.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  363. INFO:hf-to-gguf:blk.21.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  364. INFO:hf-to-gguf:gguf: loading model part 'model-23-of-61.safetensors'
  365. INFO:hf-to-gguf:blk.22.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  366. INFO:hf-to-gguf:blk.22.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  367. INFO:hf-to-gguf:blk.22.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  368. INFO:hf-to-gguf:blk.22.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  369. INFO:hf-to-gguf:blk.22.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  370. INFO:hf-to-gguf:blk.22.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  371. INFO:hf-to-gguf:blk.22.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  372. INFO:hf-to-gguf:blk.22.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  373. INFO:hf-to-gguf:blk.22.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  374. INFO:hf-to-gguf:blk.22.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  375. INFO:hf-to-gguf:blk.22.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  376. INFO:hf-to-gguf:blk.22.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  377. INFO:hf-to-gguf:blk.22.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  378. INFO:hf-to-gguf:blk.22.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  379. INFO:hf-to-gguf:blk.22.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  380. INFO:hf-to-gguf:blk.22.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  381. INFO:hf-to-gguf:blk.22.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  382. INFO:hf-to-gguf:blk.22.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  383. INFO:hf-to-gguf:blk.22.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  384. INFO:hf-to-gguf:gguf: loading model part 'model-24-of-61.safetensors'
  385. INFO:hf-to-gguf:blk.23.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  386. INFO:hf-to-gguf:blk.23.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  387. INFO:hf-to-gguf:blk.23.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  388. INFO:hf-to-gguf:blk.23.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  389. INFO:hf-to-gguf:blk.23.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  390. INFO:hf-to-gguf:blk.23.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  391. INFO:hf-to-gguf:blk.23.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  392. INFO:hf-to-gguf:blk.23.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  393. INFO:hf-to-gguf:blk.23.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  394. INFO:hf-to-gguf:blk.23.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  395. INFO:hf-to-gguf:blk.23.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  396. INFO:hf-to-gguf:blk.23.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  397. INFO:hf-to-gguf:blk.23.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  398. INFO:hf-to-gguf:blk.23.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  399. INFO:hf-to-gguf:blk.23.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  400. INFO:hf-to-gguf:blk.23.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  401. INFO:hf-to-gguf:blk.23.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  402. INFO:hf-to-gguf:blk.23.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  403. INFO:hf-to-gguf:blk.23.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  404. INFO:hf-to-gguf:gguf: loading model part 'model-25-of-61.safetensors'
  405. INFO:hf-to-gguf:blk.24.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  406. INFO:hf-to-gguf:blk.24.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  407. INFO:hf-to-gguf:blk.24.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  408. INFO:hf-to-gguf:blk.24.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  409. INFO:hf-to-gguf:blk.24.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  410. INFO:hf-to-gguf:blk.24.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  411. INFO:hf-to-gguf:blk.24.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  412. INFO:hf-to-gguf:blk.24.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  413. INFO:hf-to-gguf:blk.24.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  414. INFO:hf-to-gguf:blk.24.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  415. INFO:hf-to-gguf:blk.24.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  416. INFO:hf-to-gguf:blk.24.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  417. INFO:hf-to-gguf:blk.24.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  418. INFO:hf-to-gguf:blk.24.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  419. INFO:hf-to-gguf:blk.24.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  420. INFO:hf-to-gguf:blk.24.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  421. INFO:hf-to-gguf:blk.24.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  422. INFO:hf-to-gguf:blk.24.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  423. INFO:hf-to-gguf:blk.24.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  424. INFO:hf-to-gguf:gguf: loading model part 'model-26-of-61.safetensors'
  425. INFO:hf-to-gguf:blk.25.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  426. INFO:hf-to-gguf:blk.25.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  427. INFO:hf-to-gguf:blk.25.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  428. INFO:hf-to-gguf:blk.25.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  429. INFO:hf-to-gguf:blk.25.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  430. INFO:hf-to-gguf:blk.25.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  431. INFO:hf-to-gguf:blk.25.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  432. INFO:hf-to-gguf:blk.25.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  433. INFO:hf-to-gguf:blk.25.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  434. INFO:hf-to-gguf:blk.25.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  435. INFO:hf-to-gguf:blk.25.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  436. INFO:hf-to-gguf:blk.25.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  437. INFO:hf-to-gguf:blk.25.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  438. INFO:hf-to-gguf:blk.25.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  439. INFO:hf-to-gguf:blk.25.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  440. INFO:hf-to-gguf:blk.25.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  441. INFO:hf-to-gguf:blk.25.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  442. INFO:hf-to-gguf:blk.25.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  443. INFO:hf-to-gguf:blk.25.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  444. INFO:hf-to-gguf:gguf: loading model part 'model-27-of-61.safetensors'
  445. INFO:hf-to-gguf:blk.26.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  446. INFO:hf-to-gguf:blk.26.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  447. INFO:hf-to-gguf:blk.26.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  448. INFO:hf-to-gguf:blk.26.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  449. INFO:hf-to-gguf:blk.26.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  450. INFO:hf-to-gguf:blk.26.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  451. INFO:hf-to-gguf:blk.26.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  452. INFO:hf-to-gguf:blk.26.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  453. INFO:hf-to-gguf:blk.26.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  454. INFO:hf-to-gguf:blk.26.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  455. INFO:hf-to-gguf:blk.26.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  456. INFO:hf-to-gguf:blk.26.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  457. INFO:hf-to-gguf:blk.26.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  458. INFO:hf-to-gguf:blk.26.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  459. INFO:hf-to-gguf:blk.26.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  460. INFO:hf-to-gguf:blk.26.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  461. INFO:hf-to-gguf:blk.26.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  462. INFO:hf-to-gguf:blk.26.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  463. INFO:hf-to-gguf:blk.26.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  464. INFO:hf-to-gguf:gguf: loading model part 'model-28-of-61.safetensors'
  465. INFO:hf-to-gguf:blk.27.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  466. INFO:hf-to-gguf:blk.27.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  467. INFO:hf-to-gguf:blk.27.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  468. INFO:hf-to-gguf:blk.27.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  469. INFO:hf-to-gguf:blk.27.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  470. INFO:hf-to-gguf:blk.27.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  471. INFO:hf-to-gguf:blk.27.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  472. INFO:hf-to-gguf:blk.27.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  473. INFO:hf-to-gguf:blk.27.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  474. INFO:hf-to-gguf:blk.27.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  475. INFO:hf-to-gguf:blk.27.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  476. INFO:hf-to-gguf:blk.27.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  477. INFO:hf-to-gguf:blk.27.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  478. INFO:hf-to-gguf:blk.27.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  479. INFO:hf-to-gguf:blk.27.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  480. INFO:hf-to-gguf:blk.27.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  481. INFO:hf-to-gguf:blk.27.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  482. INFO:hf-to-gguf:blk.27.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  483. INFO:hf-to-gguf:blk.27.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  484. INFO:hf-to-gguf:gguf: loading model part 'model-29-of-61.safetensors'
  485. INFO:hf-to-gguf:blk.28.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  486. INFO:hf-to-gguf:blk.28.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  487. INFO:hf-to-gguf:blk.28.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  488. INFO:hf-to-gguf:blk.28.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  489. INFO:hf-to-gguf:blk.28.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  490. INFO:hf-to-gguf:blk.28.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  491. INFO:hf-to-gguf:blk.28.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  492. INFO:hf-to-gguf:blk.28.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  493. INFO:hf-to-gguf:blk.28.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  494. INFO:hf-to-gguf:blk.28.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  495. INFO:hf-to-gguf:blk.28.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  496. INFO:hf-to-gguf:blk.28.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  497. INFO:hf-to-gguf:blk.28.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  498. INFO:hf-to-gguf:blk.28.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  499. INFO:hf-to-gguf:blk.28.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  500. INFO:hf-to-gguf:blk.28.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  501. INFO:hf-to-gguf:blk.28.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  502. INFO:hf-to-gguf:blk.28.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  503. INFO:hf-to-gguf:blk.28.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  504. INFO:hf-to-gguf:gguf: loading model part 'model-3-of-61.safetensors'
  505. INFO:hf-to-gguf:blk.2.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  506. INFO:hf-to-gguf:blk.2.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  507. INFO:hf-to-gguf:blk.2.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  508. INFO:hf-to-gguf:blk.2.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  509. INFO:hf-to-gguf:blk.2.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  510. INFO:hf-to-gguf:blk.2.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  511. INFO:hf-to-gguf:blk.2.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  512. INFO:hf-to-gguf:blk.2.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  513. INFO:hf-to-gguf:blk.2.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  514. INFO:hf-to-gguf:blk.2.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  515. INFO:hf-to-gguf:blk.2.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  516. INFO:hf-to-gguf:blk.2.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  517. INFO:hf-to-gguf:blk.2.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  518. INFO:hf-to-gguf:blk.2.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  519. INFO:hf-to-gguf:blk.2.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  520. INFO:hf-to-gguf:blk.2.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  521. INFO:hf-to-gguf:blk.2.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  522. INFO:hf-to-gguf:blk.2.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  523. INFO:hf-to-gguf:blk.2.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  524. INFO:hf-to-gguf:gguf: loading model part 'model-30-of-61.safetensors'
  525. INFO:hf-to-gguf:blk.29.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  526. INFO:hf-to-gguf:blk.29.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  527. INFO:hf-to-gguf:blk.29.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  528. INFO:hf-to-gguf:blk.29.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  529. INFO:hf-to-gguf:blk.29.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  530. INFO:hf-to-gguf:blk.29.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  531. INFO:hf-to-gguf:blk.29.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  532. INFO:hf-to-gguf:blk.29.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  533. INFO:hf-to-gguf:blk.29.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  534. INFO:hf-to-gguf:blk.29.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  535. INFO:hf-to-gguf:blk.29.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  536. INFO:hf-to-gguf:blk.29.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  537. INFO:hf-to-gguf:blk.29.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  538. INFO:hf-to-gguf:blk.29.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  539. INFO:hf-to-gguf:blk.29.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  540. INFO:hf-to-gguf:blk.29.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  541. INFO:hf-to-gguf:blk.29.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  542. INFO:hf-to-gguf:blk.29.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  543. INFO:hf-to-gguf:blk.29.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  544. INFO:hf-to-gguf:gguf: loading model part 'model-31-of-61.safetensors'
  545. INFO:hf-to-gguf:blk.30.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  546. INFO:hf-to-gguf:blk.30.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  547. INFO:hf-to-gguf:blk.30.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  548. INFO:hf-to-gguf:blk.30.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  549. INFO:hf-to-gguf:blk.30.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  550. INFO:hf-to-gguf:blk.30.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  551. INFO:hf-to-gguf:blk.30.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  552. INFO:hf-to-gguf:blk.30.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  553. INFO:hf-to-gguf:blk.30.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  554. INFO:hf-to-gguf:blk.30.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  555. INFO:hf-to-gguf:blk.30.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  556. INFO:hf-to-gguf:blk.30.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  557. INFO:hf-to-gguf:blk.30.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  558. INFO:hf-to-gguf:blk.30.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  559. INFO:hf-to-gguf:blk.30.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  560. INFO:hf-to-gguf:blk.30.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  561. INFO:hf-to-gguf:blk.30.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  562. INFO:hf-to-gguf:blk.30.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  563. INFO:hf-to-gguf:blk.30.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  564. INFO:hf-to-gguf:gguf: loading model part 'model-32-of-61.safetensors'
  565. INFO:hf-to-gguf:blk.31.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  566. INFO:hf-to-gguf:blk.31.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  567. INFO:hf-to-gguf:blk.31.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  568. INFO:hf-to-gguf:blk.31.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  569. INFO:hf-to-gguf:blk.31.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  570. INFO:hf-to-gguf:blk.31.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  571. INFO:hf-to-gguf:blk.31.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  572. INFO:hf-to-gguf:blk.31.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  573. INFO:hf-to-gguf:blk.31.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  574. INFO:hf-to-gguf:blk.31.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  575. INFO:hf-to-gguf:blk.31.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  576. INFO:hf-to-gguf:blk.31.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  577. INFO:hf-to-gguf:blk.31.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  578. INFO:hf-to-gguf:blk.31.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  579. INFO:hf-to-gguf:blk.31.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  580. INFO:hf-to-gguf:blk.31.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  581. INFO:hf-to-gguf:blk.31.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  582. INFO:hf-to-gguf:blk.31.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  583. INFO:hf-to-gguf:blk.31.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  584. INFO:hf-to-gguf:gguf: loading model part 'model-33-of-61.safetensors'
  585. INFO:hf-to-gguf:blk.32.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  586. INFO:hf-to-gguf:blk.32.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  587. INFO:hf-to-gguf:blk.32.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  588. INFO:hf-to-gguf:blk.32.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  589. INFO:hf-to-gguf:blk.32.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  590. INFO:hf-to-gguf:blk.32.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  591. INFO:hf-to-gguf:blk.32.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  592. INFO:hf-to-gguf:blk.32.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  593. INFO:hf-to-gguf:blk.32.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  594. INFO:hf-to-gguf:blk.32.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  595. INFO:hf-to-gguf:blk.32.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  596. INFO:hf-to-gguf:blk.32.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  597. INFO:hf-to-gguf:blk.32.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  598. INFO:hf-to-gguf:blk.32.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  599. INFO:hf-to-gguf:blk.32.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  600. INFO:hf-to-gguf:blk.32.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  601. INFO:hf-to-gguf:blk.32.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  602. INFO:hf-to-gguf:blk.32.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  603. INFO:hf-to-gguf:blk.32.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  604. INFO:hf-to-gguf:gguf: loading model part 'model-34-of-61.safetensors'
  605. INFO:hf-to-gguf:blk.33.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  606. INFO:hf-to-gguf:blk.33.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  607. INFO:hf-to-gguf:blk.33.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  608. INFO:hf-to-gguf:blk.33.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  609. INFO:hf-to-gguf:blk.33.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  610. INFO:hf-to-gguf:blk.33.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  611. INFO:hf-to-gguf:blk.33.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  612. INFO:hf-to-gguf:blk.33.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  613. INFO:hf-to-gguf:blk.33.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  614. INFO:hf-to-gguf:blk.33.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  615. INFO:hf-to-gguf:blk.33.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  616. INFO:hf-to-gguf:blk.33.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  617. INFO:hf-to-gguf:blk.33.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  618. INFO:hf-to-gguf:blk.33.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  619. INFO:hf-to-gguf:blk.33.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  620. INFO:hf-to-gguf:blk.33.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  621. INFO:hf-to-gguf:blk.33.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  622. INFO:hf-to-gguf:blk.33.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  623. INFO:hf-to-gguf:blk.33.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  624. INFO:hf-to-gguf:gguf: loading model part 'model-35-of-61.safetensors'
  625. INFO:hf-to-gguf:blk.34.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  626. INFO:hf-to-gguf:blk.34.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  627. INFO:hf-to-gguf:blk.34.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  628. INFO:hf-to-gguf:blk.34.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  629. INFO:hf-to-gguf:blk.34.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  630. INFO:hf-to-gguf:blk.34.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  631. INFO:hf-to-gguf:blk.34.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  632. INFO:hf-to-gguf:blk.34.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  633. INFO:hf-to-gguf:blk.34.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  634. INFO:hf-to-gguf:blk.34.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  635. INFO:hf-to-gguf:blk.34.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  636. INFO:hf-to-gguf:blk.34.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  637. INFO:hf-to-gguf:blk.34.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  638. INFO:hf-to-gguf:blk.34.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  639. INFO:hf-to-gguf:blk.34.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  640. INFO:hf-to-gguf:blk.34.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  641. INFO:hf-to-gguf:blk.34.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  642. INFO:hf-to-gguf:blk.34.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  643. INFO:hf-to-gguf:blk.34.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  644. INFO:hf-to-gguf:gguf: loading model part 'model-36-of-61.safetensors'
  645. INFO:hf-to-gguf:blk.35.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  646. INFO:hf-to-gguf:blk.35.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  647. INFO:hf-to-gguf:blk.35.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  648. INFO:hf-to-gguf:blk.35.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  649. INFO:hf-to-gguf:blk.35.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  650. INFO:hf-to-gguf:blk.35.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  651. INFO:hf-to-gguf:blk.35.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  652. INFO:hf-to-gguf:blk.35.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  653. INFO:hf-to-gguf:blk.35.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  654. INFO:hf-to-gguf:blk.35.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  655. INFO:hf-to-gguf:blk.35.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  656. INFO:hf-to-gguf:blk.35.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  657. INFO:hf-to-gguf:blk.35.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  658. INFO:hf-to-gguf:blk.35.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  659. INFO:hf-to-gguf:blk.35.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  660. INFO:hf-to-gguf:blk.35.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  661. INFO:hf-to-gguf:blk.35.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  662. INFO:hf-to-gguf:blk.35.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  663. INFO:hf-to-gguf:blk.35.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  664. INFO:hf-to-gguf:gguf: loading model part 'model-37-of-61.safetensors'
  665. INFO:hf-to-gguf:blk.36.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  666. INFO:hf-to-gguf:blk.36.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  667. INFO:hf-to-gguf:blk.36.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  668. INFO:hf-to-gguf:blk.36.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  669. INFO:hf-to-gguf:blk.36.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  670. INFO:hf-to-gguf:blk.36.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  671. INFO:hf-to-gguf:blk.36.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  672. INFO:hf-to-gguf:blk.36.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  673. INFO:hf-to-gguf:blk.36.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  674. INFO:hf-to-gguf:blk.36.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  675. INFO:hf-to-gguf:blk.36.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  676. INFO:hf-to-gguf:blk.36.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  677. INFO:hf-to-gguf:blk.36.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  678. INFO:hf-to-gguf:blk.36.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  679. INFO:hf-to-gguf:blk.36.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  680. INFO:hf-to-gguf:blk.36.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  681. INFO:hf-to-gguf:blk.36.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  682. INFO:hf-to-gguf:blk.36.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  683. INFO:hf-to-gguf:blk.36.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  684. INFO:hf-to-gguf:gguf: loading model part 'model-38-of-61.safetensors'
  685. INFO:hf-to-gguf:blk.37.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  686. INFO:hf-to-gguf:blk.37.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  687. INFO:hf-to-gguf:blk.37.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  688. INFO:hf-to-gguf:blk.37.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  689. INFO:hf-to-gguf:blk.37.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  690. INFO:hf-to-gguf:blk.37.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  691. INFO:hf-to-gguf:blk.37.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  692. INFO:hf-to-gguf:blk.37.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  693. INFO:hf-to-gguf:blk.37.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  694. INFO:hf-to-gguf:blk.37.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  695. INFO:hf-to-gguf:blk.37.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  696. INFO:hf-to-gguf:blk.37.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  697. INFO:hf-to-gguf:blk.37.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  698. INFO:hf-to-gguf:blk.37.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  699. INFO:hf-to-gguf:blk.37.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  700. INFO:hf-to-gguf:blk.37.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  701. INFO:hf-to-gguf:blk.37.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  702. INFO:hf-to-gguf:blk.37.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  703. INFO:hf-to-gguf:blk.37.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  704. INFO:hf-to-gguf:gguf: loading model part 'model-39-of-61.safetensors'
  705. INFO:hf-to-gguf:blk.38.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  706. INFO:hf-to-gguf:blk.38.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  707. INFO:hf-to-gguf:blk.38.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  708. INFO:hf-to-gguf:blk.38.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  709. INFO:hf-to-gguf:blk.38.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  710. INFO:hf-to-gguf:blk.38.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  711. INFO:hf-to-gguf:blk.38.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  712. INFO:hf-to-gguf:blk.38.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  713. INFO:hf-to-gguf:blk.38.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  714. INFO:hf-to-gguf:blk.38.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  715. INFO:hf-to-gguf:blk.38.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  716. INFO:hf-to-gguf:blk.38.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  717. INFO:hf-to-gguf:blk.38.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  718. INFO:hf-to-gguf:blk.38.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  719. INFO:hf-to-gguf:blk.38.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  720. INFO:hf-to-gguf:blk.38.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  721. INFO:hf-to-gguf:blk.38.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  722. INFO:hf-to-gguf:blk.38.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  723. INFO:hf-to-gguf:blk.38.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  724. INFO:hf-to-gguf:gguf: loading model part 'model-4-of-61.safetensors'
  725. INFO:hf-to-gguf:blk.3.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  726. INFO:hf-to-gguf:blk.3.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  727. INFO:hf-to-gguf:blk.3.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  728. INFO:hf-to-gguf:blk.3.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  729. INFO:hf-to-gguf:blk.3.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  730. INFO:hf-to-gguf:blk.3.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  731. INFO:hf-to-gguf:blk.3.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  732. INFO:hf-to-gguf:blk.3.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  733. INFO:hf-to-gguf:blk.3.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  734. INFO:hf-to-gguf:blk.3.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  735. INFO:hf-to-gguf:blk.3.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  736. INFO:hf-to-gguf:blk.3.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  737. INFO:hf-to-gguf:blk.3.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  738. INFO:hf-to-gguf:blk.3.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  739. INFO:hf-to-gguf:blk.3.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  740. INFO:hf-to-gguf:blk.3.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  741. INFO:hf-to-gguf:blk.3.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  742. INFO:hf-to-gguf:blk.3.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  743. INFO:hf-to-gguf:blk.3.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  744. INFO:hf-to-gguf:gguf: loading model part 'model-40-of-61.safetensors'
  745. INFO:hf-to-gguf:blk.39.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  746. INFO:hf-to-gguf:blk.39.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  747. INFO:hf-to-gguf:blk.39.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  748. INFO:hf-to-gguf:blk.39.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  749. INFO:hf-to-gguf:blk.39.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  750. INFO:hf-to-gguf:blk.39.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  751. INFO:hf-to-gguf:blk.39.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  752. INFO:hf-to-gguf:blk.39.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  753. INFO:hf-to-gguf:blk.39.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  754. INFO:hf-to-gguf:blk.39.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  755. INFO:hf-to-gguf:blk.39.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  756. INFO:hf-to-gguf:blk.39.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  757. INFO:hf-to-gguf:blk.39.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  758. INFO:hf-to-gguf:blk.39.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  759. INFO:hf-to-gguf:blk.39.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  760. INFO:hf-to-gguf:blk.39.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  761. INFO:hf-to-gguf:blk.39.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  762. INFO:hf-to-gguf:blk.39.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  763. INFO:hf-to-gguf:blk.39.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  764. INFO:hf-to-gguf:gguf: loading model part 'model-41-of-61.safetensors'
  765. INFO:hf-to-gguf:blk.40.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  766. INFO:hf-to-gguf:blk.40.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  767. INFO:hf-to-gguf:blk.40.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  768. INFO:hf-to-gguf:blk.40.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  769. INFO:hf-to-gguf:blk.40.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  770. INFO:hf-to-gguf:blk.40.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  771. INFO:hf-to-gguf:blk.40.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  772. INFO:hf-to-gguf:blk.40.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  773. INFO:hf-to-gguf:blk.40.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  774. INFO:hf-to-gguf:blk.40.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  775. INFO:hf-to-gguf:blk.40.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  776. INFO:hf-to-gguf:blk.40.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  777. INFO:hf-to-gguf:blk.40.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  778. INFO:hf-to-gguf:blk.40.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  779. INFO:hf-to-gguf:blk.40.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  780. INFO:hf-to-gguf:blk.40.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  781. INFO:hf-to-gguf:blk.40.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  782. INFO:hf-to-gguf:blk.40.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  783. INFO:hf-to-gguf:blk.40.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  784. INFO:hf-to-gguf:gguf: loading model part 'model-42-of-61.safetensors'
  785. INFO:hf-to-gguf:blk.41.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  786. INFO:hf-to-gguf:blk.41.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  787. INFO:hf-to-gguf:blk.41.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  788. INFO:hf-to-gguf:blk.41.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  789. INFO:hf-to-gguf:blk.41.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  790. INFO:hf-to-gguf:blk.41.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  791. INFO:hf-to-gguf:blk.41.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  792. INFO:hf-to-gguf:blk.41.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  793. INFO:hf-to-gguf:blk.41.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  794. INFO:hf-to-gguf:blk.41.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  795. INFO:hf-to-gguf:blk.41.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  796. INFO:hf-to-gguf:blk.41.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  797. INFO:hf-to-gguf:blk.41.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  798. INFO:hf-to-gguf:blk.41.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  799. INFO:hf-to-gguf:blk.41.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  800. INFO:hf-to-gguf:blk.41.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  801. INFO:hf-to-gguf:blk.41.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  802. INFO:hf-to-gguf:blk.41.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  803. INFO:hf-to-gguf:blk.41.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  804. INFO:hf-to-gguf:gguf: loading model part 'model-43-of-61.safetensors'
  805. INFO:hf-to-gguf:blk.42.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  806. INFO:hf-to-gguf:blk.42.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  807. INFO:hf-to-gguf:blk.42.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  808. INFO:hf-to-gguf:blk.42.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  809. INFO:hf-to-gguf:blk.42.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  810. INFO:hf-to-gguf:blk.42.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  811. INFO:hf-to-gguf:blk.42.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  812. INFO:hf-to-gguf:blk.42.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  813. INFO:hf-to-gguf:blk.42.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  814. INFO:hf-to-gguf:blk.42.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  815. INFO:hf-to-gguf:blk.42.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  816. INFO:hf-to-gguf:blk.42.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  817. INFO:hf-to-gguf:blk.42.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  818. INFO:hf-to-gguf:blk.42.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  819. INFO:hf-to-gguf:blk.42.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  820. INFO:hf-to-gguf:blk.42.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  821. INFO:hf-to-gguf:blk.42.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  822. INFO:hf-to-gguf:blk.42.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  823. INFO:hf-to-gguf:blk.42.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  824. INFO:hf-to-gguf:gguf: loading model part 'model-44-of-61.safetensors'
  825. INFO:hf-to-gguf:blk.43.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  826. INFO:hf-to-gguf:blk.43.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  827. INFO:hf-to-gguf:blk.43.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  828. INFO:hf-to-gguf:blk.43.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  829. INFO:hf-to-gguf:blk.43.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  830. INFO:hf-to-gguf:blk.43.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  831. INFO:hf-to-gguf:blk.43.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  832. INFO:hf-to-gguf:blk.43.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  833. INFO:hf-to-gguf:blk.43.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  834. INFO:hf-to-gguf:blk.43.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  835. INFO:hf-to-gguf:blk.43.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  836. INFO:hf-to-gguf:blk.43.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  837. INFO:hf-to-gguf:blk.43.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  838. INFO:hf-to-gguf:blk.43.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  839. INFO:hf-to-gguf:blk.43.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  840. INFO:hf-to-gguf:blk.43.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  841. INFO:hf-to-gguf:blk.43.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  842. INFO:hf-to-gguf:blk.43.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  843. INFO:hf-to-gguf:blk.43.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  844. INFO:hf-to-gguf:gguf: loading model part 'model-45-of-61.safetensors'
  845. INFO:hf-to-gguf:blk.44.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  846. INFO:hf-to-gguf:blk.44.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  847. INFO:hf-to-gguf:blk.44.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  848. INFO:hf-to-gguf:blk.44.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  849. INFO:hf-to-gguf:blk.44.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  850. INFO:hf-to-gguf:blk.44.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  851. INFO:hf-to-gguf:blk.44.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  852. INFO:hf-to-gguf:blk.44.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  853. INFO:hf-to-gguf:blk.44.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  854. INFO:hf-to-gguf:blk.44.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  855. INFO:hf-to-gguf:blk.44.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  856. INFO:hf-to-gguf:blk.44.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  857. INFO:hf-to-gguf:blk.44.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  858. INFO:hf-to-gguf:blk.44.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  859. INFO:hf-to-gguf:blk.44.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  860. INFO:hf-to-gguf:blk.44.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  861. INFO:hf-to-gguf:blk.44.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  862. INFO:hf-to-gguf:blk.44.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  863. INFO:hf-to-gguf:blk.44.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  864. INFO:hf-to-gguf:gguf: loading model part 'model-46-of-61.safetensors'
  865. INFO:hf-to-gguf:blk.45.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  866. INFO:hf-to-gguf:blk.45.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  867. INFO:hf-to-gguf:blk.45.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  868. INFO:hf-to-gguf:blk.45.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  869. INFO:hf-to-gguf:blk.45.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  870. INFO:hf-to-gguf:blk.45.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  871. INFO:hf-to-gguf:blk.45.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  872. INFO:hf-to-gguf:blk.45.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  873. INFO:hf-to-gguf:blk.45.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  874. INFO:hf-to-gguf:blk.45.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  875. INFO:hf-to-gguf:blk.45.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  876. INFO:hf-to-gguf:blk.45.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  877. INFO:hf-to-gguf:blk.45.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  878. INFO:hf-to-gguf:blk.45.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  879. INFO:hf-to-gguf:blk.45.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  880. INFO:hf-to-gguf:blk.45.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  881. INFO:hf-to-gguf:blk.45.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  882. INFO:hf-to-gguf:blk.45.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  883. INFO:hf-to-gguf:blk.45.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  884. INFO:hf-to-gguf:gguf: loading model part 'model-47-of-61.safetensors'
  885. INFO:hf-to-gguf:blk.46.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  886. INFO:hf-to-gguf:blk.46.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  887. INFO:hf-to-gguf:blk.46.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  888. INFO:hf-to-gguf:blk.46.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  889. INFO:hf-to-gguf:blk.46.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  890. INFO:hf-to-gguf:blk.46.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  891. INFO:hf-to-gguf:blk.46.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  892. INFO:hf-to-gguf:blk.46.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  893. INFO:hf-to-gguf:blk.46.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  894. INFO:hf-to-gguf:blk.46.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  895. INFO:hf-to-gguf:blk.46.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  896. INFO:hf-to-gguf:blk.46.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  897. INFO:hf-to-gguf:blk.46.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  898. INFO:hf-to-gguf:blk.46.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  899. INFO:hf-to-gguf:blk.46.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  900. INFO:hf-to-gguf:blk.46.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  901. INFO:hf-to-gguf:blk.46.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  902. INFO:hf-to-gguf:blk.46.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  903. INFO:hf-to-gguf:blk.46.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  904. INFO:hf-to-gguf:gguf: loading model part 'model-48-of-61.safetensors'
  905. INFO:hf-to-gguf:blk.47.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  906. INFO:hf-to-gguf:blk.47.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  907. INFO:hf-to-gguf:blk.47.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  908. INFO:hf-to-gguf:blk.47.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  909. INFO:hf-to-gguf:blk.47.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  910. INFO:hf-to-gguf:blk.47.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  911. INFO:hf-to-gguf:blk.47.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  912. INFO:hf-to-gguf:blk.47.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  913. INFO:hf-to-gguf:blk.47.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  914. INFO:hf-to-gguf:blk.47.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  915. INFO:hf-to-gguf:blk.47.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  916. INFO:hf-to-gguf:blk.47.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  917. INFO:hf-to-gguf:blk.47.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  918. INFO:hf-to-gguf:blk.47.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  919. INFO:hf-to-gguf:blk.47.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  920. INFO:hf-to-gguf:blk.47.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  921. INFO:hf-to-gguf:blk.47.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  922. INFO:hf-to-gguf:blk.47.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  923. INFO:hf-to-gguf:blk.47.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  924. INFO:hf-to-gguf:gguf: loading model part 'model-49-of-61.safetensors'
  925. INFO:hf-to-gguf:blk.48.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  926. INFO:hf-to-gguf:blk.48.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  927. INFO:hf-to-gguf:blk.48.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  928. INFO:hf-to-gguf:blk.48.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  929. INFO:hf-to-gguf:blk.48.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  930. INFO:hf-to-gguf:blk.48.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  931. INFO:hf-to-gguf:blk.48.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  932. INFO:hf-to-gguf:blk.48.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  933. INFO:hf-to-gguf:blk.48.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  934. INFO:hf-to-gguf:blk.48.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  935. INFO:hf-to-gguf:blk.48.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  936. INFO:hf-to-gguf:blk.48.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  937. INFO:hf-to-gguf:blk.48.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  938. INFO:hf-to-gguf:blk.48.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  939. INFO:hf-to-gguf:blk.48.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  940. INFO:hf-to-gguf:blk.48.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  941. INFO:hf-to-gguf:blk.48.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  942. INFO:hf-to-gguf:blk.48.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  943. INFO:hf-to-gguf:blk.48.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  944. INFO:hf-to-gguf:gguf: loading model part 'model-5-of-61.safetensors'
  945. INFO:hf-to-gguf:blk.4.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  946. INFO:hf-to-gguf:blk.4.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  947. INFO:hf-to-gguf:blk.4.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  948. INFO:hf-to-gguf:blk.4.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  949. INFO:hf-to-gguf:blk.4.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  950. INFO:hf-to-gguf:blk.4.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  951. INFO:hf-to-gguf:blk.4.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  952. INFO:hf-to-gguf:blk.4.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  953. INFO:hf-to-gguf:blk.4.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  954. INFO:hf-to-gguf:blk.4.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  955. INFO:hf-to-gguf:blk.4.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  956. INFO:hf-to-gguf:blk.4.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  957. INFO:hf-to-gguf:blk.4.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  958. INFO:hf-to-gguf:blk.4.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  959. INFO:hf-to-gguf:blk.4.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  960. INFO:hf-to-gguf:blk.4.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  961. INFO:hf-to-gguf:blk.4.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  962. INFO:hf-to-gguf:blk.4.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  963. INFO:hf-to-gguf:blk.4.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  964. INFO:hf-to-gguf:gguf: loading model part 'model-50-of-61.safetensors'
  965. INFO:hf-to-gguf:blk.49.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  966. INFO:hf-to-gguf:blk.49.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  967. INFO:hf-to-gguf:blk.49.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  968. INFO:hf-to-gguf:blk.49.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  969. INFO:hf-to-gguf:blk.49.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  970. INFO:hf-to-gguf:blk.49.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  971. INFO:hf-to-gguf:blk.49.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  972. INFO:hf-to-gguf:blk.49.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  973. INFO:hf-to-gguf:blk.49.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  974. INFO:hf-to-gguf:blk.49.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  975. INFO:hf-to-gguf:blk.49.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  976. INFO:hf-to-gguf:blk.49.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  977. INFO:hf-to-gguf:blk.49.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  978. INFO:hf-to-gguf:blk.49.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  979. INFO:hf-to-gguf:blk.49.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  980. INFO:hf-to-gguf:blk.49.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  981. INFO:hf-to-gguf:blk.49.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  982. INFO:hf-to-gguf:blk.49.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  983. INFO:hf-to-gguf:blk.49.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  984. INFO:hf-to-gguf:gguf: loading model part 'model-51-of-61.safetensors'
  985. INFO:hf-to-gguf:blk.50.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  986. INFO:hf-to-gguf:blk.50.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  987. INFO:hf-to-gguf:blk.50.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  988. INFO:hf-to-gguf:blk.50.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  989. INFO:hf-to-gguf:blk.50.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  990. INFO:hf-to-gguf:blk.50.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  991. INFO:hf-to-gguf:blk.50.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  992. INFO:hf-to-gguf:blk.50.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  993. INFO:hf-to-gguf:blk.50.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  994. INFO:hf-to-gguf:blk.50.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  995. INFO:hf-to-gguf:blk.50.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  996. INFO:hf-to-gguf:blk.50.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  997. INFO:hf-to-gguf:blk.50.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  998. INFO:hf-to-gguf:blk.50.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  999. INFO:hf-to-gguf:blk.50.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  1000. INFO:hf-to-gguf:blk.50.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  1001. INFO:hf-to-gguf:blk.50.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  1002. INFO:hf-to-gguf:blk.50.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  1003. INFO:hf-to-gguf:blk.50.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  1004. INFO:hf-to-gguf:gguf: loading model part 'model-52-of-61.safetensors'
  1005. INFO:hf-to-gguf:blk.51.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1006. INFO:hf-to-gguf:blk.51.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  1007. INFO:hf-to-gguf:blk.51.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1008. INFO:hf-to-gguf:blk.51.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1009. INFO:hf-to-gguf:blk.51.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  1010. INFO:hf-to-gguf:blk.51.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  1011. INFO:hf-to-gguf:blk.51.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  1012. INFO:hf-to-gguf:blk.51.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1013. INFO:hf-to-gguf:blk.51.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1014. INFO:hf-to-gguf:blk.51.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1015. INFO:hf-to-gguf:blk.51.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  1016. INFO:hf-to-gguf:blk.51.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  1017. INFO:hf-to-gguf:blk.51.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  1018. INFO:hf-to-gguf:blk.51.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  1019. INFO:hf-to-gguf:blk.51.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  1020. INFO:hf-to-gguf:blk.51.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  1021. INFO:hf-to-gguf:blk.51.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  1022. INFO:hf-to-gguf:blk.51.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  1023. INFO:hf-to-gguf:blk.51.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  1024. INFO:hf-to-gguf:gguf: loading model part 'model-53-of-61.safetensors'
  1025. INFO:hf-to-gguf:blk.52.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1026. INFO:hf-to-gguf:blk.52.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  1027. INFO:hf-to-gguf:blk.52.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1028. INFO:hf-to-gguf:blk.52.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1029. INFO:hf-to-gguf:blk.52.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  1030. INFO:hf-to-gguf:blk.52.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  1031. INFO:hf-to-gguf:blk.52.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  1032. INFO:hf-to-gguf:blk.52.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1033. INFO:hf-to-gguf:blk.52.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1034. INFO:hf-to-gguf:blk.52.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1035. INFO:hf-to-gguf:blk.52.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  1036. INFO:hf-to-gguf:blk.52.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  1037. INFO:hf-to-gguf:blk.52.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  1038. INFO:hf-to-gguf:blk.52.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  1039. INFO:hf-to-gguf:blk.52.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  1040. INFO:hf-to-gguf:blk.52.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  1041. INFO:hf-to-gguf:blk.52.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  1042. INFO:hf-to-gguf:blk.52.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  1043. INFO:hf-to-gguf:blk.52.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  1044. INFO:hf-to-gguf:gguf: loading model part 'model-54-of-61.safetensors'
  1045. INFO:hf-to-gguf:blk.53.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1046. INFO:hf-to-gguf:blk.53.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  1047. INFO:hf-to-gguf:blk.53.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1048. INFO:hf-to-gguf:blk.53.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1049. INFO:hf-to-gguf:blk.53.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  1050. INFO:hf-to-gguf:blk.53.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  1051. INFO:hf-to-gguf:blk.53.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  1052. INFO:hf-to-gguf:blk.53.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1053. INFO:hf-to-gguf:blk.53.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1054. INFO:hf-to-gguf:blk.53.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1055. INFO:hf-to-gguf:blk.53.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  1056. INFO:hf-to-gguf:blk.53.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  1057. INFO:hf-to-gguf:blk.53.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  1058. INFO:hf-to-gguf:blk.53.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  1059. INFO:hf-to-gguf:blk.53.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  1060. INFO:hf-to-gguf:blk.53.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  1061. INFO:hf-to-gguf:blk.53.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  1062. INFO:hf-to-gguf:blk.53.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  1063. INFO:hf-to-gguf:blk.53.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  1064. INFO:hf-to-gguf:gguf: loading model part 'model-55-of-61.safetensors'
  1065. INFO:hf-to-gguf:blk.54.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1066. INFO:hf-to-gguf:blk.54.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  1067. INFO:hf-to-gguf:blk.54.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1068. INFO:hf-to-gguf:blk.54.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1069. INFO:hf-to-gguf:blk.54.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  1070. INFO:hf-to-gguf:blk.54.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  1071. INFO:hf-to-gguf:blk.54.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  1072. INFO:hf-to-gguf:blk.54.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1073. INFO:hf-to-gguf:blk.54.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1074. INFO:hf-to-gguf:blk.54.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1075. INFO:hf-to-gguf:blk.54.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  1076. INFO:hf-to-gguf:blk.54.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  1077. INFO:hf-to-gguf:blk.54.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  1078. INFO:hf-to-gguf:blk.54.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  1079. INFO:hf-to-gguf:blk.54.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  1080. INFO:hf-to-gguf:blk.54.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  1081. INFO:hf-to-gguf:blk.54.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  1082. INFO:hf-to-gguf:blk.54.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  1083. INFO:hf-to-gguf:blk.54.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  1084. INFO:hf-to-gguf:gguf: loading model part 'model-56-of-61.safetensors'
  1085. INFO:hf-to-gguf:blk.55.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1086. INFO:hf-to-gguf:blk.55.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  1087. INFO:hf-to-gguf:blk.55.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1088. INFO:hf-to-gguf:blk.55.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1089. INFO:hf-to-gguf:blk.55.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  1090. INFO:hf-to-gguf:blk.55.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  1091. INFO:hf-to-gguf:blk.55.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  1092. INFO:hf-to-gguf:blk.55.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1093. INFO:hf-to-gguf:blk.55.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1094. INFO:hf-to-gguf:blk.55.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1095. INFO:hf-to-gguf:blk.55.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  1096. INFO:hf-to-gguf:blk.55.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  1097. INFO:hf-to-gguf:blk.55.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  1098. INFO:hf-to-gguf:blk.55.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  1099. INFO:hf-to-gguf:blk.55.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  1100. INFO:hf-to-gguf:blk.55.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  1101. INFO:hf-to-gguf:blk.55.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  1102. INFO:hf-to-gguf:blk.55.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  1103. INFO:hf-to-gguf:blk.55.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  1104. INFO:hf-to-gguf:gguf: loading model part 'model-57-of-61.safetensors'
  1105. INFO:hf-to-gguf:blk.56.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1106. INFO:hf-to-gguf:blk.56.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  1107. INFO:hf-to-gguf:blk.56.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1108. INFO:hf-to-gguf:blk.56.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1109. INFO:hf-to-gguf:blk.56.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  1110. INFO:hf-to-gguf:blk.56.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  1111. INFO:hf-to-gguf:blk.56.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  1112. INFO:hf-to-gguf:blk.56.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1113. INFO:hf-to-gguf:blk.56.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1114. INFO:hf-to-gguf:blk.56.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1115. INFO:hf-to-gguf:blk.56.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  1116. INFO:hf-to-gguf:blk.56.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  1117. INFO:hf-to-gguf:blk.56.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  1118. INFO:hf-to-gguf:blk.56.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  1119. INFO:hf-to-gguf:blk.56.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  1120. INFO:hf-to-gguf:blk.56.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  1121. INFO:hf-to-gguf:blk.56.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  1122. INFO:hf-to-gguf:blk.56.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  1123. INFO:hf-to-gguf:blk.56.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  1124. INFO:hf-to-gguf:gguf: loading model part 'model-58-of-61.safetensors'
  1125. INFO:hf-to-gguf:blk.57.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1126. INFO:hf-to-gguf:blk.57.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  1127. INFO:hf-to-gguf:blk.57.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1128. INFO:hf-to-gguf:blk.57.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1129. INFO:hf-to-gguf:blk.57.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  1130. INFO:hf-to-gguf:blk.57.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  1131. INFO:hf-to-gguf:blk.57.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  1132. INFO:hf-to-gguf:blk.57.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1133. INFO:hf-to-gguf:blk.57.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1134. INFO:hf-to-gguf:blk.57.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1135. INFO:hf-to-gguf:blk.57.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  1136. INFO:hf-to-gguf:blk.57.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  1137. INFO:hf-to-gguf:blk.57.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  1138. INFO:hf-to-gguf:blk.57.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  1139. INFO:hf-to-gguf:blk.57.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  1140. INFO:hf-to-gguf:blk.57.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  1141. INFO:hf-to-gguf:blk.57.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  1142. INFO:hf-to-gguf:blk.57.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  1143. INFO:hf-to-gguf:blk.57.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  1144. INFO:hf-to-gguf:gguf: loading model part 'model-59-of-61.safetensors'
  1145. INFO:hf-to-gguf:blk.58.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1146. INFO:hf-to-gguf:blk.58.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  1147. INFO:hf-to-gguf:blk.58.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1148. INFO:hf-to-gguf:blk.58.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1149. INFO:hf-to-gguf:blk.58.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  1150. INFO:hf-to-gguf:blk.58.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  1151. INFO:hf-to-gguf:blk.58.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  1152. INFO:hf-to-gguf:blk.58.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1153. INFO:hf-to-gguf:blk.58.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1154. INFO:hf-to-gguf:blk.58.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1155. INFO:hf-to-gguf:blk.58.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  1156. INFO:hf-to-gguf:blk.58.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  1157. INFO:hf-to-gguf:blk.58.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  1158. INFO:hf-to-gguf:blk.58.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  1159. INFO:hf-to-gguf:blk.58.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  1160. INFO:hf-to-gguf:blk.58.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  1161. INFO:hf-to-gguf:blk.58.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  1162. INFO:hf-to-gguf:blk.58.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  1163. INFO:hf-to-gguf:blk.58.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  1164. INFO:hf-to-gguf:gguf: loading model part 'model-6-of-61.safetensors'
  1165. INFO:hf-to-gguf:blk.5.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1166. INFO:hf-to-gguf:blk.5.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  1167. INFO:hf-to-gguf:blk.5.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1168. INFO:hf-to-gguf:blk.5.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1169. INFO:hf-to-gguf:blk.5.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  1170. INFO:hf-to-gguf:blk.5.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  1171. INFO:hf-to-gguf:blk.5.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  1172. INFO:hf-to-gguf:blk.5.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1173. INFO:hf-to-gguf:blk.5.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1174. INFO:hf-to-gguf:blk.5.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1175. INFO:hf-to-gguf:blk.5.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  1176. INFO:hf-to-gguf:blk.5.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  1177. INFO:hf-to-gguf:blk.5.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  1178. INFO:hf-to-gguf:blk.5.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  1179. INFO:hf-to-gguf:blk.5.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  1180. INFO:hf-to-gguf:blk.5.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  1181. INFO:hf-to-gguf:blk.5.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  1182. INFO:hf-to-gguf:blk.5.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  1183. INFO:hf-to-gguf:blk.5.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  1184. INFO:hf-to-gguf:gguf: loading model part 'model-60-of-61.safetensors'
  1185. INFO:hf-to-gguf:blk.59.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1186. INFO:hf-to-gguf:blk.59.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  1187. INFO:hf-to-gguf:blk.59.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1188. INFO:hf-to-gguf:blk.59.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1189. INFO:hf-to-gguf:blk.59.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  1190. INFO:hf-to-gguf:blk.59.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  1191. INFO:hf-to-gguf:blk.59.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  1192. INFO:hf-to-gguf:blk.59.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1193. INFO:hf-to-gguf:blk.59.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1194. INFO:hf-to-gguf:blk.59.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1195. INFO:hf-to-gguf:blk.59.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  1196. INFO:hf-to-gguf:blk.59.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  1197. INFO:hf-to-gguf:blk.59.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  1198. INFO:hf-to-gguf:blk.59.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  1199. INFO:hf-to-gguf:blk.59.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  1200. INFO:hf-to-gguf:blk.59.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  1201. INFO:hf-to-gguf:blk.59.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  1202. INFO:hf-to-gguf:blk.59.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  1203. INFO:hf-to-gguf:blk.59.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  1204. INFO:hf-to-gguf:gguf: loading model part 'model-61-of-61.safetensors'
  1205. INFO:hf-to-gguf:output.weight, torch.bfloat16 --> BF16, shape = {7168, 163840}
  1206. INFO:hf-to-gguf:blk.60.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1207. INFO:hf-to-gguf:blk.60.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  1208. INFO:hf-to-gguf:blk.60.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1209. INFO:hf-to-gguf:blk.60.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1210. INFO:hf-to-gguf:blk.60.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  1211. INFO:hf-to-gguf:blk.60.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  1212. INFO:hf-to-gguf:blk.60.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  1213. INFO:hf-to-gguf:blk.60.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1214. INFO:hf-to-gguf:blk.60.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1215. INFO:hf-to-gguf:blk.60.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1216. INFO:hf-to-gguf:blk.60.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  1217. INFO:hf-to-gguf:blk.60.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  1218. INFO:hf-to-gguf:blk.60.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  1219. INFO:hf-to-gguf:blk.60.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  1220. INFO:hf-to-gguf:blk.60.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  1221. INFO:hf-to-gguf:blk.60.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  1222. INFO:hf-to-gguf:blk.60.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  1223. INFO:hf-to-gguf:blk.60.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  1224. INFO:hf-to-gguf:blk.60.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  1225. INFO:hf-to-gguf:output_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1226. INFO:hf-to-gguf:gguf: loading model part 'model-7-of-61.safetensors'
  1227. INFO:hf-to-gguf:blk.6.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1228. INFO:hf-to-gguf:blk.6.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  1229. INFO:hf-to-gguf:blk.6.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1230. INFO:hf-to-gguf:blk.6.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1231. INFO:hf-to-gguf:blk.6.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  1232. INFO:hf-to-gguf:blk.6.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  1233. INFO:hf-to-gguf:blk.6.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  1234. INFO:hf-to-gguf:blk.6.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1235. INFO:hf-to-gguf:blk.6.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1236. INFO:hf-to-gguf:blk.6.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1237. INFO:hf-to-gguf:blk.6.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  1238. INFO:hf-to-gguf:blk.6.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  1239. INFO:hf-to-gguf:blk.6.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  1240. INFO:hf-to-gguf:blk.6.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  1241. INFO:hf-to-gguf:blk.6.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  1242. INFO:hf-to-gguf:blk.6.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  1243. INFO:hf-to-gguf:blk.6.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  1244. INFO:hf-to-gguf:blk.6.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  1245. INFO:hf-to-gguf:blk.6.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  1246. INFO:hf-to-gguf:gguf: loading model part 'model-8-of-61.safetensors'
  1247. INFO:hf-to-gguf:blk.7.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1248. INFO:hf-to-gguf:blk.7.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  1249. INFO:hf-to-gguf:blk.7.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1250. INFO:hf-to-gguf:blk.7.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1251. INFO:hf-to-gguf:blk.7.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  1252. INFO:hf-to-gguf:blk.7.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  1253. INFO:hf-to-gguf:blk.7.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  1254. INFO:hf-to-gguf:blk.7.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1255. INFO:hf-to-gguf:blk.7.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1256. INFO:hf-to-gguf:blk.7.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1257. INFO:hf-to-gguf:blk.7.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  1258. INFO:hf-to-gguf:blk.7.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  1259. INFO:hf-to-gguf:blk.7.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  1260. INFO:hf-to-gguf:blk.7.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  1261. INFO:hf-to-gguf:blk.7.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  1262. INFO:hf-to-gguf:blk.7.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  1263. INFO:hf-to-gguf:blk.7.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  1264. INFO:hf-to-gguf:blk.7.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  1265. INFO:hf-to-gguf:blk.7.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  1266. INFO:hf-to-gguf:gguf: loading model part 'model-9-of-61.safetensors'
  1267. INFO:hf-to-gguf:blk.8.attn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1268. INFO:hf-to-gguf:blk.8.ffn_down_exps.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168, 384}
  1269. INFO:hf-to-gguf:blk.8.ffn_gate_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1270. INFO:hf-to-gguf:blk.8.ffn_up_exps.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048, 384}
  1271. INFO:hf-to-gguf:blk.8.exp_probs_b.bias, torch.float32 --> F32, shape = {384}
  1272. INFO:hf-to-gguf:blk.8.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {7168, 384}
  1273. INFO:hf-to-gguf:blk.8.ffn_down_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {2048, 7168}
  1274. INFO:hf-to-gguf:blk.8.ffn_gate_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1275. INFO:hf-to-gguf:blk.8.ffn_up_shexp.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 2048}
  1276. INFO:hf-to-gguf:blk.8.ffn_norm.weight, torch.bfloat16 --> F32, shape = {7168}
  1277. INFO:hf-to-gguf:blk.8.attn_kv_a_norm.weight, torch.bfloat16 --> F32, shape = {512}
  1278. INFO:hf-to-gguf:blk.8.attn_kv_a_mqa.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 576}
  1279. INFO:hf-to-gguf:blk.8.attn_kv_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 16384}
  1280. INFO:hf-to-gguf:blk.8.attn_k_b.weight, torch.float8_e4m3fn --> BF16, shape = {128, 32768}
  1281. INFO:hf-to-gguf:blk.8.attn_v_b.weight, torch.float8_e4m3fn --> BF16, shape = {512, 8192}
  1282. INFO:hf-to-gguf:blk.8.attn_output.weight, torch.float8_e4m3fn --> BF16, shape = {8192, 7168}
  1283. INFO:hf-to-gguf:blk.8.attn_q_a_norm.weight, torch.bfloat16 --> F32, shape = {1536}
  1284. INFO:hf-to-gguf:blk.8.attn_q_a.weight, torch.float8_e4m3fn --> BF16, shape = {7168, 1536}
  1285. INFO:hf-to-gguf:blk.8.attn_q_b.weight, torch.float8_e4m3fn --> BF16, shape = {1536, 12288}
  1286. INFO:hf-to-gguf:Set meta model
  1287. INFO:hf-to-gguf:Set model parameters
  1288. INFO:hf-to-gguf:gguf: context length = 131072
  1289. INFO:hf-to-gguf:gguf: embedding length = 7168
  1290. INFO:hf-to-gguf:gguf: feed forward length = 18432
  1291. INFO:hf-to-gguf:gguf: head count = 64
  1292. INFO:hf-to-gguf:gguf: key-value head count = 64
  1293. INFO:hf-to-gguf:gguf: rope theta = 50000.0
  1294. INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-06
  1295. INFO:hf-to-gguf:gguf: experts used count = 8
  1296. INFO:hf-to-gguf:gguf: file type = 32
  1297. INFO:hf-to-gguf:Set model tokenizer
  1298. Traceback (most recent call last):
  1299. File "/home/lissanro/pkgs/llama.cpp-fp8-to-bf16/llama.cpp/convert_hf_to_gguf.py", line 5244, in <module>
  1300. main()
  1301. ~~~~^^
  1302. File "/home/lissanro/pkgs/llama.cpp-fp8-to-bf16/llama.cpp/convert_hf_to_gguf.py", line 5238, in main
  1303. model_instance.write()
  1304. ~~~~~~~~~~~~~~~~~~~~^^
  1305. File "/home/lissanro/pkgs/llama.cpp-fp8-to-bf16/llama.cpp/convert_hf_to_gguf.py", line 440, in write
  1306. self.prepare_metadata(vocab_only=False)
  1307. ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  1308. File "/home/lissanro/pkgs/llama.cpp-fp8-to-bf16/llama.cpp/convert_hf_to_gguf.py", line 433, in prepare_metadata
  1309. self.set_vocab()
  1310. ~~~~~~~~~~~~~~^^
  1311. File "/home/lissanro/pkgs/llama.cpp-fp8-to-bf16/llama.cpp/convert_hf_to_gguf.py", line 4058, in set_vocab
  1312. self._set_vocab_gpt2()
  1313. ~~~~~~~~~~~~~~~~~~~~^^
  1314. File "/home/lissanro/pkgs/llama.cpp-fp8-to-bf16/llama.cpp/convert_hf_to_gguf.py", line 728, in _set_vocab_gpt2
  1315. tokens, toktypes, tokpre = self.get_vocab_base()
  1316. ~~~~~~~~~~~~~~~~~~~^^
  1317. File "/home/lissanro/pkgs/llama.cpp-fp8-to-bf16/llama.cpp/convert_hf_to_gguf.py", line 522, in get_vocab_base
  1318. tokenizer = AutoTokenizer.from_pretrained(self.dir_model)
  1319. File "/home/lissanro/.local/lib/python3.13/site-packages/transformers/models/auto/tokenization_auto.py", line 946, in from_pretrained
  1320. tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
  1321. File "/home/lissanro/.local/lib/python3.13/site-packages/transformers/models/auto/tokenization_auto.py", line 800, in get_tokenizer_config
  1322. result = json.load(reader)
  1323. File "/usr/lib/python3.13/json/__init__.py", line 293, in load
  1324. return loads(fp.read(),
  1325. cls=cls, object_hook=object_hook,
  1326. parse_float=parse_float, parse_int=parse_int,
  1327. parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  1328. File "/usr/lib/python3.13/json/__init__.py", line 346, in loads
  1329. return _default_decoder.decode(s)
  1330. ~~~~~~~~~~~~~~~~~~~~~~~^^^
  1331. File "/usr/lib/python3.13/json/decoder.py", line 348, in decode
  1332. raise JSONDecodeError("Extra data", s, end)
  1333. json.decoder.JSONDecodeError: Extra data: line 165 column 2 (char 5595)
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!