Allowed quantization types:
   2  or  Q4_0    :  3.56G, +0.2166 ppl @ LLaMA-v1-7B
   3  or  Q4_1    :  3.90G, +0.1585 ppl @ LLaMA-v1-7B
   8  or  Q5_0    :  4.33G, +0.0683 ppl @ LLaMA-v1-7B
   9  or  Q5_1    :  4.70G, +0.0349 ppl @ LLaMA-v1-7B
  135  or  Q6_0    :  6.5 bpw quantization
  38  or  MXFP4   :  4.25 bpw 4-bit float quantization
  19  or  IQ2_XXS :  2.06 bpw quantization
  219  or  IQ2_XXS_R4 : IQ2_XXS repacked
  20  or  IQ2_XS  :  2.31 bpw quantization
  220  or  IQ2_XS_R4 : IQ2_XS repacked
  28  or  IQ2_S   :  2.5  bpw quantization
  29  or  IQ2_M   :  2.7  bpw quantization
  229  or  IQ2_M_R4 :  2.7  bpw quantization
  24  or  IQ1_S   :  1.56 bpw quantization
  224  or  IQ1_S_R4 :  1.5 bpw quantization
  231  or  IQ1_M_R4 :  1.75 bpw quantization
  31  or  IQ1_M   :  1.75 bpw quantization
  136  or  IQ1_BN  :  1.62 bpw quantization (Bitnet)
  137  or  IQ2_BN  :  2.00 bpw quantization (Bitnet)
  337  or  IQ2_BN_R4 :  2.00 bpw quantization (Bitnet)
  10  or  Q2_K    :  2.63G, +0.6717 ppl @ LLaMA-v1-7B
  210  or  Q2_K_R4 : Q2_K_S repacked
  21  or  Q2_K_S  :  2.16G, +9.0634 ppl @ LLaMA-v1-7B
  23  or  IQ3_XXS :  3.06 bpw quantization
  152  or  IQ3_KT  :  3.125 bpw trellis quantization
  153  or  IQ4_KT  :  4.0 bpw trellis quantization
  223  or  IQ3_XXS_R4 : IQ3_XXS repacked
  26  or  IQ3_S   :  3.44 bpw quantization
  226  or  IQ3_S_R4 : IQ3_S repacked
  27  or  IQ3_M   :  3.66 bpw quantization mix
  12  or  Q3_K    : alias for Q3_K_M
  211  or  Q3_K_R4 : Q3_K_S repacked
  22  or  IQ3_XS  :  3.3 bpw quantization
  11  or  Q3_K_S  :  2.75G, +0.5551 ppl @ LLaMA-v1-7B
  12  or  Q3_K_M  :  3.07G, +0.2496 ppl @ LLaMA-v1-7B
  13  or  Q3_K_L  :  3.35G, +0.1764 ppl @ LLaMA-v1-7B
  25  or  IQ4_NL  :  4.50 bpw non-linear quantization
  225  or  IQ4_NL_R4 :  4.50 bpw non-linear quantization
  230  or  IQ4_XS_R8 :  4.25 bpw non-linear quantization
  202  or  Q4_0_R8 :  4.50 bpw quantization
  208  or  Q5_0_R4 :  5.50 bpw quantization
  335  or  Q6_0_R4 :  6.50 bpw quantization
  207  or  Q8_0_R8 :  8.50 bpw quantization
  149  or  Q8_KV   :  8.00 bpw quantization
  30  or  IQ4_XS  :  4.25 bpw non-linear quantization
  145  or  IQ4_KS  :  4.25 bpw non-linear quantization
  345  or  IQ4_KS_R4 : IQ4_KS repacked
  350  or  IQ5_KS_R4 : IQ5_KS repacked
  148  or  IQ4_KSS :  4.0 bpw non-linear quantization
  150  or  IQ5_KS  :  5.25 bpw non-linear quantization
  138  or  IQ2_K   :  2.375 bpw non-linear quantization
  338  or  IQ2_K_R4 : IQ2_K repacked
  147  or  IQ2_KS  :  2.1875 bpw non-linear quantization
  156  or  IQ1_KT  :  1.75 bpw trellis quantization
  151  or  IQ2_KT  :  2.125 bpw trellis quantization
  155  or  IQ2_KL  :  2.69 bpw non-linear quantization
  154  or  IQ3_KS  :  3.19 bpw non-linear quantization
  139  or  IQ3_K   :  3.44 bpw non-linear quantization
  339  or  IQ3_K_R4 : IQ3_K repacked
  146  or  IQ3_KL  :  4 bpw non-linear quantization mix
  140  or  IQ4_K   :  4.5 bpw non-linear quantization
  340  or  IQ4_K_R4 : IQ4_K repacked
  141  or  IQ5_K   :  5.5 bpw non-linear quantization
  341  or  IQ5_K_R4 : IQ5_K repacked
  142  or  IQ6_K   :  6.6 bpw non-linear quantization
  15  or  Q4_K    : alias for Q4_K_M
  214  or  Q4_K_R4 : Q4_K_S repacked
  14  or  Q4_K_S  :  3.59G, +0.0992 ppl @ LLaMA-v1-7B
  15  or  Q4_K_M  :  3.80G, +0.0532 ppl @ LLaMA-v1-7B
  17  or  Q5_K    : alias for Q5_K_M
  216  or  Q5_K_R4 : Q5_K_S repacked
  16  or  Q5_K_S  :  4.33G, +0.0400 ppl @ LLaMA-v1-7B
  17  or  Q5_K_M  :  4.45G, +0.0122 ppl @ LLaMA-v1-7B
  18  or  Q6_K    :  5.15G, +0.0008 ppl @ LLaMA-v1-7B
  218  or  Q6_K_R4 : Q6_K repacked
  399  or  Q8_K_R8 : Q8_K repacked
  398  or  Q8_KV_R8 : Q8_KV repacked
   7  or  Q8_0    :  6.70G, +0.0004 ppl @ LLaMA-v1-7B
  33  or  Q4_0_4_4 :  4.34G, +0.4685 ppl @ Llama-3-8B
  34  or  Q4_0_4_8 :  4.34G, +0.4685 ppl @ Llama-3-8B
  35  or  Q4_0_8_8 :  4.34G, +0.4685 ppl @ Llama-3-8B
   1  or  F16     : 14.00G, -0.0020 ppl @ Mistral-7B
  32  or  BF16    : 14.00G, -0.0050 ppl @ Mistral-7B
  232  or  BF16_R16 : 14.00G, -0.0050 ppl @ Mistral-7B
   0  or  F32     : 26.00G              @ 7B
          COPY    : only copy tensors, no quantizing