A _Q8_K_XL quant?
#4
by
rtzurtz
- opened
Hi bartowski, unsloth offers a Q8_K_XL quant, could you please too? I usually use ur quants.
Why it may be relevant having higher quants of active 3B parameter LLMs or higher quants of this particular 30B-A3B LLM: I recently noticed a difference in knowledge of Qwen3-30B-A3B-Q6_K.gguf vs Qwen3-30B-A3B-Q8_0.gguf, where only Q8_0 answered correctly, other fields may be affected too.
Another reason: I heard that smaller LLMs are more affected by quantization than bigger ones, in particular also this 30B, active 3B, LLM.
PS: Any higher quants in between Q8_K_XL and BF16?