gguf model?
#1
by
segmond
- opened
Can you upload a gguf model?
q8 uploadig right now (see https://huggingface.co/volker-mauel/Kimi-Dev-72B-GGUF - might take another 10m)
f32, f16, tq1_0 and tq2_0 will be there in the next 30m-1h
f32, f16, tq1_0 and tq2_0 will be there in the next 30m-1h
Hey wait, tq1_0
and tq2_0
are for ternary models only!! don't upload those they might not even work! See
@compilade
s note here: https://www.reddit.com/r/LocalLLaMA/comments/1la1v4d/comment/mxioh75/
I'd suggest you try like q4_K
or something as a better quant choice.
Anyway, bullerwins has some usable models up now!
Cheers!