Any plan to integrate with GPTQ?
#6
by changtimwu - opened
Hi! LaMini Team:
First of all, thanks for the great work. While beyond Alpaca with only 1.5B parameters is indeed quite a breakthrough, have you consider making it even “mini” by applying the well-known GPTQ technique on it? I know that’s more about the implementation side work. It’s still useful for embedded system where an LM is expected to be intelligent instead of informative.
changtimwu changed discussion title from Any plan to integrate with GPTQ to Any plan to integrate with GPTQ?
Try AutoGPTQ, so that you don't have to write your own GPTQ implementation