|
|
--- |
|
|
license: llama2 |
|
|
--- |
|
|
|
|
|
GPTQ 4-bit actor order version that works in textgen-webui with exllamav2 |
|
|
|
|
|
Generated by using scripts from https://gitee.com/yhyu13/llama_-tools |
|
|
|
|
|
Original weight : https://huggingface.co/Xwin-LM/Xwin-Math-7B-V1.0 |
|
|
|
|
|
--- |
|
|
|
|
|
| Branch | Bits | GS | Act Order | Damp % | GPTQ Dataset | Seq Len | Size | ExLlama | Desc | |
|
|
| ------ | ---- | -- | --------- | ------ | ------------ | ------- | ---- | ------- | ---- | |
|
|
| [main](https://huggingface.co/Yhyu13/Xwin-Math-7B-V1.0-GPTQ-4bit/tree/main) | 4 | 128 | Yes | 0.1 | [code](wikitext2) | 4096 | 4.4GB | Yes | 4-bit, with Act Order. 128 group size | |
|
|
|
|
|
--- |
|
|
|
|
|
Here is some testing done in textgen-webui, I was using Q&A from this dataset https://huggingface.co/datasets/TIGER-Lab/MathInstruct |
|
|
|
|
|
Basic arithmatic, the answer (A) is correct |
|
|
 |
|
|
|
|
|
|
|
|
Does not follow the instruction to write python code. And hulluciate an answer not in exists in options |
|
|
 |
|
|
|
|
|
Answer is correct, but hulluciate with a not existing option |
|
|
 |
