HIGGS
Collection
Models prequantized with [HIGGS](https://arxiv.org/abs/2411.17525) zero-shot quantization. Requires the latest `transformers` to run.
•
18 items
•
Updated
•
14
| Groups | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| mmlu | N/A | none | 0 | acc | ↑ | 0.8202 | ± | 0.0031 |
| - humanities | N/A | none | 5 | acc | ↑ | 0.8013 | ± | 0.0056 |
| - other | N/A | none | 5 | acc | ↑ | 0.8455 | ± | 0.0062 |
| - social_sciences | N/A | none | 5 | acc | ↑ | 0.8898 | ± | 0.0055 |
| - stem | N/A | none | 5 | acc | ↑ | 0.7555 | ± | 0.0073 |