metadata
license: mit
base_model:
- deepseek-ai/DeepSeek-V3-0324
tags:
- ik_llama.cpp
ik_llama.cpp quantizations of DeepSeek-V3-0324
Quantized using ik_llama.cpp build = 3788 (4622fadc)
NOTE: These quants MUST be run using the llama.cpp fork, ik_llama.cpp
Credits to @ubergarm for his DeepSeek quant recipes for which these quants were based on.
| name | file size | quant type | bpw |
|---|---|---|---|
| DeepSeek-V3-0324-IQ4_KT | 322.355 GiB | IQ4_KT (97.5%) / Q8_0 (2.5%) |
4.127 |
| DeepSeek-V3-0324-IQ4_XS_R8 | 340.764 GiB | IQ4_XS_R8 (97.5%) / Q8_0 (2.5%) |
4.362 |
| DeepSeek-V3-0324-D-IQ4_KS_R4 | 366.762 GiB | IQ4_KS_R4 (65%) / IQ5_KS_R4 (32.5%) / Q8_0 (2.5%) |
4.695 |
| DeepSeek-V3-0324-D-Q4_K_R4 | 412.131 GiB | Q4_K_R4 (65%) / Q6_K_R4 (32.5%) / Q8_0 (2.5%) |
5.276 |
| DeepSeek-V3-0324-Q8_0_R8 | 664.295 GiB | Q8_0_R8 (100%) |
8.504 |