Hugging Face's logo

bobchenyx
/
DeepSeek-V3-0324-MLA-GGUF

Text Generation
Transformers
GGUF
deepseek_v3
deepseek
GGUF
imatrix
conversational
Model card Files
xet
Community

Llamacpp Quantizations of DeepSeek-V3-0324 (MLA version)

Original model: Adopting BF16 & Imatrix from unsloth/DeepSeek-V3-0324-GGUF-UD.

All quants made with modification of llama.cpp based on bartowski1182-llama.cpp.

  • IQ1_M / Q4_K / Q8_0 : 144.24 GiB (1.85 BPW)

  • IQ1_S / Q4_K / Q8_0 : 129.94 GiB (1.66 BPW)

  • Q2_K : 222.01 GiB (2.84 BPW)

  • Q4_K_M : 381.64 GiB (4.89 BPW)

Downloads last month
229
GGUF
Model size
671B params
Architecture
deepseek2
Hardware compatibility
Log In to view the estimation

1-bit

2-bit

4-bit

Inference Providers NEW
Text Generation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for bobchenyx/DeepSeek-V3-0324-MLA-GGUF

Base model

deepseek-ai/DeepSeek-V3-0324
Quantized
(19)
this model

Collection including bobchenyx/DeepSeek-V3-0324-MLA-GGUF