We have quantised the model in 8-bit to make it inferenceable in low-end GPU cards at scale. It was achieved thanks to llama.cpp library.

GGUF

Model size

1.54B params

Architecture

qwen2

Hardware compatibility

8-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for sleeping-ai/Qwen2-Math-1.5B-GGUF

Base model

Quantized

(9)

this model