OpenReasoning-Nemotron-32B-W8A8-INT8-Dynamic

Method

Quantised using vllm-project/llm-compressor and the following configs:

recipe = [
    SmoothQuantModifier(smoothing_strength=0.8),
    GPTQModifier(targets="Linear", scheme="W8A8", ignore=["lm_head"]),
]

Safetensors

Model size

33B params

Tensor type

BF16

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Base model

Qwen/Qwen2.5-32B

Finetuned

Quantized

(31)

this model