Usage of GPU by abliterated model
#4
by
hdev27n
- opened
Hey!
Thanks for huihui-ai to create and share all the ablitedated models! :)
Beginners question:
When I use the original gemma-3-27b, it uses my GPU and works really fast.
When I use gemma-3-27b-it-abliterated, it uses almost my CPU only and - guess what - it calms down a lot.
Is there a hint how to a accelerate the abliterated model by using GPU?
I use Open WebUI with Ollama.
Came here to see if anyone else was experiencing this as well. It's nearly 4x the size of the OG gemma3:27b model, hence why it gets offloaded.
gemma3-abliterated:27b corresponds to gemma3-abliterated:27b-q8_0
Q4_K_M was not uploaded because the quality was not too good.