Gaivoronsky
/

ruGPT-3.5-13B-8bit

Text Generation

Model card Files Files and versions

Gaivoronsky commited on Aug 23, 2023

Commit

2183335

·

1 Parent(s): 6a3a844

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -18,7 +18,7 @@ This is a generative model converted to fp16 format based on [ai-forever/ruGPT-3
 from transformers import AutoTokenizer
 from auto_gptq import AutoGPTQForCausalLM
-model = AutoGPTQForCausalLM.from_pretrained('Gaivoronsky/ruGPT-3.5-13B-8bit', device="cuda:0", use_triton=False)
 tokenizer = AutoTokenizer.from_pretrained('Gaivoronsky/ruGPT-3.5-13B-8bit')
 request = "Человек: Сколько весит жираф? Помощник: "

 from transformers import AutoTokenizer
 from auto_gptq import AutoGPTQForCausalLM
+model = AutoGPTQForCausalLM.from_quantized('Gaivoronsky/ruGPT-3.5-13B-8bit', device="cuda:0", use_triton=False)
 tokenizer = AutoTokenizer.from_pretrained('Gaivoronsky/ruGPT-3.5-13B-8bit')
 request = "Человек: Сколько весит жираф? Помощник: "