iproskurina commited on
Commit
6290f5d
·
verified ·
1 Parent(s): 3e99ff2

Update README.md to include GPTQModel usage.

Browse files
Files changed (1) hide show
  1. README.md +0 -14
README.md CHANGED
@@ -128,17 +128,3 @@ model = GPTQModel.load(model_id)
128
  result = model.generate("Uncovering deep insights")[0] # tokens
129
  print(model.tokenizer.decode(result)) # string output
130
  ```
131
-
132
- ### Run the model with GPTQModel
133
-
134
- GPTQModel package: https://github.com/ModelCloud/GPTQModel
135
-
136
- ```
137
- pip install -v gptqmodel=="1.8.0" --no-build-isolation
138
- from gptqmodel import GPTQModel
139
-
140
- model_id = 'iproskurina/bloom-1b7-GPTQ-4bit-g128'
141
- model = GPTQModel.load(model_id)
142
- result = model.generate("Uncovering deep insights")[0] # tokens
143
- print(model.tokenizer.decode(result)) # string output
144
- ```
 
128
  result = model.generate("Uncovering deep insights")[0] # tokens
129
  print(model.tokenizer.decode(result)) # string output
130
  ```