princeton-nlp
/

Sheared-Pythia-160m

Text Generation

text-generation-inference

Model card Files Files and versions

Felladrin commited on Nov 24, 2023

Commit

7456015

·

1 Parent(s): 2ba06d5

Fix the model identifier

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -10,7 +10,7 @@ License: Must comply with license of Pythia since it's a model derived from Pyth
 Sheared-Pythia-160m is a model pruned and further pre-trained from [EleutherAI/pythia-410m](https://huggingface.co/EleutherAI/pythia-410m). We dynamically load data from different domains in the Pile dataset to prune and contune pre-train the model. We use 0.4B tokens for pruning and 50B tokens for continued pre-training the pruned model. This model can be loaded with HuggingFace via
 ```
-model = GPTNeoXForCausalLM.from_pretrained("princeton-nlp/Sheared-Pythia-140m")
 ```
 The model's overall performance is better than EleutherAI/pythia-160m.

 Sheared-Pythia-160m is a model pruned and further pre-trained from [EleutherAI/pythia-410m](https://huggingface.co/EleutherAI/pythia-410m). We dynamically load data from different domains in the Pile dataset to prune and contune pre-train the model. We use 0.4B tokens for pruning and 50B tokens for continued pre-training the pruned model. This model can be loaded with HuggingFace via
 ```
+model = GPTNeoXForCausalLM.from_pretrained("princeton-nlp/Sheared-Pythia-160m")
 ```
 The model's overall performance is better than EleutherAI/pythia-160m.