yujiepan
/

microllama-0.06B

Text Generation

Model card Files Files and versions

yujiepan commited on May 16

Commit

9871009

·

verified ·

1 Parent(s): 8544322

Update README.md

Files changed (1) hide show

README.md +4 -4

README.md CHANGED Viewed

@@ -19,7 +19,7 @@ Special thanks to the original author [OuteAI](https://huggingface.co/OuteAI/) f
 ```bash
 lm_eval --model hf \
-  --model_args pretrained=yujiepan/microllama-0.06B,max_length=4096,dtype="<dtype>" \
   --tasks wikitext \
   --device cuda:0 \
   --batch_size 1
@@ -27,8 +27,8 @@ lm_eval --model hf \
 | Model dtype | Word perplexity |
 | ----------- | --------------- |
-| FP32        | 97.3325         |
-| BF16        | 97.2494         |
-| FP16        | 97.3342         |
 Tested on A100 with `lm-eval==0.4.7`.

 ```bash
 lm_eval --model hf \
+  --model_args pretrained=yujiepan/microllama-0.06B,max_length=2048,dtype="<dtype>" \
   --tasks wikitext \
   --device cuda:0 \
   --batch_size 1
 | Model dtype | Word perplexity |
 | ----------- | --------------- |
+| FP32        | 59.1905         |
+| BF16        | 59.1187         |
+| FP16        | 59.1902         |
 Tested on A100 with `lm-eval==0.4.7`.