Intel
/

Phi-3-mini-4k-instruct-int4-inc

Model card Files Files and versions

yintongl commited on May 21, 2024

Commit

cb8311d

·

verified ·

1 Parent(s): 6d3c73a

Update README.md

Files changed (1) hide show

README.md +11 -11

README.md CHANGED Viewed

@@ -53,17 +53,17 @@ lm_eval --model hf --model_args pretrained="Intel/Phi-3-mini-4k-instruct-int4-in
 | Metric         | BF16   | INT4   |
 | -------------- | ------ | ------ |
-| Avg.           | 0.6547 | 0.6509 |
-| mmlu           | 0.6812 | 0.6662 |
-| lambada_openai | 0.6819 | 0.6814 |
-| hellaswag      | 0.6059 | 0.5945 |
-| winogrande     | 0.7403 | 0.7348 |
-| piqa           | 0.8008 | 0.7933 |
-| truthfulqa_mc1 | 0.3929 | 0.3868 |
-| openbookqa     | 0.3900 | 0.3860 |
-| boolq          | 0.8642 | 0.8618 |
-| arc_easy       | 0.8346 | 0.8333 |
-| arc_challenge  | 0.5555 | 0.5708 |

 | Metric         | BF16   | INT4   |
 | -------------- | ------ | ------ |
+| Avg.           | 0.6534 | 0.6509 |
+| mmlu           | 0.6795 | 0.6662 |
+| lambada_openai | 0.6835 | 0.6814 |
+| hellaswag      | 0.6050 | 0.5945 |
+| winogrande     | 0.7364 | 0.7348 |
+| piqa           | 0.8025 | 0.7933 |
+| truthfulqa_mc1 | 0.3917 | 0.3868 |
+| openbookqa     | 0.3860 | 0.3860 |
+| boolq          | 0.8621 | 0.8618 |
+| arc_easy       | 0.8325 | 0.8333 |
+| arc_challenge  | 0.5546 | 0.5708 |