taronklm/Qwen2.5-0.5B-Instruct-chatbot

README.md

@@ -20,7 +20,10 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) on the generator dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.5766
+- Loss: 0.5403
+- Bertscore Precision: 0.9330
+- Bertscore Recall: 0.9366
+- Bertscore F1: 0.9348
 
 ## Model description
 
@@ -51,13 +54,13 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch  | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| No log        | 0.9796 | 18   | 1.1009          |
-| 1.6789        | 1.9592 | 36   | 0.7237          |
-| 1.6789        | 2.9932 | 55   | 0.6249          |
-| 0.578         | 3.9728 | 73   | 0.5859          |
-| 0.4758        | 4.8980 | 90   | 0.5766          |
+| Training Loss | Epoch  | Step | Validation Loss | Bertscore Precision | Bertscore Recall | Bertscore F1 |
+|:-------------:|:------:|:----:|:---------------:|:-------------------:|:----------------:|:------------:|
+| No log        | 0.9664 | 18   | 1.0760          | 0.8860              | 0.8931           | 0.8895       |
+| 1.6935        | 1.9866 | 37   | 0.6704          | 0.9215              | 0.9234           | 0.9224       |
+| 1.6935        | 2.9530 | 55   | 0.5852          | 0.9287              | 0.9322           | 0.9304       |
+| 0.5756        | 3.9732 | 74   | 0.5481          | 0.9346              | 0.9373           | 0.9359       |
+| 0.4437        | 4.8322 | 90   | 0.5403          | 0.9330              | 0.9366           | 0.9348       |
 
 
 ### Framework versions

adapter_config.json

@@ -20,13 +20,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "down_proj",
+    "k_proj",
     "v_proj",
-    "q_proj",
-    "gate_proj",
+    "up_proj",
+    "down_proj",
     "o_proj",
-    "k_proj",
-    "up_proj"
+    "gate_proj",
+    "q_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d78fab658d78d713dfd2e383d47899ceda0f4bef3f8316f83428c349c4ba4858
+oid sha256:6410d0cd89e1e71e3db546964930ea28dbd607c080bdf521371a834bbe38c30b
 size 17640136