Update README.md
Browse files
README.md
CHANGED
|
@@ -13,6 +13,9 @@ datasets:
|
|
| 13 |
- teknium/OpenHermes-2.5
|
| 14 |
- meta-math/MetaMathQA
|
| 15 |
- trl-lib/ultrafeedback-gpt-3.5-turbo-helpfulness
|
|
|
|
|
|
|
|
|
|
| 16 |
---
|
| 17 |
|
| 18 |
# Model Card for xlstm-7b-instruct-phase-2
|
|
@@ -21,7 +24,7 @@ This model is a fine-tuned version of [ethicalabs/xLSTM-7b-Instruct](https://hug
|
|
| 21 |
|
| 22 |
It has been trained using [TRL](https://github.com/huggingface/trl) using SFT on assistant-only tokens.
|
| 23 |
|
| 24 |
-
The k_proj and v_proj matrices have been frozen to isolate and preserve the model's pre-trained knowledge base.
|
| 25 |
|
| 26 |
This fine-tuning focused only on the `q_proj` (query) and FFN matrices, adapting the model's reasoning and query-retrieval mechanisms without overwriting its core, frozen knowledge.
|
| 27 |
|
|
|
|
| 13 |
- teknium/OpenHermes-2.5
|
| 14 |
- meta-math/MetaMathQA
|
| 15 |
- trl-lib/ultrafeedback-gpt-3.5-turbo-helpfulness
|
| 16 |
+
license: mit
|
| 17 |
+
language:
|
| 18 |
+
- en
|
| 19 |
---
|
| 20 |
|
| 21 |
# Model Card for xlstm-7b-instruct-phase-2
|
|
|
|
| 24 |
|
| 25 |
It has been trained using [TRL](https://github.com/huggingface/trl) using SFT on assistant-only tokens.
|
| 26 |
|
| 27 |
+
The `k_proj` and `v_proj` matrices have been frozen to isolate and preserve the model's pre-trained knowledge base.
|
| 28 |
|
| 29 |
This fine-tuning focused only on the `q_proj` (query) and FFN matrices, adapting the model's reasoning and query-retrieval mechanisms without overwriting its core, frozen knowledge.
|
| 30 |
|