Text Generation
PEFT
Safetensors
Transformers
English
lora
sft
trl
🇪🇺 Region: EU
mrs83 commited on
Commit
4f5371c
·
verified ·
1 Parent(s): 84c3b20

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -1
README.md CHANGED
@@ -13,6 +13,9 @@ datasets:
13
  - teknium/OpenHermes-2.5
14
  - meta-math/MetaMathQA
15
  - trl-lib/ultrafeedback-gpt-3.5-turbo-helpfulness
 
 
 
16
  ---
17
 
18
  # Model Card for xlstm-7b-instruct-phase-2
@@ -21,7 +24,7 @@ This model is a fine-tuned version of [ethicalabs/xLSTM-7b-Instruct](https://hug
21
 
22
  It has been trained using [TRL](https://github.com/huggingface/trl) using SFT on assistant-only tokens.
23
 
24
- The k_proj and v_proj matrices have been frozen to isolate and preserve the model's pre-trained knowledge base.
25
 
26
  This fine-tuning focused only on the `q_proj` (query) and FFN matrices, adapting the model's reasoning and query-retrieval mechanisms without overwriting its core, frozen knowledge.
27
 
 
13
  - teknium/OpenHermes-2.5
14
  - meta-math/MetaMathQA
15
  - trl-lib/ultrafeedback-gpt-3.5-turbo-helpfulness
16
+ license: mit
17
+ language:
18
+ - en
19
  ---
20
 
21
  # Model Card for xlstm-7b-instruct-phase-2
 
24
 
25
  It has been trained using [TRL](https://github.com/huggingface/trl) using SFT on assistant-only tokens.
26
 
27
+ The `k_proj` and `v_proj` matrices have been frozen to isolate and preserve the model's pre-trained knowledge base.
28
 
29
  This fine-tuning focused only on the `q_proj` (query) and FFN matrices, adapting the model's reasoning and query-retrieval mechanisms without overwriting its core, frozen knowledge.
30