lbourdois commited on
Commit
7d591e4
·
verified ·
1 Parent(s): d5d1d14

Improve language tag

Browse files

Hi! As the model is multilingual, this is a PR to add other languages than English to the language tag to improve the referencing. Note that 29 languages are announced in the README, but only 13 are explicitly listed. I was therefore only able to add these 13 languages.

Files changed (1) hide show
  1. README.md +159 -145
README.md CHANGED
@@ -1,146 +1,160 @@
1
- ---
2
- license: apache-2.0
3
- library_name: peft
4
- tags:
5
- - axolotl
6
- - generated_from_trainer
7
- base_model: Qwen/Qwen2.5-7B-Instruct
8
- model-index:
9
- - name: qwen2.5-7B-instruct-ner-tuned
10
- results: []
11
- ---
12
-
13
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
14
- should probably proofread and complete it, then remove this comment. -->
15
-
16
- [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
17
- <details><summary>See axolotl config</summary>
18
-
19
- axolotl version: `0.4.1`
20
- ```yaml
21
- base_model: Qwen/Qwen2.5-7B-Instruct
22
- model_type: AutoModelForCausalLM
23
- tokenizer_type: AutoTokenizer
24
-
25
- trust_remote_code: true
26
-
27
- load_in_8bit: false
28
- load_in_4bit: true
29
- strict: false
30
-
31
- datasets:
32
- - path: data.jsonl
33
- ds_type: json
34
- type: alpaca
35
-
36
- dataset_prepared_path:
37
- val_set_size: 0.05
38
- output_dir: ./outputs/lora-out
39
- hub_model_id: femT-data/qwen2.5-7B-instruct-ner-tuned
40
-
41
- sequence_len: 4096 # supports up to 8192
42
- sample_packing: false
43
- pad_to_sequence_len:
44
-
45
- adapter: qlora
46
- lora_model_dir:
47
- lora_r: 32
48
- lora_alpha: 16
49
- lora_dropout: 0.05
50
- lora_target_linear: true
51
- lora_fan_in_fan_out:
52
-
53
- wandb_project:
54
- wandb_entity:
55
- wandb_watch:
56
- wandb_name:
57
- wandb_log_model:
58
-
59
- gradient_accumulation_steps: 4
60
- micro_batch_size: 2
61
- num_epochs: 1
62
- optimizer: adamw_bnb_8bit
63
- lr_scheduler: cosine
64
- learning_rate: 0.0002
65
-
66
- train_on_inputs: false
67
- group_by_length: false
68
- bf16: auto
69
- fp16:
70
- tf32: false
71
-
72
- gradient_checkpointing: true
73
- early_stopping_patience:
74
- resume_from_checkpoint:
75
- local_rank:
76
- logging_steps: 1
77
- xformers_attention:
78
- flash_attention:
79
-
80
- warmup_steps: 10
81
- evals_per_epoch: 1
82
- eval_table_size:
83
- eval_max_new_tokens: 128
84
- saves_per_epoch: 1
85
- debug:
86
- deepspeed:
87
- weight_decay: 0.0
88
- fsdp:
89
- fsdp_config:
90
- special_tokens:
91
-
92
- ```
93
-
94
- </details><br>
95
-
96
- # qwen2.5-7B-instruct-ner-tuned
97
-
98
- This model is a fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) on the None dataset.
99
- It achieves the following results on the evaluation set:
100
- - Loss: 0.1159
101
-
102
- ## Model description
103
-
104
- More information needed
105
-
106
- ## Intended uses & limitations
107
-
108
- More information needed
109
-
110
- ## Training and evaluation data
111
-
112
- More information needed
113
-
114
- ## Training procedure
115
-
116
- ### Training hyperparameters
117
-
118
- The following hyperparameters were used during training:
119
- - learning_rate: 0.0002
120
- - train_batch_size: 2
121
- - eval_batch_size: 2
122
- - seed: 42
123
- - distributed_type: multi-GPU
124
- - num_devices: 4
125
- - gradient_accumulation_steps: 4
126
- - total_train_batch_size: 32
127
- - total_eval_batch_size: 8
128
- - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
129
- - lr_scheduler_type: cosine
130
- - lr_scheduler_warmup_steps: 10
131
- - num_epochs: 1
132
-
133
- ### Training results
134
-
135
- | Training Loss | Epoch | Step | Validation Loss |
136
- |:-------------:|:------:|:----:|:---------------:|
137
- | 0.1314 | 0.9630 | 13 | 0.1159 |
138
-
139
-
140
- ### Framework versions
141
-
142
- - PEFT 0.11.1
143
- - Transformers 4.43.1
144
- - Pytorch 2.3.0+cu121
145
- - Datasets 2.19.1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
146
  - Tokenizers 0.19.1
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: peft
4
+ tags:
5
+ - axolotl
6
+ - generated_from_trainer
7
+ base_model: Qwen/Qwen2.5-7B-Instruct
8
+ language:
9
+ - zho
10
+ - eng
11
+ - fra
12
+ - spa
13
+ - por
14
+ - deu
15
+ - ita
16
+ - rus
17
+ - jpn
18
+ - kor
19
+ - vie
20
+ - tha
21
+ - ara
22
+ model-index:
23
+ - name: qwen2.5-7B-instruct-ner-tuned
24
+ results: []
25
+ ---
26
+
27
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
28
+ should probably proofread and complete it, then remove this comment. -->
29
+
30
+ [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
31
+ <details><summary>See axolotl config</summary>
32
+
33
+ axolotl version: `0.4.1`
34
+ ```yaml
35
+ base_model: Qwen/Qwen2.5-7B-Instruct
36
+ model_type: AutoModelForCausalLM
37
+ tokenizer_type: AutoTokenizer
38
+
39
+ trust_remote_code: true
40
+
41
+ load_in_8bit: false
42
+ load_in_4bit: true
43
+ strict: false
44
+
45
+ datasets:
46
+ - path: data.jsonl
47
+ ds_type: json
48
+ type: alpaca
49
+
50
+ dataset_prepared_path:
51
+ val_set_size: 0.05
52
+ output_dir: ./outputs/lora-out
53
+ hub_model_id: femT-data/qwen2.5-7B-instruct-ner-tuned
54
+
55
+ sequence_len: 4096 # supports up to 8192
56
+ sample_packing: false
57
+ pad_to_sequence_len:
58
+
59
+ adapter: qlora
60
+ lora_model_dir:
61
+ lora_r: 32
62
+ lora_alpha: 16
63
+ lora_dropout: 0.05
64
+ lora_target_linear: true
65
+ lora_fan_in_fan_out:
66
+
67
+ wandb_project:
68
+ wandb_entity:
69
+ wandb_watch:
70
+ wandb_name:
71
+ wandb_log_model:
72
+
73
+ gradient_accumulation_steps: 4
74
+ micro_batch_size: 2
75
+ num_epochs: 1
76
+ optimizer: adamw_bnb_8bit
77
+ lr_scheduler: cosine
78
+ learning_rate: 0.0002
79
+
80
+ train_on_inputs: false
81
+ group_by_length: false
82
+ bf16: auto
83
+ fp16:
84
+ tf32: false
85
+
86
+ gradient_checkpointing: true
87
+ early_stopping_patience:
88
+ resume_from_checkpoint:
89
+ local_rank:
90
+ logging_steps: 1
91
+ xformers_attention:
92
+ flash_attention:
93
+
94
+ warmup_steps: 10
95
+ evals_per_epoch: 1
96
+ eval_table_size:
97
+ eval_max_new_tokens: 128
98
+ saves_per_epoch: 1
99
+ debug:
100
+ deepspeed:
101
+ weight_decay: 0.0
102
+ fsdp:
103
+ fsdp_config:
104
+ special_tokens:
105
+
106
+ ```
107
+
108
+ </details><br>
109
+
110
+ # qwen2.5-7B-instruct-ner-tuned
111
+
112
+ This model is a fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) on the None dataset.
113
+ It achieves the following results on the evaluation set:
114
+ - Loss: 0.1159
115
+
116
+ ## Model description
117
+
118
+ More information needed
119
+
120
+ ## Intended uses & limitations
121
+
122
+ More information needed
123
+
124
+ ## Training and evaluation data
125
+
126
+ More information needed
127
+
128
+ ## Training procedure
129
+
130
+ ### Training hyperparameters
131
+
132
+ The following hyperparameters were used during training:
133
+ - learning_rate: 0.0002
134
+ - train_batch_size: 2
135
+ - eval_batch_size: 2
136
+ - seed: 42
137
+ - distributed_type: multi-GPU
138
+ - num_devices: 4
139
+ - gradient_accumulation_steps: 4
140
+ - total_train_batch_size: 32
141
+ - total_eval_batch_size: 8
142
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
143
+ - lr_scheduler_type: cosine
144
+ - lr_scheduler_warmup_steps: 10
145
+ - num_epochs: 1
146
+
147
+ ### Training results
148
+
149
+ | Training Loss | Epoch | Step | Validation Loss |
150
+ |:-------------:|:------:|:----:|:---------------:|
151
+ | 0.1314 | 0.9630 | 13 | 0.1159 |
152
+
153
+
154
+ ### Framework versions
155
+
156
+ - PEFT 0.11.1
157
+ - Transformers 4.43.1
158
+ - Pytorch 2.3.0+cu121
159
+ - Datasets 2.19.1
160
  - Tokenizers 0.19.1