Hugging Face's logo

NV9523
/
MentalGPT_SFT

Text Generation
Transformers
Safetensors
English
Vietnamese
medical
mental
Model card Files
xet
Community
NV9523 commited on
Commit
0a13680
·
verified ·
1 Parent(s): 9753289

Delete checkpoint-98500

Browse files
Files changed (12) hide show
  1. checkpoint-98500/README.md +0 -202
  2. checkpoint-98500/adapter_config.json +0 -39
  3. checkpoint-98500/adapter_model.safetensors +0 -3
  4. checkpoint-98500/chat_template.jinja +0 -93
  5. checkpoint-98500/optimizer.pt +0 -3
  6. checkpoint-98500/rng_state.pth +0 -3
  7. checkpoint-98500/scheduler.pt +0 -3
  8. checkpoint-98500/special_tokens_map.json +0 -17
  9. checkpoint-98500/tokenizer.json +0 -3
  10. checkpoint-98500/tokenizer_config.json +0 -2063
  11. checkpoint-98500/trainer_state.json +0 -1756
  12. checkpoint-98500/training_args.bin +0 -3
checkpoint-98500/README.md DELETED
@@ -1,202 +0,0 @@
1
- ---
2
- base_model: meta-llama/Llama-3.2-1B-Instruct
3
- library_name: peft
4
- ---
5
-
6
- # Model Card for Model ID
7
-
8
- <!-- Provide a quick summary of what the model is/does. -->
9
-
10
-
11
-
12
- ## Model Details
13
-
14
- ### Model Description
15
-
16
- <!-- Provide a longer summary of what this model is. -->
17
-
18
-
19
-
20
- - **Developed by:** [More Information Needed]
21
- - **Funded by [optional]:** [More Information Needed]
22
- - **Shared by [optional]:** [More Information Needed]
23
- - **Model type:** [More Information Needed]
24
- - **Language(s) (NLP):** [More Information Needed]
25
- - **License:** [More Information Needed]
26
- - **Finetuned from model [optional]:** [More Information Needed]
27
-
28
- ### Model Sources [optional]
29
-
30
- <!-- Provide the basic links for the model. -->
31
-
32
- - **Repository:** [More Information Needed]
33
- - **Paper [optional]:** [More Information Needed]
34
- - **Demo [optional]:** [More Information Needed]
35
-
36
- ## Uses
37
-
38
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
-
40
- ### Direct Use
41
-
42
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
-
44
- [More Information Needed]
45
-
46
- ### Downstream Use [optional]
47
-
48
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
-
50
- [More Information Needed]
51
-
52
- ### Out-of-Scope Use
53
-
54
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
-
56
- [More Information Needed]
57
-
58
- ## Bias, Risks, and Limitations
59
-
60
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
-
62
- [More Information Needed]
63
-
64
- ### Recommendations
65
-
66
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
-
68
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
-
70
- ## How to Get Started with the Model
71
-
72
- Use the code below to get started with the model.
73
-
74
- [More Information Needed]
75
-
76
- ## Training Details
77
-
78
- ### Training Data
79
-
80
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
-
82
- [More Information Needed]
83
-
84
- ### Training Procedure
85
-
86
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
-
88
- #### Preprocessing [optional]
89
-
90
- [More Information Needed]
91
-
92
-
93
- #### Training Hyperparameters
94
-
95
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
-
97
- #### Speeds, Sizes, Times [optional]
98
-
99
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
-
101
- [More Information Needed]
102
-
103
- ## Evaluation
104
-
105
- <!-- This section describes the evaluation protocols and provides the results. -->
106
-
107
- ### Testing Data, Factors & Metrics
108
-
109
- #### Testing Data
110
-
111
- <!-- This should link to a Dataset Card if possible. -->
112
-
113
- [More Information Needed]
114
-
115
- #### Factors
116
-
117
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
-
119
- [More Information Needed]
120
-
121
- #### Metrics
122
-
123
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
-
125
- [More Information Needed]
126
-
127
- ### Results
128
-
129
- [More Information Needed]
130
-
131
- #### Summary
132
-
133
-
134
-
135
- ## Model Examination [optional]
136
-
137
- <!-- Relevant interpretability work for the model goes here -->
138
-
139
- [More Information Needed]
140
-
141
- ## Environmental Impact
142
-
143
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
-
145
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
-
147
- - **Hardware Type:** [More Information Needed]
148
- - **Hours used:** [More Information Needed]
149
- - **Cloud Provider:** [More Information Needed]
150
- - **Compute Region:** [More Information Needed]
151
- - **Carbon Emitted:** [More Information Needed]
152
-
153
- ## Technical Specifications [optional]
154
-
155
- ### Model Architecture and Objective
156
-
157
- [More Information Needed]
158
-
159
- ### Compute Infrastructure
160
-
161
- [More Information Needed]
162
-
163
- #### Hardware
164
-
165
- [More Information Needed]
166
-
167
- #### Software
168
-
169
- [More Information Needed]
170
-
171
- ## Citation [optional]
172
-
173
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
-
175
- **BibTeX:**
176
-
177
- [More Information Needed]
178
-
179
- **APA:**
180
-
181
- [More Information Needed]
182
-
183
- ## Glossary [optional]
184
-
185
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
-
187
- [More Information Needed]
188
-
189
- ## More Information [optional]
190
-
191
- [More Information Needed]
192
-
193
- ## Model Card Authors [optional]
194
-
195
- [More Information Needed]
196
-
197
- ## Model Card Contact
198
-
199
- [More Information Needed]
200
- ### Framework versions
201
-
202
- - PEFT 0.15.2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-98500/adapter_config.json DELETED
@@ -1,39 +0,0 @@
1
- {
2
- "alpha_pattern": {},
3
- "auto_mapping": null,
4
- "base_model_name_or_path": "meta-llama/Llama-3.2-1B-Instruct",
5
- "bias": "none",
6
- "corda_config": null,
7
- "eva_config": null,
8
- "exclude_modules": null,
9
- "fan_in_fan_out": false,
10
- "inference_mode": true,
11
- "init_lora_weights": true,
12
- "layer_replication": null,
13
- "layers_pattern": null,
14
- "layers_to_transform": null,
15
- "loftq_config": {},
16
- "lora_alpha": 128,
17
- "lora_bias": false,
18
- "lora_dropout": 0.05,
19
- "megatron_config": null,
20
- "megatron_core": "megatron.core",
21
- "modules_to_save": null,
22
- "peft_type": "LORA",
23
- "r": 64,
24
- "rank_pattern": {},
25
- "revision": null,
26
- "target_modules": [
27
- "v_proj",
28
- "gate_proj",
29
- "o_proj",
30
- "down_proj",
31
- "q_proj",
32
- "up_proj",
33
- "k_proj"
34
- ],
35
- "task_type": "CAUSAL_LM",
36
- "trainable_token_indices": null,
37
- "use_dora": false,
38
- "use_rslora": false
39
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-98500/adapter_model.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:e0e8c65cd69760782c22f081cc5f4e3f97d8c1eb0f5e39eb606378c1033ca9d3
3
- size 180385008
 
 
 
 
checkpoint-98500/chat_template.jinja DELETED
@@ -1,93 +0,0 @@
1
- {{- bos_token }}
2
- {%- if custom_tools is defined %}
3
- {%- set tools = custom_tools %}
4
- {%- endif %}
5
- {%- if not tools_in_user_message is defined %}
6
- {%- set tools_in_user_message = true %}
7
- {%- endif %}
8
- {%- if not date_string is defined %}
9
- {%- if strftime_now is defined %}
10
- {%- set date_string = strftime_now("%d %b %Y") %}
11
- {%- else %}
12
- {%- set date_string = "26 Jul 2024" %}
13
- {%- endif %}
14
- {%- endif %}
15
- {%- if not tools is defined %}
16
- {%- set tools = none %}
17
- {%- endif %}
18
-
19
- {#- This block extracts the system message, so we can slot it into the right place. #}
20
- {%- if messages[0]['role'] == 'system' %}
21
- {%- set system_message = messages[0]['content']|trim %}
22
- {%- set messages = messages[1:] %}
23
- {%- else %}
24
- {%- set system_message = "" %}
25
- {%- endif %}
26
-
27
- {#- System message #}
28
- {{- "<|start_header_id|>system<|end_header_id|>\n\n" }}
29
- {%- if tools is not none %}
30
- {{- "Environment: ipython\n" }}
31
- {%- endif %}
32
- {{- "Cutting Knowledge Date: December 2023\n" }}
33
- {{- "Today Date: " + date_string + "\n\n" }}
34
- {%- if tools is not none and not tools_in_user_message %}
35
- {{- "You have access to the following functions. To call a function, please respond with JSON for a function call." }}
36
- {{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
37
- {{- "Do not use variables.\n\n" }}
38
- {%- for t in tools %}
39
- {{- t | tojson(indent=4) }}
40
- {{- "\n\n" }}
41
- {%- endfor %}
42
- {%- endif %}
43
- {{- system_message }}
44
- {{- "<|eot_id|>" }}
45
-
46
- {#- Custom tools are passed in a user message with some extra guidance #}
47
- {%- if tools_in_user_message and not tools is none %}
48
- {#- Extract the first user message so we can plug it in here #}
49
- {%- if messages | length != 0 %}
50
- {%- set first_user_message = messages[0]['content']|trim %}
51
- {%- set messages = messages[1:] %}
52
- {%- else %}
53
- {{- raise_exception("Cannot put tools in the first user message when there's no first user message!") }}
54
- {%- endif %}
55
- {{- '<|start_header_id|>user<|end_header_id|>\n\n' -}}
56
- {{- "Given the following functions, please respond with a JSON for a function call " }}
57
- {{- "with its proper arguments that best answers the given prompt.\n\n" }}
58
- {{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
59
- {{- "Do not use variables.\n\n" }}
60
- {%- for t in tools %}
61
- {{- t | tojson(indent=4) }}
62
- {{- "\n\n" }}
63
- {%- endfor %}
64
- {{- first_user_message + "<|eot_id|>"}}
65
- {%- endif %}
66
-
67
- {%- for message in messages %}
68
- {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}
69
- {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' }}
70
- {%- elif 'tool_calls' in message %}
71
- {%- if not message.tool_calls|length == 1 %}
72
- {{- raise_exception("This model only supports single tool-calls at once!") }}
73
- {%- endif %}
74
- {%- set tool_call = message.tool_calls[0].function %}
75
- {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' -}}
76
- {{- '{"name": "' + tool_call.name + '", ' }}
77
- {{- '"parameters": ' }}
78
- {{- tool_call.arguments | tojson }}
79
- {{- "}" }}
80
- {{- "<|eot_id|>" }}
81
- {%- elif message.role == "tool" or message.role == "ipython" %}
82
- {{- "<|start_header_id|>ipython<|end_header_id|>\n\n" }}
83
- {%- if message.content is mapping or message.content is iterable %}
84
- {{- message.content | tojson }}
85
- {%- else %}
86
- {{- message.content }}
87
- {%- endif %}
88
- {{- "<|eot_id|>" }}
89
- {%- endif %}
90
- {%- endfor %}
91
- {%- if add_generation_prompt %}
92
- {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' }}
93
- {%- endif %}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-98500/optimizer.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:d3445b08b4e3f3c399899251e277a507cb1064e07250cd6b015185ac1f2d89fe
3
- size 92432402
 
 
 
 
checkpoint-98500/rng_state.pth DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:f5453b152958f4dd1d018cac63ff1b140d5cd13a1a97cbe4d967ec63115d70ce
3
- size 14244
 
 
 
 
checkpoint-98500/scheduler.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:bc6acdb14a72cb6b51a8d6d9de15a63664ed684ba84dde5b83d06e1323523156
3
- size 1064
 
 
 
 
checkpoint-98500/special_tokens_map.json DELETED
@@ -1,17 +0,0 @@
1
- {
2
- "bos_token": {
3
- "content": "<|begin_of_text|>",
4
- "lstrip": false,
5
- "normalized": false,
6
- "rstrip": false,
7
- "single_word": false
8
- },
9
- "eos_token": {
10
- "content": "<|eot_id|>",
11
- "lstrip": false,
12
- "normalized": false,
13
- "rstrip": false,
14
- "single_word": false
15
- },
16
- "pad_token": "<|eot_id|>"
17
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-98500/tokenizer.json DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:90883524dbec2e8c465564ac46b4e5298235668a5cf8523690f06f45f51646fe
3
- size 17210197
 
 
 
 
checkpoint-98500/tokenizer_config.json DELETED
@@ -1,2063 +0,0 @@
1
- {
2
- "added_tokens_decoder": {
3
- "128000": {
4
- "content": "<|begin_of_text|>",
5
- "lstrip": false,
6
- "normalized": false,
7
- "rstrip": false,
8
- "single_word": false,
9
- "special": true
10
- },
11
- "128001": {
12
- "content": "<|end_of_text|>",
13
- "lstrip": false,
14
- "normalized": false,
15
- "rstrip": false,
16
- "single_word": false,
17
- "special": true
18
- },
19
- "128002": {
20
- "content": "<|reserved_special_token_0|>",
21
- "lstrip": false,
22
- "normalized": false,
23
- "rstrip": false,
24
- "single_word": false,
25
- "special": true
26
- },
27
- "128003": {
28
- "content": "<|reserved_special_token_1|>",
29
- "lstrip": false,
30
- "normalized": false,
31
- "rstrip": false,
32
- "single_word": false,
33
- "special": true
34
- },
35
- "128004": {
36
- "content": "<|finetune_right_pad_id|>",
37
- "lstrip": false,
38
- "normalized": false,
39
- "rstrip": false,
40
- "single_word": false,
41
- "special": true
42
- },
43
- "128005": {
44
- "content": "<|reserved_special_token_2|>",
45
- "lstrip": false,
46
- "normalized": false,
47
- "rstrip": false,
48
- "single_word": false,
49
- "special": true
50
- },
51
- "128006": {
52
- "content": "<|start_header_id|>",
53
- "lstrip": false,
54
- "normalized": false,
55
- "rstrip": false,
56
- "single_word": false,
57
- "special": true
58
- },
59
- "128007": {
60
- "content": "<|end_header_id|>",
61
- "lstrip": false,
62
- "normalized": false,
63
- "rstrip": false,
64
- "single_word": false,
65
- "special": true
66
- },
67
- "128008": {
68
- "content": "<|eom_id|>",
69
- "lstrip": false,
70
- "normalized": false,
71
- "rstrip": false,
72
- "single_word": false,
73
- "special": true
74
- },
75
- "128009": {
76
- "content": "<|eot_id|>",
77
- "lstrip": false,
78
- "normalized": false,
79
- "rstrip": false,
80
- "single_word": false,
81
- "special": true
82
- },
83
- "128010": {
84
- "content": "<|python_tag|>",
85
- "lstrip": false,
86
- "normalized": false,
87
- "rstrip": false,
88
- "single_word": false,
89
- "special": true
90
- },
91
- "128011": {
92
- "content": "<|reserved_special_token_3|>",
93
- "lstrip": false,
94
- "normalized": false,
95
- "rstrip": false,
96
- "single_word": false,
97
- "special": true
98
- },
99
- "128012": {
100
- "content": "<|reserved_special_token_4|>",
101
- "lstrip": false,
102
- "normalized": false,
103
- "rstrip": false,
104
- "single_word": false,
105
- "special": true
106
- },
107
- "128013": {
108
- "content": "<|reserved_special_token_5|>",
109
- "lstrip": false,
110
- "normalized": false,
111
- "rstrip": false,
112
- "single_word": false,
113
- "special": true
114
- },
115
- "128014": {
116
- "content": "<|reserved_special_token_6|>",
117
- "lstrip": false,
118
- "normalized": false,
119
- "rstrip": false,
120
- "single_word": false,
121
- "special": true
122
- },
123
- "128015": {
124
- "content": "<|reserved_special_token_7|>",
125
- "lstrip": false,
126
- "normalized": false,
127
- "rstrip": false,
128
- "single_word": false,
129
- "special": true
130
- },
131
- "128016": {
132
- "content": "<|reserved_special_token_8|>",
133
- "lstrip": false,
134
- "normalized": false,
135
- "rstrip": false,
136
- "single_word": false,
137
- "special": true
138
- },
139
- "128017": {
140
- "content": "<|reserved_special_token_9|>",
141
- "lstrip": false,
142
- "normalized": false,
143
- "rstrip": false,
144
- "single_word": false,
145
- "special": true
146
- },
147
- "128018": {
148
- "content": "<|reserved_special_token_10|>",
149
- "lstrip": false,
150
- "normalized": false,
151
- "rstrip": false,
152
- "single_word": false,
153
- "special": true
154
- },
155
- "128019": {
156
- "content": "<|reserved_special_token_11|>",
157
- "lstrip": false,
158
- "normalized": false,
159
- "rstrip": false,
160
- "single_word": false,
161
- "special": true
162
- },
163
- "128020": {
164
- "content": "<|reserved_special_token_12|>",
165
- "lstrip": false,
166
- "normalized": false,
167
- "rstrip": false,
168
- "single_word": false,
169
- "special": true
170
- },
171
- "128021": {
172
- "content": "<|reserved_special_token_13|>",
173
- "lstrip": false,
174
- "normalized": false,
175
- "rstrip": false,
176
- "single_word": false,
177
- "special": true
178
- },
179
- "128022": {
180
- "content": "<|reserved_special_token_14|>",
181
- "lstrip": false,
182
- "normalized": false,
183
- "rstrip": false,
184
- "single_word": false,
185
- "special": true
186
- },
187
- "128023": {
188
- "content": "<|reserved_special_token_15|>",
189
- "lstrip": false,
190
- "normalized": false,
191
- "rstrip": false,
192
- "single_word": false,
193
- "special": true
194
- },
195
- "128024": {
196
- "content": "<|reserved_special_token_16|>",
197
- "lstrip": false,
198
- "normalized": false,
199
- "rstrip": false,
200
- "single_word": false,
201
- "special": true
202
- },
203
- "128025": {
204
- "content": "<|reserved_special_token_17|>",
205
- "lstrip": false,
206
- "normalized": false,
207
- "rstrip": false,
208
- "single_word": false,
209
- "special": true
210
- },
211
- "128026": {
212
- "content": "<|reserved_special_token_18|>",
213
- "lstrip": false,
214
- "normalized": false,
215
- "rstrip": false,
216
- "single_word": false,
217
- "special": true
218
- },
219
- "128027": {
220
- "content": "<|reserved_special_token_19|>",
221
- "lstrip": false,
222
- "normalized": false,
223
- "rstrip": false,
224
- "single_word": false,
225
- "special": true
226
- },
227
- "128028": {
228
- "content": "<|reserved_special_token_20|>",
229
- "lstrip": false,
230
- "normalized": false,
231
- "rstrip": false,
232
- "single_word": false,
233
- "special": true
234
- },
235
- "128029": {
236
- "content": "<|reserved_special_token_21|>",
237
- "lstrip": false,
238
- "normalized": false,
239
- "rstrip": false,
240
- "single_word": false,
241
- "special": true
242
- },
243
- "128030": {
244
- "content": "<|reserved_special_token_22|>",
245
- "lstrip": false,
246
- "normalized": false,
247
- "rstrip": false,
248
- "single_word": false,
249
- "special": true
250
- },
251
- "128031": {
252
- "content": "<|reserved_special_token_23|>",
253
- "lstrip": false,
254
- "normalized": false,
255
- "rstrip": false,
256
- "single_word": false,
257
- "special": true
258
- },
259
- "128032": {
260
- "content": "<|reserved_special_token_24|>",
261
- "lstrip": false,
262
- "normalized": false,
263
- "rstrip": false,
264
- "single_word": false,
265
- "special": true
266
- },
267
- "128033": {
268
- "content": "<|reserved_special_token_25|>",
269
- "lstrip": false,
270
- "normalized": false,
271
- "rstrip": false,
272
- "single_word": false,
273
- "special": true
274
- },
275
- "128034": {
276
- "content": "<|reserved_special_token_26|>",
277
- "lstrip": false,
278
- "normalized": false,
279
- "rstrip": false,
280
- "single_word": false,
281
- "special": true
282
- },
283
- "128035": {
284
- "content": "<|reserved_special_token_27|>",
285
- "lstrip": false,
286
- "normalized": false,
287
- "rstrip": false,
288
- "single_word": false,
289
- "special": true
290
- },
291
- "128036": {
292
- "content": "<|reserved_special_token_28|>",
293
- "lstrip": false,
294
- "normalized": false,
295
- "rstrip": false,
296
- "single_word": false,
297
- "special": true
298
- },
299
- "128037": {
300
- "content": "<|reserved_special_token_29|>",
301
- "lstrip": false,
302
- "normalized": false,
303
- "rstrip": false,
304
- "single_word": false,
305
- "special": true
306
- },
307
- "128038": {
308
- "content": "<|reserved_special_token_30|>",
309
- "lstrip": false,
310
- "normalized": false,
311
- "rstrip": false,
312
- "single_word": false,
313
- "special": true
314
- },
315
- "128039": {
316
- "content": "<|reserved_special_token_31|>",
317
- "lstrip": false,
318
- "normalized": false,
319
- "rstrip": false,
320
- "single_word": false,
321
- "special": true
322
- },
323
- "128040": {
324
- "content": "<|reserved_special_token_32|>",
325
- "lstrip": false,
326
- "normalized": false,
327
- "rstrip": false,
328
- "single_word": false,
329
- "special": true
330
- },
331
- "128041": {
332
- "content": "<|reserved_special_token_33|>",
333
- "lstrip": false,
334
- "normalized": false,
335
- "rstrip": false,
336
- "single_word": false,
337
- "special": true
338
- },
339
- "128042": {
340
- "content": "<|reserved_special_token_34|>",
341
- "lstrip": false,
342
- "normalized": false,
343
- "rstrip": false,
344
- "single_word": false,
345
- "special": true
346
- },
347
- "128043": {
348
- "content": "<|reserved_special_token_35|>",
349
- "lstrip": false,
350
- "normalized": false,
351
- "rstrip": false,
352
- "single_word": false,
353
- "special": true
354
- },
355
- "128044": {
356
- "content": "<|reserved_special_token_36|>",
357
- "lstrip": false,
358
- "normalized": false,
359
- "rstrip": false,
360
- "single_word": false,
361
- "special": true
362
- },
363
- "128045": {
364
- "content": "<|reserved_special_token_37|>",
365
- "lstrip": false,
366
- "normalized": false,
367
- "rstrip": false,
368
- "single_word": false,
369
- "special": true
370
- },
371
- "128046": {
372
- "content": "<|reserved_special_token_38|>",
373
- "lstrip": false,
374
- "normalized": false,
375
- "rstrip": false,
376
- "single_word": false,
377
- "special": true
378
- },
379
- "128047": {
380
- "content": "<|reserved_special_token_39|>",
381
- "lstrip": false,
382
- "normalized": false,
383
- "rstrip": false,
384
- "single_word": false,
385
- "special": true
386
- },
387
- "128048": {
388
- "content": "<|reserved_special_token_40|>",
389
- "lstrip": false,
390
- "normalized": false,
391
- "rstrip": false,
392
- "single_word": false,
393
- "special": true
394
- },
395
- "128049": {
396
- "content": "<|reserved_special_token_41|>",
397
- "lstrip": false,
398
- "normalized": false,
399
- "rstrip": false,
400
- "single_word": false,
401
- "special": true
402
- },
403
- "128050": {
404
- "content": "<|reserved_special_token_42|>",
405
- "lstrip": false,
406
- "normalized": false,
407
- "rstrip": false,
408
- "single_word": false,
409
- "special": true
410
- },
411
- "128051": {
412
- "content": "<|reserved_special_token_43|>",
413
- "lstrip": false,
414
- "normalized": false,
415
- "rstrip": false,
416
- "single_word": false,
417
- "special": true
418
- },
419
- "128052": {
420
- "content": "<|reserved_special_token_44|>",
421
- "lstrip": false,
422
- "normalized": false,
423
- "rstrip": false,
424
- "single_word": false,
425
- "special": true
426
- },
427
- "128053": {
428
- "content": "<|reserved_special_token_45|>",
429
- "lstrip": false,
430
- "normalized": false,
431
- "rstrip": false,
432
- "single_word": false,
433
- "special": true
434
- },
435
- "128054": {
436
- "content": "<|reserved_special_token_46|>",
437
- "lstrip": false,
438
- "normalized": false,
439
- "rstrip": false,
440
- "single_word": false,
441
- "special": true
442
- },
443
- "128055": {
444
- "content": "<|reserved_special_token_47|>",
445
- "lstrip": false,
446
- "normalized": false,
447
- "rstrip": false,
448
- "single_word": false,
449
- "special": true
450
- },
451
- "128056": {
452
- "content": "<|reserved_special_token_48|>",
453
- "lstrip": false,
454
- "normalized": false,
455
- "rstrip": false,
456
- "single_word": false,
457
- "special": true
458
- },
459
- "128057": {
460
- "content": "<|reserved_special_token_49|>",
461
- "lstrip": false,
462
- "normalized": false,
463
- "rstrip": false,
464
- "single_word": false,
465
- "special": true
466
- },
467
- "128058": {
468
- "content": "<|reserved_special_token_50|>",
469
- "lstrip": false,
470
- "normalized": false,
471
- "rstrip": false,
472
- "single_word": false,
473
- "special": true
474
- },
475
- "128059": {
476
- "content": "<|reserved_special_token_51|>",
477
- "lstrip": false,
478
- "normalized": false,
479
- "rstrip": false,
480
- "single_word": false,
481
- "special": true
482
- },
483
- "128060": {
484
- "content": "<|reserved_special_token_52|>",
485
- "lstrip": false,
486
- "normalized": false,
487
- "rstrip": false,
488
- "single_word": false,
489
- "special": true
490
- },
491
- "128061": {
492
- "content": "<|reserved_special_token_53|>",
493
- "lstrip": false,
494
- "normalized": false,
495
- "rstrip": false,
496
- "single_word": false,
497
- "special": true
498
- },
499
- "128062": {
500
- "content": "<|reserved_special_token_54|>",
501
- "lstrip": false,
502
- "normalized": false,
503
- "rstrip": false,
504
- "single_word": false,
505
- "special": true
506
- },
507
- "128063": {
508
- "content": "<|reserved_special_token_55|>",
509
- "lstrip": false,
510
- "normalized": false,
511
- "rstrip": false,
512
- "single_word": false,
513
- "special": true
514
- },
515
- "128064": {
516
- "content": "<|reserved_special_token_56|>",
517
- "lstrip": false,
518
- "normalized": false,
519
- "rstrip": false,
520
- "single_word": false,
521
- "special": true
522
- },
523
- "128065": {
524
- "content": "<|reserved_special_token_57|>",
525
- "lstrip": false,
526
- "normalized": false,
527
- "rstrip": false,
528
- "single_word": false,
529
- "special": true
530
- },
531
- "128066": {
532
- "content": "<|reserved_special_token_58|>",
533
- "lstrip": false,
534
- "normalized": false,
535
- "rstrip": false,
536
- "single_word": false,
537
- "special": true
538
- },
539
- "128067": {
540
- "content": "<|reserved_special_token_59|>",
541
- "lstrip": false,
542
- "normalized": false,
543
- "rstrip": false,
544
- "single_word": false,
545
- "special": true
546
- },
547
- "128068": {
548
- "content": "<|reserved_special_token_60|>",
549
- "lstrip": false,
550
- "normalized": false,
551
- "rstrip": false,
552
- "single_word": false,
553
- "special": true
554
- },
555
- "128069": {
556
- "content": "<|reserved_special_token_61|>",
557
- "lstrip": false,
558
- "normalized": false,
559
- "rstrip": false,
560
- "single_word": false,
561
- "special": true
562
- },
563
- "128070": {
564
- "content": "<|reserved_special_token_62|>",
565
- "lstrip": false,
566
- "normalized": false,
567
- "rstrip": false,
568
- "single_word": false,
569
- "special": true
570
- },
571
- "128071": {
572
- "content": "<|reserved_special_token_63|>",
573
- "lstrip": false,
574
- "normalized": false,
575
- "rstrip": false,
576
- "single_word": false,
577
- "special": true
578
- },
579
- "128072": {
580
- "content": "<|reserved_special_token_64|>",
581
- "lstrip": false,
582
- "normalized": false,
583
- "rstrip": false,
584
- "single_word": false,
585
- "special": true
586
- },
587
- "128073": {
588
- "content": "<|reserved_special_token_65|>",
589
- "lstrip": false,
590
- "normalized": false,
591
- "rstrip": false,
592
- "single_word": false,
593
- "special": true
594
- },
595
- "128074": {
596
- "content": "<|reserved_special_token_66|>",
597
- "lstrip": false,
598
- "normalized": false,
599
- "rstrip": false,
600
- "single_word": false,
601
- "special": true
602
- },
603
- "128075": {
604
- "content": "<|reserved_special_token_67|>",
605
- "lstrip": false,
606
- "normalized": false,
607
- "rstrip": false,
608
- "single_word": false,
609
- "special": true
610
- },
611
- "128076": {
612
- "content": "<|reserved_special_token_68|>",
613
- "lstrip": false,
614
- "normalized": false,
615
- "rstrip": false,
616
- "single_word": false,
617
- "special": true
618
- },
619
- "128077": {
620
- "content": "<|reserved_special_token_69|>",
621
- "lstrip": false,
622
- "normalized": false,
623
- "rstrip": false,
624
- "single_word": false,
625
- "special": true
626
- },
627
- "128078": {
628
- "content": "<|reserved_special_token_70|>",
629
- "lstrip": false,
630
- "normalized": false,
631
- "rstrip": false,
632
- "single_word": false,
633
- "special": true
634
- },
635
- "128079": {
636
- "content": "<|reserved_special_token_71|>",
637
- "lstrip": false,
638
- "normalized": false,
639
- "rstrip": false,
640
- "single_word": false,
641
- "special": true
642
- },
643
- "128080": {
644
- "content": "<|reserved_special_token_72|>",
645
- "lstrip": false,
646
- "normalized": false,
647
- "rstrip": false,
648
- "single_word": false,
649
- "special": true
650
- },
651
- "128081": {
652
- "content": "<|reserved_special_token_73|>",
653
- "lstrip": false,
654
- "normalized": false,
655
- "rstrip": false,
656
- "single_word": false,
657
- "special": true
658
- },
659
- "128082": {
660
- "content": "<|reserved_special_token_74|>",
661
- "lstrip": false,
662
- "normalized": false,
663
- "rstrip": false,
664
- "single_word": false,
665
- "special": true
666
- },
667
- "128083": {
668
- "content": "<|reserved_special_token_75|>",
669
- "lstrip": false,
670
- "normalized": false,
671
- "rstrip": false,
672
- "single_word": false,
673
- "special": true
674
- },
675
- "128084": {
676
- "content": "<|reserved_special_token_76|>",
677
- "lstrip": false,
678
- "normalized": false,
679
- "rstrip": false,
680
- "single_word": false,
681
- "special": true
682
- },
683
- "128085": {
684
- "content": "<|reserved_special_token_77|>",
685
- "lstrip": false,
686
- "normalized": false,
687
- "rstrip": false,
688
- "single_word": false,
689
- "special": true
690
- },
691
- "128086": {
692
- "content": "<|reserved_special_token_78|>",
693
- "lstrip": false,
694
- "normalized": false,
695
- "rstrip": false,
696
- "single_word": false,
697
- "special": true
698
- },
699
- "128087": {
700
- "content": "<|reserved_special_token_79|>",
701
- "lstrip": false,
702
- "normalized": false,
703
- "rstrip": false,
704
- "single_word": false,
705
- "special": true
706
- },
707
- "128088": {
708
- "content": "<|reserved_special_token_80|>",
709
- "lstrip": false,
710
- "normalized": false,
711
- "rstrip": false,
712
- "single_word": false,
713
- "special": true
714
- },
715
- "128089": {
716
- "content": "<|reserved_special_token_81|>",
717
- "lstrip": false,
718
- "normalized": false,
719
- "rstrip": false,
720
- "single_word": false,
721
- "special": true
722
- },
723
- "128090": {
724
- "content": "<|reserved_special_token_82|>",
725
- "lstrip": false,
726
- "normalized": false,
727
- "rstrip": false,
728
- "single_word": false,
729
- "special": true
730
- },
731
- "128091": {
732
- "content": "<|reserved_special_token_83|>",
733
- "lstrip": false,
734
- "normalized": false,
735
- "rstrip": false,
736
- "single_word": false,
737
- "special": true
738
- },
739
- "128092": {
740
- "content": "<|reserved_special_token_84|>",
741
- "lstrip": false,
742
- "normalized": false,
743
- "rstrip": false,
744
- "single_word": false,
745
- "special": true
746
- },
747
- "128093": {
748
- "content": "<|reserved_special_token_85|>",
749
- "lstrip": false,
750
- "normalized": false,
751
- "rstrip": false,
752
- "single_word": false,
753
- "special": true
754
- },
755
- "128094": {
756
- "content": "<|reserved_special_token_86|>",
757
- "lstrip": false,
758
- "normalized": false,
759
- "rstrip": false,
760
- "single_word": false,
761
- "special": true
762
- },
763
- "128095": {
764
- "content": "<|reserved_special_token_87|>",
765
- "lstrip": false,
766
- "normalized": false,
767
- "rstrip": false,
768
- "single_word": false,
769
- "special": true
770
- },
771
- "128096": {
772
- "content": "<|reserved_special_token_88|>",
773
- "lstrip": false,
774
- "normalized": false,
775
- "rstrip": false,
776
- "single_word": false,
777
- "special": true
778
- },
779
- "128097": {
780
- "content": "<|reserved_special_token_89|>",
781
- "lstrip": false,
782
- "normalized": false,
783
- "rstrip": false,
784
- "single_word": false,
785
- "special": true
786
- },
787
- "128098": {
788
- "content": "<|reserved_special_token_90|>",
789
- "lstrip": false,
790
- "normalized": false,
791
- "rstrip": false,
792
- "single_word": false,
793
- "special": true
794
- },
795
- "128099": {
796
- "content": "<|reserved_special_token_91|>",
797
- "lstrip": false,
798
- "normalized": false,
799
- "rstrip": false,
800
- "single_word": false,
801
- "special": true
802
- },
803
- "128100": {
804
- "content": "<|reserved_special_token_92|>",
805
- "lstrip": false,
806
- "normalized": false,
807
- "rstrip": false,
808
- "single_word": false,
809
- "special": true
810
- },
811
- "128101": {
812
- "content": "<|reserved_special_token_93|>",
813
- "lstrip": false,
814
- "normalized": false,
815
- "rstrip": false,
816
- "single_word": false,
817
- "special": true
818
- },
819
- "128102": {
820
- "content": "<|reserved_special_token_94|>",
821
- "lstrip": false,
822
- "normalized": false,
823
- "rstrip": false,
824
- "single_word": false,
825
- "special": true
826
- },
827
- "128103": {
828
- "content": "<|reserved_special_token_95|>",
829
- "lstrip": false,
830
- "normalized": false,
831
- "rstrip": false,
832
- "single_word": false,
833
- "special": true
834
- },
835
- "128104": {
836
- "content": "<|reserved_special_token_96|>",
837
- "lstrip": false,
838
- "normalized": false,
839
- "rstrip": false,
840
- "single_word": false,
841
- "special": true
842
- },
843
- "128105": {
844
- "content": "<|reserved_special_token_97|>",
845
- "lstrip": false,
846
- "normalized": false,
847
- "rstrip": false,
848
- "single_word": false,
849
- "special": true
850
- },
851
- "128106": {
852
- "content": "<|reserved_special_token_98|>",
853
- "lstrip": false,
854
- "normalized": false,
855
- "rstrip": false,
856
- "single_word": false,
857
- "special": true
858
- },
859
- "128107": {
860
- "content": "<|reserved_special_token_99|>",
861
- "lstrip": false,
862
- "normalized": false,
863
- "rstrip": false,
864
- "single_word": false,
865
- "special": true
866
- },
867
- "128108": {
868
- "content": "<|reserved_special_token_100|>",
869
- "lstrip": false,
870
- "normalized": false,
871
- "rstrip": false,
872
- "single_word": false,
873
- "special": true
874
- },
875
- "128109": {
876
- "content": "<|reserved_special_token_101|>",
877
- "lstrip": false,
878
- "normalized": false,
879
- "rstrip": false,
880
- "single_word": false,
881
- "special": true
882
- },
883
- "128110": {
884
- "content": "<|reserved_special_token_102|>",
885
- "lstrip": false,
886
- "normalized": false,
887
- "rstrip": false,
888
- "single_word": false,
889
- "special": true
890
- },
891
- "128111": {
892
- "content": "<|reserved_special_token_103|>",
893
- "lstrip": false,
894
- "normalized": false,
895
- "rstrip": false,
896
- "single_word": false,
897
- "special": true
898
- },
899
- "128112": {
900
- "content": "<|reserved_special_token_104|>",
901
- "lstrip": false,
902
- "normalized": false,
903
- "rstrip": false,
904
- "single_word": false,
905
- "special": true
906
- },
907
- "128113": {
908
- "content": "<|reserved_special_token_105|>",
909
- "lstrip": false,
910
- "normalized": false,
911
- "rstrip": false,
912
- "single_word": false,
913
- "special": true
914
- },
915
- "128114": {
916
- "content": "<|reserved_special_token_106|>",
917
- "lstrip": false,
918
- "normalized": false,
919
- "rstrip": false,
920
- "single_word": false,
921
- "special": true
922
- },
923
- "128115": {
924
- "content": "<|reserved_special_token_107|>",
925
- "lstrip": false,
926
- "normalized": false,
927
- "rstrip": false,
928
- "single_word": false,
929
- "special": true
930
- },
931
- "128116": {
932
- "content": "<|reserved_special_token_108|>",
933
- "lstrip": false,
934
- "normalized": false,
935
- "rstrip": false,
936
- "single_word": false,
937
- "special": true
938
- },
939
- "128117": {
940
- "content": "<|reserved_special_token_109|>",
941
- "lstrip": false,
942
- "normalized": false,
943
- "rstrip": false,
944
- "single_word": false,
945
- "special": true
946
- },
947
- "128118": {
948
- "content": "<|reserved_special_token_110|>",
949
- "lstrip": false,
950
- "normalized": false,
951
- "rstrip": false,
952
- "single_word": false,
953
- "special": true
954
- },
955
- "128119": {
956
- "content": "<|reserved_special_token_111|>",
957
- "lstrip": false,
958
- "normalized": false,
959
- "rstrip": false,
960
- "single_word": false,
961
- "special": true
962
- },
963
- "128120": {
964
- "content": "<|reserved_special_token_112|>",
965
- "lstrip": false,
966
- "normalized": false,
967
- "rstrip": false,
968
- "single_word": false,
969
- "special": true
970
- },
971
- "128121": {
972
- "content": "<|reserved_special_token_113|>",
973
- "lstrip": false,
974
- "normalized": false,
975
- "rstrip": false,
976
- "single_word": false,
977
- "special": true
978
- },
979
- "128122": {
980
- "content": "<|reserved_special_token_114|>",
981
- "lstrip": false,
982
- "normalized": false,
983
- "rstrip": false,
984
- "single_word": false,
985
- "special": true
986
- },
987
- "128123": {
988
- "content": "<|reserved_special_token_115|>",
989
- "lstrip": false,
990
- "normalized": false,
991
- "rstrip": false,
992
- "single_word": false,
993
- "special": true
994
- },
995
- "128124": {
996
- "content": "<|reserved_special_token_116|>",
997
- "lstrip": false,
998
- "normalized": false,
999
- "rstrip": false,
1000
- "single_word": false,
1001
- "special": true
1002
- },
1003
- "128125": {
1004
- "content": "<|reserved_special_token_117|>",
1005
- "lstrip": false,
1006
- "normalized": false,
1007
- "rstrip": false,
1008
- "single_word": false,
1009
- "special": true
1010
- },
1011
- "128126": {
1012
- "content": "<|reserved_special_token_118|>",
1013
- "lstrip": false,
1014
- "normalized": false,
1015
- "rstrip": false,
1016
- "single_word": false,
1017
- "special": true
1018
- },
1019
- "128127": {
1020
- "content": "<|reserved_special_token_119|>",
1021
- "lstrip": false,
1022
- "normalized": false,
1023
- "rstrip": false,
1024
- "single_word": false,
1025
- "special": true
1026
- },
1027
- "128128": {
1028
- "content": "<|reserved_special_token_120|>",
1029
- "lstrip": false,
1030
- "normalized": false,
1031
- "rstrip": false,
1032
- "single_word": false,
1033
- "special": true
1034
- },
1035
- "128129": {
1036
- "content": "<|reserved_special_token_121|>",
1037
- "lstrip": false,
1038
- "normalized": false,
1039
- "rstrip": false,
1040
- "single_word": false,
1041
- "special": true
1042
- },
1043
- "128130": {
1044
- "content": "<|reserved_special_token_122|>",
1045
- "lstrip": false,
1046
- "normalized": false,
1047
- "rstrip": false,
1048
- "single_word": false,
1049
- "special": true
1050
- },
1051
- "128131": {
1052
- "content": "<|reserved_special_token_123|>",
1053
- "lstrip": false,
1054
- "normalized": false,
1055
- "rstrip": false,
1056
- "single_word": false,
1057
- "special": true
1058
- },
1059
- "128132": {
1060
- "content": "<|reserved_special_token_124|>",
1061
- "lstrip": false,
1062
- "normalized": false,
1063
- "rstrip": false,
1064
- "single_word": false,
1065
- "special": true
1066
- },
1067
- "128133": {
1068
- "content": "<|reserved_special_token_125|>",
1069
- "lstrip": false,
1070
- "normalized": false,
1071
- "rstrip": false,
1072
- "single_word": false,
1073
- "special": true
1074
- },
1075
- "128134": {
1076
- "content": "<|reserved_special_token_126|>",
1077
- "lstrip": false,
1078
- "normalized": false,
1079
- "rstrip": false,
1080
- "single_word": false,
1081
- "special": true
1082
- },
1083
- "128135": {
1084
- "content": "<|reserved_special_token_127|>",
1085
- "lstrip": false,
1086
- "normalized": false,
1087
- "rstrip": false,
1088
- "single_word": false,
1089
- "special": true
1090
- },
1091
- "128136": {
1092
- "content": "<|reserved_special_token_128|>",
1093
- "lstrip": false,
1094
- "normalized": false,
1095
- "rstrip": false,
1096
- "single_word": false,
1097
- "special": true
1098
- },
1099
- "128137": {
1100
- "content": "<|reserved_special_token_129|>",
1101
- "lstrip": false,
1102
- "normalized": false,
1103
- "rstrip": false,
1104
- "single_word": false,
1105
- "special": true
1106
- },
1107
- "128138": {
1108
- "content": "<|reserved_special_token_130|>",
1109
- "lstrip": false,
1110
- "normalized": false,
1111
- "rstrip": false,
1112
- "single_word": false,
1113
- "special": true
1114
- },
1115
- "128139": {
1116
- "content": "<|reserved_special_token_131|>",
1117
- "lstrip": false,
1118
- "normalized": false,
1119
- "rstrip": false,
1120
- "single_word": false,
1121
- "special": true
1122
- },
1123
- "128140": {
1124
- "content": "<|reserved_special_token_132|>",
1125
- "lstrip": false,
1126
- "normalized": false,
1127
- "rstrip": false,
1128
- "single_word": false,
1129
- "special": true
1130
- },
1131
- "128141": {
1132
- "content": "<|reserved_special_token_133|>",
1133
- "lstrip": false,
1134
- "normalized": false,
1135
- "rstrip": false,
1136
- "single_word": false,
1137
- "special": true
1138
- },
1139
- "128142": {
1140
- "content": "<|reserved_special_token_134|>",
1141
- "lstrip": false,
1142
- "normalized": false,
1143
- "rstrip": false,
1144
- "single_word": false,
1145
- "special": true
1146
- },
1147
- "128143": {
1148
- "content": "<|reserved_special_token_135|>",
1149
- "lstrip": false,
1150
- "normalized": false,
1151
- "rstrip": false,
1152
- "single_word": false,
1153
- "special": true
1154
- },
1155
- "128144": {
1156
- "content": "<|reserved_special_token_136|>",
1157
- "lstrip": false,
1158
- "normalized": false,
1159
- "rstrip": false,
1160
- "single_word": false,
1161
- "special": true
1162
- },
1163
- "128145": {
1164
- "content": "<|reserved_special_token_137|>",
1165
- "lstrip": false,
1166
- "normalized": false,
1167
- "rstrip": false,
1168
- "single_word": false,
1169
- "special": true
1170
- },
1171
- "128146": {
1172
- "content": "<|reserved_special_token_138|>",
1173
- "lstrip": false,
1174
- "normalized": false,
1175
- "rstrip": false,
1176
- "single_word": false,
1177
- "special": true
1178
- },
1179
- "128147": {
1180
- "content": "<|reserved_special_token_139|>",
1181
- "lstrip": false,
1182
- "normalized": false,
1183
- "rstrip": false,
1184
- "single_word": false,
1185
- "special": true
1186
- },
1187
- "128148": {
1188
- "content": "<|reserved_special_token_140|>",
1189
- "lstrip": false,
1190
- "normalized": false,
1191
- "rstrip": false,
1192
- "single_word": false,
1193
- "special": true
1194
- },
1195
- "128149": {
1196
- "content": "<|reserved_special_token_141|>",
1197
- "lstrip": false,
1198
- "normalized": false,
1199
- "rstrip": false,
1200
- "single_word": false,
1201
- "special": true
1202
- },
1203
- "128150": {
1204
- "content": "<|reserved_special_token_142|>",
1205
- "lstrip": false,
1206
- "normalized": false,
1207
- "rstrip": false,
1208
- "single_word": false,
1209
- "special": true
1210
- },
1211
- "128151": {
1212
- "content": "<|reserved_special_token_143|>",
1213
- "lstrip": false,
1214
- "normalized": false,
1215
- "rstrip": false,
1216
- "single_word": false,
1217
- "special": true
1218
- },
1219
- "128152": {
1220
- "content": "<|reserved_special_token_144|>",
1221
- "lstrip": false,
1222
- "normalized": false,
1223
- "rstrip": false,
1224
- "single_word": false,
1225
- "special": true
1226
- },
1227
- "128153": {
1228
- "content": "<|reserved_special_token_145|>",
1229
- "lstrip": false,
1230
- "normalized": false,
1231
- "rstrip": false,
1232
- "single_word": false,
1233
- "special": true
1234
- },
1235
- "128154": {
1236
- "content": "<|reserved_special_token_146|>",
1237
- "lstrip": false,
1238
- "normalized": false,
1239
- "rstrip": false,
1240
- "single_word": false,
1241
- "special": true
1242
- },
1243
- "128155": {
1244
- "content": "<|reserved_special_token_147|>",
1245
- "lstrip": false,
1246
- "normalized": false,
1247
- "rstrip": false,
1248
- "single_word": false,
1249
- "special": true
1250
- },
1251
- "128156": {
1252
- "content": "<|reserved_special_token_148|>",
1253
- "lstrip": false,
1254
- "normalized": false,
1255
- "rstrip": false,
1256
- "single_word": false,
1257
- "special": true
1258
- },
1259
- "128157": {
1260
- "content": "<|reserved_special_token_149|>",
1261
- "lstrip": false,
1262
- "normalized": false,
1263
- "rstrip": false,
1264
- "single_word": false,
1265
- "special": true
1266
- },
1267
- "128158": {
1268
- "content": "<|reserved_special_token_150|>",
1269
- "lstrip": false,
1270
- "normalized": false,
1271
- "rstrip": false,
1272
- "single_word": false,
1273
- "special": true
1274
- },
1275
- "128159": {
1276
- "content": "<|reserved_special_token_151|>",
1277
- "lstrip": false,
1278
- "normalized": false,
1279
- "rstrip": false,
1280
- "single_word": false,
1281
- "special": true
1282
- },
1283
- "128160": {
1284
- "content": "<|reserved_special_token_152|>",
1285
- "lstrip": false,
1286
- "normalized": false,
1287
- "rstrip": false,
1288
- "single_word": false,
1289
- "special": true
1290
- },
1291
- "128161": {
1292
- "content": "<|reserved_special_token_153|>",
1293
- "lstrip": false,
1294
- "normalized": false,
1295
- "rstrip": false,
1296
- "single_word": false,
1297
- "special": true
1298
- },
1299
- "128162": {
1300
- "content": "<|reserved_special_token_154|>",
1301
- "lstrip": false,
1302
- "normalized": false,
1303
- "rstrip": false,
1304
- "single_word": false,
1305
- "special": true
1306
- },
1307
- "128163": {
1308
- "content": "<|reserved_special_token_155|>",
1309
- "lstrip": false,
1310
- "normalized": false,
1311
- "rstrip": false,
1312
- "single_word": false,
1313
- "special": true
1314
- },
1315
- "128164": {
1316
- "content": "<|reserved_special_token_156|>",
1317
- "lstrip": false,
1318
- "normalized": false,
1319
- "rstrip": false,
1320
- "single_word": false,
1321
- "special": true
1322
- },
1323
- "128165": {
1324
- "content": "<|reserved_special_token_157|>",
1325
- "lstrip": false,
1326
- "normalized": false,
1327
- "rstrip": false,
1328
- "single_word": false,
1329
- "special": true
1330
- },
1331
- "128166": {
1332
- "content": "<|reserved_special_token_158|>",
1333
- "lstrip": false,
1334
- "normalized": false,
1335
- "rstrip": false,
1336
- "single_word": false,
1337
- "special": true
1338
- },
1339
- "128167": {
1340
- "content": "<|reserved_special_token_159|>",
1341
- "lstrip": false,
1342
- "normalized": false,
1343
- "rstrip": false,
1344
- "single_word": false,
1345
- "special": true
1346
- },
1347
- "128168": {
1348
- "content": "<|reserved_special_token_160|>",
1349
- "lstrip": false,
1350
- "normalized": false,
1351
- "rstrip": false,
1352
- "single_word": false,
1353
- "special": true
1354
- },
1355
- "128169": {
1356
- "content": "<|reserved_special_token_161|>",
1357
- "lstrip": false,
1358
- "normalized": false,
1359
- "rstrip": false,
1360
- "single_word": false,
1361
- "special": true
1362
- },
1363
- "128170": {
1364
- "content": "<|reserved_special_token_162|>",
1365
- "lstrip": false,
1366
- "normalized": false,
1367
- "rstrip": false,
1368
- "single_word": false,
1369
- "special": true
1370
- },
1371
- "128171": {
1372
- "content": "<|reserved_special_token_163|>",
1373
- "lstrip": false,
1374
- "normalized": false,
1375
- "rstrip": false,
1376
- "single_word": false,
1377
- "special": true
1378
- },
1379
- "128172": {
1380
- "content": "<|reserved_special_token_164|>",
1381
- "lstrip": false,
1382
- "normalized": false,
1383
- "rstrip": false,
1384
- "single_word": false,
1385
- "special": true
1386
- },
1387
- "128173": {
1388
- "content": "<|reserved_special_token_165|>",
1389
- "lstrip": false,
1390
- "normalized": false,
1391
- "rstrip": false,
1392
- "single_word": false,
1393
- "special": true
1394
- },
1395
- "128174": {
1396
- "content": "<|reserved_special_token_166|>",
1397
- "lstrip": false,
1398
- "normalized": false,
1399
- "rstrip": false,
1400
- "single_word": false,
1401
- "special": true
1402
- },
1403
- "128175": {
1404
- "content": "<|reserved_special_token_167|>",
1405
- "lstrip": false,
1406
- "normalized": false,
1407
- "rstrip": false,
1408
- "single_word": false,
1409
- "special": true
1410
- },
1411
- "128176": {
1412
- "content": "<|reserved_special_token_168|>",
1413
- "lstrip": false,
1414
- "normalized": false,
1415
- "rstrip": false,
1416
- "single_word": false,
1417
- "special": true
1418
- },
1419
- "128177": {
1420
- "content": "<|reserved_special_token_169|>",
1421
- "lstrip": false,
1422
- "normalized": false,
1423
- "rstrip": false,
1424
- "single_word": false,
1425
- "special": true
1426
- },
1427
- "128178": {
1428
- "content": "<|reserved_special_token_170|>",
1429
- "lstrip": false,
1430
- "normalized": false,
1431
- "rstrip": false,
1432
- "single_word": false,
1433
- "special": true
1434
- },
1435
- "128179": {
1436
- "content": "<|reserved_special_token_171|>",
1437
- "lstrip": false,
1438
- "normalized": false,
1439
- "rstrip": false,
1440
- "single_word": false,
1441
- "special": true
1442
- },
1443
- "128180": {
1444
- "content": "<|reserved_special_token_172|>",
1445
- "lstrip": false,
1446
- "normalized": false,
1447
- "rstrip": false,
1448
- "single_word": false,
1449
- "special": true
1450
- },
1451
- "128181": {
1452
- "content": "<|reserved_special_token_173|>",
1453
- "lstrip": false,
1454
- "normalized": false,
1455
- "rstrip": false,
1456
- "single_word": false,
1457
- "special": true
1458
- },
1459
- "128182": {
1460
- "content": "<|reserved_special_token_174|>",
1461
- "lstrip": false,
1462
- "normalized": false,
1463
- "rstrip": false,
1464
- "single_word": false,
1465
- "special": true
1466
- },
1467
- "128183": {
1468
- "content": "<|reserved_special_token_175|>",
1469
- "lstrip": false,
1470
- "normalized": false,
1471
- "rstrip": false,
1472
- "single_word": false,
1473
- "special": true
1474
- },
1475
- "128184": {
1476
- "content": "<|reserved_special_token_176|>",
1477
- "lstrip": false,
1478
- "normalized": false,
1479
- "rstrip": false,
1480
- "single_word": false,
1481
- "special": true
1482
- },
1483
- "128185": {
1484
- "content": "<|reserved_special_token_177|>",
1485
- "lstrip": false,
1486
- "normalized": false,
1487
- "rstrip": false,
1488
- "single_word": false,
1489
- "special": true
1490
- },
1491
- "128186": {
1492
- "content": "<|reserved_special_token_178|>",
1493
- "lstrip": false,
1494
- "normalized": false,
1495
- "rstrip": false,
1496
- "single_word": false,
1497
- "special": true
1498
- },
1499
- "128187": {
1500
- "content": "<|reserved_special_token_179|>",
1501
- "lstrip": false,
1502
- "normalized": false,
1503
- "rstrip": false,
1504
- "single_word": false,
1505
- "special": true
1506
- },
1507
- "128188": {
1508
- "content": "<|reserved_special_token_180|>",
1509
- "lstrip": false,
1510
- "normalized": false,
1511
- "rstrip": false,
1512
- "single_word": false,
1513
- "special": true
1514
- },
1515
- "128189": {
1516
- "content": "<|reserved_special_token_181|>",
1517
- "lstrip": false,
1518
- "normalized": false,
1519
- "rstrip": false,
1520
- "single_word": false,
1521
- "special": true
1522
- },
1523
- "128190": {
1524
- "content": "<|reserved_special_token_182|>",
1525
- "lstrip": false,
1526
- "normalized": false,
1527
- "rstrip": false,
1528
- "single_word": false,
1529
- "special": true
1530
- },
1531
- "128191": {
1532
- "content": "<|reserved_special_token_183|>",
1533
- "lstrip": false,
1534
- "normalized": false,
1535
- "rstrip": false,
1536
- "single_word": false,
1537
- "special": true
1538
- },
1539
- "128192": {
1540
- "content": "<|reserved_special_token_184|>",
1541
- "lstrip": false,
1542
- "normalized": false,
1543
- "rstrip": false,
1544
- "single_word": false,
1545
- "special": true
1546
- },
1547
- "128193": {
1548
- "content": "<|reserved_special_token_185|>",
1549
- "lstrip": false,
1550
- "normalized": false,
1551
- "rstrip": false,
1552
- "single_word": false,
1553
- "special": true
1554
- },
1555
- "128194": {
1556
- "content": "<|reserved_special_token_186|>",
1557
- "lstrip": false,
1558
- "normalized": false,
1559
- "rstrip": false,
1560
- "single_word": false,
1561
- "special": true
1562
- },
1563
- "128195": {
1564
- "content": "<|reserved_special_token_187|>",
1565
- "lstrip": false,
1566
- "normalized": false,
1567
- "rstrip": false,
1568
- "single_word": false,
1569
- "special": true
1570
- },
1571
- "128196": {
1572
- "content": "<|reserved_special_token_188|>",
1573
- "lstrip": false,
1574
- "normalized": false,
1575
- "rstrip": false,
1576
- "single_word": false,
1577
- "special": true
1578
- },
1579
- "128197": {
1580
- "content": "<|reserved_special_token_189|>",
1581
- "lstrip": false,
1582
- "normalized": false,
1583
- "rstrip": false,
1584
- "single_word": false,
1585
- "special": true
1586
- },
1587
- "128198": {
1588
- "content": "<|reserved_special_token_190|>",
1589
- "lstrip": false,
1590
- "normalized": false,
1591
- "rstrip": false,
1592
- "single_word": false,
1593
- "special": true
1594
- },
1595
- "128199": {
1596
- "content": "<|reserved_special_token_191|>",
1597
- "lstrip": false,
1598
- "normalized": false,
1599
- "rstrip": false,
1600
- "single_word": false,
1601
- "special": true
1602
- },
1603
- "128200": {
1604
- "content": "<|reserved_special_token_192|>",
1605
- "lstrip": false,
1606
- "normalized": false,
1607
- "rstrip": false,
1608
- "single_word": false,
1609
- "special": true
1610
- },
1611
- "128201": {
1612
- "content": "<|reserved_special_token_193|>",
1613
- "lstrip": false,
1614
- "normalized": false,
1615
- "rstrip": false,
1616
- "single_word": false,
1617
- "special": true
1618
- },
1619
- "128202": {
1620
- "content": "<|reserved_special_token_194|>",
1621
- "lstrip": false,
1622
- "normalized": false,
1623
- "rstrip": false,
1624
- "single_word": false,
1625
- "special": true
1626
- },
1627
- "128203": {
1628
- "content": "<|reserved_special_token_195|>",
1629
- "lstrip": false,
1630
- "normalized": false,
1631
- "rstrip": false,
1632
- "single_word": false,
1633
- "special": true
1634
- },
1635
- "128204": {
1636
- "content": "<|reserved_special_token_196|>",
1637
- "lstrip": false,
1638
- "normalized": false,
1639
- "rstrip": false,
1640
- "single_word": false,
1641
- "special": true
1642
- },
1643
- "128205": {
1644
- "content": "<|reserved_special_token_197|>",
1645
- "lstrip": false,
1646
- "normalized": false,
1647
- "rstrip": false,
1648
- "single_word": false,
1649
- "special": true
1650
- },
1651
- "128206": {
1652
- "content": "<|reserved_special_token_198|>",
1653
- "lstrip": false,
1654
- "normalized": false,
1655
- "rstrip": false,
1656
- "single_word": false,
1657
- "special": true
1658
- },
1659
- "128207": {
1660
- "content": "<|reserved_special_token_199|>",
1661
- "lstrip": false,
1662
- "normalized": false,
1663
- "rstrip": false,
1664
- "single_word": false,
1665
- "special": true
1666
- },
1667
- "128208": {
1668
- "content": "<|reserved_special_token_200|>",
1669
- "lstrip": false,
1670
- "normalized": false,
1671
- "rstrip": false,
1672
- "single_word": false,
1673
- "special": true
1674
- },
1675
- "128209": {
1676
- "content": "<|reserved_special_token_201|>",
1677
- "lstrip": false,
1678
- "normalized": false,
1679
- "rstrip": false,
1680
- "single_word": false,
1681
- "special": true
1682
- },
1683
- "128210": {
1684
- "content": "<|reserved_special_token_202|>",
1685
- "lstrip": false,
1686
- "normalized": false,
1687
- "rstrip": false,
1688
- "single_word": false,
1689
- "special": true
1690
- },
1691
- "128211": {
1692
- "content": "<|reserved_special_token_203|>",
1693
- "lstrip": false,
1694
- "normalized": false,
1695
- "rstrip": false,
1696
- "single_word": false,
1697
- "special": true
1698
- },
1699
- "128212": {
1700
- "content": "<|reserved_special_token_204|>",
1701
- "lstrip": false,
1702
- "normalized": false,
1703
- "rstrip": false,
1704
- "single_word": false,
1705
- "special": true
1706
- },
1707
- "128213": {
1708
- "content": "<|reserved_special_token_205|>",
1709
- "lstrip": false,
1710
- "normalized": false,
1711
- "rstrip": false,
1712
- "single_word": false,
1713
- "special": true
1714
- },
1715
- "128214": {
1716
- "content": "<|reserved_special_token_206|>",
1717
- "lstrip": false,
1718
- "normalized": false,
1719
- "rstrip": false,
1720
- "single_word": false,
1721
- "special": true
1722
- },
1723
- "128215": {
1724
- "content": "<|reserved_special_token_207|>",
1725
- "lstrip": false,
1726
- "normalized": false,
1727
- "rstrip": false,
1728
- "single_word": false,
1729
- "special": true
1730
- },
1731
- "128216": {
1732
- "content": "<|reserved_special_token_208|>",
1733
- "lstrip": false,
1734
- "normalized": false,
1735
- "rstrip": false,
1736
- "single_word": false,
1737
- "special": true
1738
- },
1739
- "128217": {
1740
- "content": "<|reserved_special_token_209|>",
1741
- "lstrip": false,
1742
- "normalized": false,
1743
- "rstrip": false,
1744
- "single_word": false,
1745
- "special": true
1746
- },
1747
- "128218": {
1748
- "content": "<|reserved_special_token_210|>",
1749
- "lstrip": false,
1750
- "normalized": false,
1751
- "rstrip": false,
1752
- "single_word": false,
1753
- "special": true
1754
- },
1755
- "128219": {
1756
- "content": "<|reserved_special_token_211|>",
1757
- "lstrip": false,
1758
- "normalized": false,
1759
- "rstrip": false,
1760
- "single_word": false,
1761
- "special": true
1762
- },
1763
- "128220": {
1764
- "content": "<|reserved_special_token_212|>",
1765
- "lstrip": false,
1766
- "normalized": false,
1767
- "rstrip": false,
1768
- "single_word": false,
1769
- "special": true
1770
- },
1771
- "128221": {
1772
- "content": "<|reserved_special_token_213|>",
1773
- "lstrip": false,
1774
- "normalized": false,
1775
- "rstrip": false,
1776
- "single_word": false,
1777
- "special": true
1778
- },
1779
- "128222": {
1780
- "content": "<|reserved_special_token_214|>",
1781
- "lstrip": false,
1782
- "normalized": false,
1783
- "rstrip": false,
1784
- "single_word": false,
1785
- "special": true
1786
- },
1787
- "128223": {
1788
- "content": "<|reserved_special_token_215|>",
1789
- "lstrip": false,
1790
- "normalized": false,
1791
- "rstrip": false,
1792
- "single_word": false,
1793
- "special": true
1794
- },
1795
- "128224": {
1796
- "content": "<|reserved_special_token_216|>",
1797
- "lstrip": false,
1798
- "normalized": false,
1799
- "rstrip": false,
1800
- "single_word": false,
1801
- "special": true
1802
- },
1803
- "128225": {
1804
- "content": "<|reserved_special_token_217|>",
1805
- "lstrip": false,
1806
- "normalized": false,
1807
- "rstrip": false,
1808
- "single_word": false,
1809
- "special": true
1810
- },
1811
- "128226": {
1812
- "content": "<|reserved_special_token_218|>",
1813
- "lstrip": false,
1814
- "normalized": false,
1815
- "rstrip": false,
1816
- "single_word": false,
1817
- "special": true
1818
- },
1819
- "128227": {
1820
- "content": "<|reserved_special_token_219|>",
1821
- "lstrip": false,
1822
- "normalized": false,
1823
- "rstrip": false,
1824
- "single_word": false,
1825
- "special": true
1826
- },
1827
- "128228": {
1828
- "content": "<|reserved_special_token_220|>",
1829
- "lstrip": false,
1830
- "normalized": false,
1831
- "rstrip": false,
1832
- "single_word": false,
1833
- "special": true
1834
- },
1835
- "128229": {
1836
- "content": "<|reserved_special_token_221|>",
1837
- "lstrip": false,
1838
- "normalized": false,
1839
- "rstrip": false,
1840
- "single_word": false,
1841
- "special": true
1842
- },
1843
- "128230": {
1844
- "content": "<|reserved_special_token_222|>",
1845
- "lstrip": false,
1846
- "normalized": false,
1847
- "rstrip": false,
1848
- "single_word": false,
1849
- "special": true
1850
- },
1851
- "128231": {
1852
- "content": "<|reserved_special_token_223|>",
1853
- "lstrip": false,
1854
- "normalized": false,
1855
- "rstrip": false,
1856
- "single_word": false,
1857
- "special": true
1858
- },
1859
- "128232": {
1860
- "content": "<|reserved_special_token_224|>",
1861
- "lstrip": false,
1862
- "normalized": false,
1863
- "rstrip": false,
1864
- "single_word": false,
1865
- "special": true
1866
- },
1867
- "128233": {
1868
- "content": "<|reserved_special_token_225|>",
1869
- "lstrip": false,
1870
- "normalized": false,
1871
- "rstrip": false,
1872
- "single_word": false,
1873
- "special": true
1874
- },
1875
- "128234": {
1876
- "content": "<|reserved_special_token_226|>",
1877
- "lstrip": false,
1878
- "normalized": false,
1879
- "rstrip": false,
1880
- "single_word": false,
1881
- "special": true
1882
- },
1883
- "128235": {
1884
- "content": "<|reserved_special_token_227|>",
1885
- "lstrip": false,
1886
- "normalized": false,
1887
- "rstrip": false,
1888
- "single_word": false,
1889
- "special": true
1890
- },
1891
- "128236": {
1892
- "content": "<|reserved_special_token_228|>",
1893
- "lstrip": false,
1894
- "normalized": false,
1895
- "rstrip": false,
1896
- "single_word": false,
1897
- "special": true
1898
- },
1899
- "128237": {
1900
- "content": "<|reserved_special_token_229|>",
1901
- "lstrip": false,
1902
- "normalized": false,
1903
- "rstrip": false,
1904
- "single_word": false,
1905
- "special": true
1906
- },
1907
- "128238": {
1908
- "content": "<|reserved_special_token_230|>",
1909
- "lstrip": false,
1910
- "normalized": false,
1911
- "rstrip": false,
1912
- "single_word": false,
1913
- "special": true
1914
- },
1915
- "128239": {
1916
- "content": "<|reserved_special_token_231|>",
1917
- "lstrip": false,
1918
- "normalized": false,
1919
- "rstrip": false,
1920
- "single_word": false,
1921
- "special": true
1922
- },
1923
- "128240": {
1924
- "content": "<|reserved_special_token_232|>",
1925
- "lstrip": false,
1926
- "normalized": false,
1927
- "rstrip": false,
1928
- "single_word": false,
1929
- "special": true
1930
- },
1931
- "128241": {
1932
- "content": "<|reserved_special_token_233|>",
1933
- "lstrip": false,
1934
- "normalized": false,
1935
- "rstrip": false,
1936
- "single_word": false,
1937
- "special": true
1938
- },
1939
- "128242": {
1940
- "content": "<|reserved_special_token_234|>",
1941
- "lstrip": false,
1942
- "normalized": false,
1943
- "rstrip": false,
1944
- "single_word": false,
1945
- "special": true
1946
- },
1947
- "128243": {
1948
- "content": "<|reserved_special_token_235|>",
1949
- "lstrip": false,
1950
- "normalized": false,
1951
- "rstrip": false,
1952
- "single_word": false,
1953
- "special": true
1954
- },
1955
- "128244": {
1956
- "content": "<|reserved_special_token_236|>",
1957
- "lstrip": false,
1958
- "normalized": false,
1959
- "rstrip": false,
1960
- "single_word": false,
1961
- "special": true
1962
- },
1963
- "128245": {
1964
- "content": "<|reserved_special_token_237|>",
1965
- "lstrip": false,
1966
- "normalized": false,
1967
- "rstrip": false,
1968
- "single_word": false,
1969
- "special": true
1970
- },
1971
- "128246": {
1972
- "content": "<|reserved_special_token_238|>",
1973
- "lstrip": false,
1974
- "normalized": false,
1975
- "rstrip": false,
1976
- "single_word": false,
1977
- "special": true
1978
- },
1979
- "128247": {
1980
- "content": "<|reserved_special_token_239|>",
1981
- "lstrip": false,
1982
- "normalized": false,
1983
- "rstrip": false,
1984
- "single_word": false,
1985
- "special": true
1986
- },
1987
- "128248": {
1988
- "content": "<|reserved_special_token_240|>",
1989
- "lstrip": false,
1990
- "normalized": false,
1991
- "rstrip": false,
1992
- "single_word": false,
1993
- "special": true
1994
- },
1995
- "128249": {
1996
- "content": "<|reserved_special_token_241|>",
1997
- "lstrip": false,
1998
- "normalized": false,
1999
- "rstrip": false,
2000
- "single_word": false,
2001
- "special": true
2002
- },
2003
- "128250": {
2004
- "content": "<|reserved_special_token_242|>",
2005
- "lstrip": false,
2006
- "normalized": false,
2007
- "rstrip": false,
2008
- "single_word": false,
2009
- "special": true
2010
- },
2011
- "128251": {
2012
- "content": "<|reserved_special_token_243|>",
2013
- "lstrip": false,
2014
- "normalized": false,
2015
- "rstrip": false,
2016
- "single_word": false,
2017
- "special": true
2018
- },
2019
- "128252": {
2020
- "content": "<|reserved_special_token_244|>",
2021
- "lstrip": false,
2022
- "normalized": false,
2023
- "rstrip": false,
2024
- "single_word": false,
2025
- "special": true
2026
- },
2027
- "128253": {
2028
- "content": "<|reserved_special_token_245|>",
2029
- "lstrip": false,
2030
- "normalized": false,
2031
- "rstrip": false,
2032
- "single_word": false,
2033
- "special": true
2034
- },
2035
- "128254": {
2036
- "content": "<|reserved_special_token_246|>",
2037
- "lstrip": false,
2038
- "normalized": false,
2039
- "rstrip": false,
2040
- "single_word": false,
2041
- "special": true
2042
- },
2043
- "128255": {
2044
- "content": "<|reserved_special_token_247|>",
2045
- "lstrip": false,
2046
- "normalized": false,
2047
- "rstrip": false,
2048
- "single_word": false,
2049
- "special": true
2050
- }
2051
- },
2052
- "bos_token": "<|begin_of_text|>",
2053
- "clean_up_tokenization_spaces": true,
2054
- "eos_token": "<|eot_id|>",
2055
- "extra_special_tokens": {},
2056
- "model_input_names": [
2057
- "input_ids",
2058
- "attention_mask"
2059
- ],
2060
- "model_max_length": 131072,
2061
- "pad_token": "<|eot_id|>",
2062
- "tokenizer_class": "PreTrainedTokenizer"
2063
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-98500/trainer_state.json DELETED
@@ -1,1756 +0,0 @@
1
- {
2
- "best_global_step": null,
3
- "best_metric": null,
4
- "best_model_checkpoint": null,
5
- "epoch": 0.22556098275889708,
6
- "eval_steps": 500,
7
- "global_step": 98500,
8
- "is_hyper_param_search": false,
9
- "is_local_process_zero": true,
10
- "is_world_process_zero": true,
11
- "log_history": [
12
- {
13
- "epoch": 0.0009159836863305464,
14
- "grad_norm": 3.5712358951568604,
15
- "learning_rate": 0.0001596,
16
- "loss": 1.5506,
17
- "step": 400
18
- },
19
- {
20
- "epoch": 0.0018319673726610928,
21
- "grad_norm": 2.6055777072906494,
22
- "learning_rate": 0.0001999999420964769,
23
- "loss": 1.4925,
24
- "step": 800
25
- },
26
- {
27
- "epoch": 0.0027479510589916395,
28
- "grad_norm": 2.598275899887085,
29
- "learning_rate": 0.00019999968354149158,
30
- "loss": 1.4543,
31
- "step": 1200
32
- },
33
- {
34
- "epoch": 0.0036639347453221857,
35
- "grad_norm": 3.6462411880493164,
36
- "learning_rate": 0.00019999921772856336,
37
- "loss": 1.4282,
38
- "step": 1600
39
- },
40
- {
41
- "epoch": 0.004579918431652732,
42
- "grad_norm": 3.534177780151367,
43
- "learning_rate": 0.00019999854465865773,
44
- "loss": 1.3997,
45
- "step": 2000
46
- },
47
- {
48
- "epoch": 0.005495902117983279,
49
- "grad_norm": 4.041103363037109,
50
- "learning_rate": 0.00019999766433316963,
51
- "loss": 1.4071,
52
- "step": 2400
53
- },
54
- {
55
- "epoch": 0.006411885804313826,
56
- "grad_norm": 2.65792179107666,
57
- "learning_rate": 0.0001999965767539236,
58
- "loss": 1.3747,
59
- "step": 2800
60
- },
61
- {
62
- "epoch": 0.007327869490644371,
63
- "grad_norm": 2.7183997631073,
64
- "learning_rate": 0.0001999952819231738,
65
- "loss": 1.3796,
66
- "step": 3200
67
- },
68
- {
69
- "epoch": 0.008243853176974918,
70
- "grad_norm": 4.0687174797058105,
71
- "learning_rate": 0.00019999377984360382,
72
- "loss": 1.387,
73
- "step": 3600
74
- },
75
- {
76
- "epoch": 0.009159836863305465,
77
- "grad_norm": 2.770643472671509,
78
- "learning_rate": 0.00019999207051832687,
79
- "loss": 1.3559,
80
- "step": 4000
81
- },
82
- {
83
- "epoch": 0.010075820549636011,
84
- "grad_norm": 2.8189332485198975,
85
- "learning_rate": 0.00019999015395088566,
86
- "loss": 1.3373,
87
- "step": 4400
88
- },
89
- {
90
- "epoch": 0.010991804235966558,
91
- "grad_norm": 3.2458949089050293,
92
- "learning_rate": 0.0001999880301452525,
93
- "loss": 1.3498,
94
- "step": 4800
95
- },
96
- {
97
- "epoch": 0.011907787922297105,
98
- "grad_norm": 2.921995162963867,
99
- "learning_rate": 0.0001999856991058291,
100
- "loss": 1.3519,
101
- "step": 5200
102
- },
103
- {
104
- "epoch": 0.012823771608627651,
105
- "grad_norm": 2.910311460494995,
106
- "learning_rate": 0.00019998316083744678,
107
- "loss": 1.3468,
108
- "step": 5600
109
- },
110
- {
111
- "epoch": 0.013739755294958196,
112
- "grad_norm": 2.994410753250122,
113
- "learning_rate": 0.00019998041534536625,
114
- "loss": 1.3274,
115
- "step": 6000
116
- },
117
- {
118
- "epoch": 0.014655738981288743,
119
- "grad_norm": 2.5082335472106934,
120
- "learning_rate": 0.00019997746263527785,
121
- "loss": 1.3324,
122
- "step": 6400
123
- },
124
- {
125
- "epoch": 0.01557172266761929,
126
- "grad_norm": 2.9452273845672607,
127
- "learning_rate": 0.0001999743027133013,
128
- "loss": 1.3379,
129
- "step": 6800
130
- },
131
- {
132
- "epoch": 0.016487706353949836,
133
- "grad_norm": 2.3184399604797363,
134
- "learning_rate": 0.0001999709355859858,
135
- "loss": 1.324,
136
- "step": 7200
137
- },
138
- {
139
- "epoch": 0.017403690040280383,
140
- "grad_norm": 2.8244330883026123,
141
- "learning_rate": 0.00019996736126031,
142
- "loss": 1.3279,
143
- "step": 7600
144
- },
145
- {
146
- "epoch": 0.01831967372661093,
147
- "grad_norm": 2.5758519172668457,
148
- "learning_rate": 0.00019996357974368204,
149
- "loss": 1.312,
150
- "step": 8000
151
- },
152
- {
153
- "epoch": 0.019235657412941476,
154
- "grad_norm": 2.689647674560547,
155
- "learning_rate": 0.00019995959104393945,
156
- "loss": 1.313,
157
- "step": 8400
158
- },
159
- {
160
- "epoch": 0.020151641099272023,
161
- "grad_norm": 2.9180686473846436,
162
- "learning_rate": 0.00019995539516934904,
163
- "loss": 1.313,
164
- "step": 8800
165
- },
166
- {
167
- "epoch": 0.02106762478560257,
168
- "grad_norm": 2.66694974899292,
169
- "learning_rate": 0.00019995099212860725,
170
- "loss": 1.3179,
171
- "step": 9200
172
- },
173
- {
174
- "epoch": 0.021983608471933116,
175
- "grad_norm": 2.296940803527832,
176
- "learning_rate": 0.0001999463819308397,
177
- "loss": 1.3188,
178
- "step": 9600
179
- },
180
- {
181
- "epoch": 0.022899592158263662,
182
- "grad_norm": 3.979454278945923,
183
- "learning_rate": 0.00019994156458560147,
184
- "loss": 1.3347,
185
- "step": 10000
186
- },
187
- {
188
- "epoch": 0.02381557584459421,
189
- "grad_norm": 2.359060764312744,
190
- "learning_rate": 0.00019993654010287687,
191
- "loss": 1.3452,
192
- "step": 10400
193
- },
194
- {
195
- "epoch": 0.024731559530924756,
196
- "grad_norm": 2.784097194671631,
197
- "learning_rate": 0.00019993130849307956,
198
- "loss": 1.3668,
199
- "step": 10800
200
- },
201
- {
202
- "epoch": 0.025647543217255302,
203
- "grad_norm": 2.0763189792633057,
204
- "learning_rate": 0.0001999258697670525,
205
- "loss": 1.3175,
206
- "step": 11200
207
- },
208
- {
209
- "epoch": 0.026563526903585846,
210
- "grad_norm": 2.903045415878296,
211
- "learning_rate": 0.000199920223936068,
212
- "loss": 1.3224,
213
- "step": 11600
214
- },
215
- {
216
- "epoch": 0.027479510589916392,
217
- "grad_norm": 2.770756244659424,
218
- "learning_rate": 0.00019991437101182747,
219
- "loss": 1.3096,
220
- "step": 12000
221
- },
222
- {
223
- "epoch": 0.02839549427624694,
224
- "grad_norm": 3.1364219188690186,
225
- "learning_rate": 0.0001999083110064616,
226
- "loss": 1.3298,
227
- "step": 12400
228
- },
229
- {
230
- "epoch": 0.029311477962577485,
231
- "grad_norm": 3.2954201698303223,
232
- "learning_rate": 0.00019990204393253026,
233
- "loss": 1.3198,
234
- "step": 12800
235
- },
236
- {
237
- "epoch": 0.030227461648908032,
238
- "grad_norm": 5.423952579498291,
239
- "learning_rate": 0.00019989556980302254,
240
- "loss": 1.3087,
241
- "step": 13200
242
- },
243
- {
244
- "epoch": 0.03114344533523858,
245
- "grad_norm": 2.3389499187469482,
246
- "learning_rate": 0.00019988888863135658,
247
- "loss": 1.3093,
248
- "step": 13600
249
- },
250
- {
251
- "epoch": 0.03205942902156913,
252
- "grad_norm": 2.3693130016326904,
253
- "learning_rate": 0.00019988200043137973,
254
- "loss": 1.306,
255
- "step": 14000
256
- },
257
- {
258
- "epoch": 0.03297541270789967,
259
- "grad_norm": 2.584962844848633,
260
- "learning_rate": 0.00019987490521736833,
261
- "loss": 1.295,
262
- "step": 14400
263
- },
264
- {
265
- "epoch": 0.03389139639423022,
266
- "grad_norm": 3.4960484504699707,
267
- "learning_rate": 0.0001998676030040279,
268
- "loss": 1.3255,
269
- "step": 14800
270
- },
271
- {
272
- "epoch": 0.034807380080560765,
273
- "grad_norm": 5.452722072601318,
274
- "learning_rate": 0.0001998600938064928,
275
- "loss": 1.2998,
276
- "step": 15200
277
- },
278
- {
279
- "epoch": 0.03572336376689131,
280
- "grad_norm": 2.527163028717041,
281
- "learning_rate": 0.00019985237764032657,
282
- "loss": 1.2932,
283
- "step": 15600
284
- },
285
- {
286
- "epoch": 0.03663934745322186,
287
- "grad_norm": 3.5106911659240723,
288
- "learning_rate": 0.0001998444545215216,
289
- "loss": 1.3189,
290
- "step": 16000
291
- },
292
- {
293
- "epoch": 0.0375553311395524,
294
- "grad_norm": 2.078429698944092,
295
- "learning_rate": 0.0001998363244664992,
296
- "loss": 1.3213,
297
- "step": 16400
298
- },
299
- {
300
- "epoch": 0.03847131482588295,
301
- "grad_norm": 3.225316047668457,
302
- "learning_rate": 0.00019982798749210965,
303
- "loss": 1.2817,
304
- "step": 16800
305
- },
306
- {
307
- "epoch": 0.039387298512213495,
308
- "grad_norm": 3.7959091663360596,
309
- "learning_rate": 0.00019981944361563203,
310
- "loss": 1.3254,
311
- "step": 17200
312
- },
313
- {
314
- "epoch": 0.040303282198544045,
315
- "grad_norm": 2.8166396617889404,
316
- "learning_rate": 0.00019981069285477429,
317
- "loss": 1.2932,
318
- "step": 17600
319
- },
320
- {
321
- "epoch": 0.04121926588487459,
322
- "grad_norm": 3.156550645828247,
323
- "learning_rate": 0.0001998017352276731,
324
- "loss": 1.33,
325
- "step": 18000
326
- },
327
- {
328
- "epoch": 0.04213524957120514,
329
- "grad_norm": 2.330085277557373,
330
- "learning_rate": 0.0001997925707528939,
331
- "loss": 1.3186,
332
- "step": 18400
333
- },
334
- {
335
- "epoch": 0.04305123325753568,
336
- "grad_norm": 3.9666330814361572,
337
- "learning_rate": 0.0001997831994494309,
338
- "loss": 1.314,
339
- "step": 18800
340
- },
341
- {
342
- "epoch": 0.04396721694386623,
343
- "grad_norm": 3.022580146789551,
344
- "learning_rate": 0.00019977362133670692,
345
- "loss": 1.3444,
346
- "step": 19200
347
- },
348
- {
349
- "epoch": 0.044883200630196775,
350
- "grad_norm": 4.415026664733887,
351
- "learning_rate": 0.0001997638364345734,
352
- "loss": 1.3019,
353
- "step": 19600
354
- },
355
- {
356
- "epoch": 0.045799184316527325,
357
- "grad_norm": 2.2941715717315674,
358
- "learning_rate": 0.00019975384476331035,
359
- "loss": 1.3267,
360
- "step": 20000
361
- },
362
- {
363
- "epoch": 0.04671516800285787,
364
- "grad_norm": 2.7769296169281006,
365
- "learning_rate": 0.0001997436463436265,
366
- "loss": 1.2882,
367
- "step": 20400
368
- },
369
- {
370
- "epoch": 0.04763115168918842,
371
- "grad_norm": 2.3512470722198486,
372
- "learning_rate": 0.00019973324119665882,
373
- "loss": 1.3305,
374
- "step": 20800
375
- },
376
- {
377
- "epoch": 0.04854713537551896,
378
- "grad_norm": 3.1230602264404297,
379
- "learning_rate": 0.00019972262934397293,
380
- "loss": 1.3018,
381
- "step": 21200
382
- },
383
- {
384
- "epoch": 0.04946311906184951,
385
- "grad_norm": 3.0036442279815674,
386
- "learning_rate": 0.00019971181080756279,
387
- "loss": 1.2998,
388
- "step": 21600
389
- },
390
- {
391
- "epoch": 0.050379102748180055,
392
- "grad_norm": 11.243963241577148,
393
- "learning_rate": 0.00019970078560985078,
394
- "loss": 1.3189,
395
- "step": 22000
396
- },
397
- {
398
- "epoch": 0.051295086434510605,
399
- "grad_norm": 2.546957015991211,
400
- "learning_rate": 0.0001996895537736875,
401
- "loss": 1.3066,
402
- "step": 22400
403
- },
404
- {
405
- "epoch": 0.05221107012084115,
406
- "grad_norm": 2.777229070663452,
407
- "learning_rate": 0.00019967811532235196,
408
- "loss": 1.2995,
409
- "step": 22800
410
- },
411
- {
412
- "epoch": 0.05312705380717169,
413
- "grad_norm": 2.9132065773010254,
414
- "learning_rate": 0.00019966647027955134,
415
- "loss": 1.3097,
416
- "step": 23200
417
- },
418
- {
419
- "epoch": 0.05404303749350224,
420
- "grad_norm": 3.6997904777526855,
421
- "learning_rate": 0.00019965461866942093,
422
- "loss": 1.2968,
423
- "step": 23600
424
- },
425
- {
426
- "epoch": 0.054959021179832784,
427
- "grad_norm": 3.026538133621216,
428
- "learning_rate": 0.0001996425605165243,
429
- "loss": 1.3492,
430
- "step": 24000
431
- },
432
- {
433
- "epoch": 0.055875004866163334,
434
- "grad_norm": 3.052229881286621,
435
- "learning_rate": 0.00019963029584585293,
436
- "loss": 1.2992,
437
- "step": 24400
438
- },
439
- {
440
- "epoch": 0.05679098855249388,
441
- "grad_norm": 2.065054178237915,
442
- "learning_rate": 0.00019961782468282645,
443
- "loss": 1.3266,
444
- "step": 24800
445
- },
446
- {
447
- "epoch": 0.05770697223882443,
448
- "grad_norm": 3.4963061809539795,
449
- "learning_rate": 0.0001996051470532924,
450
- "loss": 1.3034,
451
- "step": 25200
452
- },
453
- {
454
- "epoch": 0.05862295592515497,
455
- "grad_norm": 2.485971689224243,
456
- "learning_rate": 0.00019959226298352621,
457
- "loss": 1.3158,
458
- "step": 25600
459
- },
460
- {
461
- "epoch": 0.05953893961148552,
462
- "grad_norm": 2.431988000869751,
463
- "learning_rate": 0.0001995791725002313,
464
- "loss": 1.2833,
465
- "step": 26000
466
- },
467
- {
468
- "epoch": 0.060454923297816064,
469
- "grad_norm": 2.6740353107452393,
470
- "learning_rate": 0.00019956587563053883,
471
- "loss": 1.2865,
472
- "step": 26400
473
- },
474
- {
475
- "epoch": 0.061370906984146614,
476
- "grad_norm": 4.834751129150391,
477
- "learning_rate": 0.00019955237240200762,
478
- "loss": 1.2959,
479
- "step": 26800
480
- },
481
- {
482
- "epoch": 0.06228689067047716,
483
- "grad_norm": 5.183393955230713,
484
- "learning_rate": 0.00019953866284262432,
485
- "loss": 1.3262,
486
- "step": 27200
487
- },
488
- {
489
- "epoch": 0.0632028743568077,
490
- "grad_norm": 5.2344489097595215,
491
- "learning_rate": 0.00019952474698080322,
492
- "loss": 1.3133,
493
- "step": 27600
494
- },
495
- {
496
- "epoch": 0.06411885804313826,
497
- "grad_norm": 3.3201353549957275,
498
- "learning_rate": 0.00019951062484538604,
499
- "loss": 1.3156,
500
- "step": 28000
501
- },
502
- {
503
- "epoch": 0.0650348417294688,
504
- "grad_norm": 3.141674041748047,
505
- "learning_rate": 0.00019949629646564217,
506
- "loss": 1.3074,
507
- "step": 28400
508
- },
509
- {
510
- "epoch": 0.06595082541579934,
511
- "grad_norm": 3.8046987056732178,
512
- "learning_rate": 0.00019948176187126846,
513
- "loss": 1.273,
514
- "step": 28800
515
- },
516
- {
517
- "epoch": 0.06686680910212989,
518
- "grad_norm": 2.2140769958496094,
519
- "learning_rate": 0.000199467021092389,
520
- "loss": 1.307,
521
- "step": 29200
522
- },
523
- {
524
- "epoch": 0.06778279278846044,
525
- "grad_norm": 2.583613872528076,
526
- "learning_rate": 0.0001994520741595554,
527
- "loss": 1.2793,
528
- "step": 29600
529
- },
530
- {
531
- "epoch": 0.06869877647479099,
532
- "grad_norm": 3.466653347015381,
533
- "learning_rate": 0.00019943692110374646,
534
- "loss": 1.2848,
535
- "step": 30000
536
- },
537
- {
538
- "epoch": 0.06961476016112153,
539
- "grad_norm": 2.8881075382232666,
540
- "learning_rate": 0.00019942156195636819,
541
- "loss": 1.3187,
542
- "step": 30400
543
- },
544
- {
545
- "epoch": 0.07053074384745207,
546
- "grad_norm": 2.6795177459716797,
547
- "learning_rate": 0.00019940599674925368,
548
- "loss": 1.3043,
549
- "step": 30800
550
- },
551
- {
552
- "epoch": 0.07144672753378262,
553
- "grad_norm": 2.6429250240325928,
554
- "learning_rate": 0.00019939022551466323,
555
- "loss": 1.2888,
556
- "step": 31200
557
- },
558
- {
559
- "epoch": 0.07236271122011317,
560
- "grad_norm": 2.6998791694641113,
561
- "learning_rate": 0.00019937424828528406,
562
- "loss": 1.2778,
563
- "step": 31600
564
- },
565
- {
566
- "epoch": 0.07327869490644372,
567
- "grad_norm": 2.7572948932647705,
568
- "learning_rate": 0.00019935806509423034,
569
- "loss": 1.3302,
570
- "step": 32000
571
- },
572
- {
573
- "epoch": 0.07419467859277426,
574
- "grad_norm": 5.08123779296875,
575
- "learning_rate": 0.00019934167597504315,
576
- "loss": 1.3033,
577
- "step": 32400
578
- },
579
- {
580
- "epoch": 0.0751106622791048,
581
- "grad_norm": 3.8141486644744873,
582
- "learning_rate": 0.00019932508096169032,
583
- "loss": 1.3028,
584
- "step": 32800
585
- },
586
- {
587
- "epoch": 0.07602664596543536,
588
- "grad_norm": 4.578782558441162,
589
- "learning_rate": 0.00019930828008856645,
590
- "loss": 1.3011,
591
- "step": 33200
592
- },
593
- {
594
- "epoch": 0.0769426296517659,
595
- "grad_norm": 4.768449306488037,
596
- "learning_rate": 0.0001992912733904928,
597
- "loss": 1.3303,
598
- "step": 33600
599
- },
600
- {
601
- "epoch": 0.07785861333809645,
602
- "grad_norm": 3.1697211265563965,
603
- "learning_rate": 0.0001992740609027172,
604
- "loss": 1.2926,
605
- "step": 34000
606
- },
607
- {
608
- "epoch": 0.07877459702442699,
609
- "grad_norm": 2.7294230461120605,
610
- "learning_rate": 0.00019925664266091407,
611
- "loss": 1.3147,
612
- "step": 34400
613
- },
614
- {
615
- "epoch": 0.07969058071075755,
616
- "grad_norm": 2.3929479122161865,
617
- "learning_rate": 0.0001992390187011841,
618
- "loss": 1.3007,
619
- "step": 34800
620
- },
621
- {
622
- "epoch": 0.08060656439708809,
623
- "grad_norm": 3.5411980152130127,
624
- "learning_rate": 0.0001992211890600546,
625
- "loss": 1.2973,
626
- "step": 35200
627
- },
628
- {
629
- "epoch": 0.08152254808341863,
630
- "grad_norm": 2.940880537033081,
631
- "learning_rate": 0.00019920315377447895,
632
- "loss": 1.3072,
633
- "step": 35600
634
- },
635
- {
636
- "epoch": 0.08243853176974918,
637
- "grad_norm": 2.7717535495758057,
638
- "learning_rate": 0.00019918491288183684,
639
- "loss": 1.2908,
640
- "step": 36000
641
- },
642
- {
643
- "epoch": 0.08335451545607973,
644
- "grad_norm": 3.153017520904541,
645
- "learning_rate": 0.00019916646641993406,
646
- "loss": 1.3313,
647
- "step": 36400
648
- },
649
- {
650
- "epoch": 0.08427049914241028,
651
- "grad_norm": 3.065833568572998,
652
- "learning_rate": 0.00019914781442700253,
653
- "loss": 1.3237,
654
- "step": 36800
655
- },
656
- {
657
- "epoch": 0.08518648282874082,
658
- "grad_norm": 2.692406177520752,
659
- "learning_rate": 0.00019912895694170014,
660
- "loss": 1.3175,
661
- "step": 37200
662
- },
663
- {
664
- "epoch": 0.08610246651507136,
665
- "grad_norm": 2.9629924297332764,
666
- "learning_rate": 0.00019910989400311058,
667
- "loss": 1.3247,
668
- "step": 37600
669
- },
670
- {
671
- "epoch": 0.0870184502014019,
672
- "grad_norm": 3.1538310050964355,
673
- "learning_rate": 0.00019909062565074345,
674
- "loss": 1.2894,
675
- "step": 38000
676
- },
677
- {
678
- "epoch": 0.08793443388773246,
679
- "grad_norm": 2.88925838470459,
680
- "learning_rate": 0.00019907115192453412,
681
- "loss": 1.2974,
682
- "step": 38400
683
- },
684
- {
685
- "epoch": 0.088850417574063,
686
- "grad_norm": 3.396505832672119,
687
- "learning_rate": 0.00019905147286484348,
688
- "loss": 1.3409,
689
- "step": 38800
690
- },
691
- {
692
- "epoch": 0.08976640126039355,
693
- "grad_norm": 5.7697978019714355,
694
- "learning_rate": 0.0001990315885124581,
695
- "loss": 1.3222,
696
- "step": 39200
697
- },
698
- {
699
- "epoch": 0.09068238494672409,
700
- "grad_norm": 2.9619033336639404,
701
- "learning_rate": 0.00019901149890859006,
702
- "loss": 1.2918,
703
- "step": 39600
704
- },
705
- {
706
- "epoch": 0.09159836863305465,
707
- "grad_norm": 2.1384809017181396,
708
- "learning_rate": 0.00019899120409487675,
709
- "loss": 1.2941,
710
- "step": 40000
711
- },
712
- {
713
- "epoch": 0.09251435231938519,
714
- "grad_norm": 2.692207098007202,
715
- "learning_rate": 0.0001989707041133809,
716
- "loss": 1.3278,
717
- "step": 40400
718
- },
719
- {
720
- "epoch": 0.09343033600571574,
721
- "grad_norm": 2.231248140335083,
722
- "learning_rate": 0.00019894999900659053,
723
- "loss": 1.2966,
724
- "step": 40800
725
- },
726
- {
727
- "epoch": 0.09434631969204628,
728
- "grad_norm": 4.9003801345825195,
729
- "learning_rate": 0.00019892908881741873,
730
- "loss": 1.3301,
731
- "step": 41200
732
- },
733
- {
734
- "epoch": 0.09526230337837684,
735
- "grad_norm": 2.9206295013427734,
736
- "learning_rate": 0.00019890797358920365,
737
- "loss": 1.3225,
738
- "step": 41600
739
- },
740
- {
741
- "epoch": 0.09617828706470738,
742
- "grad_norm": 3.754241704940796,
743
- "learning_rate": 0.0001988866533657084,
744
- "loss": 1.3073,
745
- "step": 42000
746
- },
747
- {
748
- "epoch": 0.09709427075103792,
749
- "grad_norm": 2.3221588134765625,
750
- "learning_rate": 0.00019886512819112108,
751
- "loss": 1.3078,
752
- "step": 42400
753
- },
754
- {
755
- "epoch": 0.09801025443736847,
756
- "grad_norm": 3.1331708431243896,
757
- "learning_rate": 0.00019884339811005432,
758
- "loss": 1.3036,
759
- "step": 42800
760
- },
761
- {
762
- "epoch": 0.09892623812369902,
763
- "grad_norm": 8.973402976989746,
764
- "learning_rate": 0.0001988214631675457,
765
- "loss": 1.3162,
766
- "step": 43200
767
- },
768
- {
769
- "epoch": 0.09984222181002957,
770
- "grad_norm": 2.487173557281494,
771
- "learning_rate": 0.00019879932340905723,
772
- "loss": 1.3379,
773
- "step": 43600
774
- },
775
- {
776
- "epoch": 0.10075820549636011,
777
- "grad_norm": 2.4204483032226562,
778
- "learning_rate": 0.00019877697888047544,
779
- "loss": 1.2983,
780
- "step": 44000
781
- },
782
- {
783
- "epoch": 0.10167418918269065,
784
- "grad_norm": 2.4077422618865967,
785
- "learning_rate": 0.00019875442962811131,
786
- "loss": 1.2975,
787
- "step": 44400
788
- },
789
- {
790
- "epoch": 0.10259017286902121,
791
- "grad_norm": 3.7924795150756836,
792
- "learning_rate": 0.0001987316756987001,
793
- "loss": 1.3049,
794
- "step": 44800
795
- },
796
- {
797
- "epoch": 0.10350615655535175,
798
- "grad_norm": 2.4726195335388184,
799
- "learning_rate": 0.00019870871713940133,
800
- "loss": 1.3108,
801
- "step": 45200
802
- },
803
- {
804
- "epoch": 0.1044221402416823,
805
- "grad_norm": 2.7203783988952637,
806
- "learning_rate": 0.0001986855539977985,
807
- "loss": 1.3079,
808
- "step": 45600
809
- },
810
- {
811
- "epoch": 0.10533812392801284,
812
- "grad_norm": 3.3892195224761963,
813
- "learning_rate": 0.00019866218632189926,
814
- "loss": 1.2998,
815
- "step": 46000
816
- },
817
- {
818
- "epoch": 0.10625410761434338,
819
- "grad_norm": 7.186534881591797,
820
- "learning_rate": 0.0001986386141601351,
821
- "loss": 1.3204,
822
- "step": 46400
823
- },
824
- {
825
- "epoch": 0.10717009130067394,
826
- "grad_norm": 2.1294145584106445,
827
- "learning_rate": 0.00019861483756136145,
828
- "loss": 1.3159,
829
- "step": 46800
830
- },
831
- {
832
- "epoch": 0.10808607498700448,
833
- "grad_norm": 2.179487943649292,
834
- "learning_rate": 0.00019859085657485725,
835
- "loss": 1.3221,
836
- "step": 47200
837
- },
838
- {
839
- "epoch": 0.10900205867333503,
840
- "grad_norm": 2.950897216796875,
841
- "learning_rate": 0.0001985666712503252,
842
- "loss": 1.3029,
843
- "step": 47600
844
- },
845
- {
846
- "epoch": 0.10991804235966557,
847
- "grad_norm": 3.8391480445861816,
848
- "learning_rate": 0.00019854228163789143,
849
- "loss": 1.3144,
850
- "step": 48000
851
- },
852
- {
853
- "epoch": 0.11083402604599613,
854
- "grad_norm": 3.4370920658111572,
855
- "learning_rate": 0.00019851768778810555,
856
- "loss": 1.3077,
857
- "step": 48400
858
- },
859
- {
860
- "epoch": 0.11175000973232667,
861
- "grad_norm": 3.353703260421753,
862
- "learning_rate": 0.00019849288975194043,
863
- "loss": 1.3034,
864
- "step": 48800
865
- },
866
- {
867
- "epoch": 0.11266599341865721,
868
- "grad_norm": 3.25929856300354,
869
- "learning_rate": 0.00019846788758079212,
870
- "loss": 1.2789,
871
- "step": 49200
872
- },
873
- {
874
- "epoch": 0.11358197710498776,
875
- "grad_norm": 2.8102946281433105,
876
- "learning_rate": 0.00019844268132647975,
877
- "loss": 1.3224,
878
- "step": 49600
879
- },
880
- {
881
- "epoch": 0.11449796079131831,
882
- "grad_norm": 3.392300605773926,
883
- "learning_rate": 0.00019841727104124543,
884
- "loss": 1.3119,
885
- "step": 50000
886
- },
887
- {
888
- "epoch": 0.11541394447764886,
889
- "grad_norm": 2.5362913608551025,
890
- "learning_rate": 0.00019839165677775422,
891
- "loss": 1.3027,
892
- "step": 50400
893
- },
894
- {
895
- "epoch": 0.1163299281639794,
896
- "grad_norm": 1.9659497737884521,
897
- "learning_rate": 0.00019836583858909388,
898
- "loss": 1.2823,
899
- "step": 50800
900
- },
901
- {
902
- "epoch": 0.11724591185030994,
903
- "grad_norm": 2.3742833137512207,
904
- "learning_rate": 0.00019833981652877478,
905
- "loss": 1.3226,
906
- "step": 51200
907
- },
908
- {
909
- "epoch": 0.1181618955366405,
910
- "grad_norm": 2.898440361022949,
911
- "learning_rate": 0.00019831359065072991,
912
- "loss": 1.3127,
913
- "step": 51600
914
- },
915
- {
916
- "epoch": 0.11907787922297104,
917
- "grad_norm": 3.8529317378997803,
918
- "learning_rate": 0.00019828716100931467,
919
- "loss": 1.3176,
920
- "step": 52000
921
- },
922
- {
923
- "epoch": 0.11999386290930159,
924
- "grad_norm": 2.7400405406951904,
925
- "learning_rate": 0.00019826052765930673,
926
- "loss": 1.3108,
927
- "step": 52400
928
- },
929
- {
930
- "epoch": 0.12090984659563213,
931
- "grad_norm": 3.10595965385437,
932
- "learning_rate": 0.000198233690655906,
933
- "loss": 1.3142,
934
- "step": 52800
935
- },
936
- {
937
- "epoch": 0.12182583028196267,
938
- "grad_norm": 2.706665515899658,
939
- "learning_rate": 0.00019820665005473452,
940
- "loss": 1.3015,
941
- "step": 53200
942
- },
943
- {
944
- "epoch": 0.12274181396829323,
945
- "grad_norm": 2.9297852516174316,
946
- "learning_rate": 0.00019817940591183622,
947
- "loss": 1.316,
948
- "step": 53600
949
- },
950
- {
951
- "epoch": 0.12365779765462377,
952
- "grad_norm": 3.5302863121032715,
953
- "learning_rate": 0.00019815195828367692,
954
- "loss": 1.3253,
955
- "step": 54000
956
- },
957
- {
958
- "epoch": 0.12457378134095431,
959
- "grad_norm": 2.3251285552978516,
960
- "learning_rate": 0.0001981243072271442,
961
- "loss": 1.3374,
962
- "step": 54400
963
- },
964
- {
965
- "epoch": 0.12548976502728487,
966
- "grad_norm": 2.586763620376587,
967
- "learning_rate": 0.0001980964527995473,
968
- "loss": 1.3103,
969
- "step": 54800
970
- },
971
- {
972
- "epoch": 0.1264057487136154,
973
- "grad_norm": 3.486201286315918,
974
- "learning_rate": 0.00019806839505861684,
975
- "loss": 1.3404,
976
- "step": 55200
977
- },
978
- {
979
- "epoch": 0.12732173239994596,
980
- "grad_norm": 2.812666893005371,
981
- "learning_rate": 0.00019804013406250496,
982
- "loss": 1.2929,
983
- "step": 55600
984
- },
985
- {
986
- "epoch": 0.12823771608627652,
987
- "grad_norm": 3.6000659465789795,
988
- "learning_rate": 0.00019801166986978502,
989
- "loss": 1.3145,
990
- "step": 56000
991
- },
992
- {
993
- "epoch": 0.12915369977260704,
994
- "grad_norm": 2.623760938644409,
995
- "learning_rate": 0.00019798300253945142,
996
- "loss": 1.2905,
997
- "step": 56400
998
- },
999
- {
1000
- "epoch": 0.1300696834589376,
1001
- "grad_norm": 2.5378899574279785,
1002
- "learning_rate": 0.00019795413213091973,
1003
- "loss": 1.3345,
1004
- "step": 56800
1005
- },
1006
- {
1007
- "epoch": 0.13098566714526813,
1008
- "grad_norm": 3.966965913772583,
1009
- "learning_rate": 0.00019792505870402634,
1010
- "loss": 1.2805,
1011
- "step": 57200
1012
- },
1013
- {
1014
- "epoch": 0.1319016508315987,
1015
- "grad_norm": 3.2356481552124023,
1016
- "learning_rate": 0.00019789578231902842,
1017
- "loss": 1.2846,
1018
- "step": 57600
1019
- },
1020
- {
1021
- "epoch": 0.13281763451792925,
1022
- "grad_norm": 3.4602396488189697,
1023
- "learning_rate": 0.00019786630303660383,
1024
- "loss": 1.308,
1025
- "step": 58000
1026
- },
1027
- {
1028
- "epoch": 0.13373361820425977,
1029
- "grad_norm": 4.348870754241943,
1030
- "learning_rate": 0.00019783662091785086,
1031
- "loss": 1.2911,
1032
- "step": 58400
1033
- },
1034
- {
1035
- "epoch": 0.13464960189059033,
1036
- "grad_norm": 3.970092296600342,
1037
- "learning_rate": 0.0001978067360242883,
1038
- "loss": 1.3088,
1039
- "step": 58800
1040
- },
1041
- {
1042
- "epoch": 0.1355655855769209,
1043
- "grad_norm": 3.6399309635162354,
1044
- "learning_rate": 0.00019777664841785515,
1045
- "loss": 1.3451,
1046
- "step": 59200
1047
- },
1048
- {
1049
- "epoch": 0.13648156926325142,
1050
- "grad_norm": 2.8347928524017334,
1051
- "learning_rate": 0.00019774635816091057,
1052
- "loss": 1.3221,
1053
- "step": 59600
1054
- },
1055
- {
1056
- "epoch": 0.13739755294958197,
1057
- "grad_norm": 2.156590461730957,
1058
- "learning_rate": 0.0001977158653162337,
1059
- "loss": 1.3071,
1060
- "step": 60000
1061
- },
1062
- {
1063
- "epoch": 0.1383135366359125,
1064
- "grad_norm": 3.543612241744995,
1065
- "learning_rate": 0.0001976851699470236,
1066
- "loss": 1.3316,
1067
- "step": 60400
1068
- },
1069
- {
1070
- "epoch": 0.13922952032224306,
1071
- "grad_norm": 2.555818796157837,
1072
- "learning_rate": 0.00019765427211689902,
1073
- "loss": 1.3102,
1074
- "step": 60800
1075
- },
1076
- {
1077
- "epoch": 0.14014550400857362,
1078
- "grad_norm": 2.2213761806488037,
1079
- "learning_rate": 0.00019762317188989843,
1080
- "loss": 1.3198,
1081
- "step": 61200
1082
- },
1083
- {
1084
- "epoch": 0.14106148769490415,
1085
- "grad_norm": 2.6470835208892822,
1086
- "learning_rate": 0.0001975918693304797,
1087
- "loss": 1.279,
1088
- "step": 61600
1089
- },
1090
- {
1091
- "epoch": 0.1419774713812347,
1092
- "grad_norm": 2.345820903778076,
1093
- "learning_rate": 0.00019756036450352009,
1094
- "loss": 1.3239,
1095
- "step": 62000
1096
- },
1097
- {
1098
- "epoch": 0.14289345506756523,
1099
- "grad_norm": 2.149404287338257,
1100
- "learning_rate": 0.00019752865747431605,
1101
- "loss": 1.3445,
1102
- "step": 62400
1103
- },
1104
- {
1105
- "epoch": 0.1438094387538958,
1106
- "grad_norm": 2.8613338470458984,
1107
- "learning_rate": 0.0001974967483085831,
1108
- "loss": 1.371,
1109
- "step": 62800
1110
- },
1111
- {
1112
- "epoch": 0.14472542244022635,
1113
- "grad_norm": 2.959139585494995,
1114
- "learning_rate": 0.00019746463707245575,
1115
- "loss": 1.3083,
1116
- "step": 63200
1117
- },
1118
- {
1119
- "epoch": 0.14564140612655688,
1120
- "grad_norm": 2.392615795135498,
1121
- "learning_rate": 0.00019743232383248732,
1122
- "loss": 1.3123,
1123
- "step": 63600
1124
- },
1125
- {
1126
- "epoch": 0.14655738981288743,
1127
- "grad_norm": 3.202287435531616,
1128
- "learning_rate": 0.00019739980865564972,
1129
- "loss": 1.3188,
1130
- "step": 64000
1131
- },
1132
- {
1133
- "epoch": 0.147473373499218,
1134
- "grad_norm": 2.6809701919555664,
1135
- "learning_rate": 0.00019736709160933353,
1136
- "loss": 1.3307,
1137
- "step": 64400
1138
- },
1139
- {
1140
- "epoch": 0.14838935718554852,
1141
- "grad_norm": 2.6796672344207764,
1142
- "learning_rate": 0.0001973341727613476,
1143
- "loss": 1.3257,
1144
- "step": 64800
1145
- },
1146
- {
1147
- "epoch": 0.14930534087187908,
1148
- "grad_norm": 2.986403465270996,
1149
- "learning_rate": 0.00019730105217991906,
1150
- "loss": 1.3302,
1151
- "step": 65200
1152
- },
1153
- {
1154
- "epoch": 0.1502213245582096,
1155
- "grad_norm": 4.021244525909424,
1156
- "learning_rate": 0.00019726772993369318,
1157
- "loss": 1.3299,
1158
- "step": 65600
1159
- },
1160
- {
1161
- "epoch": 0.15113730824454016,
1162
- "grad_norm": 2.3918700218200684,
1163
- "learning_rate": 0.00019723420609173319,
1164
- "loss": 1.3285,
1165
- "step": 66000
1166
- },
1167
- {
1168
- "epoch": 0.15205329193087072,
1169
- "grad_norm": 2.162391185760498,
1170
- "learning_rate": 0.0001972004807235201,
1171
- "loss": 1.3017,
1172
- "step": 66400
1173
- },
1174
- {
1175
- "epoch": 0.15296927561720125,
1176
- "grad_norm": 3.2081525325775146,
1177
- "learning_rate": 0.00019716655389895267,
1178
- "loss": 1.3482,
1179
- "step": 66800
1180
- },
1181
- {
1182
- "epoch": 0.1538852593035318,
1183
- "grad_norm": 3.2288455963134766,
1184
- "learning_rate": 0.00019713242568834716,
1185
- "loss": 1.3169,
1186
- "step": 67200
1187
- },
1188
- {
1189
- "epoch": 0.15480124298986236,
1190
- "grad_norm": 4.8006792068481445,
1191
- "learning_rate": 0.00019709809616243716,
1192
- "loss": 1.3135,
1193
- "step": 67600
1194
- },
1195
- {
1196
- "epoch": 0.1557172266761929,
1197
- "grad_norm": 2.4130496978759766,
1198
- "learning_rate": 0.00019706356539237366,
1199
- "loss": 1.3152,
1200
- "step": 68000
1201
- },
1202
- {
1203
- "epoch": 0.15663321036252345,
1204
- "grad_norm": 3.030592441558838,
1205
- "learning_rate": 0.00019702883344972463,
1206
- "loss": 1.2712,
1207
- "step": 68400
1208
- },
1209
- {
1210
- "epoch": 0.15754919404885398,
1211
- "grad_norm": 2.5594406127929688,
1212
- "learning_rate": 0.00019699390040647493,
1213
- "loss": 1.3191,
1214
- "step": 68800
1215
- },
1216
- {
1217
- "epoch": 0.15846517773518454,
1218
- "grad_norm": 2.589397668838501,
1219
- "learning_rate": 0.0001969587663350264,
1220
- "loss": 1.3356,
1221
- "step": 69200
1222
- },
1223
- {
1224
- "epoch": 0.1593811614215151,
1225
- "grad_norm": 3.142416000366211,
1226
- "learning_rate": 0.00019692343130819733,
1227
- "loss": 1.2929,
1228
- "step": 69600
1229
- },
1230
- {
1231
- "epoch": 0.16029714510784562,
1232
- "grad_norm": 2.8887860774993896,
1233
- "learning_rate": 0.00019688789539922265,
1234
- "loss": 1.2715,
1235
- "step": 70000
1236
- },
1237
- {
1238
- "epoch": 0.16121312879417618,
1239
- "grad_norm": 2.9049878120422363,
1240
- "learning_rate": 0.0001968521586817536,
1241
- "loss": 1.3102,
1242
- "step": 70400
1243
- },
1244
- {
1245
- "epoch": 0.1621291124805067,
1246
- "grad_norm": 5.651445388793945,
1247
- "learning_rate": 0.00019681622122985756,
1248
- "loss": 1.3383,
1249
- "step": 70800
1250
- },
1251
- {
1252
- "epoch": 0.16304509616683727,
1253
- "grad_norm": 2.6682519912719727,
1254
- "learning_rate": 0.00019678008311801803,
1255
- "loss": 1.3298,
1256
- "step": 71200
1257
- },
1258
- {
1259
- "epoch": 0.16396107985316782,
1260
- "grad_norm": 2.235239267349243,
1261
- "learning_rate": 0.0001967437444211343,
1262
- "loss": 1.3179,
1263
- "step": 71600
1264
- },
1265
- {
1266
- "epoch": 0.16487706353949835,
1267
- "grad_norm": 2.893455982208252,
1268
- "learning_rate": 0.00019670720521452152,
1269
- "loss": 1.3156,
1270
- "step": 72000
1271
- },
1272
- {
1273
- "epoch": 0.1657930472258289,
1274
- "grad_norm": 2.421581268310547,
1275
- "learning_rate": 0.00019667046557391026,
1276
- "loss": 1.3122,
1277
- "step": 72400
1278
- },
1279
- {
1280
- "epoch": 0.16670903091215947,
1281
- "grad_norm": 2.4242067337036133,
1282
- "learning_rate": 0.00019663352557544665,
1283
- "loss": 1.3144,
1284
- "step": 72800
1285
- },
1286
- {
1287
- "epoch": 0.16762501459849,
1288
- "grad_norm": 2.6495418548583984,
1289
- "learning_rate": 0.00019659638529569197,
1290
- "loss": 1.3041,
1291
- "step": 73200
1292
- },
1293
- {
1294
- "epoch": 0.16854099828482055,
1295
- "grad_norm": 2.511672019958496,
1296
- "learning_rate": 0.0001965590448116227,
1297
- "loss": 1.3138,
1298
- "step": 73600
1299
- },
1300
- {
1301
- "epoch": 0.16945698197115108,
1302
- "grad_norm": 1.9795838594436646,
1303
- "learning_rate": 0.0001965215042006301,
1304
- "loss": 1.3126,
1305
- "step": 74000
1306
- },
1307
- {
1308
- "epoch": 0.17037296565748164,
1309
- "grad_norm": 2.0971930027008057,
1310
- "learning_rate": 0.0001964837635405204,
1311
- "loss": 1.3481,
1312
- "step": 74400
1313
- },
1314
- {
1315
- "epoch": 0.1712889493438122,
1316
- "grad_norm": 2.5095582008361816,
1317
- "learning_rate": 0.00019644582290951436,
1318
- "loss": 1.3169,
1319
- "step": 74800
1320
- },
1321
- {
1322
- "epoch": 0.17220493303014273,
1323
- "grad_norm": 2.5341243743896484,
1324
- "learning_rate": 0.00019640768238624715,
1325
- "loss": 1.3021,
1326
- "step": 75200
1327
- },
1328
- {
1329
- "epoch": 0.17312091671647328,
1330
- "grad_norm": 4.831284999847412,
1331
- "learning_rate": 0.00019636934204976831,
1332
- "loss": 1.3097,
1333
- "step": 75600
1334
- },
1335
- {
1336
- "epoch": 0.1740369004028038,
1337
- "grad_norm": 2.303138494491577,
1338
- "learning_rate": 0.0001963308019795415,
1339
- "loss": 1.3239,
1340
- "step": 76000
1341
- },
1342
- {
1343
- "epoch": 0.17495288408913437,
1344
- "grad_norm": 4.299285888671875,
1345
- "learning_rate": 0.00019629206225544433,
1346
- "loss": 1.3131,
1347
- "step": 76400
1348
- },
1349
- {
1350
- "epoch": 0.17586886777546493,
1351
- "grad_norm": 3.942025661468506,
1352
- "learning_rate": 0.00019625312295776817,
1353
- "loss": 1.3413,
1354
- "step": 76800
1355
- },
1356
- {
1357
- "epoch": 0.17678485146179546,
1358
- "grad_norm": 3.2219340801239014,
1359
- "learning_rate": 0.0001962139841672181,
1360
- "loss": 1.3132,
1361
- "step": 77200
1362
- },
1363
- {
1364
- "epoch": 0.177700835148126,
1365
- "grad_norm": 2.0211946964263916,
1366
- "learning_rate": 0.0001961746459649126,
1367
- "loss": 1.2732,
1368
- "step": 77600
1369
- },
1370
- {
1371
- "epoch": 0.17861681883445657,
1372
- "grad_norm": 3.735818862915039,
1373
- "learning_rate": 0.00019613510843238352,
1374
- "loss": 1.3029,
1375
- "step": 78000
1376
- },
1377
- {
1378
- "epoch": 0.1795328025207871,
1379
- "grad_norm": 2.5477070808410645,
1380
- "learning_rate": 0.00019609537165157575,
1381
- "loss": 1.3028,
1382
- "step": 78400
1383
- },
1384
- {
1385
- "epoch": 0.18044878620711766,
1386
- "grad_norm": 2.3577768802642822,
1387
- "learning_rate": 0.00019605543570484725,
1388
- "loss": 1.3296,
1389
- "step": 78800
1390
- },
1391
- {
1392
- "epoch": 0.18136476989344819,
1393
- "grad_norm": 2.8222320079803467,
1394
- "learning_rate": 0.0001960153006749686,
1395
- "loss": 1.3247,
1396
- "step": 79200
1397
- },
1398
- {
1399
- "epoch": 0.18228075357977874,
1400
- "grad_norm": 3.6256015300750732,
1401
- "learning_rate": 0.0001959749666451232,
1402
- "loss": 1.3482,
1403
- "step": 79600
1404
- },
1405
- {
1406
- "epoch": 0.1831967372661093,
1407
- "grad_norm": 3.6244571208953857,
1408
- "learning_rate": 0.00019593443369890671,
1409
- "loss": 1.2999,
1410
- "step": 80000
1411
- },
1412
- {
1413
- "epoch": 0.18411272095243983,
1414
- "grad_norm": 2.577220916748047,
1415
- "learning_rate": 0.00019589370192032724,
1416
- "loss": 1.2971,
1417
- "step": 80400
1418
- },
1419
- {
1420
- "epoch": 0.18502870463877039,
1421
- "grad_norm": 2.6459338665008545,
1422
- "learning_rate": 0.00019585277139380484,
1423
- "loss": 1.3178,
1424
- "step": 80800
1425
- },
1426
- {
1427
- "epoch": 0.18594468832510094,
1428
- "grad_norm": 2.8962595462799072,
1429
- "learning_rate": 0.00019581164220417157,
1430
- "loss": 1.2918,
1431
- "step": 81200
1432
- },
1433
- {
1434
- "epoch": 0.18686067201143147,
1435
- "grad_norm": 2.5094783306121826,
1436
- "learning_rate": 0.00019577031443667118,
1437
- "loss": 1.3126,
1438
- "step": 81600
1439
- },
1440
- {
1441
- "epoch": 0.18777665569776203,
1442
- "grad_norm": 4.931221008300781,
1443
- "learning_rate": 0.00019572878817695905,
1444
- "loss": 1.3178,
1445
- "step": 82000
1446
- },
1447
- {
1448
- "epoch": 0.18869263938409256,
1449
- "grad_norm": 2.7248241901397705,
1450
- "learning_rate": 0.00019568706351110192,
1451
- "loss": 1.3037,
1452
- "step": 82400
1453
- },
1454
- {
1455
- "epoch": 0.18960862307042312,
1456
- "grad_norm": 2.4585657119750977,
1457
- "learning_rate": 0.00019564514052557776,
1458
- "loss": 1.266,
1459
- "step": 82800
1460
- },
1461
- {
1462
- "epoch": 0.19052460675675367,
1463
- "grad_norm": 4.284319877624512,
1464
- "learning_rate": 0.00019560301930727554,
1465
- "loss": 1.3234,
1466
- "step": 83200
1467
- },
1468
- {
1469
- "epoch": 0.1914405904430842,
1470
- "grad_norm": 3.2392733097076416,
1471
- "learning_rate": 0.00019556069994349512,
1472
- "loss": 1.2925,
1473
- "step": 83600
1474
- },
1475
- {
1476
- "epoch": 0.19235657412941476,
1477
- "grad_norm": 3.9642889499664307,
1478
- "learning_rate": 0.000195518182521947,
1479
- "loss": 1.325,
1480
- "step": 84000
1481
- },
1482
- {
1483
- "epoch": 0.1932725578157453,
1484
- "grad_norm": 3.1390507221221924,
1485
- "learning_rate": 0.00019547546713075223,
1486
- "loss": 1.3283,
1487
- "step": 84400
1488
- },
1489
- {
1490
- "epoch": 0.19418854150207585,
1491
- "grad_norm": 2.6130847930908203,
1492
- "learning_rate": 0.0001954325538584421,
1493
- "loss": 1.315,
1494
- "step": 84800
1495
- },
1496
- {
1497
- "epoch": 0.1951045251884064,
1498
- "grad_norm": 2.1440110206604004,
1499
- "learning_rate": 0.00019538944279395804,
1500
- "loss": 1.2824,
1501
- "step": 85200
1502
- },
1503
- {
1504
- "epoch": 0.19602050887473693,
1505
- "grad_norm": 4.09576416015625,
1506
- "learning_rate": 0.0001953461340266515,
1507
- "loss": 1.2979,
1508
- "step": 85600
1509
- },
1510
- {
1511
- "epoch": 0.1969364925610675,
1512
- "grad_norm": 3.521430015563965,
1513
- "learning_rate": 0.0001953026276462836,
1514
- "loss": 1.2959,
1515
- "step": 86000
1516
- },
1517
- {
1518
- "epoch": 0.19785247624739805,
1519
- "grad_norm": 2.449007987976074,
1520
- "learning_rate": 0.000195258923743025,
1521
- "loss": 1.2622,
1522
- "step": 86400
1523
- },
1524
- {
1525
- "epoch": 0.19876845993372858,
1526
- "grad_norm": 2.7827935218811035,
1527
- "learning_rate": 0.00019521502240745585,
1528
- "loss": 1.3096,
1529
- "step": 86800
1530
- },
1531
- {
1532
- "epoch": 0.19968444362005913,
1533
- "grad_norm": 2.0877015590667725,
1534
- "learning_rate": 0.00019517092373056544,
1535
- "loss": 1.2967,
1536
- "step": 87200
1537
- },
1538
- {
1539
- "epoch": 0.20060042730638966,
1540
- "grad_norm": 3.6653809547424316,
1541
- "learning_rate": 0.00019512662780375207,
1542
- "loss": 1.2924,
1543
- "step": 87600
1544
- },
1545
- {
1546
- "epoch": 0.20151641099272022,
1547
- "grad_norm": 2.5417637825012207,
1548
- "learning_rate": 0.00019508213471882286,
1549
- "loss": 1.3262,
1550
- "step": 88000
1551
- },
1552
- {
1553
- "epoch": 0.20243239467905078,
1554
- "grad_norm": 2.624288320541382,
1555
- "learning_rate": 0.00019503744456799354,
1556
- "loss": 1.3025,
1557
- "step": 88400
1558
- },
1559
- {
1560
- "epoch": 0.2033483783653813,
1561
- "grad_norm": 2.456366777420044,
1562
- "learning_rate": 0.00019499255744388826,
1563
- "loss": 1.3399,
1564
- "step": 88800
1565
- },
1566
- {
1567
- "epoch": 0.20426436205171186,
1568
- "grad_norm": 4.339550018310547,
1569
- "learning_rate": 0.00019494747343953953,
1570
- "loss": 1.3008,
1571
- "step": 89200
1572
- },
1573
- {
1574
- "epoch": 0.20518034573804242,
1575
- "grad_norm": 5.688206195831299,
1576
- "learning_rate": 0.00019490219264838775,
1577
- "loss": 1.31,
1578
- "step": 89600
1579
- },
1580
- {
1581
- "epoch": 0.20609632942437295,
1582
- "grad_norm": 2.1266727447509766,
1583
- "learning_rate": 0.0001948567151642813,
1584
- "loss": 1.3074,
1585
- "step": 90000
1586
- },
1587
- {
1588
- "epoch": 0.2070123131107035,
1589
- "grad_norm": 2.642869710922241,
1590
- "learning_rate": 0.00019481104108147611,
1591
- "loss": 1.3326,
1592
- "step": 90400
1593
- },
1594
- {
1595
- "epoch": 0.20792829679703403,
1596
- "grad_norm": 2.08663010597229,
1597
- "learning_rate": 0.00019476517049463573,
1598
- "loss": 1.3104,
1599
- "step": 90800
1600
- },
1601
- {
1602
- "epoch": 0.2088442804833646,
1603
- "grad_norm": 2.1731367111206055,
1604
- "learning_rate": 0.00019471910349883083,
1605
- "loss": 1.2756,
1606
- "step": 91200
1607
- },
1608
- {
1609
- "epoch": 0.20976026416969515,
1610
- "grad_norm": 4.205643177032471,
1611
- "learning_rate": 0.00019467284018953923,
1612
- "loss": 1.3519,
1613
- "step": 91600
1614
- },
1615
- {
1616
- "epoch": 0.21067624785602568,
1617
- "grad_norm": 3.688913345336914,
1618
- "learning_rate": 0.00019462638066264568,
1619
- "loss": 1.3051,
1620
- "step": 92000
1621
- },
1622
- {
1623
- "epoch": 0.21159223154235623,
1624
- "grad_norm": 2.8318068981170654,
1625
- "learning_rate": 0.00019457972501444145,
1626
- "loss": 1.2898,
1627
- "step": 92400
1628
- },
1629
- {
1630
- "epoch": 0.21250821522868676,
1631
- "grad_norm": 2.134383201599121,
1632
- "learning_rate": 0.00019453287334162447,
1633
- "loss": 1.3033,
1634
- "step": 92800
1635
- },
1636
- {
1637
- "epoch": 0.21342419891501732,
1638
- "grad_norm": 2.209117889404297,
1639
- "learning_rate": 0.0001944858257412988,
1640
- "loss": 1.3375,
1641
- "step": 93200
1642
- },
1643
- {
1644
- "epoch": 0.21434018260134788,
1645
- "grad_norm": 2.902921438217163,
1646
- "learning_rate": 0.00019443858231097462,
1647
- "loss": 1.3301,
1648
- "step": 93600
1649
- },
1650
- {
1651
- "epoch": 0.2152561662876784,
1652
- "grad_norm": 2.680903911590576,
1653
- "learning_rate": 0.0001943911431485681,
1654
- "loss": 1.3035,
1655
- "step": 94000
1656
- },
1657
- {
1658
- "epoch": 0.21617214997400896,
1659
- "grad_norm": 2.6961417198181152,
1660
- "learning_rate": 0.00019434350835240087,
1661
- "loss": 1.325,
1662
- "step": 94400
1663
- },
1664
- {
1665
- "epoch": 0.21708813366033952,
1666
- "grad_norm": 2.705254316329956,
1667
- "learning_rate": 0.00019429567802120023,
1668
- "loss": 1.3125,
1669
- "step": 94800
1670
- },
1671
- {
1672
- "epoch": 0.21800411734667005,
1673
- "grad_norm": 2.9032137393951416,
1674
- "learning_rate": 0.0001942476522540986,
1675
- "loss": 1.3146,
1676
- "step": 95200
1677
- },
1678
- {
1679
- "epoch": 0.2189201010330006,
1680
- "grad_norm": 2.815955877304077,
1681
- "learning_rate": 0.00019419943115063352,
1682
- "loss": 1.2994,
1683
- "step": 95600
1684
- },
1685
- {
1686
- "epoch": 0.21983608471933114,
1687
- "grad_norm": 2.9431276321411133,
1688
- "learning_rate": 0.0001941510148107474,
1689
- "loss": 1.3166,
1690
- "step": 96000
1691
- },
1692
- {
1693
- "epoch": 0.2207520684056617,
1694
- "grad_norm": 2.2290589809417725,
1695
- "learning_rate": 0.0001941024033347872,
1696
- "loss": 1.3363,
1697
- "step": 96400
1698
- },
1699
- {
1700
- "epoch": 0.22166805209199225,
1701
- "grad_norm": 6.320208549499512,
1702
- "learning_rate": 0.0001940535968235045,
1703
- "loss": 1.3342,
1704
- "step": 96800
1705
- },
1706
- {
1707
- "epoch": 0.22258403577832278,
1708
- "grad_norm": 2.5221080780029297,
1709
- "learning_rate": 0.0001940045953780549,
1710
- "loss": 1.3139,
1711
- "step": 97200
1712
- },
1713
- {
1714
- "epoch": 0.22350001946465334,
1715
- "grad_norm": 1.954762578010559,
1716
- "learning_rate": 0.00019395539909999814,
1717
- "loss": 1.3196,
1718
- "step": 97600
1719
- },
1720
- {
1721
- "epoch": 0.2244160031509839,
1722
- "grad_norm": 6.8120436668396,
1723
- "learning_rate": 0.00019390600809129772,
1724
- "loss": 1.3372,
1725
- "step": 98000
1726
- },
1727
- {
1728
- "epoch": 0.22533198683731442,
1729
- "grad_norm": 3.1833598613739014,
1730
- "learning_rate": 0.00019385642245432077,
1731
- "loss": 1.3251,
1732
- "step": 98400
1733
- }
1734
- ],
1735
- "logging_steps": 400,
1736
- "max_steps": 873378,
1737
- "num_input_tokens_seen": 0,
1738
- "num_train_epochs": 2,
1739
- "save_steps": 50,
1740
- "stateful_callbacks": {
1741
- "TrainerControl": {
1742
- "args": {
1743
- "should_epoch_stop": false,
1744
- "should_evaluate": false,
1745
- "should_log": false,
1746
- "should_save": true,
1747
- "should_training_stop": false
1748
- },
1749
- "attributes": {}
1750
- }
1751
- },
1752
- "total_flos": 1.23243891523584e+18,
1753
- "train_batch_size": 2,
1754
- "trial_name": null,
1755
- "trial_params": null
1756
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-98500/training_args.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:d41cf245f1cbaed3057de22d80908b9da1fb1f2b5972cc6b8230f38f3f7323c2
3
- size 5304