deberta-v3-large__sst2__train-8-6
This model is a fine-tuned version of microsoft/deberta-v3-large on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.4331
- Accuracy: 0.7106
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 50
- mixed_precision_training: Native AMP
Training results
| Training Loss |
Epoch |
Step |
Validation Loss |
Accuracy |
| 0.6486 |
1.0 |
3 |
0.7901 |
0.25 |
| 0.6418 |
2.0 |
6 |
0.9259 |
0.25 |
| 0.6169 |
3.0 |
9 |
1.0574 |
0.25 |
| 0.5639 |
4.0 |
12 |
1.1372 |
0.25 |
| 0.4562 |
5.0 |
15 |
0.6090 |
0.5 |
| 0.3105 |
6.0 |
18 |
0.4435 |
1.0 |
| 0.2303 |
7.0 |
21 |
0.2804 |
1.0 |
| 0.1388 |
8.0 |
24 |
0.2205 |
1.0 |
| 0.0918 |
9.0 |
27 |
0.1282 |
1.0 |
| 0.0447 |
10.0 |
30 |
0.0643 |
1.0 |
| 0.0297 |
11.0 |
33 |
0.0361 |
1.0 |
| 0.0159 |
12.0 |
36 |
0.0211 |
1.0 |
| 0.0102 |
13.0 |
39 |
0.0155 |
1.0 |
| 0.0061 |
14.0 |
42 |
0.0158 |
1.0 |
| 0.0049 |
15.0 |
45 |
0.0189 |
1.0 |
| 0.0035 |
16.0 |
48 |
0.0254 |
1.0 |
| 0.0027 |
17.0 |
51 |
0.0305 |
1.0 |
| 0.0021 |
18.0 |
54 |
0.0287 |
1.0 |
| 0.0016 |
19.0 |
57 |
0.0215 |
1.0 |
| 0.0016 |
20.0 |
60 |
0.0163 |
1.0 |
| 0.0014 |
21.0 |
63 |
0.0138 |
1.0 |
| 0.0015 |
22.0 |
66 |
0.0131 |
1.0 |
| 0.001 |
23.0 |
69 |
0.0132 |
1.0 |
| 0.0014 |
24.0 |
72 |
0.0126 |
1.0 |
| 0.0011 |
25.0 |
75 |
0.0125 |
1.0 |
| 0.001 |
26.0 |
78 |
0.0119 |
1.0 |
| 0.0008 |
27.0 |
81 |
0.0110 |
1.0 |
| 0.0007 |
28.0 |
84 |
0.0106 |
1.0 |
| 0.0008 |
29.0 |
87 |
0.0095 |
1.0 |
| 0.0009 |
30.0 |
90 |
0.0089 |
1.0 |
| 0.0008 |
31.0 |
93 |
0.0083 |
1.0 |
| 0.0007 |
32.0 |
96 |
0.0075 |
1.0 |
| 0.0008 |
33.0 |
99 |
0.0066 |
1.0 |
| 0.0006 |
34.0 |
102 |
0.0059 |
1.0 |
| 0.0007 |
35.0 |
105 |
0.0054 |
1.0 |
| 0.0008 |
36.0 |
108 |
0.0051 |
1.0 |
| 0.0007 |
37.0 |
111 |
0.0049 |
1.0 |
| 0.0007 |
38.0 |
114 |
0.0047 |
1.0 |
| 0.0006 |
39.0 |
117 |
0.0045 |
1.0 |
| 0.0006 |
40.0 |
120 |
0.0046 |
1.0 |
| 0.0005 |
41.0 |
123 |
0.0045 |
1.0 |
| 0.0006 |
42.0 |
126 |
0.0044 |
1.0 |
| 0.0006 |
43.0 |
129 |
0.0043 |
1.0 |
| 0.0006 |
44.0 |
132 |
0.0044 |
1.0 |
| 0.0005 |
45.0 |
135 |
0.0045 |
1.0 |
| 0.0006 |
46.0 |
138 |
0.0043 |
1.0 |
| 0.0006 |
47.0 |
141 |
0.0043 |
1.0 |
| 0.0006 |
48.0 |
144 |
0.0041 |
1.0 |
| 0.0007 |
49.0 |
147 |
0.0042 |
1.0 |
| 0.0005 |
50.0 |
150 |
0.0042 |
1.0 |
Framework versions
- Transformers 4.15.0
- Pytorch 1.10.2+cu102
- Datasets 1.18.2
- Tokenizers 0.10.3
