distilbert-classn-LAlg-multihead-context-width-1

This model is a fine-tuned version of dslim/distilbert-NER on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7685
• Accuracy: 0.8016
• F1: 0.7968
• Precision: 0.8118
• Recall: 0.8016

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 25
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
2.4353	1.3514	50	2.3841	0.1032	0.0764	0.0688	0.1032
2.4183	2.7027	100	2.3609	0.1270	0.1157	0.1493	0.1270
2.3684	4.0541	150	2.3155	0.1746	0.1538	0.1718	0.1746
2.3091	5.4054	200	2.2475	0.2460	0.2314	0.2715	0.2460
2.1631	6.7568	250	2.1282	0.3571	0.3272	0.3831	0.3571
1.95	8.1081	300	1.8926	0.5317	0.5254	0.5465	0.5317
1.5956	9.4595	350	1.5340	0.6429	0.6333	0.6574	0.6429
1.1292	10.8108	400	1.1863	0.7302	0.7269	0.7479	0.7302
0.735	12.1622	450	0.9370	0.7698	0.7687	0.7884	0.7698
0.4303	13.5135	500	0.7836	0.7937	0.7902	0.8107	0.7937
0.2449	14.8649	550	0.7493	0.7857	0.7818	0.7900	0.7857
0.1215	16.2162	600	0.7686	0.7857	0.7816	0.7886	0.7857
0.0739	17.5676	650	0.7501	0.7937	0.7900	0.8058	0.7937
0.0452	18.9189	700	0.7628	0.8016	0.7960	0.8097	0.8016
0.0278	20.2703	750	0.7612	0.7937	0.7900	0.8073	0.7937
0.0212	21.6216	800	0.7714	0.7937	0.7884	0.8028	0.7937
0.0197	22.9730	850	0.7743	0.7937	0.7882	0.8014	0.7937
0.0151	24.3243	900	0.7685	0.8016	0.7968	0.8118	0.8016

Framework versions

• Transformers 4.48.3
• Pytorch 2.5.1+cu124
• Datasets 3.3.1
• Tokenizers 0.21.0