xlsr-a-nomimo-ag

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3557
• Wer: 0.3488

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0004
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 132
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
4.6325	1.9275	200	2.6167	1.0
1.6918	3.8502	400	0.5002	0.6674
0.4587	5.7729	600	0.2962	0.5378
0.2519	7.6957	800	0.3525	0.4776
0.1919	9.6184	1000	0.3203	0.3588
0.172	11.5411	1200	0.3277	0.3796
0.1073	13.4638	1400	0.3040	0.3951
0.1129	15.3865	1600	0.2973	0.3634
0.0811	17.3092	1800	0.3278	0.3696
0.0931	19.2319	2000	0.2994	0.3619
0.0755	21.1546	2200	0.3818	0.3627
0.0723	23.0773	2400	0.3354	0.3673
0.0606	25.0	2600	0.2995	0.3557
0.0595	26.9275	2800	0.3685	0.3557
0.0527	28.8502	3000	0.3813	0.3866
0.0451	30.7729	3200	0.3707	0.3611
0.0374	32.6957	3400	0.3243	0.3619
0.0411	34.6184	3600	0.3236	0.3573
0.0319	36.5411	3800	0.3399	0.3557
0.0289	38.4638	4000	0.3429	0.3495
0.0296	40.3865	4200	0.3523	0.3603
0.0302	42.3092	4400	0.3489	0.3503
0.0238	44.2319	4600	0.3400	0.3519
0.0246	46.1546	4800	0.3541	0.3495
0.0215	48.0773	5000	0.3557	0.3488

Framework versions

• Transformers 4.47.0.dev0
• Pytorch 2.4.0
• Datasets 3.0.1
• Tokenizers 0.20.0