ssc-bew-mms-model

This model is a fine-tuned version of facebook/mms-1b-all on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7747
• Cer: 0.2038
• Wer: 0.6111

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 12
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Cer	Wer
1.6619	0.3063	200	0.9694	0.2219	0.6669
1.5435	0.6126	400	0.8620	0.2158	0.6542
1.5126	0.9188	600	0.8212	0.2101	0.6326
1.4544	1.2251	800	0.8192	0.2101	0.6346
1.4753	1.5314	1000	0.8128	0.2096	0.6393
1.4388	1.8377	1200	0.8112	0.2098	0.6292
1.4494	2.1440	1400	0.8120	0.2084	0.6268
1.4289	2.4502	1600	0.7985	0.2075	0.6254
1.4736	2.7565	1800	0.7951	0.2070	0.6263
1.4217	3.0628	2000	0.8154	0.2104	0.6369
1.4146	3.3691	2200	0.8092	0.2094	0.6295
1.4205	3.6753	2400	0.8056	0.2092	0.6298
1.3951	3.9816	2600	0.8070	0.2087	0.6384
1.3695	4.2879	2800	0.7980	0.2078	0.6261
1.4743	4.5942	3000	0.8138	0.2099	0.6376
1.4499	4.9005	3200	0.7828	0.2041	0.6168
1.3801	5.2067	3400	0.8019	0.2086	0.6324
1.4191	5.5130	3600	0.7931	0.2064	0.6229
1.4359	5.8193	3800	0.7794	0.2034	0.6144
1.407	6.1256	4000	0.7877	0.2064	0.6243
1.4079	6.4319	4200	0.7963	0.2071	0.6221
1.417	6.7381	4400	0.7839	0.2048	0.6139
1.4031	7.0444	4600	0.7759	0.2033	0.6098
1.3999	7.3507	4800	0.7781	0.2043	0.6137
1.4052	7.6570	5000	0.7769	0.2041	0.6110
1.4102	7.9632	5200	0.7750	0.2031	0.6085
1.392	8.2695	5400	0.7776	0.2041	0.6115
1.4032	8.5758	5600	0.7733	0.2033	0.6096
1.3956	8.8821	5800	0.7761	0.2038	0.6101
1.4148	9.1884	6000	0.7761	0.2040	0.6102
1.3921	9.4946	6200	0.7746	0.2036	0.6099
1.4109	9.8009	6400	0.7747	0.2038	0.6111

Framework versions

• Transformers 4.57.2
• Pytorch 2.9.1+cu128
• Datasets 3.6.0
• Tokenizers 0.22.0