train_multirc_456_1767257752

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the multirc dataset. It achieves the following results on the evaluation set:

• Loss: 1.4579
• Num Input Tokens Seen: 235044016

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3159	2.0	10898	0.3301	23491904
0.2466	4.0	21796	0.2969	47003136
0.186	6.0	32694	0.2781	70491792
0.2792	8.0	43592	0.3070	93981376
0.1407	10.0	54490	0.4988	117461104
0.1463	12.0	65388	0.6823	140981152
0.0003	14.0	76286	1.0003	164493408
0.0001	16.0	87184	1.2439	188018640
0.2313	18.0	98082	1.4316	211522080
0.0	20.0	108980	1.4579	235044016

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1