train_record_456_1766622579

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the record dataset. It achieves the following results on the evaluation set:

• Loss: 0.2735
• Num Input Tokens Seen: 928892640

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.203	1.0	31242	0.3015	46454656
0.2078	2.0	62484	0.2735	92898208
0.2747	3.0	93726	0.2799	139330944
0.2688	4.0	124968	0.3303	185787424
0.2789	5.0	156210	0.3290	232232736
0.1039	6.0	187452	0.4111	278675168
0.2495	7.0	218694	0.3815	325124320
0.1111	8.0	249936	0.4473	371565312
0.3756	9.0	281178	0.4629	418010016
0.178	10.0	312420	0.4743	464454880
0.2079	11.0	343662	0.5030	510906784
0.0945	12.0	374904	0.5430	557340128
0.1262	13.0	406146	0.5626	603790528
0.0913	14.0	437388	0.6708	650253184
0.1518	15.0	468630	0.7122	696691296
0.1106	16.0	499872	0.7963	743122464
0.0548	17.0	531114	0.8800	789557088
0.1011	18.0	562356	0.9980	835994816
0.0432	19.0	593598	1.1224	882444928
0.1182	20.0	624840	1.1603	928892640

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1