kp-mt5-base

This model is a fine-tuned version of google/mt5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.9680
• Bleu: 29.4263
• Gen Len: 44.3157

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2.59e-05
• train_batch_size: 24
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 96
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Bleu	Gen Len	Validation Loss
1.8289	0.4026	20001	17.9948	45.585	1.3952
1.569	0.8052	40002	22.2856	45.2362	1.2204
1.463	1.2078	60003	24.6884	44.4264	1.1352
1.3552	1.6104	80004	26.0272	44.3574	1.0864
1.3187	2.0130	100005	27.0619	45.005	1.0543
1.3011	2.4156	120006	1.0286	27.63	44.1082
1.2552	2.8182	140007	0.9847	29.0405	44.4054

Framework versions

• Transformers 4.56.1
• Pytorch 2.8.0+cu126
• Datasets 4.0.0
• Tokenizers 0.22.0