# Resume training from checkpoint

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fresume-training-from-checkpoint%2F19764%3E%5C
**Category:** Beginners\
**Created:** [June 29, 2022, 7:42am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fresume-training-from-checkpoint%2F19764 "2022-06-29T07:42:26Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![mzyMMMM](https://avatars.discourse-cdn.com/v4/letter/m/e95f7d/32.png) [@mzyMMMM](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FmzyMMMM)%5C
**Post date:** [June 29, 2022, 7:42am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fresume-training-from-checkpoint%2F19764%2F1 "2022-06-29T07:42:26Z")

</div>

Hi, all!  
I want to resume training from a checkpoint and I use the method trainer.train(resume\_from\_checkpoint=True)  
(also tried trainer.train(resume\_from\_checkpoint=checkpoint\_dir))  
the train code is:

```auto
    training_args = TrainingArguments(
        output_dir=base_path, overwrite_output_dir=True, num_train_epochs=num_train_epochs,
        learning_rate=5e-5, weight_decay=0.01, warmup_steps=10000, local_rank=args.local_rank,
        per_device_train_batch_size=train_batch_size, per_device_eval_batch_size=eval_batch_size,save_total_limit=5,
        save_strategy="steps", evaluation_strategy="steps",logging_steps=2500,save_steps=2500,eval_steps=2500,
        load_best_model_at_end =True,
        metric_for_best_model="eval_loss", logging_dir="temp",seed=1994,data_seed=1994)

    custom_dataset = mydataset(args_file_path)
    eval_ratio = 0.005
    train_size =int( len(custom_dataset)*(1-eval_ratio))
    eval_size = len(custom_dataset)-train_size
    print(f"the evaluation size is {eval_size}")
    train_dataset, eval_dataset = random_split(custom_dataset,[train_size,eval_size],generator=torch.Generator().manual_seed(1994))
    trainer = TripletLossTrainer(
        model=model, args=training_args,
        data_collator=collator,
        train_dataset=train_dataset,
        eval_dataset=eval_dataset
    )

    transformers.utils.logging.set_verbosity_info()
    trainer.train(resume_from_checkpoint=True)
    trainer.save_model(base_path)

```

It truly loaded the latest model, but the training progress bar shows it starts training on step 1, not where it stopped before

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/2X/3/34c77c5c1136b7497bfbda830ad63c1385b79d85.png)

---

<div class="post-metadata">

**Author:** ![corbt](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/corbt/32/13358_2.png) [@corbt](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcorbt)%5C
**Post date:** [January 5, 2023, 2:07am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fresume-training-from-checkpoint%2F19764%2F2 "2023-01-05T02:07:31Z")

</div>

I’m seeing the same behavior. Transformers logs `Loading model from models/.../checkpoint-13000`, but when training begins the epoch/step count ends up reset to 0 anyway.