# Train a CausalLM for machine translation

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrain-a-causallm-for-machine-translation%2F133865%3E%5C
**Category:** Beginners\
**Created:** [January 1, 2025, 5:45pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrain-a-causallm-for-machine-translation%2F133865 "2025-01-01T17:45:58Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [January 1, 2025, 11:22pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrain-a-causallm-for-machine-translation%2F133865%2F2 "2025-01-01T23:22:47Z")

</div>

Maybe unresolved issue? Or maybe it hasn’t been made into an issue on the transformers github.

> **[There were missing keys in the checkpoint model loaded: \['proj\_out.weight'\]....](https://github.com/openai/whisper/discussions/2302)**
>
> I am trying to fine tune the model using the technique shown in this tutorial: https://huggingface.co/blog/fine-tune-whisper My dataset is much larger and my PC cannot handle it all at once so I am...

> [@Unable to load checkpoint after finetuning](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funable-to-load-checkpoint-after-finetuning%2F50693%2F)%3A
>
> Hi, So I’m running into trouble when trying to load the model weights after finetuning a pretrained base model, steps I’m taking: model= BertForSequenceClassification.from\_pretrained("model/repo") model.classifier = torch.nn.Sequential( torch.nn.Dropout(0.4), torch.nn.Linear(model.config.hidden\_size, 256), torch.nn.ReLU(), torch.nn.Dropout(0.4), torch.nn.Linear(256, 2), ) getting a warning here which I guess is fine for the moment: Some weights of the model checkpoint at…

> [@Resuming training: There were missing keys in the checkpoint model loaded: \['lm\_head.weight'\]](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fresuming-training-there-were-missing-keys-in-the-checkpoint-model-loaded-lm-head-weight%2F103831%2F)%3A
>
> Hey all, Trying to learn a bit about training AI models, so I figured I would try to kind of recreate the “Nothing, Forever” AI but with a whatsapp friend group of mine as training set. However, the text set that i’m using is large (100k lines) so training is taking a while and I would like to interrupt it at times. It all seems to work fine, except when i resume a checkpoint I always see this message: There were missing keys in the checkpoint model loaded: ['lm\_head.weight']. Should I be wo…

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrain-a-causallm-for-machine-translation%2F133865)._