# Train a transformer from scratch

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrain-a-transformer-from-scratch%2F9095%3E%5C
**Category:** 🤗Transformers\
**Created:** [August 9, 2021, 2:42pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrain-a-transformer-from-scratch%2F9095 "2021-08-09T14:42:50Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hannibal046](https://avatars.discourse-cdn.com/v4/letter/h/ccd318/32.png) [@Hannibal046](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FHannibal046)%5C
**Post date:** [August 9, 2021, 2:42pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrain-a-transformer-from-scratch%2F9095%2F1 "2021-08-09T14:42:51Z")

</div>

Hello,guys!  
`Huggingface/transformers` is such a convenient library to use when it comes to all sorts of pertained model. But I am wondering is there a convenient way to train a model from scratch ?  
If I want to rebuild the model in `Attention is all you need` , the first thought came into my mind is change `modeling_bart.py` to adapt to `Attention is all you need` setting and do not `from_pretrained`, Is there any better way to do it ?  
I am looking forward to your reply