# ALBERT Pretraining example (Tensorflow)

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Falbert-pretraining-example-tensorflow%2F1427%3E%5C
**Category:** Beginners\
**Created:** [October 8, 2020, 3:43pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Falbert-pretraining-example-tensorflow%2F1427 "2020-10-08T15:43:34Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![UmarSpa](https://avatars.discourse-cdn.com/v4/letter/u/e47774/32.png) [@UmarSpa](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FUmarSpa)%5C
**Post date:** [October 8, 2020, 3:43pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Falbert-pretraining-example-tensorflow%2F1427%2F1 "2020-10-08T15:43:34Z")

</div>

Hello everyone,

Is there any guide / example of how to do pretraining of ALBERT (in Tensorflow) using code only from huggingface?

I would really appreciate any help.

Thanks

---

<div class="post-metadata">

**Author:** ![UmarSpa](https://avatars.discourse-cdn.com/v4/letter/u/e47774/32.png) [@UmarSpa](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FUmarSpa)%5C
**Post date:** [October 23, 2020, 12:05pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Falbert-pretraining-example-tensorflow%2F1427%2F2 "2020-10-23T12:05:15Z")

</div>

@sgugger any help here would be very appreciated

---

<div class="post-metadata">

**Author:** ![UmarSpa](https://avatars.discourse-cdn.com/v4/letter/u/e47774/32.png) [@UmarSpa](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FUmarSpa)%5C
**Post date:** [November 23, 2020, 10:15am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Falbert-pretraining-example-tensorflow%2F1427%2F3 "2020-11-23T10:15:30Z")

</div>

Still waiting a reply on this