# Questions about loading checkpoint by \`.from\_pretrained\`

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestions-about-loading-checkpoint-by-from-pretrained%2F168787%3E%5C
**Category:** 🤗Transformers\
**Created:** [September 29, 2025, 8:55pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestions-about-loading-checkpoint-by-from-pretrained%2F168787 "2025-09-29T20:55:07Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Zoe0427](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/zoe0427/32/53729_2.png) [@Zoe0427](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FZoe0427)%5C
**Post date:** [September 29, 2025, 8:55pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestions-about-loading-checkpoint-by-from-pretrained%2F168787%2F1 "2025-09-29T20:55:07Z")

</div>

I tried to use BERT -large-uncased to finetune on GLUE bechmark, and I loaded checkpoint from Hub. When I saw the details:

 ![2685b2c07b56768f00bbb3b603f25612](https://us1.discourse-cdn.com/hellohellohello/original/3X/5/7/5726b3e3f3762befb6a7a7cc63879f45ac3f366b.png)

My understand is that the finetuning model for GLUE is BERT + classifier, the chechpoint is only for BERT. So missing keys is belong to classifer. But I think there is a pooler behind BERT attention, so pooler keys are also missing, right? (But why I didn’t see them in missing keys?) And I think the unexpected keys actually can be assigned to pooler.

And I also noticed that during the finetuning, not loading pooler’s weights (keeping the weights initialized normally) is better.🤔

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [September 29, 2025, 11:54pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestions-about-loading-checkpoint-by-from-pretrained%2F168787%2F2 "2025-09-29T23:54:40Z")

</div>

Seems [basically expected behavior](https://huggingface.co/datasets/John6666/forum1/blob/main/cp_loading_missing_keys.md).