# Naming inconsistency in Distilbert config

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fnaming-inconsistency-in-distilbert-config%2F2220%3E%5C
**Category:** 🤗Transformers\
**Created:** [November 25, 2020, 11:08pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fnaming-inconsistency-in-distilbert-config%2F2220 "2020-11-25T23:08:04Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![dropout05](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/dropout05/32/5465_2.png) [@dropout05](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdropout05)%5C
**Post date:** [November 25, 2020, 11:08pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fnaming-inconsistency-in-distilbert-config%2F2220%2F1 "2020-11-25T23:08:04Z")

</div>

Hi!  
I noticed one inconsistency between Distilbert and bert configs. Distilbert config stores output hidden size as `hidden_size` and ffn dim as `hidden_dim` while BERT and RoBERTa use `hidden_dim` for output and `intermediate_size` for ffn dim.

I know such thing can be hard to fix without breaking backcompatibiliby, but such behavior makes is a bit harder to get your model output size upfront.  
E.g. if I want to be able to use both DistilBERT and BERT as an encoder in my model like this

```python
class MySuperCustomModel(nn.Module):
    def __init__ (self, encoder, n_classes):
        super(). __init__ ()
        self.encoder = encoder
        hidden_size = ... # I wish it would be as simple as encoder.config.hidden_dim
        self.logit_network = nn.Linear(hidden_size, n_classes)

```

the code to get the encoder output size is kind of ugly, because you need to use `isinstance` or something like it.  
Or course, for classification you can use \*ModelForClassification, but what if you want to use a pre-trained model as a seq2seq encoder or to write some other custom model.

I feel like solving this issue can make quite some people a bit happier, as they would be able to have to experiment with using different pre-trained models without code modifications and without thinking about the differences between transformer configs.

Is there a better way to get the output dimension of the model or any fixes planned? I can help with a PR too.

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [November 30, 2020, 2:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fnaming-inconsistency-in-distilbert-config%2F2220%2F2 "2020-11-30T14:00:01Z")

</div>

You can make a PR with new properties for those configs (like ` hidden_size` for `DistilBert`) but we can’t change the name of the arguments of the configs as it would be a severe breaking change.  
I agree that consistent named properties would be useful!