# Unexpected result from transformer model prediction

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funexpected-result-from-transformer-model-prediction%2F12125%3E%5C
**Category:** Beginners\
**Created:** [November 21, 2021, 2:43pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funexpected-result-from-transformer-model-prediction%2F12125 "2021-11-21T14:43:28Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Roman](https://avatars.discourse-cdn.com/v4/letter/r/ed8c4c/32.png) [@Roman](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FRoman)%5C
**Post date:** [November 21, 2021, 2:43pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funexpected-result-from-transformer-model-prediction%2F12125%2F1 "2021-11-21T14:43:28Z")

</div>

Hello.  
Working with huggingface transformers for Masked Language Task I have expected that prediction will return the same input string plus tokens for masked ones:

```auto
    from transformers import BertConfig, BertTokenizer, BertForMaskedLM
    
    model1 = BertForMaskedLM.from_pretrained("bert-base-uncased")
    tokenizer1 = BertTokenizer.from_pretrained("bert-base-uncased")
    
    # Read the rest of this [MASK] to understand things in more detail
    text = ["Read the rest of this [MASK] to understand things in more detail"]
    encoding1 = tokenizer1(text, return_tensors="pt")
    
    # forward pass
    outputs1 = model1(**encoding1)
    outputs1.logits.argmax(-1)

```

The output is :

```auto
    tensor([[1012, 3191, 1996, 2717, 1997, 2023, 2338, 2000, 3305, 2477, 1999, 2062,
             1012, 1012]])

```

But when I have decoded the output I did not find last input token _ **detail** _ :

` tokenizer1.convert_ids_to_tokens([1012, 3191, 1996, 2717, 1997, 2023, 2338, 2000, 3305, 2477, 1999, 2062, 1012, 1012])`

```auto

    ['.',
     'read',
     'the',
     'rest',
     'of',
     'this',
     'book',
     'to',
     'understand',
     'things',
     'in',
     'more',
     '.',
     '.']

```

Maybe I’m using it incorrectly model? Any other reason?