# Code example of getting cross attention from T5?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcode-example-of-getting-cross-attention-from-t5%2F31740%3E%5C
**Category:** Intermediate\
**Created:** [February 15, 2023, 5:16am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcode-example-of-getting-cross-attention-from-t5%2F31740 "2023-02-15T05:16:21Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![veritas2019](https://avatars.discourse-cdn.com/v4/letter/v/e47774/32.png) [@veritas2019](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fveritas2019)%5C
**Post date:** [February 15, 2023, 5:16am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcode-example-of-getting-cross-attention-from-t5%2F31740%2F1 "2023-02-15T05:16:21Z")

</div>

Hi fellow hackers – could you point me to a short code snippet showing how to obtain cross attention weights from a T5 model (T5ForConditionalGeneration)? I’d like to get the attention distribution for every decoded token on all input tokens.

Thanks a lot!