# Specify different attention masks for different layers

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fspecify-different-attention-masks-for-different-layers%2F29731%3E%5C
**Category:** 🤗Transformers\
**Created:** [January 16, 2023, 6:04am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fspecify-different-attention-masks-for-different-layers%2F29731 "2023-01-16T06:04:41Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![nachtsky](https://avatars.discourse-cdn.com/v4/letter/n/b9e5f3/32.png) [@nachtsky](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnachtsky)%5C
**Post date:** [January 16, 2023, 6:04am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fspecify-different-attention-masks-for-different-layers%2F29731%2F1 "2023-01-16T06:04:41Z")

</div>

Hi, I am experimenting with some ideas of adding different attention masks for different transformer layers. Currently, I am trying to implement a customized model that accepts two attention masks. But I’m wondering if there’s any simpler method to do so? Any suggestion is welcome! Thanks!