# Masking task with BERT on time serires

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmasking-task-with-bert-on-time-serires%2F113171%3E%5C
**Category:** Research\
**Created:** [October 21, 2024, 12:16pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmasking-task-with-bert-on-time-serires%2F113171 "2024-10-21T12:16:35Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![LeoBigelli](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/leobigelli/32/33212_2.png) [@LeoBigelli](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FLeoBigelli)%5C
**Post date:** [October 21, 2024, 12:16pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmasking-task-with-bert-on-time-serires%2F113171%2F1 "2024-10-21T12:16:35Z")

</div>

Hi everone,  
I started to pre-trained BERT for a masking task in a time series domain. I used a custom tokenizzation (not an usual model) to masking some samples with a special token. But durig the training process the loss is too much constant (like 6.7, I used the SparseCategoricalCrossentropy).  
Could anyone help me?

Thank guys.