# Setting up a timeseries transformer

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsetting-up-a-timeseries-transformer%2F37083%3E%5C
**Category:** 🤗Transformers\
**Created:** [April 19, 2023, 7:49am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsetting-up-a-timeseries-transformer%2F37083 "2023-04-19T07:49:07Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Imbecilcode](https://avatars.discourse-cdn.com/v4/letter/i/c89c15/32.png) [@Imbecilcode](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FImbecilcode)%5C
**Post date:** [April 19, 2023, 7:49am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsetting-up-a-timeseries-transformer%2F37083%2F1 "2023-04-19T07:49:07Z")

</div>

I’ve been trying to set up the TimeSeriesTransformerForPrediction model with a time series dataset I’ve preprocessed for it.  
After a painstaking process I’ve managed to get to the point in the script where I’m just about ready to initiate the training of said model.

however I am thuroughly stuck on this issue:

File “C:\Users\difrf\ASSEMBLE TRANSFORMERS\lib\site-packages\torch\nn\modules\module.py”, line 1501, in \_call\_impl  
return forward\_call(\*args, \*\*kwargs)  
File “C:\Users\difrf\ASSEMBLE TRANSFORMERS\lib\site-packages\transformers\models\time\_series\_transformer\modeling\_time\_series\_transformer.py”, line 1430, in forward  
transformer\_inputs, loc, scale, static\_feat = self.create\_network\_inputs(  
File “C:\Users\difrf\ASSEMBLE TRANSFORMERS\lib\site-packages\transformers\models\time\_series\_transformer\modeling\_time\_series\_transformer.py”, line 1303, in create\_network\_inputs  
torch.cat(  
TypeError: expected Tensor as element 1 in argument 0, but got NoneType  
0%| | 0/81910 [00:00\<?, ?it/s]

After triplechecking that I’m passing: [past\_values, future\_values, past\_time\_features and past\_observed\_mask] as tensors I’m still receiving this error.

I’m starting to suspect there might be another required positional argument I’m not aware off or maybe I’m just hopelessly stupid. Any help or feedback on how to get past this issue would be greatly appreciated

---

<div class="post-metadata">

**Author:** ![cardboardaardvark](https://avatars.discourse-cdn.com/v4/letter/c/df705f/32.png) [@cardboardaardvark](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcardboardaardvark)%5C
**Post date:** [April 21, 2023, 9:10am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsetting-up-a-timeseries-transformer%2F37083%2F2 "2023-04-21T09:10:34Z")

</div>

You need to also pass in future\_time\_features.

I just got an Informer model to start training and had to solve this problem as well. I figured it out by looking at the library code around where the torch.cat() is being called.

I’m now encountering the rather odd behavior of having negative loss show up during training. I’m quite curious if you encounter the same behavior after your training starts. [/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpositive-loss-value-changes-to-negative-loss-while-training-informer-or-timeseriestransformer-model%2F37304%5D(https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpositive-loss-value-changes-to-negative-loss-while-training-informer-or-timeseriestransformer-model%2F37304)

---

<div class="post-metadata">

**Author:** ![tdobrxl](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/tdobrxl/32/4762_2.png) [@tdobrxl](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Ftdobrxl)%5C
**Post date:** [April 25, 2023, 1:54pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsetting-up-a-timeseries-transformer%2F37083%2F3 "2023-04-25T13:54:22Z")

</div>

I had the same problem with Time series transformer.  
It’s a bug from the old version of transformers lib. Upgrading to 2.28 solved the problem for me.

---

<div class="post-metadata">

**Author:** ![gian-michele](https://avatars.discourse-cdn.com/v4/letter/g/5f9b8f/32.png) [@gian-michele](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fgian-michele)%5C
**Post date:** [February 8, 2024, 9:57am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsetting-up-a-timeseries-transformer%2F37083%2F4 "2024-02-08T09:57:12Z")

</div>

Hi,

I am not acquainted with the Informer network but if the Informer is optimised through Maximum-Likelihood on a continuous probability density ( student or normal), like the TimeSeries Transformer for prediction, a negative loss function can happen.  
It means the model is assigning very low fluctuation to the predictions, which makes the negative log-likelihood go negative.