# Time series for prediction: generate RuntimeError **URL:** **Author:** ![gian-michele](https://avatars.discourse-cdn.com/v4/letter/g/5f9b8f/32.png) [@gian-michele](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fgian-michele)%5C **Post date:** [February 8, 2024, 9:27pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftime-series-for-prediction-generate-runtimeerror%2F72489%2F1 "2024-02-08T21:27:37Z") Hello everyone, I trained a Time Series Transformer for prediction. I am trying to do something that is a bit unorthodox because my interest is not really forecasting, but rather learning the conditional distribution associated with a stationary time series with the objective of extrapolating its long-term autocorrelation function with precision. For this purpose, my inputs are 3-d historical trajectories of length `context_length=128` steps with no lags and I am trying to predict the subsequent first difference, the variation Delta X\_t if you wish, instead of the absolute value X\_t. The prediction length is therefore only 1. The NLL looks good, It becomes negative and stabilizes to a certain value after around 100 epochs. My problem is that when I try use the generate method, it gives me an error. The architecture config dict is the following: ```auto TimeSeriesTransformerConfig { "activation_dropout": 0.1, "activation_function": "gelu", "attention_dropout": 0.1, "cardinality": [ 0 ], "context_length": 128, "d_model": 32, "decoder_attention_heads": 1, "decoder_ffn_dim": 32, "decoder_layerdrop": 0.1, "decoder_layers": 1, "distribution_output": "normal", "dropout": 0.1, "embedding_dimension": [ 0 ], "encoder_attention_heads": 1, "encoder_ffn_dim": 32, "encoder_layerdrop": 0.1, "encoder_layers": 1, "feature_size": 11, "init_std": 0.02, "input_size": 3, "is_encoder_decoder": true, "lags_sequence": [ 0 ], "loss": "nll", "model_type": "time_series_transformer", "num_dynamic_real_features": 0, "num_parallel_samples": 1, "num_static_categorical_features": 0, "num_static_real_features": 1, "num_time_features": 1, "prediction_length": 1, "scaler": false, "scaling": "mean", "transformers_version": "4.37.2", "use_cache": true } ``` At the inference phase: ```auto def sample(self, history_input, features): past_time_features = torch.ones(history_input.size(0), history_input.size(1),1).cumsum(dim=1).to(history_input) past_obs_mask = torch.ones(history_input.size(0), history_input.size(1),3, dtype=torch.bool).to(history_input) future_time_features = 129*torch.ones(history_input.size(0), 1, 1).to(history_input) outputs = self.timeseries_transformer.generate( past_values=history_input, past_time_features=past_time_features, past_observed_mask=past_obs_mask, static_real_features=features[:,:self.modes + self.temp], future_time_features=future_time_features, ) return outputs ``` And the following error appears ```auto Traceback (most recent call last): File "", line 1, in File "/home/gmcherch/.cache/pypoetry/virtualenvs/neural-ar-p_BuPOL2-py3.9/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/home/gmcherch/.cache/pypoetry/virtualenvs/neural-ar-p_BuPOL2-py3.9/lib/python3.9/site-packages/transformers/models/time_series_transformer/modeling_time_series_transformer.py", line 1767, in generate decoder_input = torch.cat((reshaped_lagged_sequence, repeated_features[:, : k + 1]), dim=-1) RuntimeError: Sizes of tensors must match except in dimension 2. Expected size 128 but got size 1 for tensor number 1 in the list. ``` I don’t understand why this errors appear since it since I am feeding the correct tensors as inputs. Is there something I didn’t set properly in the configuration phase? I would like to thank anyone who could provide any useful advice. ---
**Author:** ![gian-michele](https://avatars.discourse-cdn.com/v4/letter/g/5f9b8f/32.png) [@gian-michele](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fgian-michele)%5C **Post date:** [February 10, 2024, 8:52pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftime-series-for-prediction-generate-runtimeerror%2F72489%2F2 "2024-02-10T20:52:19Z")
[EDIT] The issue was already filed and answered here: > > > \### System Info > > \- \`transformers\` version: 4.28.1 > \- Platform: Windows-10-10.0.1…9041-SP0 > \- Python version: 3.7.9 > \- Huggingface\_hub version: 0.13.4 > \- Safetensors version: not installed > \- PyTorch version (GPU?): 1.13.1+cu117 (True) > \- Tensorflow version (GPU?): not installed (NA) > \- Flax version (CPU?/GPU?/TPU?): not installed (NA) > \- Jax version: not installed > \- JaxLib version: not installed > \- Using GPU in script?: \ > \- Using distributed or parallel set-up in script?: \ > > \### Who can help? > > @ArthurZucker, @younesbelkada, @sgugger > > \### Information > > \- \[X\] The official example scripts > \- \[\] My own modified scripts > > \### Tasks > > \- \[\] An officially supported task in the \`examples\` folder (such as GLUE/SQuAD, ...) > \- \[\] My own task or dataset (give details below) > > \### Reproduction > > \`\`\` > import torch > from transformers import TimeSeriesTransformerConfig, TimeSeriesTransformerForPrediction > > > batch\_size = 32 > context\_length = 100 > prediction\_length = 1 > input\_size = 25 > num\_time\_features = 1 > > config = TimeSeriesTransformerConfig(prediction\_length=prediction\_length, > context\_length=context\_length, > input\_size=input\_size, > lags\_sequence=\[0\], > num\_time\_features=num\_time\_features, > num\_static\_categorical\_features=0, > num\_static\_real\_features=0, > num\_dynamic\_real\_features=0, > embedding\_dimension=64, > encoder\_ffn\_dim=32, > decoder\_ffn\_dim=32, > encoder\_attention\_heads=2, > decoder\_attention\_heads=2, > encoder\_layers=2, > decoder\_layers=2, > is\_encoder\_decoder=True, > activation\_function="gelu", > d\_model=64, > dropout=0.1, > encoder\_layerdrop=0.1, > decoder\_layerdrop=0.1, > attention\_dropout=0.1, > activation\_dropout=0.1, > num\_parallel\_samples=100, > init\_std=0.02 > ) > > model = TimeSeriesTransformerForPrediction(config) > > outputs = model.generate(past\_values=torch.empty((batch\_size, context\_length, input\_size)), > past\_time\_features=torch.empty((batch\_size, context\_length, num\_time\_features)), > past\_observed\_mask=torch.ones((batch\_size, context\_length, input\_size)), > future\_time\_features=torch.empty((batch\_size, prediction\_length, input\_size)), > ) > > print(outputs.keys()) > \`\`\` > > \`\`\` > File ".\\venv\\lib\\site-packages\\transformers\\models\\time\_series\_transformer\\modeling\_time\_series\_transformer.py", line 1807, in generate > decoder\_input = torch.cat((reshaped\_lagged\_sequence, repeated\_features\[:, : k + 1\]), dim=-1) > RuntimeError: Sizes of tensors must match except in dimension 2. Expected size 100 but got size 1 for tensor number 1 in the list. > \`\`\` > > Error in this row: > \`\`\` > decoder\_input = torch.cat((reshaped\_lagged\_sequence, repeated\_features\[:, : k + 1\]), dim=-1) > \`\`\` > > An attempt to concatenate tensors with dimensions \`\[3200, 100, 25\]\` and \`\[3200, 1, 75\]\`. > > \### Expected behavior > > I expected to get the correct result of the model.