TimesNet: Runtime error when trying to forecast more than 12 months

[elenamanth]

What happened + What you expected to happen

I am using the example provided in the documentation for the Times Net model that uses the Air passengers monthly data. While experimenting, i tried to increase the forecast horizon (h) above 12 months but i keep getting a Runtime error that the maximum size of tensor at dimension 2 is not enough. The model is working fine if i use h<= 12.

In the attached script i have used h=24 and input_size=48 and the error i get is the following:

---

RuntimeError                              Traceback (most recent call last)
Cell In[187], [line 20](vscode-notebook-cell:?execution_count=187&line=20)
      [4](vscode-notebook-cell:?execution_count=187&line=4) model = TimesNet(h=24,
      [5](vscode-notebook-cell:?execution_count=187&line=5)                  input_size=48,
      [6](vscode-notebook-cell:?execution_count=187&line=6)                  hidden_size = 16,
   (...)
     [13](vscode-notebook-cell:?execution_count=187&line=13)                  val_check_steps=50,
     [14](vscode-notebook-cell:?execution_count=187&line=14)                  early_stop_patience_steps=2)
     [16](vscode-notebook-cell:?execution_count=187&line=16) nf = NeuralForecast(
     [17](vscode-notebook-cell:?execution_count=187&line=17)     models=[model],
     [18](vscode-notebook-cell:?execution_count=187&line=18)     freq='M'
     [19](vscode-notebook-cell:?execution_count=187&line=19) )
---> [20](vscode-notebook-cell:?execution_count=187&line=20) nf.fit(df=Y_train_df, static_df=AirPassengersStatic, val_size=12)
     [21](vscode-notebook-cell:?execution_count=187&line=21) forecasts = nf.predict(futr_df=Y_test_df)

File [c:\Users\.venv\lib\site-packages\neuralforecast\core.py:486, in NeuralForecast.fit(self, df, static_df, val_size, sort_df, use_init_models, verbose, id_col, time_col, target_col, distributed_config)
    [483](file:///C:/Users/.venv/lib/site-packages/neuralforecast/core.py:483)     self._reset_models()
    [485](file:///C:/Users/.venv/lib/site-packages/neuralforecast/core.py:485) for i, model in enumerate(self.models):
--> [486](file:///C:/Users/.venv/lib/site-packages/neuralforecast/core.py:486)     self.models[i] = model.fit(
    [487](file:///C:/Users/.venv/lib/site-packages/neuralforecast/core.py:487)         self.dataset, val_size=val_size, distributed_config=distributed_config
    [488](file:///C:/Users/.venv/lib/site-packages/neuralforecast/core.py:488)     )
    [490](file:///C:/Users/.venv/lib/site-packages/neuralforecast/core.py:490) self._fitted = True

File [c:\Users\.venv\lib\site-packages\neuralforecast\common\_base_windows.py:661, in BaseWindows.fit(self, dataset, val_size, test_size, random_seed, distributed_config)
    [632](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:632) def fit(
    [633](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:633)     self,
    [634](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:634)     dataset,
   (...)
    [638](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:638)     distributed_config=None,
    [639](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:639) ):
    [640](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:640)     """Fit.
    [641](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:641) 
    [642](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:642)     The `fit` method, optimizes the neural network's weights using the
   (...)
    [659](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:659)     `test_size`: int, test size for temporal cross-validation.<br>
    [660](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:660)     """
--> [661](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:661)     return self._fit(
    [662](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:662)         dataset=dataset,
    [663](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:663)         batch_size=self.batch_size,
    [664](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:664)         valid_batch_size=self.valid_batch_size,
    [665](file:///C:/Users.venv/lib/site-packages/neuralforecast/common/_base_windows.py:665)         val_size=val_size,
    [666](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:666)         test_size=test_size,
    [667](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:667)         random_seed=random_seed,
    [668](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:668)         distributed_config=distributed_config,
    [669](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:669)     )

File [c:\Users\.venv\lib\site-packages\neuralforecast\common\_base_model.py:357, in BaseModel._fit(self, dataset, batch_size, valid_batch_size, val_size, test_size, random_seed, shuffle_train, distributed_config)
    [355](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_model.py:355) model = self
    [356](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_model.py:356) trainer = pl.Trainer(**model.trainer_kwargs)
--> [357](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_model.py:357) trainer.fit(model, datamodule=datamodule)
    [358](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_model.py:358) model.metrics = trainer.callback_metrics
    [359](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_model.py:359) model.__dict__.pop("_trainer", None)

File [c:\Users\.venv\lib\site-packages\pytorch_lightning\trainer\trainer.py:543, in Trainer.fit(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
    [541](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:541) self.state.status = TrainerStatus.RUNNING
    [542](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:542) self.training = True
--> [543](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:543) call._call_and_handle_interrupt(
    [544](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:544)     self, self._fit_impl, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path
    [545](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:545) )

File [c:\Users\.venv\lib\site-packages\pytorch_lightning\trainer\call.py:44, in _call_and_handle_interrupt(trainer, trainer_fn, *args, **kwargs)
     [42](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:42)     if trainer.strategy.launcher is not None:
     [43](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:43)         return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs)
---> [44](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:44)     return trainer_fn(*args, **kwargs)
     [46](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:46) except _TunerExitException:
     [47](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:47)     _call_teardown_hook(trainer)

File [c:\Users\.venv\lib\site-packages\pytorch_lightning\trainer\trainer.py:579, in Trainer._fit_impl(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
    [572](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:572) assert self.state.fn is not None
    [573](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:573) ckpt_path = self._checkpoint_connector._select_ckpt_path(
    [574](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:574)     self.state.fn,
    [575](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:575)     ckpt_path,
    [576](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:576)     model_provided=True,
    [577](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:577)     model_connected=self.lightning_module is not None,
    [578](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:578) )
--> [579](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:579) self._run(model, ckpt_path=ckpt_path)
    [581](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:581) assert self.state.stopped
    [582](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:582) self.training = False

File [c:\Users\.venv\lib\site-packages\pytorch_lightning\trainer\trainer.py:986, in Trainer._run(self, model, ckpt_path)
    [981](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:981) self._signal_connector.register_signal_handlers()
    [983](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:983) # ----------------------------
    [984](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:984) # RUN THE TRAINER
    [985](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:985) # ----------------------------
--> [986](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:986) results = self._run_stage()
    [988](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:988) # ----------------------------
    [989](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:989) # POST-Training CLEAN UP
    [990](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:990) # ----------------------------
    [991](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:991) log.debug(f"{self.__class__.__name__}: trainer tearing down")

File [c:\Users\.venv\lib\site-packages\pytorch_lightning\trainer\trainer.py:1028, in Trainer._run_stage(self)
   [1026](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1026) if self.training:
   [1027](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1027)     with isolate_rng():
-> [1028](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1028)         self._run_sanity_check()
   [1029](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1029)     with torch.autograd.set_detect_anomaly(self._detect_anomaly):
   [1030](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1030)         self.fit_loop.run()

File [c:\Users\.venv\lib\site-packages\pytorch_lightning\trainer\trainer.py:1057, in Trainer._run_sanity_check(self)
   [1054](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1054) call._call_callback_hooks(self, "on_sanity_check_start")
   [1056](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1056) # run eval step
-> [1057](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1057) val_loop.run()
   [1059](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1059) call._call_callback_hooks(self, "on_sanity_check_end")
   [1061](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1061) # reset logger connector

File [c:\Users\.venv\lib\site-packages\pytorch_lightning\loops\utilities.py:182, in _no_grad_context.<locals>._decorator(self, *args, **kwargs)
    [180](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/utilities.py:180)     context_manager = torch.no_grad
    [181](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/utilities.py:181) with context_manager():
--> [182](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/utilities.py:182)     return loop_run(self, *args, **kwargs)

File [c:\Users\.venv\lib\site-packages\pytorch_lightning\loops\evaluation_loop.py:135, in _EvaluationLoop.run(self)
    [133](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:133)     self.batch_progress.is_last_batch = data_fetcher.done
    [134](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:134)     # run step hooks
--> [135](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:135)     self._evaluation_step(batch, batch_idx, dataloader_idx, dataloader_iter)
    [136](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:136) except StopIteration:
    [137](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:137)     # this needs to wrap the `*_step` call too (not just `next`) for `dataloader_iter` support
    [138](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:138)     break

File [c:\Users\.venv\lib\site-packages\pytorch_lightning\loops\evaluation_loop.py:396, in _EvaluationLoop._evaluation_step(self, batch, batch_idx, dataloader_idx, dataloader_iter)
    [390](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:390) hook_name = "test_step" if trainer.testing else "validation_step"
    [391](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:391) step_args = (
    [392](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:392)     self._build_step_args_from_hook_kwargs(hook_kwargs, hook_name)
    [393](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:393)     if not using_dataloader_iter
    [394](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:394)     else (dataloader_iter,)
    [395](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:395) )
--> [396](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:396) output = call._call_strategy_hook(trainer, hook_name, *step_args)
    [398](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:398) self.batch_progress.increment_processed()
    [400](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:400) if using_dataloader_iter:
    [401](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:401)     # update the hook kwargs now that the step method might have consumed the iterator

File [c:\Users\.venv\lib\site-packages\pytorch_lightning\trainer\call.py:311, in _call_strategy_hook(trainer, hook_name, *args, **kwargs)
    [308](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:308)     return None
    [310](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:310) with trainer.profiler.profile(f"[Strategy]{trainer.strategy.__class__.__name__}.{hook_name}"):
--> [311](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:311)     output = fn(*args, **kwargs)
    [313](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:313) # restore current_fx when nested context
    [314](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:314) pl_module._current_fx_name = prev_fx_name

File [c:\Users\.venv\lib\site-packages\pytorch_lightning\strategies\strategy.py:411, in Strategy.validation_step(self, *args, **kwargs)
    [409](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/strategies/strategy.py:409) if self.model != self.lightning_module:
    [410](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/strategies/strategy.py:410)     return self._forward_redirection(self.model, self.lightning_module, "validation_step", *args, **kwargs)
--> [411](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/strategies/strategy.py:411) return self.lightning_module.validation_step(*args, **kwargs)

File [c:\Users\.venv\lib\site-packages\neuralforecast\common\_base_windows.py:492, in BaseWindows.validation_step(self, batch, batch_idx)
    [489](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:489)     return np.nan
    [491](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:491) # TODO: Hack to compute number of windows
--> [492](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:492) windows = self._create_windows(batch, step="val")
    [493](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:493) n_windows = len(windows["temporal"])
    [494](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:494) y_idx = batch["y_idx"]

File [c:\Users\.venv\lib\site-packages\neuralforecast\common\_base_windows.py:248, in BaseWindows._create_windows(self, batch, step, w_idxs)
    [245](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:245)     padder_right = nn.ConstantPad1d(padding=(0, self.h), value=0)
    [246](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:246)     temporal = padder_right(temporal)
--> [248](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:248) windows = temporal.unfold(
    [249](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:249)     dimension=-1, size=window_size, step=predict_step_size
    [250](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:250) )
    [252](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:252) # [batch, channels, windows, window_size] 0, 1, 2, 3
    [253](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:253) # -> [batch * windows, window_size, channels] 0, 2, 3, 1
    [254](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:254) windows_per_serie = windows.shape[2]

RuntimeError: maximum size for tensor at dimension 2 is 60 but size is 72

Versions / Dependencies

pandas==1.5.3
torch==2.0.0
neuralforecast==1.7.3
numpy==1.23.5
python== 3.8.10

Reproduction script

import numpy as np
import pandas as pd
from neuralforecast import NeuralForecast
from neuralforecast.losses.pytorch import DistributionLoss
from neuralforecast.utils import AirPassengers, AirPassengersPanel, AirPassengersStatic, augment_calendar_df
from neuralforecast.models import TimesNet  

#### Augment the DataFrame with calendar features
AirPassengersPanel, calendar_cols = augment_calendar_df(df=AirPassengersPanel, freq='M')

#### Split the data into training and testing sets
Y_train_df = AirPassengersPanel[AirPassengersPanel.ds < AirPassengersPanel['ds'].values[-24]]
Y_test_df = AirPassengersPanel[AirPassengersPanel.ds >= AirPassengersPanel['ds'].values[-24]].reset_index(drop=True)

#### TimesNet model
model = TimesNet(h=24,
                 input_size=48,
                 hidden_size = 16,
                 conv_hidden_size = 32,
                 loss=DistributionLoss(distribution='Normal', level=[80, 90]),
                 futr_exog_list=calendar_cols,
                 scaler_type='standard',
                 learning_rate=1e-3,
                 max_steps=5,
                 val_check_steps=50,
                 early_stop_patience_steps=2)

nf = NeuralForecast(
    models=[model],
    freq='M'
)
nf.fit(df=Y_train_df, static_df=AirPassengersStatic, val_size=12)
forecasts = nf.predict(futr_df=Y_test_df)

Issue Severity

High: It blocks me from completing my task.

[marcopeix]

I'm able to reproduce the bug, but not sure why it happens. Will have to investigate further when I have more time!