y-levy / swa-in-ltsf

M.Sc Thesis

The recent boom of forecasters is fueling the ongoing pursuit for improvements through yet more architectural modifications. Ensemble learning stands as a popular alternative, renowned for its ability to oftentimes outperform any of its base-models. Indeed, deep ensembles have been applied successfully in time-series forecasting through various methods. Albeit, despite their considerable gains, prevailing methods suffer major computational overhead preventing them from widespread use. Stochastic Weight Averaging (SWA) is an ensemble method that averages weights along the optimization trajectory. Since its release, SWA has had many follow-up papers in a wide range of applications, to the best of our knowledge, excluding forecasting. This work, evaluates the performance of SWA on three selected transformer-based architectures for long-term time-series forecasting for the first time.