zshhans / msd-mixer

A Multi-Scale Decomposition MLP-Mixer for Time Series Analysis

MSD-Mixer: A Multi-Scale Decomposition MLP-Mixer for Time Series Analysis

This is the PyTorch and Lightning implementation of our paper: A Multi-Scale Decomposition MLP-Mixer for Time Series Analysis. (https://arxiv.org/abs/2310.11959)

If you find this repo useful, please consider citing our paper:

@misc{zhong2023multiscale,
      title={A Multi-Scale Decomposition MLP-Mixer for Time Series Analysis}, 
      author={Shuhan Zhong and Sizhe Song and Guanyao Li and Weipeng Zhuo and Yang Liu and S. -H. Gary Chan},
      year={2023},
      eprint={2310.11959},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

Table of Contents

• Abstract
• Dependency Setup
• Dataset Preparation
• Run MSD-Mixer
• Baselines
• Acknoledgements

Abstract

Time series data, including univariate and multivariate ones, are characterized by unique composition and complex multi-scale temporal variations. They often require special consideration of decomposition and multi-scale modeling to analyze. Existing deep learning methods on this best fit to univariate time series only, and have not sufficiently considered sub-series level modeling and decomposition completeness. To address these challenges, we propose MSD-Mixer, a Multi-Scale Decomposition MLP-Mixer, which learns to explicitly decompose the input time series into different components, and represent the components in different layers. To handle the multi-scale temporal patterns and multivariate dependencies, we propose a novel temporal patching approach to model the time series as multi-scale sub-series (i.e., patches), and employ MLPs to capture intra- and inter-patch variations and channel-wise correlations. In addition, we propose a novel loss function to constrain both the magnitude and autocorrelation of the decomposition residual for better decomposition completeness. Through extensive experiments on various real-world datasets for five common time series analysis tasks, we demonstrate that MSD-Mixer consistently and significantly outperforms other state-of-the-art algorithms.

Dependency Setup

• Create a conda virtual environment

  conda create -n msd-mixer python=3.10
  conda activate msd-mixer

• Install Python Packages

  pip install -r requirements.txt

Dataset Preparation

Please download the datasets from dataset.zip, unzip the content into the dataset folder and structure the directory as follows:

/path/to/MSD-Mixer/dataset/
  electricity/
  ETT-small/
  exchange_rate/
  m4/
  MSL/
  Multivariate_ts/
  PSM/
  SMAP/
  SMD/
  SWaT/
  traffic/
  weather/

Run MSD-Mixer

Please use python main.py to run the experiments. Please use the -h or --help argument for details.

Long-Term Forecasting

Example training commands:

• Run all benchmarks

  python main.py ltf
  
  # equivalent
  python main.py ltf --dataset all --pred_len all

• Run specific benchmarks

  python main.py ltf --dataset etth1 etth2 --pred_len 96 192 336

Logs, results, and model checkpoints will be saved in /path/to/MSD-Mixer/logs/ltf

Short-Term Forecasting

Example training commands:

• Run all benchmarks

  python main.py stf
  
  # equivalent
  python main.py stf --dataset all

• Run specific benchmarks

  python main.py stf --dataset yearly quarterly monthly

Logs, results, and model checkpoints will be saved in /path/to/MSD-Mixer/logs/stf

Imputation

Example training commands:

• Run all benchmarks

  python main.py imp
  
  # equivalent
  python main.py imp --dataset all --mask_rate all

• Run specific benchmarks

  python main.py imp --dataset ecl ettm1 --mask_rate 0.25 0.5

Logs, results, and model checkpoints will be saved in /path/to/MSD-Mixer/logs/imp

Anomaly Detection

Example training commands:

• Run all benchmarks

  python main.py ad
  
  # equivalent
  python main.py ad --dataset all

• Run specific benchmarks

  python main.py ad --dataset smd msl swat

Logs, results, and model checkpoints will be saved in /path/to/MSD-Mixer/logs/ad

Classification

Example training commands:

• Run all benchmarks

  python main.py cls
  
  # equivalent
  python main.py cls --dataset all

• Run specific benchmarks

  python main.py cls --dataset awr scp1 scp2

Logs, results, and model checkpoints will be saved in /path/to/MSD-Mixer/logs/cls

Baselines

• For task-general baselines, including TimesNet, PatchTST, DLinear, LightTS, ETSformer, FEDformer, and NST, as well as N-BEATS and N-HiTS for short-term forecasting, and Anomaly Transformer for anomaly detection, we follow and reuse the unified implementations from Time Series Library (TSlib).

• For Scaleformer for long-term forecasting, we follow the official implementation from the Scaleformer paper.

• For TARNet, DTWD, TapNet, MiniRocket, and TST for classification, we follow the implementations from the TARNet paper. And for FormerTime for classification, we follow the implementation from the FormerTime paper

Acknowledgements

• Time Series Library (TSlib): datasets, experiment settings, and data processing
• UEA Time Series Classification datasets: datasets

msd-mixer's Issues

Packet Installing

Hi authors,
if I use miniconda instead of conda, would the experiments run normally without any troubles?
Thanks

help

Please make a notebook on how to run a model with Short-Term Forecasting