atik-ahamed / timemachine Goto Github PK

View Code? Open in Web Editor NEW

137.0 137.0 17.0 236 KB

TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecasting

Home Page: https://arxiv.org/abs/2403.09898

License: Apache License 2.0

Python 78.54% Shell 21.46%

time-series-forecasting

timemachine's Issues

not able to achieve the same excellent results

Hello, I appreciate the compliment on your ideas, but I have run the etth1.sh file using the parameters you provided without any modifications, and I'm not able to achieve the same excellent results as described in your paper.

$MTOBUUYSF)7F){S_O2R$C$

问题求助

为什么这个代码运行的很慢了，是原本就这样吗？

Nan loss！

I run the script "sh ./scripts/TimeMachine/weather.sh"，and at the second training epoch, it occured nan training loss problem.

I print the per step loss, can you help me solve this problem?

TimeMachine_weather_96_96_128_16_0.1_1_1_256_2_1.log

debug

How to install causal conv1d and mamba-ssm

How to install causal conv1d and mamba-sms

What data shape could be mamba's correct input?

import torch
from mamba_ssm import Mamba

batch, length, dim = 2, 64, 16
x = torch.randn(batch, length, dim).to("cuda")
model = Mamba(
# This module uses roughly 3 * expand * d_model^2 parameters
d_model=dim, # Model dimension d_model
d_state=16, # SSM state expansion factor
d_conv=4, # Local convolution width
expand=2, # Block expansion factor
).to("cuda")
y = model(x)
assert y.shape == x.shape
This is the official introduction of mamba-ssm,so i think the input shape should be B L d,while in timemachine,the input data's shape is B M ni
I am confused at this about this,could you please provide some guidance for me.Thank you so much for your time and assistance.

Multi-GPU Execution Error: Invalid device id

First off, thank you for your excellent work on this project! I've encountered an issue while attempting to utilize the multi-GPU functionality. Specifically, the program runs successfully when GPUs are specified in a sequential order starting from zero using the --devices parameter (e.g., "0,1,2"). However, errors occur when the GPUs are specified in a non-sequential order or do not start with zero (e.g., "0,2" or "1,2,3").

Full error message:

Traceback (most recent call last):
  File "/home/mateus/repos/forks/TimeMachine/TimeMachine_supervised/run_longExp.py", line 117, in <module>
    exp = Exp(args)  # set experiments
  File "/home/mateus/repos/forks/TimeMachine/TimeMachine_supervised/exp/exp_main.py", line 24, in __init__
    super(Exp_Main, self).__init__(args)
  File "/home/mateus/repos/forks/TimeMachine/TimeMachine_supervised/exp/exp_basic.py", line 10, in __init__
    self.model = self._build_model().to(self.device)
  File "/home/mateus/repos/forks/TimeMachine/TimeMachine_supervised/exp/exp_main.py", line 33, in _build_model
    model = nn.DataParallel(model, device_ids=self.args.device_ids)
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/nn/parallel/data_parallel.py", line 159, in __init__
    _check_balance(self.device_ids)
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/nn/parallel/data_parallel.py", line 26, in _check_balance
    dev_props = _get_devices_properties(device_ids)
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/_utils.py", line 734, in _get_devices_properties
    return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids]
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/_utils.py", line 734, in <listcomp>
    return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids]
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/_utils.py", line 713, in _get_device_attr
    return get_member(torch.cuda)
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/_utils.py", line 734, in <lambda>
    return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids]
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/cuda/__init__.py", line 452, in get_device_properties
    raise AssertionError("Invalid device id")
AssertionError: Invalid device id
Traceback (most recent call last):
  File "/home/mateus/repos/forks/TimeMachine/TimeMachine_supervised/run_longExp.py", line 117, in <module>
    exp = Exp(args)  # set experiments
  File "/home/mateus/repos/forks/TimeMachine/TimeMachine_supervised/exp/exp_main.py", line 24, in __init__
    super(Exp_Main, self).__init__(args)
  File "/home/mateus/repos/forks/TimeMachine/TimeMachine_supervised/exp/exp_basic.py", line 10, in __init__
    self.model = self._build_model().to(self.device)
  File "/home/mateus/repos/forks/TimeMachine/TimeMachine_supervised/exp/exp_main.py", line 33, in _build_model
    model = nn.DataParallel(model, device_ids=self.args.device_ids)
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/nn/parallel/data_parallel.py", line 159, in __init__
    _check_balance(self.device_ids)
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/nn/parallel/data_parallel.py", line 26, in _check_balance
    dev_props = _get_devices_properties(device_ids)
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/_utils.py", line 734, in _get_devices_properties
    return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids]
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/_utils.py", line 734, in <listcomp>
    return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids]
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/_utils.py", line 713, in _get_device_attr
    return get_member(torch.cuda)
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/_utils.py", line 734, in <lambda>
    return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids]
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/cuda/__init__.py", line 452, in get_device_properties
    raise AssertionError("Invalid device id")
AssertionError: Invalid device id
Traceback (most recent call last):
  File "/home/mateus/repos/forks/TimeMachine/TimeMachine_supervised/run_longExp.py", line 117, in <module>
    exp = Exp(args)  # set experiments
  File "/home/mateus/repos/forks/TimeMachine/TimeMachine_supervised/exp/exp_main.py", line 24, in __init__
    super(Exp_Main, self).__init__(args)
  File "/home/mateus/repos/forks/TimeMachine/TimeMachine_supervised/exp/exp_basic.py", line 10, in __init__
    self.model = self._build_model().to(self.device)
  File "/home/mateus/repos/forks/TimeMachine/TimeMachine_supervised/exp/exp_main.py", line 33, in _build_model
    model = nn.DataParallel(model, device_ids=self.args.device_ids)
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/nn/parallel/data_parallel.py", line 159, in __init__
    _check_balance(self.device_ids)
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/nn/parallel/data_parallel.py", line 26, in _check_balance
    dev_props = _get_devices_properties(device_ids)
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/_utils.py", line 734, in _get_devices_properties
    return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids]
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/_utils.py", line 734, in <listcomp>
    return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids]
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/_utils.py", line 713, in _get_device_attr
    return get_member(torch.cuda)
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/_utils.py", line 734, in <lambda>
    return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids]
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/cuda/__init__.py", line 452, in get_device_properties
    raise AssertionError("Invalid device id")
AssertionError: Invalid device id
Traceback (most recent call last):
  File "/home/mateus/repos/forks/TimeMachine/TimeMachine_supervised/run_longExp.py", line 117, in <module>
    exp = Exp(args)  # set experiments
  File "/home/mateus/repos/forks/TimeMachine/TimeMachine_supervised/exp/exp_main.py", line 24, in __init__
    super(Exp_Main, self).__init__(args)
  File "/home/mateus/repos/forks/TimeMachine/TimeMachine_supervised/exp/exp_basic.py", line 10, in __init__
    self.model = self._build_model().to(self.device)
  File "/home/mateus/repos/forks/TimeMachine/TimeMachine_supervised/exp/exp_main.py", line 33, in _build_model
    model = nn.DataParallel(model, device_ids=self.args.device_ids)
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/nn/parallel/data_parallel.py", line 159, in __init__
    _check_balance(self.device_ids)
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/nn/parallel/data_parallel.py", line 26, in _check_balance
    dev_props = _get_devices_properties(device_ids)
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/_utils.py", line 734, in _get_devices_properties
    return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids]
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/_utils.py", line 734, in <listcomp>
    return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids]
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/_utils.py", line 713, in _get_device_attr
    return get_member(torch.cuda)
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/_utils.py", line 734, in <lambda>
    return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids]
  File "/home/mateus/miniconda3/envs/TimeMachine_supervised/lib/python3.10/site-packages/torch/cuda/__init__.py", line 452, in get_device_properties
    raise AssertionError("Invalid device id")
AssertionError: Invalid device id

My etth1.sh file:

if [ ! -d "./logs" ]; then
    mkdir ./logs
fi

if [ ! -d "./logs/LongForecasting" ]; then
    mkdir ./logs/LongForecasting
fi
if [ ! -d "./csv_results" ]; then
    mkdir ./csv_results
fi
if [ ! -d "./results" ]; then
    mkdir ./results
fi
if [ ! -d "./test_results" ]; then
    mkdir ./test_results
fi
model_name=TimeMachine

root_path_name=../data/ETT-small
data_path_name=ETTh1.csv
model_id_name=ETTh1
data_name=ETTh1

rin=1
random_seed=2024
one=96
two=192
three=336
four=720
residual=1
fc_drop=0.7
dstate=256
dconv=2
for seq_len in 96
do
    for pred_len in 96 192 336 720
    do  
        for e_fact in 1
        do

            if [ $pred_len -eq $one ]
            then
                n1=512
                n2=32
            fi
            if [ $pred_len -eq $two ]
            then
                n1=512
                n2=64
            fi
            if [ $pred_len -eq $three ]
            then
                n1=512
                n2=128
            fi
            if [ $pred_len -eq $four ]
            then
                n1=128
                n2=16
            fi
            python -u run_longExp.py \
            --random_seed $random_seed \
            --is_training 1 \
            --root_path $root_path_name \
            --data_path $data_path_name \
            --model_id $model_id_name_$seq_len'_'$pred_len \
            --model $model_name \
            --data $data_name \
            --features M \
            --seq_len $seq_len \
            --pred_len $pred_len \
            --enc_in 7 \
            --n1 $n1 \
            --n2 $n2 \
            --dropout $fc_drop\
            --revin 1\
            --ch_ind 1\
            --residual $residual\
            --dconv $dconv \
            --d_state $dstate\
            --e_fact $e_fact\
            --des 'Exp' \
            --train_epochs 100\
            --itr 1 --batch_size 64 --learning_rate 0.001 --use_multi_gpu --devices  "1,2" >logs/LongForecasting/$model_name'_'$model_id_name'_'$seq_len'_'$pred_len'_'$n1'_'$n2'_'$fc_drop'_'$rin'_'$residual'_'$dstate'_'$dconv'_'$e_fact.log 
        
        done        
    done
done

I appreciate any guidance or updates you can provide on this issue. Thank you!

When I was running the code, I encountered the following problems, could you help me solve them? I used python=3.10, torch=2.0.1,cuda=11.8. Thank you very much import selective_scan_cuda ImportError: /home/hp/miniforge3/envs/mamba/lib/python3.10/site-packages/selective_scan_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda9SetDeviceEi

The training time of the model

Hi, I recently run the code and the MSE results(and so on) is OK. However, although the model costs relatively low gpu memory, its training speed is really really slow, have you ever encountered a similar situation? Take Weather_96_96 as an example, TimeMachine costs 380s for one epoch while PatchTST only needs 40s...

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.