salesforce / etsformer Goto Github PK

Hi team,
great work first of all. I would like to use your framework for the domain of sports. I have a time series data of players in regards to different parameters. How would I use the data witht the current model? In what input format does the data have to be in? Is a dataframe sufficient that is grouped by player and his parameters? Can the model be trained on the data of an entire team and then predict a parameter of a given player?
Thank's a lot in advance!

I gain this error, when I try to rebuild this model in other library
File "...../models/ETSformer/encoder.py", line 87, in topk_freq x_freq = x_freq[index_tuple] RuntimeError: index does not support automatic differentiation for outputs with complex dtype.

I'm getting the following error.

Traceback (most recent call last):
  File "run.py", line 117, in <module>
    exp.train(setting)
  File "/nvme/git/ETSformer/exp/exp_main.py", line 140, in train
    dec_inp = torch.zeros_like(batch_y[:, -self.args.pred_len:, :]).float()
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

I believe this error is caused by the CUDA version being 11.7 (the newest), and is incompatible with the older torch version (1.11.0 specified in the requirements.txt).

I tried upgrading the torch version but found errors. Would you please update the torch code to the latest? I know that it breaks the transformer code as mentioned here:
pytorch/pytorch#80569

How does it handle holidays or other flags that might drive the data? It's not always visible for the Fourier transformation.

Hi, I'm trying to run ETTm2.sh script, but this is what I get. I've put ETTm2.csv file to dataset/ETT-small

Traceback (most recent call last):
  File "C:\python3\lib\site-packages\einops\einops.py", line 410, in reduce
    return _apply_recipe(recipe, tensor, reduction_type=reduction)
  File "C:\python3\lib\site-packages\einops\einops.py", line 233, in _apply_recipe
    _reconstruct_from_shape(recipe, backend.shape(tensor))
  File "C:\python3\lib\site-packages\einops\einops.py", line 163, in _reconstruct_from_shape_uncached
    raise EinopsError('Expected {} dimensions, got {}'.format(len(self.input_composite_axes), len(shape)))      
einops.EinopsError: Expected 2 dimensions, got 4

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Senti\Desktop\ETSformer\run.py", line 117, in <module>
    exp.train(setting)
  File "C:\Users\Senti\Desktop\ETSformer\exp\exp_main.py", line 144, in train
    outputs = self.model(batch_x, batch_x_mark, dec_inp, batch_y_mark)
  File "C:\python3\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\Senti\Desktop\ETSformer\models\etsformer\model.py", line 72, in forward
    level, growths, seasons = self.encoder(res, x_enc, attn_mask=enc_self_mask)
  File "C:\python3\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\Senti\Desktop\ETSformer\models\etsformer\encoder.py", line 169, in forward
    res, level, growth, season = layer(res, level, attn_mask=None)
  File "C:\python3\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\Senti\Desktop\ETSformer\models\etsformer\encoder.py", line 142, in forward
    growth = self._growth_block(res)
  File "C:\Users\Senti\Desktop\ETSformer\models\etsformer\encoder.py", line 151, in _growth_block
    x = self.growth_layer(x)
  File "C:\python3\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\Senti\Desktop\ETSformer\models\etsformer\encoder.py", line 38, in forward
    out = torch.cat([repeat(self.es.v0, 'h d -> b 1 h d', b=b), out], dim=1)
  File "C:\python3\lib\site-packages\einops\einops.py", line 537, in repeat
    return reduce(tensor, pattern, reduction='repeat', **axes_lengths)
  File "C:\python3\lib\site-packages\einops\einops.py", line 418, in reduce
    raise EinopsError(message + '\n {}'.format(e))
einops.EinopsError:  Error while processing repeat-reduction pattern "h d -> b 1 h d".
 Input tensor shape: torch.Size([1, 1, 8, 64]). Additional info: {'b': 32}.
 Expected 2 dimensions, got 4

This part of code seems to cause the problem:

   def forward(self, inputs):
       """
       :param inputs: shape: (batch, seq_len, dim)
       :return: shape: (batch, seq_len, dim)
       """
       b, t, d = inputs.shape
       values = self.in_proj(inputs).view(b, t, self.nhead, -1)
       values = torch.cat([repeat(self.z0, 'h d -> b 1 h d', b=b), values], dim=1)
       values = values[:, 1:] - values[:, :-1]
       out = self.es(values)
       out = torch.cat([repeat(self.es.v0, 'h d -> b 1 h d', b=b), out], dim=1)
       out = rearrange(out, 'b t h d -> b t (h d)')
       return self.out_proj(out)

Hi,

I was wondering if it is possible to use your model with a multivariate input while predicting a univariate variable. If not, do you know what code I should change to make it work? As you are using some of the code from Informer, I was thinking about using the 'MS' features parameter, but this gives the following error in the encoder on line 109:

level = level.view(b, t, self.c_out, 1)
RuntimeError: shape '[32, 192, 1, 1]' is invalid for input of size 36864

Now I could reshape this level variable so it would be consistent with my data, but I don't know if your model is capable of handling that. Please let me know what you think.

Thanks for your time and contribution,

Rico

Thanks for sharing, this series of timeseries models are really interesting. I especially like deeptime which works well for me (and I've tried adding multivariate, past only, inputs).

I particularly like the fact that you've testing on challenging multivariate weather and financial data. Many timeseries papers skip these difficult domains in favor of trivial problems. That's why I <3 Deeptime and the *Former papers.

I do have a question. I can't get ETSFormer to work. It seems like the current code mainly just predicts level, perhaps there is a bug in the uploaded code?

To replicate this I used a notebook and no substantial modifications. And you can see it's not predicting nice smooth ARIMA-like lines like in the paper. Instead it seems like it's all level with a tiny bit of growth in the first few steps. This happens at multiple lr's and with multiple datasets.

Am I missing something. Any ideas why this might be?

https://github.com/wassname/ETSformer/blob/w_notebook/notebook/run.ipynb

Hi, gorold. I gone through your repo i appreciate your work TSF. Please upload a sample notebook how to use the ETSformer. It will help a lot, learners like me.

Also please confirm ETSformer can be run on pc/laptop? If forecasting horizon less than 7-steps?

Thanks in advance! Your early reply awaiting...

Hello, thank you very much for your work. I have a few unclear points about the code：
（1）Why is the value of true.npy saved in the exp_main.py file not the same as the corresponding original value?
（2）How and where do I print the training set and test set?