Exception: Reached maximum number of idle transformation calls. about pytorch-ts

[domainsense]

dear Kashif Rasul! When I run the 11th cell of Time-Grad-Electricity.ipynb with the 0.7.0 branch and updated gluonts, I encounter the following issue,. Could you please assist me again? the 11th cell: estimator = TimeGradEstimator( target_dim=int(dataset.metadata.feat_static_cat[0].cardinality), prediction_length=dataset.metadata.prediction_length, context_length=dataset.metadata.prediction_length, cell_type='GRU', input_size=1484, freq=dataset.metadata.freq, loss_type='l2', scaling=True, diff_steps=100, beta_end=0.1, beta_schedule="linear", trainer=Trainer(device=device, epochs=20, learning_rate=1e-3, num_batches_per_epoch=100, batch_size=64,) ) the issue:

…

--------------------------------------------------------------------------- ValidationError Traceback (most recent call last) Cell In[11], line 1 ----> 1 estimator = TimeGradEstimator( 2 target_dim=int(dataset.metadata.feat_static_cat[0].cardinality), 3 prediction_length=dataset.metadata.prediction_length, 4 context_length=dataset.metadata.prediction_length, 5 cell_type='GRU', 6 input_size=1484, 7 freq=dataset.metadata.freq, 8 loss_type='l2', 9 scaling=True, 10 diff_steps=100, 11 beta_end=0.1, 12 beta_schedule="linear", 13 trainer=Trainer(device=device, 14 epochs=20, 15 learning_rate=1e-3, 16 num_batches_per_epoch=100, 17 batch_size=64,) 18 ) File ~\Anaconda3\envs\msconda311\Lib\site-packages\gluonts\core\component.py:340, in validated.<locals>.validator.<locals>.init_wrapper(*args, **kwargs) 331 self, *args = args 333 nmargs = { 334 name: arg 335 for (name, param), arg in zip( (...) 338 if name != "self" 339 } --> 340 model = PydanticModel(**{**nmargs, **kwargs}) 342 # merge nmargs, kwargs, and the model fields into a single dict 343 all_args = {**nmargs, **kwargs, **model.__dict__} File ~\Anaconda3\envs\msconda311\Lib\site-packages\pydantic\main.py:341, in pydantic.main.BaseModel.__init__() ValidationError: 1 validation error for TimeGradEstimatorModel scheduler field required (type=value_error.missing) Thank you very much! 方义Fangyi ***@***.*** From: Kashif Rasul Date: 2023-08-18 16:03 To: zalandoresearch/pytorch-ts CC: opensource012; Author Subject: Re: [zalandoresearch/pytorch-ts] Exception: Reached maximum number of idle transformation calls. (Issue #145) pip install diffusers perhaps i forgot to add it to the setup.py — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[domainsense]

This is my runtime environment. The above issue still hasn't been resolved. 方义Fangyi ***@***.*** From: ***@***.*** Date: 2023-08-19 10:47 To: reply+ar4lkbq7xnghxoyz3qrnai6c5rmglevbnhhg5i7s6a Subject: Re: Re: [zalandoresearch/pytorch-ts] Exception: Reached maximum number of idle transformation calls. (Issue #145) dear Kashif Rasul! When I run the 11th cell of Time-Grad-Electricity.ipynb with the 0.7.0 branch and updated gluonts, I encounter the following issue,. Could you please assist me again? the 11th cell: estimator = TimeGradEstimator( target_dim=int(dataset.metadata.feat_static_cat[0].cardinality), prediction_length=dataset.metadata.prediction_length, context_length=dataset.metadata.prediction_length, cell_type='GRU', input_size=1484, freq=dataset.metadata.freq, loss_type='l2', scaling=True, diff_steps=100, beta_end=0.1, beta_schedule="linear", trainer=Trainer(device=device, epochs=20, learning_rate=1e-3, num_batches_per_epoch=100, batch_size=64,) ) the issue:

…

--------------------------------------------------------------------------- ValidationError Traceback (most recent call last) Cell In[11], line 1 ----> 1 estimator = TimeGradEstimator( 2 target_dim=int(dataset.metadata.feat_static_cat[0].cardinality), 3 prediction_length=dataset.metadata.prediction_length, 4 context_length=dataset.metadata.prediction_length, 5 cell_type='GRU', 6 input_size=1484, 7 freq=dataset.metadata.freq, 8 loss_type='l2', 9 scaling=True, 10 diff_steps=100, 11 beta_end=0.1, 12 beta_schedule="linear", 13 trainer=Trainer(device=device, 14 epochs=20, 15 learning_rate=1e-3, 16 num_batches_per_epoch=100, 17 batch_size=64,) 18 ) File ~\Anaconda3\envs\msconda311\Lib\site-packages\gluonts\core\component.py:340, in validated.<locals>.validator.<locals>.init_wrapper(*args, **kwargs) 331 self, *args = args 333 nmargs = { 334 name: arg 335 for (name, param), arg in zip( (...) 338 if name != "self" 339 } --> 340 model = PydanticModel(**{**nmargs, **kwargs}) 342 # merge nmargs, kwargs, and the model fields into a single dict 343 all_args = {**nmargs, **kwargs, **model.__dict__} File ~\Anaconda3\envs\msconda311\Lib\site-packages\pydantic\main.py:341, in pydantic.main.BaseModel.__init__() ValidationError: 1 validation error for TimeGradEstimatorModel scheduler field required (type=value_error.missing) Thank you very much! 方义Fangyi ***@***.*** From: Kashif Rasul Date: 2023-08-18 16:03 To: zalandoresearch/pytorch-ts CC: opensource012; Author Subject: Re: [zalandoresearch/pytorch-ts] Exception: Reached maximum number of idle transformation calls. (Issue #145) pip install diffusers perhaps i forgot to add it to the setup.py — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[kashif]

can you try with the 0.7.0 branch and updated gluonts?

[domainsense]

Very glad to meet you! I tried using the 0.7.0 branch and updated GluonTS, but I encountered this issue: File "D:\资料\计算机博士\工作及研究成果\基于Informer预测\参考文献\Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting\pytorch-ts-version-0.7.0\examples\pts\model\time_grad\estimator.py", line 44, in <module> from diffusers import SchedulerMixin ModuleNotFoundError: No module named 'diffusers' Process finished with exit code 1 Could you help me! Thank you! 方义Fangyi ***@***.*** From: Kashif Rasul Date: 2023-08-18 15:48 To: zalandoresearch/pytorch-ts CC: opensource012; Author Subject: Re: [zalandoresearch/pytorch-ts] Exception: Reached maximum number of idle transformation calls. (Issue #145) can you try with the 0.7.0 branch and updated gluonts? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[kashif]

pip install diffusers perhaps i forgot to add it to the setup.py

[domainsense]

Think you ！

…

---Original--- From: "Kashif ***@***.***&gt; Date: Fri, Aug 18, 2023 16:03 PM To: ***@***.***&gt;; Cc: ***@***.******@***.***&gt;; Subject: Re: [zalandoresearch/pytorch-ts] Exception: Reached maximum number ofidle transformation calls. (Issue #145) pip install diffusers perhaps i forgot to add it to the setup.py — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***&gt;

[xiyuanzh]

Hi @kashif, I want to use DeepVAR model which is not supported in 0.7.0 branch. If I use the default branch, I get the error of "Reached maximum number of idle transformation calls". Do you have any suggestions? Thanks a lot!

[ProRedCat]

Version 0.7.0 does not have the updated model creation for the TimeGrad example. You need to import a scheduler (the diffusion solver) from the diffusers library https://huggingface.co/docs/diffusers/api/schedulers/overview.

For the issue of "Reached maximum number of idle transformation calls." it's possible you do not have enough data, one solution is to just duplicate the data. Another issues is that if you're doing custom data you must ensure that the provided data is in the shape (input_size, timesteps) instead of (timesteps, input_size). It is possible that 0.7.0 has solved this issue as I have not encountered it since moving versions but I am using custom data.

Example on how to instantiate TimeGrad in 0.7.0

from diffusers import DEISMultistepScheduler

scheduler = DEISMultistepScheduler(
    num_train_timesteps=150,
    beta_end=0.1,
)

estimator = TimeGradEstimator(
    input_size=int(dataset.metadata.feat_static_cat[0].cardinality),
    hidden_size=64,
    num_layers=2,
    dropout_rate=0.1,
    lags_seq=[1],
    scheduler=scheduler,
    num_inference_steps=150,
    prediction_length=dataset.metadata.prediction_length,
    context_length=dataset.metadata.prediction_length,
    freq=dataset.metadata.freq,
    scaling="mean",
    trainer_kwargs=dict(max_epochs=200, accelerator="gpu", devices="1"),
)

[nonconvexopt]

@ProRedCat I get bad performance at version-0.7.0. Can you reproduce the similar performance recorded at timegrad-electricity notebook file?

[ProRedCat]

Could be the solver you're using, DEISMultistepSchedular is a fast ODE solver but may perform worse than some other solvers.

I was able to get a lower score on Electricity of 0.018 with the DEISMultistepSchedular but the number of epochs was set to 200. What sort of performance numbers are you getting?

[nonconvexopt]

@ProRedCat I used the DDPMScheduler but I ran only 20 epochs. That might be the reason. I will try with DEISMultistepSchedular and large epochs. Thanks.