Prediction images don't change. about predrnn-pytorch

[leelew]

Actually, I think this is caused by that input X(t) have more information than h(t-1), c(t-1), thus the model tend to forecast a similar image with X(t) (i.e., true image of previous timestep). When we turn train mode to inference mode, the forecasting steps doesn't have corresponding true images from previous timesteps, thus they tend to forecast a similar images as prediction image of previous timestep. [This is define as difficulty in learning long-term dynamic problem in your paper].

Can you give me some suggestion?
Thanks!

[wuhaixu2016]

Hi, thanks for your interest.
(1) I think your case is quite difficult for prediction since the inputs show an accumulation trend (right-top) while the future tends to dissipate.
(2) To emphasis the motion, I think predicting the change between two adjacent frames might be helpful, that is X_{t}-X_{t-1}.
(3) Since this problem is quite challenging, maybe you can change the schedule sampling strategy as the input mask start from a high value, such as 0.8. This will not bring too much difficulty for prediction and still force the model to learn the long-term dependecies.

[leelew]

Thanks for your insightful suggestion!

[lsteffenel]

Hello, I'm facing the same problem (I'm using meteorological images), but I could not understand exactly which parameters to change.
I'm using the "moving mnist" predrnn_v2_mnist_train.sh as starting point, can you give me some hints on which parameters to change (and which values)?
Thank you!