About the step_loss == nan about svd_xtend

[pixeli99]

Could you please provide some more details, such as your specific settings, device information, and so on?

[maobenz]

Thanks a lot!

I tried different resolution of bdd images but all step_loss is nan. I just use one video clip of the bdd and split the videos into some images to be fed into the model. I have tried the GTX3090 and A100.

When I use the fp32 model , the step loss is not nan but fp16'model' s loss is still nan. However, in the last block of upsample_block, query @ key.transpose(-1, -2) is too large to show nan.

My model id is "stabilityai/stable-video-diffusion-img2vid-xt", but when i tried other model is , it also doesn't work.

[maobenz]

My torch version is 1.13.1+cu116, and my diffusers version is 0.25.0. Even if I input the all zeros as input, the loss is also nan.

[maobenz]

OK， i have found the issue, the torch version should be 2.0.1 rather than 1.13.1. When I change the version of pytorch, the problem has been solved.

[pixeli99]

Ah, I see, but in fact, I might not be able to answer why modifications to the PyTorch version would cause this issue.😢

[xiliu8006]

I upgraded the PyTorch to 2.1.2 but still has this problem, I can only train on the bf16. Any solutions?

[Sibo2rr]

I upgraded the PyTorch to 2.1.2 but still has this problem, I can only train on the bf16. Any solutions?

hi did you get any solutions? I get similar problem but the loss is nan