The cross attention module part I faced the problem of cross attention map about datasetdm

[Sunting78]

Have you solved ? I meet the same issue.

[hanyuxuan123]

Have you solved ? I meet the same issue.

No, I haven't solved. Now I am trying to replace the Unet and cross attention part with the single CNN part, I wonder maybe the Unet part has problems

[hanyuxuan123]

I have solved the problems above today, by changing several codes. Firstly, the diffusers in requirement.txt is not 0.14.2. You may download the 0.10.2 in model and tap into the diffuser folder in the code
and run "pip install ." to install diffuser 0.10.2.
Then the unet_2D_block.py has some problems. Especially the CrossAttnUp and Upsample block. You can refer to the CrossAttnDown and DownSample to change the code forward part.
Then the training code needs several changes. Later I will update the code I changed. I also test the final result, the decorder can perform well.

[T-S-Liang]

Hi there, thank you for opening up this issue and I encountered the same problem.

Would you mind sharing the code for fixing the problem? Thank you so much!

[T-S-Liang]

Hi guys,

Thank you for your assistance. I resolved the issue by referring to this comment.

The problem was due to a mismatch of the UNet blocks from diffusers. In newer versions, the "CrossAttention" module has been replaced with "Attention".

Thank you all again for your help!