hxlh50k / u-net-transformer Goto Github PK

Hello:)
Thank you so much for sharing your code. It has been very useful in understanding the paper.

There is still something I don't quite get from the paper and the code. It From my understanding, the paper mentioned that the attention is calculated separately on multiple heads before being combined. However, I could not find how many heads are used in the code (not sure if I am confused about multi-head part). Could you point to where you defined the number of heads? I understand that in the paper, they did an experiment to analyse the number of attention heads for MHSA from 1 to 8.

Thank you.
Charlene

I want to know serious bugs is what?

Hi! Thank you for sharing the wonderful job.

When I ran the train.py, I got the errors like the following:

Traceback (most recent call last):
File "train.py", line 12, in
from torch.cuda import amp
ImportError: cannot import name 'amp' from 'torch.cuda' (/usr/local/lib/python3.7/dist-packages/torch/cuda/init.py)

I searched a lot and tried to find the resolution. It seems there is an issue with the version compatibility. What are your PyTorch version and CUDA version?
Thanks a lot!

Could you provide the training weights？

Have you ever done some test or on the dataset of official paper?

Thank you for sharing the code. I'd like to ask a question regarding the MHSA module. In the original text, the output description seems to suggest attention weights, but why does the code directly output the result of V multiplied by attention?