haoyuc / a2n

PyTorch code for our paper "Attention in Attention Network for Image Super-Resolution"

Under construction...
If you have any question or suggestion, welcome to email me at here.

Attention in Attention Network for Image Super-Resolution (A2N)

This repository is an PyTorch implementation of the paper

"Attention in Attention Network for Image Super-Resolution" [arXiv]

Visual results in the paper are availble at Google Drive or Baidu Netdisk (password: 7t74).

Unofficial TensorFlow implementation: https://github.com/Anuj040/superres

Test

Dependecies: PyTorch==0.4.1 (Will be updated to support PyTorch>1.0 in the future)

You can download the test sets from Google Drive. Put the test data in ../Data/benchmark/.

python main.py  --scale 4 --data_test Set5 --pre_train ./experiment/model/aan_x4.pt --chop --test_only

If you use CPU, please add "--cpu".

Train

Training data preparation

1. Download DIV2K training data from DIV2K dataset or SNU_CVLab.
2. Specify '--dir_data' in option.py based on the data path.

For more informaiton, please refer to EDSR(PyTorch).

Training

# SR x2
python main.py --scale 2 --patch_size 128 --reset --chop --batch_size 32  --lr 5e-4

# SR x3
python main.py --scale 3 --patch_size 192 --reset --chop --batch_size 32  --lr 5e-4

# SR x4
python main.py --scale 4 --patch_size 256 --reset --chop --batch_size 32  --lr 5e-4

A2N-M (Recommended, fewer parameters)

For A2N-M, use 1x1 conv instead of 3x3 conv in non-attention branch, the code is here

Experiments

Enhanced and suppressed attention

Left: The most enhanced attention maps. Right: The most suppressed attention maps.

Visual results

Citation

If you have any question or suggestion, welcome to email me at here.

If you find our work helpful in your resarch or work, please cite the following papers.

@misc{chen2021attention,
      title={Attention in Attention Network for Image Super-Resolution}, 
      author={Haoyu Chen and Jinjin Gu and Zhi Zhang},
      year={2021},
      eprint={2104.09497},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Acknowledgements

This code is built on EDSR (PyTorch) and PAN. We thank the authors for sharing their codes.

a2n's Issues

关于MutiAdds的问题

作者您好！十分感谢您极具意义的工作，我想请教一个问题，文章中的MultiAdds计算是怎样的，和FLOPs的区别有什么区别吗？还有它的输入大小是多少，知道这些这让我感觉很重要，十分感谢！

Mismatch in the model architecture implementation and the one given in the paper

Thanks a lot for an interesting work on super-resolution and an easy to understand repository.
I am working to implement this repo in tensorflow (link). I noticed that the following skip connection is not shown in the Figure 3. of https://arxiv.org/abs/2104.09497.
Just wanted to confirm that.

A2N/code/model/aan.py

Line 152 in a04e78a

fea = fea + trunk

Thanks a lot for your time.
PS: If you feel my implementation worthy of reference, please consider that. Would be immensely grateful.

Problem with loading data

Thank you for sharing codes to us. While reproducing the code, I came across some minor issues that I would like to ask you for advice on, as follows:

I carefully checked the code and found that the output of the data loader had only three parts, namely hr, lr and filename, and I was puzzled as to why there were four parts (lr,hr,_,idx_scale) of output in the code.
Looking forward to your reply, thanks!

关于复现时的问题

您好，最近阅读了您的这篇论文，感觉非常受启发，想要尝试复现一下。请问是不是目前还是只能支持pytorch=0.4.1的版本，可以使用pytorch>1.0版本吗，谢谢。

Suggestion

Hi, @haoyuc
Thanks for your work
For the convenience of comparison, can you share the pre-trained model or benchmark results.
Looking for you work and good luck!

关于注意力分支的问题

您好，在看完您的文章后，我有一些不清楚的地方想向您请教一下，Attention Branch中使用的是通道-空间注意嘛？为什么我只能看到的是空间注意力机制。还有就是我不清楚，注意力机制的最后一步，为什么又加了一个卷积，K4的作用是什么呢？我的问题可能比较基础，希望您有空的话，可以解决一下我的疑惑。谢谢您的指教。

Code for heatmap in figure2

This type of heatmap has rarely been seen before. If possible, is there a code?

About pre-trained model

Thank you for your amazing work!
I would like to know if the pre-trained model is provided? I can't find it , sry.
And I notice that the methods of reconstruction module and Upsample module are nearest-neighbor interpolation and bilinear interpolation, respectively.Have you tried using sub-pixel related methods? And how is the result?
Thanks again.

Code for LAM in fig.8

Hello, can you provide the code in Figure 8? Thank you!

how to calculate the diffuision index

please can u let me know , where in your work have u mentioned about the diffusion index

对于注意力分支的选择

对您的工作很感兴趣，请问一下看您论文里写的 非注意和通道-空间注意的组合具有最好的表现，您代码中用的是通道-空间注意吗？还有就是请问一下 单纯采用通道注意力和非注意力的组合，加上您的dropout模块，效果会变得不好吗？dropout模块只能在非注意和通道-空间注意或空间注意的组合才有正面效果吗？ 谢谢您的指教

关于论文中图6的问题

您好，在看完您的文章后，我有一些不清楚的地方想向您请教一下，图六中关于两个分支的权重值，在网络训练时该如何呈现出来？或者在网络训练完毕后，如何得知每个块中对应的两个分支的权重值？希望您有空的话，可以解决一下我的疑惑。谢谢您的指教！！！