liqiufu / wavecnet

the code for WaveCNet

WaveCNet [paper1][paper2]

The WaveCNet is designed using DWT and the commonly used CNN networks in PyTorch: https://pytorch.org/docs/stable/torchvision/models.html#classification

The main.py is revised on the PyTorch image classification code: https://github.com/pytorch/examples/blob/42e5b996718797e45c46a25c55b031e6768f8440/imagenet/main.py#L89-L101

WaveUNet for image segmentation has been renamed as WaveSNet: https://github.com/LiQiufu/WaveSNet

(0) Paper Title

Wavelet Integrated CNNs for Noise-Robust Image Classification

(1) Training WaveCNet on ImageNet

 CUDA_VISIBLE_DEVICES=0 python main.py --data /PYTHON_TO_IMANGENET -a resnet18_dwt -b 256 -w bior3.3 --gpu 0 --lr 0.1

(2) The trained weight files

The pretrained weight files have be uploaded on the website: https://pan.baidu.com/s/1T8vrOd0Zq8jv9fT5b7jdcQ (passwords: auc9)

(3) The paper

If the code or method help you in the research, please cite the following paper:

@inproceedings{li2020wavelet,
  title={Wavelet integrated CNNs for noise-robust image classification},
  author={Li, Qiufu and Shen, Linlin and Guo, Sheng and Lai, Zhihui},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={7245--7254},
  year={2020}
}

@article(qiufu_2021_TIP,
author = {Li, Qiufu and Shen, Linlin and Guo, Sheng and Lai, Zhihui},
title = {WaveCNet: Wavelet Integrated CNNs to Suppress Aliasing Effect for Noise-Robust Image Classification},
booktitle = {IEEE Transactions on Image Processing (TIP)},
month = {july},
year = {2021},
pages = {7074--7089},
vol = {30}
}

wavecnet's Issues

Is the WaveUNet code available?

Thank you for publishing this paper and making your DWT and IDWT layer code available. I am interested in wavelet transforms and previously explored their use in CNNs for cloud detection (related paper link). I have begun testing your provided sample networks with low-frequency-only DWT downsampling. Would it be possible for you to also upload an example WaveUNet model showcasing how you performed the inverse transform? Thanks again!

The code of Wavelet denoising block?

Dear author, thanks for your contribution.

I'm interested in Soft Thresholding, so I want to know whether it is possible to publish the code of Wavelet denoising block?

Thanks in advance!

About dataset processing issues?

Hello, It's very nice of you to share this code. I am very sorry because I can't find any code about dataset processing.I dowmload the ILSVRC2012 dataset and unzip the "ILSVRC2012_img_train.tar", "ILSVRC2012_img_val.tar", "ILSVRC2012_img_val.tar", respectively. How can I do next? And What is the data format of a in your folder“/raid/liqiufu/DATA/ILSVRC2012/train” and “/raid/liqiufu/DATA/ILSVRC2012/val”? Where do I enter the label of the picture in the code? Thank you very much.

Is the pretrained checkpoint based on pool_only=True or =False?

Hi. Is the pretrained checkpoint trained with pool_only=True or =False?

About backward propagation issue

Hello, i have some questions:
1, Are the gradient calculation methods of WResNet18_A and WResNet18_C same? Are these methods all in this way of "grad_input = torch.add(torch.matmul(grad_L, matrix_L), torch.matmul(grad_H, matrix_H))" ?
2, If so, why is it not divided by 2 in the WResNet18_A?

why in DWT_IDWT\DWT_IDWT_layer.py DWT using rec_lo and rec_hi ?

thx so much for sharing, i have a question:
why in DWT_IDWT\DWT_IDWT_layer.py, DWT is using rec_lo and rec_hi,
while in IDWT, using dec_lo/hi and then reverse ?
i thought that in DWT with dec_lo/hi and IDWT with rec_lo/hi,
cuz DWT is the decomposition process and IDWT is the reconstruction process.

may i have the requirements.txt?

I want to build a same environment of python to try this model

Replacing average pooling with the DWTll downsampling in ResNet models

Hi
I noticed in your paper that in WaveCnets, you mentioned replacing the conventional pooling layers with the DWTll layer. However, in the ResNet models, I observed that the average pooling layer before the fully connected layer has not been replaced by the DWTll layer. I am curious to know if there is a specific reason for this?

Thank you in advance for your response.

RuntimeError: Tensor for 'out' is on CPU, Tensor for argument #1 'self' is on CPU, but expected them to be on GPU (while checking arguments for addmm)

when I use this code in my own object,using the dwtlayer and dwtfunction, I met the problem"RuntimeError: Tensor for 'out' is on CPU, Tensor for argument #1 'self' is on CPU, but expected them to be on GPU (while checking arguments for addmm)"
The main problem is located in the class DWTFunction_2D_tiny(Function):L = torch.matmul(matrix_Low_0, input),
please tell me what I can do.
Thank you very much!

Is the WResNet18_A and WResNet18_C code available?

Hello,I am so excited when I meet this paper. Especially the design of DWT and IDWT layer. And I am courious ahout the control experiment, like WResNet18_A and WResNet18_C. If that's alright, could you please upload the code of WResNet18_A and WResNet18_C. Thank you so much.

Unable to download weight files

I am currently working on a project that requires the network weights for models like VGG, ResNet, etc., which I understand are available on Baidu Netdisk. However, it seems the access password has expired. Could you kindly assist by providing a new password or an alternative download method? My email is [email protected]. Any help would be greatly appreciated!

strided-convolution replaced by DWT operation

Traceback (most recent call last):
...
x1 = self.layer1(x)
...
out += identity
RuntimeError: The size of tensor a (16) must match the size of tensor b (64) at non-singleton dimension 3

thank you for this great work,
I tried to use resnext50_32x4d,
but it showed me this error...

How to robustify CNNs by using DWTll denoising layer against adversarial examples available in Adversarial Robustness Toolbox?

Hello,
I want to use DWTll denoising layer in an image classification CNN from Adversarial Robustness Toolbox (ART) classifiers to evaluate its robustness against different types of adversarial examples in this toolbox. I want to ask if you could suggest me how to replace downsampling layers with this DWTll denoising layer in an MNIST (or any other dataset) image classification CNN model available in ART? or maybe perform adversarial attacks available in ART on a noise robust pretrained image classification model in your repository, if it is possible?
here is a link to an example that performs FGSM adversarial attack on a simple PyTorch MNIST image classification CNN by ART toolbox.
Thanks a lot in advance.

feature maps

I would like to ask how to draw such beautiful feature maps of CNNs and WaveCNets as the article?
Does the code to draw the feature maps exist in the main.py/calidate_save function?
Looking forward to your reply, thanks!

Unable to download weight files from baidu

Hello, I am studying this paper for a course project, and was trying to run the code. I wanted to download the weights for the network and went to baidu for the same, however I am facing issues downloading the weights. I tried to download baidu and create an account also, but that did not work. Can you guide me on how to do so? Thanks!

About visual_results Issues

I am very sorry because I did not find any code for visual_results. If there is such code in the project, could you show it out for me?

Problem about code running

After running the code, the terminal shows "I AM HERE", and then there is no feedback. What is the problem? Looking forward to your reply！

No IDWT layer used in main procedure

Thanks a lot for sharing.
Looks like in the paper, there are three steps:

1. Do DWT operation
2. Directly forward low-frequency elements, and filter high-frequency elements.
3. IDWT to reconstruct the signal.

My question is how to combine the IDWT operation with the end-to-end CNNs?

Thanks & Regards!
Momo