liyingwei / shapetexturedebiasedtraining

Code and models for the paper Shape-Texture Debiased Neural Network Training (ICLR 2021)

Shape-Texture Debiased Neural Network Training

Code and models for the paper Shape-Texture Debiased Neural Network Training (ICLR 2021).

Introduction

Convolutional Neural Networks are often biased towards either texture or shape, depending on the training dataset (Geirhos et al. 2019). Our ablation shows that such bias degenerates model performance. Motivated by this observation, we develop a simple algorithm for shape-texture debiased learning.

Experiments show that our method successfully improves model performance on several image recognition benchmarks and adversarial robustness. For example, by training on ImageNet, it helps ResNet-152 achieve substantial improvements on ImageNet (+1.2%), ImageNet-A (+5.2%), ImageNet-C (+8.3%) and Stylized-ImageNet (+11.1%), and on defending against FGSM adversarial attacker on ImageNet (+14.4%). Our method also claims to be compatible to other advanced data augmentation strategies, eg, Mixup and CutMix.

Dependencies:

• PyTorch = 1.4.0 with GPU support

Model Zoo:

Shape-Texture Debiased Models	ImageNet (Top-1 Acc.)
ResNet-50 ⬇️	76.9
ResNet-101 ⬇️	78.9
ResNet-152 ⬇️	79.8
Mixup-ResNeXt-101 ⬇️	80.5
CutMix-ResNeXt-101 ⬇️	81.2

We also provide the checkpoint for the Shape-Biased ResNet-50 ⬇️ and the Texture-Biased ResNet-50 ⬇️. See issue#2 for more details.

Training & Testing:

Please see the Training recipes / Testing recipes for how to train / test the models.

Acknowledgements

Part of this code comes from pytorch-classification and AdaIN.

Citation

If you use our code, models or wish to refer to our results, please use the following BibTex entry:

@article{li2020shapetexture,
  author  = {Li, Yingwei and Yu, Qihang and Tan, Mingxing and Mei, Jieru and Tang, Peng and Shen, Wei and Yuille, Alan and Xie, Cihang},
  title   = {Shape-Texture Debiased Neural Network Training},
  journal = {arXiv preprint arXiv:2010.05981},
  year    = {2020}
}

shapetexturedebiasedtraining's Issues

FGSM Attack

Hey, could you provide the schedule of how you tested the FGSM attack against your model?
Did you generate from all ImageNet Test images and run afterwards the classification task? Or did you use another procedure?

Thanks! :)

Shape and Texture biased pretrain models

Thank you for your exciting paper and nice script! I am interested in how the shape/texture biased model perform and would like to do some experiments on those biased models.
I wonder if you save shape/texture biased model (label-gamma=[1,0])? If possible, would you mind share a ResNet50 texture/shape biased pretrain model with me?
Thank you very much! : )

imagenet-a

hello,i have a quession,if you test your model in imagenet-a directly?i find the classes in imagenet-a is different from imagenet-200?how do you slove it?

Question about training dataset

I am very interesting in your debiased model and trying to reproduce the results. However I only get 65% top1 accuracy for ResNet50 model. I also tried the pre-trained ResNet50 model provided by the repo, and I only get 0.02% top1 accuracy. So I wonder if I am using the wrong dataset. Here are what I tried.

Dataset: Stanford CS231N's tiny-ImageNet-200
Training with command:
python imagenet.py -a resnet50 --data $DATA --epochs 100 --schedule 30 60 90 --checkpoint $MODEL_DIR --gpu-id 0 --train-batch 64 --lr 0.2 --mixbn --style --multi_grid
Evaluate with command:
python imagenet.py -a resnet50 --data $DATA --mixbn --evaluate --load $CKPT
Thank you :)

RuntimeError: No CUDA GPUs are available

Hi @LiYingwei hope that is well.

When I run torch.cuda.is_available() it shows true. And Cuda is always work fine in all projects that need GPU. However, in this project when I run python imagenet.py -a resnet50 --data %DATA% --mixbn --evaluate --load %CKPT% , it returns RuntimeError: No CUDA GPUs are available

Could you please give me some clue?
Thanks

StyleTransfer problem

Hello, when I tried to reproduce your work, the output of the style transfer module often has noise, I have tried to use a smaller alpha as the style transformation parameter, but it doesn't work at all. I want to know whether you have met this problem, and if so how can you solve this.
Thanks for your reading!

CAM to visualize which image regions are attended by models

Hi @LiYingwei

Thanks for the repo, and codes. I was reading the paper and had a look at the codes as well. Specially the visualization part. I was wondering which file is related to plotting figures like Figur3 of paper where you mention to CAM. Is it the calorized function?

Thanks!

Models trained and evaluated on ImageNet-200 are Center Cropped to 224?

Hi @LiYingwei

In the paper under the section 2.2 EVALUATION AND OBSERVATION you mentioned that "all models are trained and evaluated on ImageNet-200". These images are downsized to 64×64 colored images. However, in the code there is CenterCrop(224).

Also in the following line 619; in the comment it is written "images are already 224X224"

I also looked into closed issues; in #3 it seems training was on imagenet-200 but in #1 it mentions to ImageNet itself. Which one is the case?

Thanks

Question about the label of style for original image

Hi, thank you for your nice work!

I found that the labels of style for the original image are different w/ or w/o multi_grid.
When deploying multi_grid, the label of style is labeled as '0', but when not deploying, the label is the same as the styled image.
I was wondering why does not set them both as '0'.

P.S. When deploying multi_grid,

When not deploying multi_grid,

FileNotFoundError: Couldn't find any class folder in C:/datasets/ILSVRC/Data/CLS-LOC\val

Hi Yingwei Li

Hope that all is well. I am following the TESTING.MD instructions and would like to run the following code :

python imagenet.py -a resnet50 --data $DATA --mixbn --evaluate --load $CKPT 

I downloaded the ImageNet dataset from Kaggle , and have installed python=3.11 and latest version of Pytorch. I have two questions:

1- What is your Pytorch version and other dependencies? I encountered errors like:

AttributeError: module 'torchvision.transforms' has no attribute 'RandomSizedCrop'. Did you mean: 'RandomResizedCrop'?
https://github.com/search?q=repo%3ALiYingwei%2FShapeTextureDebiasedTraining%20RandomSizedCrop&type=code


AttributeError: module 'torchvision.transforms' has no attribute 'Scale'

2- For the dataset path which folder should I use? I got the following error when I set the path to ILSVRC\Data\CLS-LOC

FileNotFoundError: Couldn't find any class folder in C:/datasets/ILSVRC/Data/CLS-LOC\val.

Thank you

Ensemble of two models

Can you please provide me detalis regarding the ensemble of two models?

1. During training: did you just compute one loss for each model?
2. During testing: how did you get the final logits?
   Thanks!