TransferAttack is a pytorch framework to boost the adversarial transferability for image classification.
Devling into Adversarial Transferability on Image Classification: A Review, Benchmark and Evaluation will be released soon.
We also release a list of papers about transfer-based attacks here.
There are a lot of reasons for TransferAttack, such as:
- A benchmark for evaluating new transfer-based attacks: TransferAttack categorizes existing transfer-based attacks into several types and fairly evaluates various transfer-based attacks under the same setting.
- Evaluate the robustness of deep models: TransferAttack provides a plug-and-play interface to verify the robustness of models, such as CNNs and ViTs.
- A summary of transfer-based attacks: TransferAttack reviews numerous transfer-based attacks, making it easy to get the whole picture of transfer-based attacks for practitioners.
- Python >= 3.6
- PyTorch >= 1.12.1
- Torchvision >= 0.13.1
- timm >= 0.6.12
pip install -r requirements.txtWe randomly sample 1,000 images from ImageNet validate set, in which each image is from one category and can be correctly classified by the adopted models. Download the data into /path/to/data. Then you can run the attack as follows:
python main.py --input_dir ./path/to/data --output_dir adv_data/mifgsm/resnet18 --attack mifgsm --model=resnet18
python main.py --input_dir ./path/to/data --output_dir adv_data/mifgsm/resnet18 --eval
| Category | Attack | Main Idea |
|---|---|---|
| Gradient-based | FGSM (Goodfellow et al., 2015) | Add a small perturbation in the direction of gradient |
| I-FGSM (Kurakin et al., 2015) | Iterative version of FGSM | |
| MI-FGSM (Dong et al., 2018) | Integrate the momentum term into the I-FGSM | |
| NI-FGSM (Lin et al., 2020) | Integrate the Nesterov's accelerated gradient into I-FGSM | |
| PI-FGSM (Gao et al., 2020) | Reusing the cut noise and apply a heuristic project strategy to generate patch-wise noise | |
| VMI-FGSM (Wang et al., 2021) | Variance tuning MI-FGSM | |
| VNI-FGSM (Wang et al., 2021) | Variance tuning NI-FGSM | |
| EMI-FGSM (Wang et al., 2021) | Accumulate the gradients of several data points linearly sampled in the direction of previous gradient | |
| I-FGS²M (Zhang et al., 2021) | Assigning staircase weights to each interval of the gradient | |
| VA-I-FGSM (Zhang et al., 2022) | Adopt a larger step size and auxiliary gradients from other categories | |
| AI-FGTM (Zou et al., 2022) | Adopt Adam to adjust the step size and momentum using the tanh function | |
| RAP (Qin et al., 2022) | Inject the worst-case perturbation when calculating the gradient. | |
| GI-FGSM (Wang et al., 2022) | Use global momentum initialization to better stablize update direction. | |
| PC-I-FGSM (Wan et al., 2023) | Gradient Prediction-Correction on MI-FGSM | |
| IE-FGSM (Peng et al., 2023) | Integrate anticipatory data point to stabilize the update direction. | |
| DTA (Yang et al., 2023) | Calculate the gradient on several examples using small stepsize | |
| GRA (Zhu et al., 2023) | Correct the gradient using the average gradient of several data points sampled in the neighborhood and adjust the update gradient with a decay indicator | |
| PGN (Ge et al., 2023) | Penalizing gradient norm on the original loss function | |
| SMI-FGRM (Han et al., 2023) | Substitute sign function with data rescaling and use the depth first sampling technique to stabilize the update direction. | |
| Input transformation-based | DIM (Xie et al., 2019) | Random resize and add padding to the input sample |
| TIM (Dong et al., 2019) | Adopt a Gaussian kernel to smooth the gradient before updating the perturbation | |
| SIM (Ling et al., 2020) | Calculate the average gradient of several scaled images | |
| ATTA (Wu et al., 2021) | Train an adversarial transformation network to perform the input-transformation | |
| Admix (Wang et al., 2021) | Mix up the images from other categories | |
| DEM (Zou et al., 2021) | Calculate the average gradient of several DIM's transformed images | |
| SSM (Long et al., 2022) | Randomly scale images and add noise in the frequency domain | |
| MaskBlock (Fan et al., 2022) | Calculate the average gradients of multiply randomly block-level masked images. | |
| SIA (Wang et al., 2023) | Split the image into blocks and apply various transformations to each block | |
| STM (Ge et al., 2023) | Transform the image using a style transfer network | |
| BSR (Wang et al., 2023) | Randomly shuffles and rotates the image blocks | |
| Advanced objective | TAP (Zhou et al., 2018) | Maximize the difference of feature maps between benign sample and adversarial example and smooth the perturbation |
| ILA (Huang et al., 2019) | Enlarge the similarity of feature difference between the original adversarial example and benign sample | |
| YAILA (Wu et al., 2020) | Establishe a linear map between intermediate-level discrepancies and classification loss | |
| FIA (Wang et al., 2021) | Minimize a weighted feature map in the intermediate layer | |
| TRAP (Wang et al., 2021) | Utilize affine transformations and reference feature map | |
| NAA (Zhang et al., 2022) | Compute the feature importance of each neuron with decomposition on integral | |
| RPA (Zhang et al., 2022) | Calculate the weight matrix in FIA on randomly patch-wise masked images | |
| TAIG (Huang et al., 2022) | Adopt the integrated gradient to update perturbation | |
| FMAA (He et al., 2022) | Utilize momentum to calculate the weight matrix in FIA | |
| ILPD (Li et al., 2023) | Decays the intermediate-level perturbation from the benign features by mixing the features of benign samples and adversarial examples | |
| Model-related | SGM (Wu et al., 2021) | Utilize more gradients from the skip connections in the residual blocks |
| DSM (Yang et al., 2022) | Train surrogate models in a knowledge distillation manner and adopt CutMix on the input | |
| MTA (Qin et al., 2023) | Train a meta-surrogate model (MSM), whose adversarial examples can maximize the loss on a single or a set of pre-trained surrogate models | |
| MUP (Yang et al., 2023) | Mask unimportant parameters of surrogate models | |
| BPA (Wang et al., 2023) | Recover the trunctaed gradient of non-linear layers | |
| DHF (Wang et al., 2023) | Mixup the feature of current examples and benign samples and randomly replaces the features with their means. | |
| PNA-PatchOut (Wei et al., 2021) | Ignore gradient of attention and randomly drop patches among the perturbation | |
| SAPR (Zhou et al., 2022) | Randomly permute input tokens at each attention layer | |
| TGR (Zhang et al., 2023) | Scale the gradient and mask the maximum or minimum gradient magnitude |
To thoroughly evaluate existing attacks, we have included various popular models, including both CNNs (ResNet-18, ResNet-101, ResNeXt-50, DenseNet-121) and ViTs (ViT, PiT, Visformer, Swin). Moreover, we also adopted four defense methods, namely AT, HGD, RS, NRP. The defense models can be downloaded from Google Drive.
Note: We adopt
| Category | Attacks | CNNs | ViTs | Defenses | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ResNet-18 | ResNet-101 | ResNeXt-50 | DenseNet-101 | ViT | PiT | Visformer | Swin | AT | HGD | RS | NRP | ||
| Gradient-based | FGSM | 97.4 | 36.2 | 43.8 | 61.0 | 15.2 | 21.2 | 28.8 | 34.4 | 31.0 | 28.0 | 20.1 | 29.8 |
| I-FGSM | 100.0 | 13.9 | 16.1 | 37.4 | 5.4 | 8.3 | 11.5 | 17.0 | 27.9 | 9.9 | 16.2 | 21.2 | |
| MI-FGSM | 100.0 | 41.3 | 48.4 | 77.2 | 16.3 | 23.9 | 34.6 | 42.0 | 30.4 | 33.9 | 19.3 | 27.6 | |
| NI-FGSM | 100.0 | 43.9 | 49.8 | 79.5 | 16.8 | 23.4 | 35.3 | 41.2 | 30.1 | 36.2 | 19.7 | 28.2 | |
| PI-FGSM | 100.0 | 37.3 | 46.7 | 74.9 | 19.9 | 18.4 | 26.3 | 35.7 | 34.1 | 35.7 | 30.0 | 34.1 | |
| VMI-FGSM | 100.0 | 62.4 | 68.8 | 91.2 | 28.3 | 41.3 | 54.5 | 58.9 | 32.9 | 55.6 | 23.7 | 47.6 | |
| VNI-FGSM | 100.0 | 61.4 | 68.5 | 92.6 | 25.3 | 38.6 | 52.0 | 56.9 | 32.3 | 52.3 | 21.5 | 36.9 | |
| EMI-FGSM | 100.0 | 56.6 | 62.4 | 90.4 | 20.9 | 32.6 | 46.8 | 53.1 | 32.4 | 46.7 | 21.3 | 34.2 | |
| I-FGS²M | 100.0 | 18.9 | 24.2 | 52.3 | 8.1 | 11.9 | 16.1 | 23.4 | 28.4 | 14.2 | 16.8 | 14.3 | |
| VA-I-FGSM | 100.0 | 19.4 | 23.0 | 44.6 | 6.8 | 11.5 | 14.3 | 21.1 | 28.8 | 11.5 | 16.9 | 18.4 | |
| AI-FGTM | 100.0 | 34.6 | 40.5 | 70.1 | 12.7 | 20.1 | 28.9 | 34.9 | 29.8 | 26.4 | 18.2 | 20.4 | |
| RAP | 100.0 | 51.8 | 58.5 | 87.5 | 21.1 | 26.9 | 43.1 | 49.3 | 32.4 | 39.7 | 22.8 | 31.0 | |
| GI-FGSM | 100.0 | 49.5 | 54.6 | 83.7 | 18.5 | 27.0 | 38.7 | 46.6 | 31.3 | 39.0 | 20.2 | 31.2 | |
| PC-I-FGSM | 100.0 | 41.3 | 48.4 | 76.7 | 16.7 | 25.0 | 35.1 | 41.4 | 30.2 | 34.1 | 19.3 | 26.6 | |
| DTA | 100.0 | 50.0 | 57.4 | 84.8 | 19.4 | 28.5 | 42.5 | 45.0 | 31.2 | 41.7 | 19.7 | 38.1 | |
| GRA | 100.0 | 65.1 | 70.6 | 93.6 | 32.6 | 39.2 | 54.0 | 63.1 | 38.3 | 59.0 | 31.2 | 49.7 | |
| PGN | 100.0 | 68.4 | 73.6 | 94.5 | 31.6 | 43.6 | 57.3 | 65.0 | 38.8 | 60.7 | 32.1 | 51.7 | |
| IE-FGSM | 100.0 | 50.8 | 56.8 | 85.9 | 22.2 | 26.9 | 41.4 | 47.0 | 30.3 | 40.9 | 19.5 | 29.0 | |
| SMI-FGRM | 99.7 | 37.4 | 41.0 | 74.5 | 15.2 | 21.8 | 29.7 | 38.8 | 32.8 | 31.1 | 24.1 | 31.3 | |
| Input transformation-based | DIM | 100.0 | 62.2 | 68.1 | 91.9 | 28.1 | 36.6 | 52.8 | 57.7 | 33.5 | 59.8 | 22.8 | 44.7 |
| TIM | 100.0 | 35.6 | 46.4 | 72.3 | 15.0 | 17.4 | 26.2 | 35.6 | 33.7 | 32.5 | 29.6 | 34.1 | |
| SIM | 100.0 | 58.4 | 64.9 | 91.3 | 22.9 | 34.4 | 47.2 | 53.5 | 33.6 | 50.1 | 22.9 | 38.2 | |
| ATTA | 100.0 | 44.2 | 51.1 | 80.6 | 18.9 | 25.9 | 37.4 | 43.4 | 31.0 | 37.6 | 20.0 | 28.8 | |
| Admix | 100.0 | 70.1 | 74.4 | 96.0 | 28.6 | 40.5 | 58.4 | 62.1 | 35.6 | 62.0 | 24.8 | 43.6 | |
| DEM | 100.0 | 74.5 | 80.7 | 98.0 | 40.0 | 45.9 | 64.9 | 65.4 | 36.7 | 78.2 | 29.0 | 45.5 | |
| SSM | 100.0 | 69.8 | 73.5 | 94.2 | 30.5 | 41.3 | 56.7 | 64.1 | 35.9 | 61.2 | 26.1 | 48.3 | |
| MaskBlock | 100.0 | 46.8 | 54.5 | 82.9 | 17.5 | 27.3 | 39.2 | 45.4 | 30.8 | 38.9 | 20.5 | 30.0 | |
| SIA | 100.0 | 88.8 | 92.1 | 99.5 | 45.1 | 61.4 | 80.7 | 80.6 | 36.0 | 82.4 | 26.3 | 50.4 | |
| STM | 100.0 | 72.9 | 78.3 | 96.7 | 35.0 | 47.5 | 62.1 | 68.3 | 37.2 | 70.0 | 29.6 | 53.2 | |
| BSR | 100.0 | 85.5 | 90.1 | 99.2 | 43.8 | 61.5 | 79.3 | 78.5 | 36.6 | 81.7 | 25.9 | 54.5 | |
| Advanced objective | TAP | 100.0 | 36.1 | 43.4 | 69.9 | 13.6 | 17.3 | 26.1 | 33.0 | 30.8 | 26.6 | 19.0 | 26.8 |
| ILA | 100.0 | 55.9 | 62.0 | 85.6 | 15.5 | 25.4 | 42.9 | 45.2 | 29.9 | 38.6 | 18.5 | 27.7 | |
| YAILA | 47.9 | 20.9 | 24.9 | 46.1 | 5.9 | 9.7 | 13.2 | 18.7 | 27.4 | 12.2 | 15.7 | 14.5 | |
| FIA | 99.8 | 29.4 | 32.2 | 61.6 | 9.6 | 16.3 | 23.5 | 30.3 | 29.6 | 18.9 | 17.8 | 27.5 | |
| TRAP | 99.8 | 80.3 | 84.0 | 96.7 | 35.9 | 49.3 | 71.4 | 67.9 | 33.5 | 76.0 | 21.8 | 37.2 | |
| NAA | 99.6 | 53.0 | 57.6 | 81.2 | 22.8 | 34.2 | 44.4 | 52.3 | 32.0 | 44.1 | 21.5 | 34.1 | |
| RPA | 100.0 | 64.9 | 68.6 | 92.5 | 26.2 | 35.5 | 53.0 | 58.6 | 34.7 | 56.8 | 24.7 | 44.7 | |
| TAIG | 100.0 | 20.3 | 25.5 | 56.6 | 7.3 | 13.3 | 18.7 | 25.5 | 36.0 | 14.6 | 17.4 | 28.5 | |
| FMAA | 100.0 | 37.0 | 41.3 | 76.3 | 10.5 | 19.1 | 28.2 | 35.2 | 29.8 | 24.1 | 17.9 | 18.9 | |
| ILPD | 73.1 | 68.3 | 70.0 | 72.7 | 35.4 | 49.2 | 55.8 | 57.0 | 47.3 | 85.2 | 22.7 | 48.8 | |
| Model-related | SGM | 100.0 | 47.2 | 52.7 | 81.6 | 21.1 | 29.8 | 42.1 | 48.7 | 32.2 | 41.1 | 21.6 | 31.4 |
| DSM | 99.2 | 62.3 | 67.6 | 93.8 | 42.6 | 36.9 | 50.8 | 56.9 | 32.5 | 51.5 | 21.9 | 35.2 | |
| MTA | 84.7 | 42.4 | 46.5 | 73.8 | 12.9 | 21.5 | 32.0 | 40.0 | 28.9 | 36.8 | 19.3 | 24.1 | |
| MUP | 100.0 | 46.9 | 54.0 | 84.6 | 17.3 | 26.4 | 38.3 | 46.3 | 30.9 | 37.2 | 20.3 | 29.8 | |
| BPA | 100.0 | 61.4 | 68.0 | 92.7 | 24.1 | 36.6 | 52.2 | 58.9 | 31.8 | 52.3 | 22.4 | 35.3 | |
| DHF | 100 | 71.8 | 76.6 | 94.1 | 31.3 | 43.5 | 61.5 | 65.2 | 32.4 | 62 | 22.6 | 40.5 | |
| PNA-PatchOut | 68.0 | 52.6 | 56.7 | 66.9 | 96.6 | 63.1 | 65.7 | 76.0 | 32.4 | 47.4 | 21.7 | 34.1 | |
| SAPR | 67.6 | 53.1 | 55.2 | 66.3 | 97.2 | 61.6 | 65.4 | 79.1 | 32.7 | 47.1 | 23.3 | 50.6 | |
| TGR | 80.0 | 58.0 | 63.4 | 77.8 | 98.8 | 69.8 | 73.8 | 86.9 | 36.1 | 54.0 | 28.7 | 41.7 | |
Xiaosen Wang |
Zeyuan Yin |
Zeliang Zhang |
Kunyu Wang |
Zhijin Ge |
Yuyang Luo |
We thank all the researchers who contribute or check the methods. See contributors for details.
We are trying to include more transfer-based attacks. We welcome suggestions and contributions! Submit an issue or pull request and we will try our best to respond in a timely manner.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent