trustworthydl / leba

[NeurIPS'20] Learning Black-Box Attackers with Transferable Priors and Query Feedback

Learning Black-Box Attackers with Transferable Priors and Query Feedback

Jiancheng Yang*, Yangzhou Jiang*, Xiaoyang Huang, Bingbing Ni, Chenglong Zhao.

Neural Information Processing Systems (NeurIPS), 2020 (arXiv)

Abstract

This paper addresses the challenging black-box adversarial attack problem, where only classification confidence of a victim model is available. Inspired by consistency of visual saliency between different vision models, a surrogate model is expected to improve the attack performance via transferability. By combining transferability-based and query-based black-box attack, we propose a surprisingly simple baseline approach (named SimBA++) using the surrogate model, which significantly outperforms several state-of-the-art methods. Moreover, to efficiently utilize the query feedback, we update the surrogate model in a novel learning scheme, named High-Order Gradient Approximation (HOGA). By constructing a high-order gradient computation graph, we update the surrogate model to approximate the victim model in both forward and backward pass. The SimBA++ and HOGA result in Learnable Black-Box Attack (LeBA), which surpasses previous state of the art by large margins: the proposed LeBA reduces 34%-78% queries, while keeping higher attack success rates close to 100% in extensive ImageNet experiments, including attacking vision benchmarks and defensive models.

Implementation

Due to the size limitation of supplementary, we do not provide weights of model and datasets here. Note that our experiments setting is similar to paper Improving Black-box Adversarial Attacks with a Transfer-based Prior, including models and attack setting. Code and experiment setting will be open source after acceptance.

Requirements

The dependent package we use include: pytorch=1.2.0, pandas=0.25.1, pillow=5.4.1, opencv-python=4.1.1.26.
Note that the key package is pytorch.

Prepare Data

You can put ImageNet images and label file in dir images or try our example images in images first.
Note that you can find dataset IMGN1 in our paper in Baidu Wangpan (Password：wawy) and Google Drive.

Prepare Models

You can prepare your own model as victim model or surrogate model, and modify function get_models in data_utils.py.
We provide pretrain ImageNet model from torchvision, note that we test with pretrained models from Tensorflow-Slim in paper.

Pretrain models and model used in exps

You can find the models we used in experiments and pretrained in this folder on Baidu Wangpan(Password:r4z6) and Google Drive.

Script of Repo

LeBA2.py: Main script of LeBA attack, incluing 5 attack mode (train, test, SimBA, SimBA+, SimBA++). run_attack.py: Wrap script to run LeBA. data_utils.py: Functions to provide data, models, and log class get_result.py: Evaluate result file. defense: Contain defense method, but currently only Jpeg Compression is provided.

Run LeBA

• RUN different (train, test, SimBA, SimBA+, SimBA++) mode in sequence with script
  Use run_attack.py, it will save all the result files to the dir like: 'note_log_all/inception_v3_Dec10_f1'. Please edit run_attack.py to specify the attack mode(train, test, SimBA, SimBA+, SimBA++), else it will run 5 attack mode in sequence.

python run_attack.py --gpu_id=0,1,2 --script=LeBA10.py --model1=inception_v3 --model2=resnet152

• To run SimBA+ mode

python LeBA10.py --mode=simba+ --model1=inception_v3 --model2=resnet152 --input_dir=images --label=labels --out_dir="your output dir" 

• To run SimBA++ mode

python LeBA10.py --mode=simba++ --model1=inception_v3 --model2=resnet152 --input_dir=images --label=labels --out_dir="your output dir" 

• To run LeBA

python LeBA10.py --mode=train --model1=inception_v3 --model2=resnet152 --input_dir=images --label=labels --out_dir="your output dir" --pretrain_weight=""

• To run LeBA(test) mode
  After run LeBA, use weight trained in LeBA to test on another dataset.

python LeBA10.py --mode=test --model1=inception_v3 --model2=resnet152 --input_dir=imagesset2 --label=labels --out_dir="same output dir as train" --pretrain_weight="this_weight"

To attack defensive model:

#for Jpeg Compression
python run_attack.py --gpu_id=0,1,2 --script=LeBA10.py --model1=inception_v3 --model2=resnet152 --defense_method=jpeg

To evaluate results: Modify out_dir in get_result.py and run python get_result.py result will be save in result dir.

leba's Issues

Problem about the loss

Hi, Yang:

I've read your Leba work and I'm very interested in it. Recently, I'm trying to reproduce the part that uses forward loss and backward loss to narrow the gap between the two models. However, I feel confused about the code. I hope you can answer it. Thank you very much

I imitate the code in Git and try to calculate the backward loss as follows:

images = images.detach().clone()
adv_images = images + diff

surrogate_logits = surrogate_model(images)
surrogate_loss = nn.CrossEntropyLoss(reduction='none')(surrogate_logits, labels)

grad = torch.autograd.grad(surrogate_loss.sum(), images, create_graph=True)[0]
s_loss = (diff.detach() * grad).view([images.shape[0], -1]).sum(dim=1)  # scalar

target_adv_logits = target_model(adv_images)
target_adv_loss = nn.CrossEntropyLoss(reduction='none')(target_adv_logits, labels)

target_ori_logits = target_model(images)
target_ori_loss = nn.CrossEntropyLoss(reduction='none')(target_ori_logits, labels)
d_loss = torch.log(target_adv_loss) - torch.log(target_ori_loss)  # scalar

backward_loss = nn.MSELoss()(s_loss / lamda, d_loss.detach())

However, based on the above implementation, the gap between the output of the surrogate model and the target model gradually widens uncontrollably. I tested the gap via torch.nn.MSELoss(reduce=True, size_average=False)(target_model(images), surrogate_model(images)).

Whats's more, I imitate the code in Git and try to calculate the forward loss as follows:

surrogate_logits = surrogate_model(images)
surrogate_prob = torch.nn.functional.softmax(surrogate_logits, dim=1)
s_score = surrogate_prob.gather(1, labels.reshape([-1, 1]))

target_logits = target_model(images)
target_prob = torch.nn.functional.softmax(target_logits, dim=1)
target_score = target_prob.gather(1, labels.reshape([-1, 1]))

forward_loss = nn.MSELoss()(s_score, target_score.detach())

And using forward loss the gap does not show a monotonous downward trend.
I would like to ask which part of my understanding is wrong. :（

Yours.

Fei

Problem about the backward loss and the forward loss training on surrogate models

Hello, yang:

Thanks for your impressive work of LeBA.

I am wondering have you tested train the surrogate models with the forward loss and backward loss separately, which means that I use several clean images to train with the forward loss, then during the attacking process with the backward loss. Will that work?
In the code, the FL_rate is set to control the loss. On the above way, how do I set the updated rate?

Yours!
Fei.

Where is the code of SimBA+ and SimBA++?

I found that you only upload LeBA, can you upload SimBA+ and SimBA++?