Batch size too large for Clean Label attack about trojanzoo

[ain-soph]

Hi, is there any concern that you don't want to directly adjust batch_size in dataset?

[TDteach]

There is a misunderstanding.
I set the batch_size through the parameters to the program, however, calc_grad function takes the whole x as an integration meaning that the batch_size I set is useless in calc_grad.

[ain-soph]

Sorry I don’t catch your idea. x is fetched from dataloader with (N, C, H, W) where N is the batch size.

You need to set batch size to make N smaller. We don’t need to further split x.
calc_grad only process one batch of data, and it doesn't use batch_size in this method for sure.

Btw, what’s the scenario? are you testing PGD or BadNet attack that uses PGD optimizer?

[ain-soph]

Will close this issue if there is no further response.

[TDteach]

The following is the running shell that induces the OOM problem

python examples/backdoor_attack.py --verbose 1 --dataset cifar10 --model resnet101_comp --attack clean_label --train_mode dataset --poison_percent 0.25 --batch_size 32 --pretrain  --epoch 100 --save

Below is the line-116 of trojanvision/attacks/backdoor/clean_label.py

poison_imgs, _ = self.pgd.optimize(_input_poison_imgs)

This line gives the _input directly the poison_imgs (using the above shell, with shape of [1250,3,32,32]).
However, neither the optimize function in trojanvision/shortcut/pgd.py nor the optimize function in trojanvision/optim/pgd.py has no action to split the _input , making OOM error for resnet101 such big network on machines with limited GPU memory

[ain-soph]

Ok, I get your idea. Clean Label attack use pgd attack for poison generation with one single batch. But if the poison ratio is too large which leads to the large batch size, it will be out of memory when using a large resnet101 model.

I’ll fix it by using a tensor dataset.

But just to note that, resnet101 is an overkill for Cifar10. So it is for the 0.25 poison ratio.