ain-soph / trojanzoo Goto Github PK

The original code is pasted below.
An apparent problem is that no batch separation is implemented which causes the OOM problem when running the large neural networks e.g., ResNet101.

def calc_grad(self, f, x: torch.Tensor, grad_method: str = None, loss_kwargs: dict[str, torch.Tensor] = {}) -> torch.Tensor:
    grad_method = grad_method if grad_method is not None else self.grad_method
    grad_func = self.whitebox_grad if grad_method == 'white' else self.blackbox_grad
    return grad_func(f, x, loss_kwargs=loss_kwargs)

@staticmethod
def whitebox_grad(f, x: torch.Tensor, loss_kwargs: dict[str, torch.Tensor] = {}) -> torch.Tensor:
    x.requires_grad_()
    loss = f(x, **loss_kwargs)
    grad = torch.autograd.grad(loss, x)[0]
    x.requires_grad_(False)
    return grad

After I git clone the repo and install it, I get an attributeError When I run the scripts ‘train.py’.
It seems like the 'summary' attribute doesn't pass correctly in ‘trojanvision/init.py’.
Do you have any idea to solve the problem? Thanks!

$ python examples/train.py --color --verbose 1 --dataset cifar10 --model resnet18_comp --lr_scheduler --cutout --grad_clip 5.0 --save Traceback (most recent call last): File "/mnt/g/Cm_code/trojanzoo/examples/train.py", line 28, in <module> trojanvision.summary(env=env, dataset=dataset, model=model, trainer=trainer) AttributeError: module 'trojanvision' has no attribute 'summary'

cannot import name 'Callable' from 'collections'
for python==3.10

In the original paper, the author say that "we pick the neuron that has the largest value of the sum of absolute weights ..."

But in trojannn.py, line 116 : "return weight.argsort(descending=False)[:self.neuron_num]" use the ascending order, is this a typo? or you reverse the sequence in other line?

I read your paper, and saw the table 2. And I found BN(BadNets), ESB(TrojanNet), TNN(TrojanNN), RB(reflection backdoor), LB(latent backdoor), IMC.

But where is the code for TB (Blended attack )and ABE(Bypassing Backdoor Detection Algorithms in Deep Learning)? (I read your code at commit cc4551f in main branch)

Thank you!

We won't enable this feature data_format='zip' until torchvision.datasets.DatasetFolder support customize make_dataset.
See pytorch/vision#3215

Actually we have all the code ready. If the pull request is merged, I'll remove the block message from ImageFolder and everything would be fine, even though I doubt why do we need zip format actually. It won't provide any significant advantage.

My ZipFolder Implementation at

the bug
python ./examples/train.py --color --verbose 1 --dataset cifar10 --model resnet18_comp --lr_scheduler --cutout --grad_clip 5.0 --save

it does't work because the dataset cifar10 can't be downloaded

Dear Sir/Madam, thanks for your great work! I'm trying to re-implement the same results that you show on the Trojanzoo paper. However, I'm confused about the res of ESB in Table4. Could you please tell me how you generate the validate target dataset for ESB method? Cause it requires the specific triggers while these triggers don't exist in the trojanvision/marks dir.

Hi, so I was trying to attack MNIST fine-tuned resnet model with trojanNN and I got this error on my terminal.

"line 395, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [64, 3, 7, 7], expected input[512, 1, 32, 32] to have 3 channels, but got 1 channels instead"

Can someone please help me resolve this, is this because MNIST is grayscale?

Describe the bug
Traceback (most recent call last):
File "/home/hkz/trojanzoo/backdoor_attack.py", line 38, in
attack.attack(**trainer)
File "/home/hkz/trojanzoo/trojanvision/attacks/backdoor/badnet.py", line 67, in attack
self.model._train(epoch, save=save,
File "/home/hkz/trojanzoo/trojanzoo/models.py", line 429, in _train
save_fn(file_path=file_path, folder_path=folder_path, suffix=suffix, verbose=verbose)
File "/home/hkz/trojanzoo/trojanvision/attacks/backdoor/badnet.py", line 107, in save
filename = self.get_filename(**kwargs)
File "/home/hkz/trojanzoo/trojanvision/attacks/backdoor/badnet.py", line 94, in get_filename
mark_name, mark_ext = os.path.splitext(mark_filename)
File "/home/hkz/anaconda3/envs/trojanzoo/lib/python3.9/posixpath.py", line 118, in splitext
p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not tuple

To Reproduce
Steps to reproduce the behavior:

1. git clone the current repo and download the cifar10 dataset
2. mv backdoor_attack.py from example dir to the repo root
3. run the quick start example
   python backdoor_attack.py --verbose 1 --validate_interval 1 --amp --dataset cifar10 --model resnetcomp18 --attack badnet --random_init --epoch 50 --lr 0.01 --save without --pretrain
4. See error

Screenshots

Desktop (please complete the following information):

• OS: ubuntu 18.04
• Version git commit id 7478061

Hi,
I've run Badnet attack, followed by ABS defense (with default settings). From the results, it seems like ABS cannot detect the attack, which is weird.

Specifically, this is the final result:

Score:  [7.156167772364617, 9.457871842956543, 4.543633675146103, 7.217127031528951, 5.883398698425293, 10.027698394012452, 10.763735179138184, 9.409244415664674, 9.147680532836914, 7.1922257972240455]
Score MAD:  tensor([0.0213, 0.7829, 0.9341, 0.0000, 0.4660, 0.9819, 1.2391, 0.7659, 0.6745,
        0.0087])

Does anybody have an idea why? Or maybe I am misinterpreting the results?

python3.9 ./examples/train.py --verbose 1 --dataset cifar10 --model resnet18_comp --lr_scheduler --cutout --grad_clip 5.0 --save --device cuda > train.txt

python3.9 ./examples/backdoor_attack.py --verbose 1 --pretrain --validate_interval 1 --dataset cifar10 --model resnet18_comp --attack badnet --random_init --epoch 50 --lr 0.01 --save --device cuda > test2_.txt

 python3.9 ./examples/backdoor_defense.py --verbose 1 --pretrain --validate_interval 1 --dataset cifar10 --model resnet18_comp --attack badnet --defense abs --random_init --device cuda > defens_abs_attack_badnet.txt

Output of the above commands:

train.txt
test2_.txt
defens_abs_attack_badnet.txt

hello author!
i met a few question when i try to use [trojanzoo]
1.

Are these 4 points all need to be done? Or just choose one to install?
2.
when i run docker pull local0state/trojanzoo or docker pull ghcr.io/ain-soph/trojanzoo,the error are

and

how can i solve them?
3.the command conda install trojanzoo shows

Do this cant install by conda?
4.the last question is when i run the quick_start
i use python ./examples/train.py --color --verbose 1 --dataset cifar10 --model resnet18_comp --lr_scheduler --cutout --grad_clip 5.0 --save and get

im hoping for your reply! thank you.

Problem Description

I downloaded the package and used pip to install. I am sure my environment has torch installed but I got an error saying "no module named torch".

Output from "conda list | grep torch"

pytorch-lightning         1.2.6                    pypi_0    pypi
torch                     1.7.1+cu110              pypi_0    pypi
torchaudio                0.7.2                    pypi_0    pypi
torchfile                 0.1.0                    pypi_0    pypi
torchmetrics              0.2.0                    pypi_0    pypi
torchsummary              1.5.1                    pypi_0    pypi
torchvision               0.8.2+cu110              pypi_0    pypi

Output from "pip3 install trojanzoo-master.zip"

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Processing ./trojanzoo-master.zip
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  ERROR: Command errored out with exit status 1:
   command: /home/zjunesa/anaconda2/envs/myh/bin/python /home/zjunesa/anaconda2/envs/myh/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmph1f7a19_
       cwd: /tmp/pip-req-build-c6kjz7z2
  Complete output (64 lines):
  Traceback (most recent call last):
    File "/tmp/pip-build-env-qrekwfu_/overlay/lib/python3.7/site-packages/setuptools/config.py", line 35, in __getattr__
      for statement in self.module.body
  StopIteration
  
  The above exception was the direct cause of the following exception:
  
  Traceback (most recent call last):
    File "/tmp/pip-build-env-qrekwfu_/overlay/lib/python3.7/site-packages/setuptools/config.py", line 387, in _parse_attr
      return getattr(StaticModule(module_name), attr_name)
    File "/tmp/pip-build-env-qrekwfu_/overlay/lib/python3.7/site-packages/setuptools/config.py", line 43, in __getattr__
      ) from e
  AttributeError: trojanzoo has no attribute __version__
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call `last):`
    File "/home/zjunesa/anaconda2/envs/myh/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
      main()
    File "/home/zjunesa/anaconda2/envs/myh/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/zjunesa/anaconda2/envs/myh/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 114, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/tmp/pip-build-env-qrekwfu_/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 155, in get_requires_for_build_wheel
      config_settings, requirements=['wheel'])
    File "/tmp/pip-build-env-qrekwfu_/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 135, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-qrekwfu_/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 150, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 4, in <module>
      setuptools.setup()
    File "/tmp/pip-build-env-qrekwfu_/overlay/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/home/zjunesa/anaconda2/envs/myh/lib/python3.7/distutils/core.py", line 121, in setup
      dist.parse_config_files()
    File "/tmp/pip-build-env-qrekwfu_/overlay/lib/python3.7/site-packages/setuptools/dist.py", line 716, in parse_config_files
      ignore_option_errors=ignore_option_errors)
    File "/tmp/pip-build-env-qrekwfu_/overlay/lib/python3.7/site-packages/setuptools/config.py", line 157, in parse_configuration
      meta.parse()
    File "/tmp/pip-build-env-qrekwfu_/overlay/lib/python3.7/site-packages/setuptools/config.py", line 463, in parse
      section_parser_method(section_options)
    File "/tmp/pip-build-env-qrekwfu_/overlay/lib/python3.7/site-packages/setuptools/config.py", line 436, in parse_section
      self[name] = value
    File "/tmp/pip-build-env-qrekwfu_/overlay/lib/python3.7/site-packages/setuptools/config.py", line 220, in __setitem__
      value = parser(value)
    File "/tmp/pip-build-env-qrekwfu_/overlay/lib/python3.7/site-packages/setuptools/config.py", line 552, in _parse_version
      version = self._parse_attr(value, self.package_dir)
    File "/tmp/pip-build-env-qrekwfu_/overlay/lib/python3.7/site-packages/setuptools/config.py", line 390, in _parse_attr
      module = importlib.import_module(module_name)
    File "/home/zjunesa/anaconda2/envs/myh/lib/python3.7/importlib/__init__.py", line 127, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
    File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
    File "<frozen importlib._bootstrap>", line 983, in _find_and_load
    File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
    File "<frozen importlib._bootstrap_external>", line 728, in exec_module
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "/tmp/pip-req-build-c6kjz7z2/trojanzoo/__init__.py", line 5, in <module>
      from trojanzoo import environ as environ
    File "/tmp/pip-req-build-c6kjz7z2/trojanzoo/environ.py", line 3, in <module>
      from trojanzoo.utils.environ import *
    File "/tmp/pip-req-build-c6kjz7z2/trojanzoo/utils/__init__.py", line 3, in <module>
      import torch.cuda
  ModuleNotFoundError: No module named 'torch'
  ----------------------------------------
ERROR: Command errored out with exit status 1: /home/zjunesa/anaconda2/envs/myh/bin/python /home/zjunesa/anaconda2/envs/myh/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmph1f7a19_ Check the logs for full command output.

System

NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

Output from "pip3 install trojanzoo"

Looking in indexes: https://pypi.org/simple/
WARNING: Keyring is skipped due to an exception: Failed to create the collection: Prompt dismissed..
ERROR: Could not find a version that satisfies the requirement trojanzoo
ERROR: No matching distribution found for trojanzoo

Describe the bug
Traceback (most recent call last):
File "/home/hkz/trojanzoo/backdoor_defense.py", line 41, in
defense.detect(**trainer)
File "/home/hkz/trojanzoo/trojanvision/defenses/backdoor/fine_pruning.py", line 65, in detect
self.prune(**kwargs)
File "/home/hkz/trojanzoo/trojanvision/defenses/backdoor/fine_pruning.py", line 76, in prune
_, target_acc, clean_acc = self.attack.validate_func()
ValueError: not enough values to unpack (expected 3, got 2)

To Reproduce
Steps to reproduce the behavior:

1. git clone the current repo and download the cifar10 dataset
2. mv backdoor_defense.py from example dir to the repo root
3. run the quick start example
   python backdoor_defense.py --verbose 1 --validate_interval 1 --dataset cifar10 --model resnetcomp18 --attack badnet --defense fine_pruning --random_init --epoch 50 --lr 0.01
4. See error

Expected behavior

Screenshots

Desktop (please complete the following information):

• OS: ubuntu 18.04
• Version git commit id 7478061

Describe the bug
I was trying to run the ABS defense on a pretrained resnetcomp model. The following bug appeared:

It seems that the preprocess method has not been defined in the model ResNetcomp or its parent classes.
Btw, I found a preprocess method defined in the activation_clustering defense. Does this method have any relationship with the process method called in resnet.py?

Desktop (please complete the following information):

• OS: ubuntu
• Browser chrome

Hi,

I'm using the 1.0.8 version. Running ABS defense on resnet_comp18, I get the following error:

  File ".../trojanzoo/trojanzoo/utils/model.py", line 102, in get_layer
    raise ValueError('Layer name not correct')
ValueError: Layer name not correct

I realized that it is looking for a layer named classifier which is absent. I did the following ad-hoc fix:

layer_output_name = [k0 for k0 in self.model.get_layer_name() if k0.startswith('classifier')][0]
result.append(self.model.get_layer(h.flatten(end_dim=1), layer_input=layer,
                                   layer_output=layer_output_name).detach().cpu())

I'm not sure if it was the right thing to do. Is there something else I should do?

Thanks.

1. Support RGBA image for watermarks.
2. According to the definition of RGBA, alpha should stand for the degree of opaque, rather than transparency.
3. class trojanvision.marks.Watermark should use 4 channel RGBA tensor, rather than so many masks and alpha masks. This could simplify the code.

Hi,
In the following line, the result is saved at each iteration, overwriting previous results:

Support SWA in trojanzoo.models.Model

Support Tensorboard in trojanzoo.models.Model._train()

The current Randomized Smoothing is a generic method, that we use the averaged logits of samples from Gaussian distribution as the prediction result. However, according to Certified Adversarial Robustness via Randomized Smoothing, it uses a vote mechanism to detect outliers. So it's not like a general mitigation method (such as adversarial training and MagNet), but a input detector like STRIP.

And another thing is to add some naive transformation as defenses(rotation, random croping, brightness change). It seems these naive methods are very effective.

We will certainly move to python 3.10 when it is released.

1. PEP604 python 3.10 will help us remove all typing.Union in the current codes.
2. PEP563 Move annotation imports into typing.TYPE_CHECKING if condition.

Backward compatibility is never considered. Please use the newest python version.

add_arguments() requires all modules have their unique argument names. The same argument name can't be registered twice.
However, e.g., adv_train and hidden_trigger both have pgd_iteration arguments, which will raise error.

What is the difference between resnet18 and resnet18_comp?

I have installed all the dependencies based on the requirements.txt file.
Then when I tried to install trojanzoo using:

pip install trojanzoo
It was showing this error

ERROR: Could not find a version that satisfies the requirement trojanzoo
ERROR: No matching distribution found for trojanzoo

I have installed all the libraries in the requirements.txt file, I am not sure why this is showing this error.

Hi,
I've run the "badnet" attack and the "neural cleanse" and "deep inspect" defenses on it several times. The results show that the defenses are not doing a very good job. Given that badnet is one of the simplest attacks, I guess that should not be the case. Could anyone please have a look at the test results?

I've used trojanzoo version 1.0.8 (with some small modifications and bug fixes of my own).
The experiment was run in a loop like this:

for ...
  run badnet
  run neural cleanse
  run deep inspect
end

About the attached files:
After each run of defense, there are a few lines that read "soft/hard median".
They show a list of outlier indexes of the results (that I think should be the detected poisoned class).
The "soft median" line refers to the outliers computed by considering median by averaging the two middle elements when there are an even number of data points (same as what NumPy does). The hard median refers to the median computed by taking floor(n/2) (the PyTorch way)
defense_di_attack_badnet_cifar_multirun2.txt
defense_di_attack_badnet_mnist_multirun2.txt
defense_nc_attack_badnet_cifar_multirun2.txt
defense_nc_attack_badnet_mnist_multirun2.txt
attack_badnet_cifar_multirun2.txt
attack_badnet_mnist_multirun2.txt
.

• PEP 673 Self type annotation

According to the author of paper TrojanNet, I may update the algorithm to make it work on alpha != 0.0 cases.
Basic idea is to make watermark in training data (3*3 for example) transparent as well. The other part of the image will be from clean training data. And the random noises still keeps for the last class.

When loading loss weights from GTSRB dataset, I found the default type of loss weights is np.float64, which will generate the following error:

RuntimeError: expected scalar type Float but found Double

And it may be fixed to add the following strict type conversion before line 182, trojanzoo/trojanzoo/models.py:

 if (isinstance(loss_weights,np.ndarray)):
     loss_weights=np.asarray(loss_weights).astype(np.float32)

"python backdoor_defense.py --color --verbose 1 --pretrained --validate_interval 1 --dataset cifar10 --model resnet18_comp --attack badnet --defense neuron_inspect --mark_random_init --epochs 50 --lr
0.01 --save --tqdm"
When I ran the command above to detect badnets with NeuronInspect, I got the error: "TypeError: only integer tensors of a single element can be converted to an index". Please help me.

Traceback (most recent call last):
File "/home/itl/Documents/xrh/backdoor/trojanzoo/examples/backdoor_defense.py", line 46, in
defense.detect(**trainer)
File "/home/itl/Documents/xrh/backdoor/trojanzoo/examples/../trojanvision/defenses/backdoor/model_inspection/neuron_inspect.py", line 48, in detect
exp_features = self.get_explation_feature()
File "/home/itl/Documents/xrh/backdoor/trojanzoo/examples/../trojanvision/defenses/backdoor/model_inspection/neuron_inspect.py", line 68, in get_explation_feature
exp_features.append(self.cal_explanation_feature(backdoor_saliency_maps, benign_saliency_maps))
File "/home/itl/Documents/xrh/backdoor/trojanzoo/examples/../trojanvision/defenses/backdoor/model_inspection/neuron_inspect.py", line 86, in cal_explanation_feature
exp_feats = self.lambd_sp * sparse_feats + self.lambd_sm * smooth_feats + self.lambd_pe * persist_feats
TypeError: only integer tensors of a single element can be converted to an index

In version 1.1, I'll reimplement existing attacks and defenses to eliminate the difference between trojanzoo and original codes.

Attacks:

• BadNet
• TrojanNN
• IMC
• LatentBackdoor
• Refool
• TrojanNet (Embarrasingly Simple Backdoor)
• BypassEmbed
• CleanLabel
• HiddenTrigger

Defenses:

• NeuralCleanse
• TABOR
• ABS
• DeepInspect
• NeuronInspect
• Strip
• Neo
• ActivationClustering
• SpectralSignature

Describe the bug
Traceback (most recent call last):
File "/home/hkz/trojanzoo/backdoor_attack.py", line 38, in
attack.attack(**trainer)
File "/home/hkz/trojanzoo/trojanvision/attacks/backdoor/reflection_backdoor.py", line 65, in attack
self.model.load()
File "/home/hkz/trojanzoo/trojanzoo/models.py", line 280, in load
raise e
File "/home/hkz/trojanzoo/trojanzoo/models.py", line 277, in load
_dict: OrderedDict[str, torch.Tensor] = torch.load(file_path, map_location=map_location, **kwargs)
File "/home/hkz/anaconda3/envs/trojanzoo/lib/python3.9/site-packages/torch/serialization.py", line 581, in load
with _open_file_like(f, 'rb') as opened_file:
File "/home/hkz/anaconda3/envs/trojanzoo/lib/python3.9/site-packages/torch/serialization.py", line 230, in _open_file_like
return _open_file(name_or_buffer, mode)
File "/home/hkz/anaconda3/envs/trojanzoo/lib/python3.9/site-packages/torch/serialization.py", line 211, in init
super(_open_file, self).init(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'data/model/image/cifar10/resnetcomp18.pth'
To Reproduce
python backdoor_attack.py --verbose 1 --color --validate_interval 1 --dataset cifar10 --model resnetcomp18 --attack reflection_backdoor --random_init --epoch 50 --lr 0.01 --save --train_mode dataset

Screenshots

Desktop (please complete the following information):

• OS: ubuntu 18.04
• version: aeb4482

Describe the bug
Traceback (most recent call last):
File "/home/workspace/backdoor_attack.py", line 37, in
attack.attack(**trainer)
File "/home/workspace/trojanvision/attacks/backdoor/reflection_backdoor.py", line 46, in attack
loader = self.dataset.get_dataloader(mode='train', batch_size=self.candidate_num, classes=[self.target_class],
File "/home/workspace/trojanvision/datasets/imageset.py", line 43, in get_dataloader
dataset = self.get_dataset(mode, **kwargs)
File "/home/workspace/trojanzoo/datasets/dataset.py", line 148, in get_dataset
dataset = self.get_class_set(dataset=dataset, classes=classes)
File "/home/workspace/trojanzoo/datasets/dataset.py", line 158, in get_class_set
_, targets = cls.to_memory(dataset=dataset, label_only=True)
File "/home/workspace/trojanvision/datasets/imageset.py", line 77, in to_memory
data, targets = super().to_memory(dataset=dataset, label_only=label_only)
TypeError: super(type, obj): obj must be an instance or subtype of type

To Reproduce
python backdoor_attack.py --verbose 1 --color --validate_interval 1 --dataset cifar10 --model resnet18 --attack reflection_backdoor --random_init --epoch 50 --lr 0.01 --save --train_mode dataset

Screenshots

Desktop
Ubuntu 5.4.0-6ubuntu1~16.04.12

Additional context
The same problem also occur when run clean_label attack

When I run "python train.py --color --verbose 1 --dataset gtsrb --lr_scheduler --cutout --grad_clip 5.0 --save --download --epoch 50", I meet the follow problem:

Traceback (most recent call last):
File "/home/itl/Documents/xrh/backdoor/trojanzoo/examples/train.py", line 30, in
dataset = trojanvision.datasets.create(**kwargs)
File "/home/itl/Documents/xrh/backdoor/trojanzoo/examples/../trojanvision/datasets/init.py", line 78, in create
return trojanzoo.datasets.create(dataset_name=dataset_name, dataset=dataset,
File "/home/itl/Documents/xrh/backdoor/trojanzoo/examples/../trojanzoo/datasets.py", line 534, in create
return DatasetType(**result)
File "/home/itl/Documents/xrh/backdoor/trojanzoo/examples/../trojanvision/datasets/folder/gtsrb.py", line 40, in init
return super().init(norm_par=norm_par, loss_weights=loss_weights, **kwargs)
File "/home/itl/Documents/xrh/backdoor/trojanzoo/examples/../trojanvision/datasets/imagefolder.py", line 79, in init
super().init(**kwargs)
File "/home/itl/Documents/xrh/backdoor/trojanzoo/examples/../trojanvision/datasets/imageset.py", line 150, in init
super().init(default_model=default_model, **kwargs)
File "/home/itl/Documents/xrh/backdoor/trojanzoo/examples/../trojanzoo/datasets.py", line 163, in init
loss_weights = self.get_loss_weights() if loss_weights else None
File "/home/itl/Documents/xrh/backdoor/trojanzoo/examples/../trojanzoo/datasets.py", line 450, in get_loss_weights
_, targets = dataset_to_tensor(dataset)
File "/home/itl/Documents/xrh/backdoor/trojanzoo/examples/../trojanzoo/utils/data.py", line 87, in dataset_to_tensor
return torch.stack(data), torch.as_tensor(targets, dtype=torch.long)
TypeError: expected Tensor as element 0 in argument 0, but got Image

I'm currently doing a bunch of breaking changes on trojanvision.marks.Watermark class, which might lead to misfunctionality of some backdoor attacks/defenses.

Will fix this very soon. And the documents are almost done!

"python backdoor_defense.py --color --verbose 1 --pretrained --validate_interval 1 --dataset cifar10 --model resnet18_comp --attack badnet --defense deep_inspect --mark_random_init --epochs 50 --lr
0.01 --save --tqdm"
When I ran the command above to detect badnets with DeepInspect, I got the error: DeepInspect.optimize_mark() got an unexpected keyword argument 'verbose'. Please help me.

Traceback (most recent call last):
File "/home/itl/Documents/xrh/backdoor/trojanzoo/examples/backdoor_defense.py", line 46, in
defense.detect(**trainer)
File "/home/itl/Documents/xrh/backdoor/trojanzoo/examples/../trojanvision/defenses/abstract.py", line 345, in detect
mark_list, loss_list, asr_list = self.get_mark_loss_list()
File "/home/itl/Documents/xrh/backdoor/trojanzoo/examples/../trojanvision/defenses/abstract.py", line 386, in get_mark_loss_list
mark, loss = self.optimize_mark(label, verbose=verbose, **kwargs)
TypeError: DeepInspect.optimize_mark() got an unexpected keyword argument 'verbose

Can somebody please let me know the transforms used in cfir10 and mnist datasets?
Thank you.

I use the recommended scrip to train model on cifar10 as below:

python ./examples/train.py --color --verbose 1 --dataset  cifar10 --model resnet18_comp --lr_scheduler --cutout --grad_clip 5.0 --save

However, I get the following error

File "~/trojanzoo/trojanzoo/utils/model.py", line 176, in activate_params                                         
    module.requires_grad_(False)                             
AttributeError: 'ResNet' object has no attribute 'requires_grad_'

There are some attacks not included in TrojanZoo, especially for those dynamic backdoors.

• Blended Attack
• WaNet
• Label-consistent Attack
• BlindBackdoor
• Input-Aware dynamic attack
• LIRA: Learnable, Imperceptible and Robust Backdoor Attacks
• SleeperAgent
• Invisible Backdoor Attack with Sample-Specific Triggers

I trained the default model on MNIST dataset, and use badnet as backdoor attack method. Then I want to do backdoor defense.
First I used your example command in README.md.
python ./examples/backdoor_defense.py --color --verbose 1 --pretrain --validate_interval 1 --dataset mnist --attack badnet --defense neural_cleanse --random_init --epoch 50 --lr 0.01
but it seemed to detect backdoor attack rather than backdoor defense.
Last I use fine_pruning to do backdoor defense, the command as following:
python ./examples/backdoor_defense.py --color --verbose 1 --pretrain --validate_interval 1 --dataset mnist --attack badnet --defense fine_pruning --random_init --epoch 50 --lr 0.01
it came with an error "IndexError: index 4694 is out of bounds for dimension 0 with size 64".
How can I solve it? or How can I do backdoor defense?

THANKS!
I'll appreciate it if you can answer it.

(1)
Describe the bug
There seems to be something wrong in the 298th line in model.py. Is the path correctly set?

To Reproduce
I tried to run this project with the command in the Quick Start:
python train.py --verbose 1 --amp --dataset cifar10 --model resnetcomp18 --epoch 150 --lr 0.1 --lr_scheduler --lr_decay_step 50 --save

Then I got an error when saving the model:

I checked the code and change the 298th line in model.py:
from
file_path = os.path.normpath(os.path.join(folder_path, self.name, suffix, '.pth'))
to
file_path = os.path.normpath(os.path.join(folder_path, suffix, self.name + '.pth'))

Then the code ran normally and the model is saved at: data\model\image\cifar10\resnetcomp18.pth

I wonder whether I am doing the right thing?

Desktop (please complete the following information):

• OS: Windows 10

(2)
Describe the bug
I found another problem about path when running backdoor_attack.py.
Just run the second command in Quick Start and I got an error saying like "os.path.split() got wrong arguments".

Then I changed the 92nd line in badnets.py to fix this, form:
mark_filename = os.path.split(self.mark.mark_path)
to:
mark_filename = os.path.split(self.mark.mark_path)[-1]

Although the code ran normally, the files about attack seemed not correctly stored:

Could the authors tell me whether there is a better way to fix this?

(3)
Describe the bug
The command to run “IMC” in Quick Start seems to be not correct, should it be:
python backdoor_attack.py --verbose 1 --pretrain --validate_interval 1 --amp --dataset cifar10 --model resnetcomp18 --attack imc --random_init --epoch 50 --lr 0.01 --save ?

Further more, I found I cannot run this command unless I change the 71st line in the trojannn.py:
from:
self.pgd = PGD(alpha=self.neuron_lr, epsilon=1.0, iteration=self.neuron_epoch, output=0)
to:
self.pgd = PGD(alpha=self.neuron_lr, epsilon=1.0, iteration=self.neuron_epoch, output=0, model=self.model)

Hi,
Sorry, is TABOR's hyperparameter augmentation in trojanzoo 1.0.8?

Hi,
What's the use of pre-train in defense, please?

Thanks.

Since you note in your paper that you conduct various of attacks and defenses on different datasets. I wonder if there are tables of result between each attack and defense on those datasets? I only see those summarized analytic results on different aspects, instead of detailed full result tables.

Thank you.

See concrete issue at:
pytorch/vision#2269
pytorch/pytorch#39088

The issue about bug will be fixed in later torchvision versions.
Therefore, I will not work around by myself but only use num_workers=0. This will be slower compared with multiple workers. 
We'll support num_workers when the bug is fixed in torchvision.

It seems like the adjusted trojanzoo/utils/train.py goes wrong in train() when comparing between the best_acc and cur_acc to save the model.

best_acc will preserve the original clean ACC at the very beginning but the attack will decrease the ACC, and therefore the backdoored model will never be saved...QAQ

It seems that adding the mark is behind normal data transformations in badnet-like attacks according to your code badnet.py. However, in real-world scenarios, adding the mark is before normal data transformations. Did I get it wrong？

Describe the bug
When I tried to load the GTSRB dataset, I got some error messages. However, when I change the input argument from '-d gtsrb' to '-d mnist' or '-d cifar10', things went to work well.

To Reproduce
Here is my test.py file:

# -*- coding: utf-8 -*-

import trojanvision.datasets
import argparse


parser = argparse.ArgumentParser()
trojanvision.datasets.add_argument(parser)
args = parser.parse_args()

dataset = trojanvision.datasets.create(**args.__dict__)
print(type(dataset))

In the terminal, when I ran:

python test.py -d mnist --download
python test.py -d cifar10 --download

The works were done successfully. The original data were downloaded and loaded into the script. However, when I tried to load GTSRB, I tried several different arguments:

python test.py -d gtsrb --download
python test.py -d gtsrb --download --data_format zip
python test.py -d gtsrb --download --data_format folder
python test.py -d gtsrb --download --data_format numpy

Different errors were reported. I also tried to manually download the GTSRB.zip and placed it at ./datasets/data/data/image/ and re-ran the script, but things did not turn to work. For example. after manually downloading the dataset and unzipping it:

python test.py -d gtsrb --data_dir ./data/data/ --data_format folder

I got:

Traceback (most recent call last):
  File "/home/yl764/datasets/test.py", line 11, in <module>
    dataset = trojanvision.datasets.create(**args.__dict__)
  File "/home/yl764/miniconda3/envs/datasets/lib/python3.9/site-packages/trojanvision/datasets/__init__.py", line 37, in create
    return trojanzoo.datasets.create(dataset_name=dataset_name, dataset=dataset,
  File "/home/yl764/miniconda3/envs/datasets/lib/python3.9/site-packages/trojanzoo/datasets.py", line 246, in create
    return DatasetType(folder_path=folder_path, **result)
  File "/home/yl764/miniconda3/envs/datasets/lib/python3.9/site-packages/trojanvision/datasets/gtsrb.py", line 17, in __init__
    super().__init__(loss_weights=loss_weights, **kwargs)
  File "/home/yl764/miniconda3/envs/datasets/lib/python3.9/site-packages/trojanvision/datasets/imagefolder.py", line 36, in __init__
    super().__init__(**kwargs)
  File "/home/yl764/miniconda3/envs/datasets/lib/python3.9/site-packages/trojanvision/datasets/imageset.py", line 26, in __init__
    super().__init__(default_model=default_model, **kwargs)
  File "/home/yl764/miniconda3/envs/datasets/lib/python3.9/site-packages/trojanzoo/datasets.py", line 89, in __init__
    self.loss_weights = self.get_loss_weights() if loss_weights else None
  File "/home/yl764/miniconda3/envs/datasets/lib/python3.9/site-packages/trojanzoo/datasets.py", line 192, in get_loss_weights
    _, targets = self.to_memory(dataset, label_only=True)
TypeError: to_memory() got multiple values for argument 'label_only'

Expected behavior
Load the dataset .

Desktop (please complete the following information):

• OS: Ubuntu 18.04