Comments (16)
Hi @EdwardRaff, sorry for not responding earlier. Somehow, I accidentally marked this issue as read.
In principle, it should be no problem to use Foolbox in this scenario. Image-specific things like the channel_axis
argument can probably simply be ignored (they are only relevant for certain image-specific attacks). Same goes for arguments that have image-specific names (e.g. image
, original_image
), just use them as if they were named in a more generic way (we will fix this in a future version).
from foolbox.
@EdwardRaff can this be closed?
from foolbox.
I don't think so. I just tried using the library as you state, and get an unhelpful error message.
Traceback (most recent call last):
File "fool_malconv.py", line 108, in <module>
adversarial = attack(batch, 1)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/attacks/base.py", line 89, in __call__
find = Adversarial(model, criterion, image, label)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/adversarial.py", line 61, in __init__
self.predictions(original_image)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/adversarial.py", line 235, in predictions
assert not strict or self.in_bounds(image)
AssertionError
where the code that does the work
fmodel = PyTorchModel(model, bounds=(0, 255), num_classes=2, cuda=args.num_gpus > 0)
attack = foolbox.attacks.FGSM(fmodel)
adversarial = attack(batch, 1)
from foolbox.
Looks like that error was for having too lose a bound, but still results in an error
Traceback (most recent call last):
File "fool_malconv.py", line 108, in <module>
adversarial = attack(batch, 1)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/attacks/base.py", line 89, in __call__
find = Adversarial(model, criterion, image, label)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/adversarial.py", line 61, in __init__
self.predictions(original_image)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/adversarial.py", line 238, in predictions
predictions = self.__model.predictions(image)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/models/base.py", line 122, in predictions
return np.squeeze(self.batch_predictions(image[np.newaxis]), axis=0)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/models/pytorch.py", line 64, in batch_predictions
predictions = self._model(images)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 206, in __call__
result = self.forward(*input, **kwargs)
File "/home/edraff/Development/SeveringMalConv/web/model.py", line 73, in forward
x = self.lookup_table(x)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 206, in __call__
result = self.forward(*input, **kwargs)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 94, in forward
)(input, self.weight)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/torch/nn/_functions/thnn/sparse.py", line 33, in forward
assert indices.dim() <= 2
AssertionError
from foolbox.
Could you please provide the full code or a minimal example to reproduce the problem.
Btw, you used bounds=(0, 255)
which looks wrong if you have non-image inputs. What kind of data do you have?
Also, you are feeding a variable called batch
to the attack, which sounds like it could be the reason for the above error message. When applying an attack, you should provide a single input and the corresponding label.
from foolbox.
My data is byte based, so there are 256 possible byte values (and a special EOF token that I think it was unhappy about).
Unfortunately, I'm not allowed to share the code at this moment in time.
I was doing batches but a batch size of 1. I used numpy.squeeze to remove the first dimension, and now get this error.
Traceback (most recent call last):
File "fool_malconv.py", line 110, in <module>
adversarial = attack(np.squeeze(batch,axis=0), 1)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/attacks/base.py", line 89, in __call__
find = Adversarial(model, criterion, image, label)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/adversarial.py", line 61, in __init__
self.predictions(original_image)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/adversarial.py", line 238, in predictions
predictions = self.__model.predictions(image)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/models/base.py", line 122, in predictions
return np.squeeze(self.batch_predictions(image[np.newaxis]), axis=0)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/models/pytorch.py", line 65, in batch_predictions
predictions = predictions.data
AttributeError: 'tuple' object has no attribute 'data'
from foolbox.
Which Foolbox and PyTorch version are you using?
Is your PyTorch model (the model
you pass to foolbox.models.PyTorch
) an instance of PyTorch's nn.Module
.
It looks like the forward
method of your nn.Module
subclass does return a tuple instead of Tensor with a data
attribute.
from foolbox.
Foolbox currently assumes that the forward
method returns a torch.autograd.Variable
that holds the tensor that contains the predictions. Maybe this is a bit too strict on our side, but so far it worked fine. Foolbox will automatically convert inputs from NumPy to a PyTorch Tensor to a PyTorch Variable holding that Tensor, then apply the PyTorch model (effectively calling the forward
method) and then unpacking the result again, i.e. it calls the PyTorch model with a PyTorch Variable and expects to get one back.
A very simple net (used in our test cases) can for example be implemented like this. It doesn't do anything useful, but shows that things should be fine as long as your forward
method just does the usual transformations.
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
def forward(self, x):
x = torch.mean(x, 3)
x = torch.squeeze(x, dim=3)
x = torch.mean(x, 2)
x = torch.squeeze(x, dim=2)
logits = x
return logits
from foolbox.
I'm using PyTorch version 1.12 and the latest version of foolbox from pip.
The information about what is expected is very useful. Please consider adding it to the documentation. I'd also encourage some error messages that indicate what the problem is.
I've adjusted my code to match your stated exceptions. But I'm still having some trouble. If i return the logits as an array of shape (2) (1 dim, 2 values for the 2 classes), I get an error in batch_predictions asserting that the predictions.ndim == 2.
Traceback (most recent call last):
File "fool_malconv.py", line 116, in <module>
adversarial = attack(np.squeeze(batch,axis=0), 1)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/attacks/base.py", line 89, in __call__
find = Adversarial(model, criterion, image, label)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/adversarial.py", line 61, in __init__
self.predictions(original_image)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/adversarial.py", line 238, in predictions
predictions = self.__model.predictions(image)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/models/base.py", line 122, in predictions
return np.squeeze(self.batch_predictions(image[np.newaxis]), axis=0)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/models/pytorch.py", line 69, in batch_predictions
assert predictions.ndim == 2
AssertionError
If I return an array of shape (1,2), i get an error in predictions_and_gradient asserting that the image dimension is == 3.
Traceback (most recent call last):
File "fool_malconv.py", line 116, in <module>
adversarial = attack(np.squeeze(batch,axis=0), 1)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/attacks/base.py", line 98, in __call__
_ = self._apply(adversarial, **kwargs)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/attacks/gradientsign.py", line 22, in _apply
gradient = a.gradient()
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/adversarial.py", line 322, in gradient
gradient = self.__model.gradient(image, label)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/models/base.py", line 208, in gradient
_, gradient = self.predictions_and_gradient(image, label)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/models/pytorch.py", line 89, in predictions_and_gradient
assert image.ndim == 3
AssertionError
from foolbox.
The second one is correct. The PyTorch model (i.e. the nn.Module
subclass) should always deal with batches, i.e. expect batches and return batches, so in your case, it should return something with shape (1, 2)
.
The error you see in that case is indeed in image-specific assertion that should not be there. I will change this as soon as possible, but it will take a bit of time (running all the tests, publishing a new version on PyPI, etc.)… For now, it would be great if you could just open the file foolbox/models/pytorch.py
and comment the lines 89 and 159 (both saying assert image.ndim == 3
) out. You can find the location of that file by opening a Python interpreter, running import foolbox
and then running foolbox
. That should print the path where the foolbox package is located in your system. Something like this:
Python 3.6.3 (default, Oct 6 2017, 08:44:35)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import foolbox
>>> foolbox
<module 'foolbox' from 'SOME PATH TO foolbox/__init__.py'>
Just go to that path and edit the two lines I mentioned. Let me know if this fixed the problem.
Sorry for all the trouble. Originally, we developed Foolbox in a very image-centric way. As I said, this will change and for TensorFlow I think I verified that it works, but apparently, for PyTorch these assertions were still left.
Next time it would great if you could create a minimal example that reproduces the error – it doesn't need to be your actual secret code ;-)
Regarding the documentation: yes, it has to improve ;-)
from foolbox.
No worries, I appreciate all the help! I made the changes you mentioned, and it now gets to what I was afraid of: what to do about the embedding layer. This is the error the comes out
Traceback (most recent call last):
File "fool_malconv.py", line 116, in <module>
adversarial = attack(np.squeeze(batch,axis=0), 1)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/attacks/base.py", line 98, in __call__
_ = self._apply(adversarial, **kwargs)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/attacks/gradientsign.py", line 22, in _apply
gradient = a.gradient()
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/adversarial.py", line 322, in gradient
gradient = self.__model.gradient(image, label)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/models/base.py", line 208, in gradient
_, gradient = self.predictions_and_gradient(image, label)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/models/pytorch.py", line 95, in predictions_and_gradient
predictions = self._model(images)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 206, in __call__
result = self.forward(*input, **kwargs)
File "/home/edraff/Development/SeveringMalConv/web/model_infer.py", line 77, in forward
x = self.lookup_table(x)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 206, in __call__
result = self.forward(*input, **kwargs)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 94, in forward
)(input, self.weight)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/torch/nn/_functions/thnn/sparse.py", line 34, in forward
assert not self.needs_input_grad[0], "Embedding doesn't " \
AssertionError: Embedding doesn't compute the gradient w.r.t. the indices
Yea, I know a simplified example would help. Unfortunately I don't get to fully dictate my time :-/ I'm working on getting approval to open source the code I'm working on as its not significant, but that takes time.
from foolbox.
I also tried the LocalSearchAttack since it seems to be non-gradient based, but it errors on line 82 asserting that the image has two axes. And the axes object is empty.
Traceback (most recent call last):
File "fool_malconv.py", line 117, in <module>
adversarial = attack(np.squeeze(batch,axis=0), 1)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/attacks/base.py", line 98, in __call__
_ = self._apply(adversarial, **kwargs)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/attacks/localsearch.py", line 87, in _apply
assert len(axes) == 2
AssertionError
from foolbox.
Well, if your model isn't differentiable w.r.t. its inputs, it's hard to apply a gradient-based adversarial attack. The problem with the LocalSearchAttack is: it has built-in notion of pixels and is therefore image-specific. I do think it should be possible to generalize the attack to arbitrary inputs, but I haven't thought about it in detail. Maybe you want to give it a try? Alternatively, it might be possible to change the shape of your data to something that looks like an image, e.g. an image with width 1, height N and 1 color channel. Again, no guarantees, but I think that should be easy and might work. Finally, you can use a decision-based attack: it requires even less than the LocalSearchattack
. To get started and make sure that your code works, I recommend something basic like the AdditiveUniformNoiseAttack
. Once everything works, you can use our recently published BoundaryAttack
. https://foolbox.readthedocs.io/en/latest/modules/attacks/decision.html
from foolbox.
Well, if your model isn't differentiable w.r.t. its inputs, it's hard to apply a gradient-based adversarial attack.
That is what I was asking about in my first post since there is an embedding layer. I wasn't sure how it would be handled.
I was able to get it to work with the AdditiveUniformNoiseAttack
attack. I did try the boundary attack after, but got the following error. I'm not sure where its getting a non float input from.
Traceback (most recent call last):
File "fool_malconv.py", line 119, in <module>
adversarial = attack(np.squeeze(batch,axis=0), 1)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/attacks/boundary_attack.py", line 165, in __call__
threaded_gen=threaded_gen)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/attacks/base.py", line 98, in __call__
_ = self._apply(adversarial, **kwargs)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/attacks/boundary_attack.py", line 183, in _apply
return self._apply_inner(pool, *args, **kwargs)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/attacks/boundary_attack.py", line 206, in _apply_inner
assert external_dtype in [np.float32, np.float64]
AssertionError
I also tried it with the GPU since the error was so strange, and got this even more confusing error. It looks like it got what it was expecting.
Traceback (most recent call last):
File "fool_malconv.py", line 119, in <module>
adversarial = attack(np.squeeze(batch,axis=0), 1)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/attacks/boundary_attack.py", line 165, in __call__
threaded_gen=threaded_gen)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/attacks/base.py", line 89, in __call__
find = Adversarial(model, criterion, image, label)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/adversarial.py", line 61, in __init__
self.predictions(original_image)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/adversarial.py", line 238, in predictions
predictions = self.__model.predictions(image)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/models/base.py", line 122, in predictions
return np.squeeze(self.batch_predictions(image[np.newaxis]), axis=0)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/foolbox/models/pytorch.py", line 64, in batch_predictions
predictions = self._model(images)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 206, in __call__
result = self.forward(*input, **kwargs)
File "/home/edraff/Development/SeveringMalConv/web/model_infer.py", line 79, in forward
x = self.lookup_table(x)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 206, in __call__
result = self.forward(*input, **kwargs)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 94, in forward
)(input, self.weight)
File "/home/edraff/anaconda3/lib/python3.6/site-packages/torch/nn/_functions/thnn/sparse.py", line 53, in forward
output = torch.index_select(weight, 0, indices.view(-1))
TypeError: torch.index_select received an invalid combination of arguments - got (torch.cuda.FloatTensor, int, torch.LongTensor), but expected (torch.cuda.FloatTensor source, int dim, torch.cuda.LongTensor index)
from foolbox.
If your model requires inputs to be integers, it's hard to apply any of these attacks directly. Sorry if I missed that detail in your first message. I think it should still be possible to do what you want, but one first needs to define adversarials in your setting properly and then think about how they could be found. Technically, it might be possible that you define a model that accepts floats and rounds them to integers and then applying the attacks to that model. Without your code and a description of the scenario, I don't think I can be much of a help here. You might also want to get familiar with Foolbox in a more common setting (say images or even audio data) first and then think about the differences to your problem.
from foolbox.
Hi,@jonasrauber. I meet several problems that confused me. My python is Python 3.6.3 on Windows64, and my computer can not connect the Internet. So I need your help. Can you give me an example? This example shows how to use foolbox to generate adversarial example for MNIST. If you can give me the code is better. Hoping the code is shorter and shows explanation. Thanks.
from foolbox.
Related Issues (20)
- Example Code Running Failed HOT 1
- [tests/test_models] The results of `transform_bounds` are inconsistent between CPU and GPU. HOT 3
- Are there any plans to support attacks on TFLite models? HOT 1
- Changing CUDA device at runtime HOT 1
- Logit optimization
- about PGD attack HOT 2
- specifying criterion fails with TypeError HOT 2
- "nll_loss_forward_no_reduce_cuda_kernel_index" not implemented for 'Float' HOT 3
- Deprecation warning using old scipy namespace for gaussian_filter
- how to define the bounds HOT 2
- About the pgd attacks HOT 1
- how to use GaussianBlurAttack HOT 1
- FGSM TargetedMisclassfication HOT 1
- Use foolbox for multi-label classification HOT 1
- Local datasets supported?
- Is there a criterion for query budget? HOT 1
- It seems like the 'success' value in the return of the 'attack' function is overconfident. HOT 2
- About Carlini-Wagner Attack
- Are the wrong classified images sorted out? HOT 1
- It seems your CI/CD has a bug. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from foolbox.