L-BFGS target classes fails about foolbox

[jonasrauber]

You calculate the reference label like this:
label = numpy.argmax(fmodel.predictions(image)),
i.e. with RGB color channel ordering, but the network expects BGR and you also use BGR when running the attack: adversarial = attack(image[:,:,::-1], label, unpack=False)
Therefore, the reference label and the prediction for the input image made by the attack differ and the original image is misclassified.

In other words: just use label = numpy.argmax(fmodel.predictions(image[:, :, ::-1])) to get the reference label.

However, I think Foolbox should abstract away the color channel ordering (RGB vs. BGR) as part of the preprocessing so that the user does not run into these issues. I will probably add this to the next Foolbox release. Thanks @neale!

P.S.: when I run the code in your gist, I do get a warning that helped me track down this problem:

.../foolbox/attacks/base.py:93: UserWarning: Not running the attack because the original image is already misclassified and the adversarial thus has a distance of 0.

Do you get this warning as well, @neale, or is it hidden because of your python configuration? In that case, it would be good to know so that we can make sure that users see the warnings.

[wielandbrendel]

Could you give a minimum working example where this error arises?

[neale]

Of course.
The attack just isn't applied in most cases, I figured it was just my images, but I'm only using the imagenet validation set, so they should be ok as far as that goes.

Here's the test code in a gist
And here is an offending image

This is with a clean pip install, to rule out my edits.
Thanks again!

[wielandbrendel]

@jonasrauber How do you want to abstract channel ordering away? I wouldn't like to introduce anything that is specific to "color channels". Quite the contrary, I'd even like to get rid of the color_channel argument as it is too specific to images. One could nonetheless make it part of the preprocessing in that one can define channel flipping with an additional argument of the type [1,1,-1] (so in this case the spatial axes would not be flipped but the color channel would).

[jonasrauber]

For now, I will add a reverse_channel_order argument (at first I thought about adding a way to specify RGB / BGR, but I also thought that's too image specific). I think the notion of "channels" is general enough for now: we have channel_axis (it's not called color_channel ;-) ) anyway and cannot easily get rid of it, and every ConvNet (not just for images) has that notion.
If we at some point really get rid of channel_axis and other channel related stuff, we might want to think about a more general preprocessing anyway.

[wielandbrendel]

But where is channel_axis actually being used? As far as I can see it's only used in two or three attacks, right?

[jonasrauber]

True, some attacks (don't know how many). But let's have this discussion somewhere else.

@neale If your problem is solved, just close the issue.

[neale]

Thanks!