TargetClassProbability classifier not working properly about foolbox

[jonasrauber]

Hi @realbadbytes! The way you apply the attack is correct. Just the interpretation of the results is slightly wrong. The returned adversarial is actually classified as 781 (scoreboard) with a probability of 84%.

After running the attack using adversarial = attack(image[:, :, ::-1], label), you can check it like this:

np.argmax(fmodel.predictions(adversarial))
This outputs 781.

Note that the returned adversarial has BGR color channel ordering (so for plotting you would need to reverse it, but not for passing it to the Keras model kmodel or its Foolbox-compatible wrapper fmodel). Also not that I am passing it to the Foolbox model that handles the preprocessing for us.

To get the the probability, you can apply the softmax to the logits returned by the Foolbox-compatible model wrapper like this: foolbox.utils.softmax(fmodel.predictions(adversarial))[781]
This outputs 0.830603.

Now you wanted to pass it directly to the original Keras model. To do that, you need to apply the preprocessing expected by Keras yourself, i.e. transform from RGB to BGR and subtracting the mean. You can either do it manually, or you use Keras preprocess_input function.

Manually:
The adversarial we have is already BGR, because the Keras Resnet model expected it like this. So all we need to do is add the 4th dimension and subtract the mean:
preprocessed_adv = adversarial[np.newaxis] - preprocessing[0].reshape(1, 1, 1, 3)
And then kmodel.predict(preprocessed_adv)[0, 781] outputs 0.83060318.

Using preprocess_input:
from keras.applications.resnet50 import preprocess_input
We need to transform the BGR adversarial back to RGB because Keras preprocess_input will transform from RGB to BGR. And we need to add the 4th dimension.
adversarial_rgb = adversarial[np.newaxis, :, :, ::-1]
And then kmodel.predict(preprocess_input(adversarial_rgb.copy()))[0, 781] outputs 0.82709205.
Note, we copy the array, because preprocess_input works inplace, for whatever reason and so without copying, we would run into problems when calling it multiply times.

Finally, we can also look at the output of decode_predictions:
from keras.applications.resnet50 import decode_predictions
preds = kmodel.predict(preprocess_input(adversarial_rgb.copy()))
print("Top 5 predictions (adversarial: ", decode_predictions(preds, top=5))
This outputs Top 5 predictions (adversarial: [[('n04149813', 'scoreboard', 0.82709193), ('n03196217', 'digital_clock', 0.03087672), ('n04152593', 'screen', 0.014944675), ('n04141975', 'scale', 0.01231501), ('n04074963', 'remote_control', 0.0091410046)]].

Summary:

import foolbox
from foolbox.models import KerasModel
from foolbox.attacks import LBFGSAttack
from foolbox.criteria import TargetClassProbability
import numpy as np
import keras
from keras.applications.resnet50 import ResNet50
from keras.applications.resnet50 import preprocess_input
from keras.applications.resnet50 import decode_predictions

keras.backend.set_learning_phase(0)
kmodel = ResNet50(weights='imagenet')
preprocessing = (np.array([104, 116, 123]), 1)
fmodel = KerasModel(kmodel, bounds=(0, 255), preprocessing=preprocessing)

image, label = foolbox.utils.imagenet_example()

# run the attack
attack = LBFGSAttack(model=fmodel, criterion=TargetClassProbability(781, p=.5))
adversarial = attack(image[:, :, ::-1], label)

# show results
print(np.argmax(fmodel.predictions(adversarial)))
print(foolbox.utils.softmax(fmodel.predictions(adversarial))[781])
adversarial_rgb = adversarial[np.newaxis, :, :, ::-1]
preds = kmodel.predict(preprocess_input(adversarial_rgb.copy()))
print("Top 5 predictions (adversarial: ", decode_predictions(preds, top=5))

outputs

781
0.832095
Top 5 predictions (adversarial:  [[('n04149813', 'scoreboard', 0.83013469), ('n03196217', 'digital_clock', 0.030192226), ('n04152593', 'screen', 0.016133979), ('n04141975', 'scale', 0.011708578), ('n03782006', 'monitor', 0.0091574294)]]

[jonasrauber]

@realbadbytes Please close this issue once you confirmed that it works.

[realbadbytes]

Thank you very much for the detailed explanation. Working as intended