How to perform defense about if-defense

[Wuziyi616]

Hi, thanks for your interest in our work. As you said, we adopted setting2 (black-box attack) where the attacker doesn't know the defense method. This is partially because IF-Defense is non-differentiable (hard to calculate the gradient of the optimization process), so we cannot adopt setting1 (white-box attach) against IF-Defense. Therefore, to perform a fair comparison, we test all the defense/attacks under setting2.

As described above, it's hard to test IF-Defense under setting1. Instead, I tried the so-called gray-box attack or adaptative attack, where the attacker has some knowledge of our defense method, and has special design to attack it. Please see our paper Section 4.2 Adaptive attack part for the results. Overall, the Adaptive attack indeed outperforms setting2 attack, but our defense accuracy is still okay.

[Wuziyi616]

Btw, I do implement the drop-200 attack in this repo here, which can reproduce the results reported in the paper.

[Daniel-xsy]

Get it, thanks for your kind reply!