Point cloud model classification accuracy decreases after hybrid training about if-defense

[Wuziyi616]

What do you mean by accuracy is reduced (the accuracy is lower compared to what)? Do you mean the accuracy of [trained pointnet on defended point cloud] is lower than [original pointnet on clean point cloud]? This is somewhat expected, because defense methods will inevitable distort point clouds a bit, thus degrading the performance. I guess as long as the degradation is not a lot (say, 2%), it is acceptable.

[jamesliu609]

Yes, we use the defense method on mn40_random2048.npz to obtain **def_mn40.npz** for hybrid training of pointnet, and then test pointnet against the attack-free point cloud data def_attack_data.npz after defense processing. Compared with the normally trained pointnet, the accuracy of the hybrid trained pointnet dropped by about 10%. This problem bothers me.

[Wuziyi616]

Hum, that's a lot. Did you only train on def_attack_data.npz, or did you train on a mix of def_attack_data.npz and mn40_random2048.npz? In my paper, I did mixed training, so that the model performance is high on both defended and clean point cloud.

Mix training is just when training the model, we randomly sample data from both sets in a batch.

[jamesliu609]

I trained on a mix of def_attack_data.npz and mn40_random2048.npz, but the model only performs well on clean data.

[Wuziyi616]

That's very weird. In my case I trained on both data, and it works well on both clean data and defended data (still slight drop in performance, but not a lot). Maybe your defended data are too different from the clean data, so there is domain gap, which makes the model hard to perform well in both cases. I suggest you check your defense method for this.

[jamesliu609]

Well, thank you for your patience in answering.