Why is "EER%" lower in MN3_large than AENET? about light-weight-face-anti-spoofing

[kprokofi]

Hi,
The lower EER the better. On CelebA_Spoof Aenet has better results. It has more parameters and better fits the data. But on cross-domain performance (LCC_FASD), the MN3 has better descriptive ability based on metrics. At the same time MN3 noticeably faster in real-time performance.

[kadirbeytorun]

Yeah MN3 is much faster, I noticed that too. Yet, since antispoofing is a very challenging task with only rgb frame, even AENET isn't very stable, but it is the most accurate one I came accross so far. It's speed is enough for me as well.

Also CelebA_Spoof dataset is the best one I have seen so far, I even found your project while searching CELEBA_Spoof, so AENET being good only on their own dataset isn't the worst thing.

Or do you think LCC_FASD is better and more general, thus MN3 trained with LCC_FASD is a better solution?

[kprokofi]

I agree CelebA_Spoof dataset is the best one for now. MN3 is trained on it and LCC_FASD is used for testing purposes. This is a good option since CelebA has more quality images. From professional camera and magazine covers. LCC_FASD has different images taken from poor quality webcams and smartphones, it's more challenging for testing. When I developed the training pipeline I came across a cross-domain problem. The better model fit the CelebA the worse results were on LCC_FASD. Applying some techniques I managed to receive almost the same metrics on CelebA as the AENET did, but It turns out that generalization dropped a lot. So, I had to choose the golden middle sacrificing metric on the CelebA.

Overall, it's a really challenging task, both of the models perform not so stable in real life and there is room for growth and improvement.

[kadirbeytorun]

Yeah I agree, its one of the most challenging tasks in ML and also unfortunately not a very popular one.

About being stable, both aenet and mn3 depend heavily on the quality of camera and lighting conditions. After trial and errors, I came to the conclusion they work the best with FHD camera (with high NIR efficiency) and external NIR lighting. Only challenge I faced is, network mistakes surgical masked face as fake.