About dataset setting for training and inference. about doctamper

[qcf-568]

Hello,

The resize function is forbidden, since this method relies on the Block Artifacts Grids (BAGs) for tampering localization. When the input image is resized (bigger or smaller), its BAGs will be destroyed, so this method does not work in this case.

However, by utilizing the BAGs, model can gain much more better fine-grained perception ability to detect visually consistent tampering, which is important for tampered text detection in documents. Moreover, by utilizing the BAGs, model can gain much more better cross-domain generalization.
So, this methods is sacrificing anti-resize for better detection and generalization abilities.

It's also notable that if an image is resized or never has BAGs (e.g. smartphone screen captures), it can be easily distinguished from the images that have BAGs in Fourier frequency domain by a simple binary classification model.
Therefore, in real-world application, we can conduct tampering localization in a Mixture of Experts manner: first, use a classifier to identify whether the input image has BAGs, if it has BAGs, then crop it to patches and feed the patches into the frequency-based model like this model; if it does not have BAGs, then feed it into a common pure RGB model that is anti-resize. By doing this, the advantages of both of the two model types can be maximized.
In some toB cases, we also can tell users not to resize the image before upload and let them re-upload one if the original one is detected to be resized.

For question 2, the performance is totally not related to "clarity", it is totally related to the existence of BAGs.

[Nomiluks]

@RobotDouble Could you please help me to understand how to test it on our own dataset?

[qcf-568]

@Nomiluks Please refer to L39~L137 https://github.com/qcf-568/DocTamper/blob/main/models/tsroie/infer_sroie.py​

and this colab notebook https://colab.research.google.com/drive/1rWaSKy2Rsy5welyvj6FbzF01o2zv8ips?usp=sharing