Request - Add support for Zits and MAT inpainting which are the current SoTa about lama-cleaner

[styler00dollar]

Looks like the comments I replied to are gone. Stating this to avoid confusion. Maybe I should remove mine too..

[Sanster]

MAT: https://github.com/Sanster/lama-cleaner/releases/tag/0.16.0

[Sanster]

ZITS: https://github.com/Sanster/lama-cleaner/releases/tag/0.15.0

[styler00dollar]

I feel like you don't properly use MAT. I used your MAT output image to draw over it with MAT again. Maybe your code is wrong.

[styler00dollar]

I created a few different images and I guess I left just one which I felt was the best one which I did in my limited testing. I just wanted to illustrate that I do not get the output you got with MAT. I just randomly drew a mask, not a square. I modded lama cleaner to have MAT which I used for this. It's still in a rough shape but functional. At the bottom I felt like trying to remove the lines since you showed color inconsistencies in the other images there. You did not even provide a mask, how am I supposed to guess what you did. My whole point was to show that MAT does not create such artifacts which you said it does, nothing else, and I think I archived that. You don't need to act rude.

[styler00dollar]

This has nothing to do with the output quality of MAT and I can't be bothered to trace the pixels you inpainted. I just made a rough mask which is bigger and also masked a few minor details. My intention was not to perfectly replicate what you did. I won't repeat that again. If you can't contribute anything useful then just stop responding.

[styler00dollar]

Who cares about the mask? I talk about the output quality of MAT. If you are looking for a fight, don't do it on Github and do it somewhere else. Github is a place for developers and not about arguing about stuff that is not even relevant. I guess I will just ignore further comments, if you can't seem to make any kind of productive comment. You could retry MAT, or whatever, instead you just pick on something that was never important.

[styler00dollar]

What part of "I just randomly drew a mask, not a square. [...] At the bottom I felt like trying to remove the lines since you showed color inconsistencies" did you not get. Also, I am just pointing out that you somehow wrongly use MAT. I do not know how, but you did. Here, I did it again. Keep in mind that I added MAT into my code.

There are no such artefacts like in your picture.

[styler00dollar]

I feel like you are trolling at this point. I took your mat as a base image to proof that the inpainted stuff does not look like your result and you did something wrong. It simply does not matter what image I use. Water looks like actual water and not this black/white whatever striped artifact. That was the only thing I wanted to show, nothing else. Absolutely nothing else. Why do you bring up stuff I never mentioned or cared about I wonder..

[Sanster]

@styler00dollar I added the MAT model in this PR. Have you found any practical use where MAT works better than LaMa? In my tests, the MAT tends to produce strange results and does not perform well on high resolution, which I don't feel is very practical. Maybe there is something wrong with my code, will take some time to check it.

[styler00dollar]

MAT is non-deterministic because of z = torch.randn(images_in.size(0), self.img_resolution), so you can generate a different image all the image, and since it is a transformer, you don't need to wait like in diffusion, which is what I liked about MAT. I didn't really do comparisons against lama. I mainly tested it on drawings and it did well there though. I only used it on 512x512 due to VRAM.