zuruoke / watermark-removal

a machine learning image inpainting task that instinctively removes watermarks from image indistinguishable from the ground truth image

Chimzuruoke Okafor

Watermark-Removal

An open source project that uses a machine learning based image inpainting methodology to remove watermark from images which is totally indistinguishable from the ground truth version of the image.

This project was inspired by the Contextual Attention (CVPR 2018) and Gated Convolution (ICCV 2019 Oral).

And also a shoutout to Chu-Tak Li for his Medium article series that really gave me a deep insight into the image inpainting papers stated above

Run

• use Google colab

• First of all, clone this repo

  !git clone https://github.com/zuruoke/watermark-removal
  

• Change Directory to the repo

  !cd watermark-removal
  

• Since Google Colab uses the latest Tensorflow 2x version and this project uses 1.15.0, downgrade to Tensorflow 1.15.0 version and restart the runtime, (although the new version of Google Colab does not need you to restart the runtime).

  !pip install tensorflow==1.15.0
  

• Install tensorflow toolkit neuralgym.

  !pip install git+https://github.com/JiahuiYu/neuralgym
  

• Download the model dirs using this link and put it under model/ (rename checkpoint.txt to checkpoint because sometimes google drive automatically adds .txt after download)

And you're all Set!!

• Now remove the watermark on the image by runing the main.py file

  !python main.py --image path-to-input-image --output path-to-output-image --checkpoint_dir model/ --watermark_type istock
  

Citing

@article{yu2018generative,
  title={Generative Image Inpainting with Contextual Attention},
  author={Yu, Jiahui and Lin, Zhe and Yang, Jimei and Shen, Xiaohui and Lu, Xin and Huang, Thomas S},
  journal={arXiv preprint arXiv:1801.07892},
  year={2018}
}

@article{yu2018free,
  title={Free-Form Image Inpainting with Gated Convolution},
  author={Yu, Jiahui and Lin, Zhe and Yang, Jimei and Shen, Xiaohui and Lu, Xin and Huang, Thomas S},
  journal={arXiv preprint arXiv:1806.03589},
  year={2018}
}

© Chimzuruoke Okafor

watermark-removal's Issues

How do we create new watermarks?

Is there a step by step guide someone can write out on how we can train a model to work with other watermarks?

Where is the mask?

Hi,
The script is failing in line 31 of the main.py
Traceback (most recent call last):
File "main.py", line 31, in
assert image.shape == mask.shape
AttributeError: 'NoneType' object has no attribute 'shape'

Looks like there is no folder file "mask.png" in the github repo?

How to generate models for other watermarks

Hey, can you tell me how can I generate models for other watermarks using this script?

Tensor Flow > 2.0

How do I run the code using stand alone python on my laptop?

Unable to install tensor flow 1.15 as the default is 2.0 onwards now.

How do I go about it, as neuralgym has 1.15

The generated image is completely incorrect

My Pictures

My Watermark

output

Package Version

Generated images

1

No module named 'PIL'

Traceback (most recent call last):
File "main.py", line 3, in
from PIL import Image
ModuleNotFoundError: No module named 'PIL'

No module named PIL on Ubuntu 22.04

I'm facing this issue on the Ubuntu server. Where I'm trying to setup this python project. Could you please make a setup guide for Ubuntu? I can't do anything. I tried it on my mac as well. No module found cv2, PIL

ValueError: Couldn't find 'checkpoint' file or checkpoints in given directory model/

Hello, hope you are doing well.

Im getting this issue even when i have the "checkpoint" file in the model/ folder. No it does not have the .txt on it, and i tried moving the model/ folder around. Here is my folder structure:

watermar-removal/
│ .gitignore
│ batch_test.py
│ guided_batch_test.py
│ image.png
│ inpaint.yml
│ inpaint_model.py
│ inpaint_ops.py
│ main.py
│ preprocess_image.py
│ README.md
│
├───model
│ checkpoint
│ txt.text
│
├───utils
│ └───istock
│ ├───landscape
│ │ mask.png
│ │
│ └───potrait
│ mask.png
│
└───__pycache__

I made the portrait version in photoshop, you can see it here if you want
" https://we.tl/t-VyVXJeQFRf "

Using python 3.7.0 and pip version 23.1.2, in a virtual env i installed all the required packages. I had to change one of the lines for the code to work, its this line in preprocess_image.py " assert image.shape == preprocessed_mask_image**.shape** "
Please write if you need more info.

Have a great rest of your day.

Video watermark removal

Can we change to gradio UI? and have video watermark removal

不可用

所以只能识别指定格式的水印？

自己的图片用不了 用了主页的示例图片也报错不知道是什么原因
Traceback (most recent call last):
File "main.py", line 32, in
input_image = preprocess_image(image, args.watermark_type)
File "/Users/fgg/code/projects/watermark-removal/preprocess_image.py", line 42, in preprocess_image
assert image.shape == preprocessed_mask_image
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Traceback (most recent call last):
File "main.py", line 32, in
input_image = preprocess_image(image, args.watermark_type)
File "/content/watermark-removal/preprocess_image.py", line 42, in preprocess_image
assert image.shape == preprocessed_mask_image
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

can you remove the istock video logo ? please help me

Missing potrait/mask.png

Hi zuruoke,

That's a great tool you are putting out there. I have run the command all the commands, your package seems to detect that I want to clean up a portrait and is looking for "utils/portrait/mask.png", but the "portrait" folder does not even exist.

Here is the error message:

image size: (1730, 1155, 3)
Traceback (most recent call last):
  File "main.py", line 32, in <module>
    input_image = preprocess_image(image, args.watermark_type)
  File "/content/watermark-removal/preprocess_image.py", line 25, in preprocess_image
    "utils/{}/{}/mask.png".format(watermark_type, image_type))
  File "/usr/local/lib/python3.7/dist-packages/PIL/Image.py", line 2843, in open
    fp = builtins.open(filename, "rb")
FileNotFoundError: [Errno 2] No such file or directory: 'utils/istock/potrait/mask.png'

Should I run some code to generate the mask?

ERROR: Config File doesn't exist

When running this code on Google Colab, I get the following:

WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/neuralgym/ops/layers.py:142: The name tf.image.resize_bilinear is deprecated. Please use tf.compat.v1.image.resize_bilinear instead.

WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/neuralgym/callbacks/npz_model_loader.py:31: The name tf.get_variable_scope is deprecated. Please use tf.compat.v1.get_variable_scope instead.

WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/neuralgym/ops/gan_ops.py:138: The name tf.layers.Conv2D is deprecated. Please use tf.compat.v1.layers.Conv2D instead.

Traceback (most recent call last):
  File "watermark-removal/main.py", line 26, in <module>
    FLAGS = ng.Config('inpaint.yml')
  File "/usr/local/lib/python3.7/dist-packages/neuralgym/utils/config.py", line 82, in __init__
    assert os.path.exists(filename), "ERROR: Config File doesn't exist."
AssertionError: ERROR: Config File doesn't exist.

Are the pretrained models specific to the istock mask?

A lot of issues have been created asking for how to create a custom mask, and I've ran into the same problem.

I created a custom mask (png, black background, with just the watermark transparent) and put it into the correct directory. The script runs without issues, but the output image is exactly the same as the input image, just with some parameters changed (like it went from being 48-bit color to 24-bit color) so it's clear that something is being done. Just not the thing I'd like it to do.

So are the models provided through the google drive link just specific to the istock mask? And if so, how can we train them ourselves? Or should the models be universal?

Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/PIL/Image.py", line 2847, in open fp.seek(0) AttributeError: 'str' object has no attribute 'seek'

Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/PIL/Image.py", line 2847, in open
fp.seek(0)
AttributeError: 'str' object has no attribute 'seek'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "main.py", line 31, in
image = Image.open(args.image)
File "/usr/local/lib/python3.7/dist-packages/PIL/Image.py", line 2849, in open
fp = io.BytesIO(fp.read())
AttributeError: 'str' object has no attribute 'read'

Please add model for Deviantart watermark remover

it would be very usefull and its also just a simple shape

Image size not supported!!!

What are the requirements of the program for the size of the picture

python3: can't open file 'main.py': [Errno 2] No such file or directory

I followed all steps carefuly, but when executing i get this error...

tensorflow.python.framework.errors_impl.OutOfRangeError: Read less bytes than requested

Traceback (most recent call last):
File "main.py", line 51, in
args.checkpoint_dir, from_name)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/contrib/framework/python/framework/checkpoint_utils.py", line 81, in load_variable
return reader.get_tensor(name)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/pywrap_tensorflow_internal.py", line 915, in get_tensor
return CheckpointReader_GetTensor(self, compat.as_bytes(tensor_str))
tensorflow.python.framework.errors_impl.OutOfRangeError: Read less bytes than requested

How to create the mask file?

If I can not get the source picture file of the watermark, I am wondering how to create the mask file?

Cannot find mask file for portrait images

ls: cannot access utils/istock/potrait/mask.png: No such file or directory

How do you train your own model on another watermark?

pip install tensorflow==1.15.0 in colab ERROR

ERROR: Could not find a version that satisfies the requirement tensorflow==1.15.0 (from versions: 2.8.0rc0, 2.8.0rc1, 2.8.0, 2.8.1, 2.8.2, 2.8.3, 2.8.4, 2.9.0rc0, 2.9.0rc1, 2.9.0rc2, 2.9.0, 2.9.1, 2.9.2, 2.9.3, 2.10.0rc0, 2.10.0rc1, 2.10.0rc2, 2.10.0rc3, 2.10.0, 2.10.1, 2.11.0rc0, 2.11.0rc1, 2.11.0rc2, 2.11.0, 2.11.1, 2.12.0rc0, 2.12.0rc1, 2.12.0, 2.12.1, 2.13.0rc0, 2.13.0rc1, 2.13.0rc2, 2.13.0, 2.13.1, 2.14.0rc0, 2.14.0rc1, 2.14.0, 2.14.1, 2.15.0rc0, 2.15.0rc1, 2.15.0)
ERROR: No matching distribution found for tensorflow==1.15.0

AttributeError: module 'tensorflow' has no attribute 'contrib'

I have installed tenserflow 2.11 with python 3.10

then when i run the code

!  python main.py --image pic1.jpeg --output res2.jpg --checkpoint_dir model/ --watermark_type istock

the following error will show

2022-12-01 17:24:41.275524: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-12-01 17:24:41.413694: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/user/fun/python/watermark-removal/venv/lib/python3.10/site-packages/cv2/../../lib64:
2022-12-01 17:24:41.413720: I tensorflow/compiler/xla/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-12-01 17:24:42.047465: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/user/fun/python/watermark-removal/venv/lib/python3.10/site-packages/cv2/../../lib64:
2022-12-01 17:24:42.047536: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/user/fun/python/watermark-removal/venv/lib/python3.10/site-packages/cv2/../../lib64:
2022-12-01 17:24:42.047547: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
Traceback (most recent call last):
  File "/home/user/fun/python/watermark-removal/main.py", line 8, in <module>
    import neuralgym as ng
  File "/home/user/fun/python/watermark-removal/venv/lib/python3.10/site-packages/neuralgym/__init__.py", line 27, in <module>
    from . import callbacks
  File "/home/user/fun/python/watermark-removal/venv/lib/python3.10/site-packages/neuralgym/callbacks/__init__.py", line 3, in <module>
    from .hyper_param_scheduler import HyperParamScheduler
  File "/home/user/fun/python/watermark-removal/venv/lib/python3.10/site-packages/neuralgym/callbacks/hyper_param_scheduler.py", line 6, in <module>
    from ..ops.summary_ops import scalar_summary
  File "/home/user/fun/python/watermark-removal/venv/lib/python3.10/site-packages/neuralgym/ops/__init__.py", line 4, in <module>
    from . import layers
  File "/home/user/fun/python/watermark-removal/venv/lib/python3.10/site-packages/neuralgym/ops/layers.py", line 93, in <module>
    activation_fn=None, w_init=tf.contrib.layers.xavier_initializer_conv2d(),
AttributeError: module 'tensorflow' has no attribute 'contrib'

How can i fix this problem?

Google Colab removed support for Tensorflow 1, and it is not possible to use %tensorflow_version 1.x magic anymore.

dependencies are not included

What dependencies are required? Usually people include this info when depositing code in github

不一样的图片和水印如何生成mask.png，本项目并不能使用于别的图片

不一样的图片和水印如何生成mask.png，本项目并不能使用于别的图片

安装不上TensorFlow，试了3.5，3.11俩Python版本

安装不上TensorFlow，试了3.5，3.11俩Python版本
Can not install TensorFlow, tried Python3.5, Python3.11

Kindly upgrade to newer versions of python, tensorflow

It's easy using tf_upgrade_v2 --intree watermark-removal/ --outtree watermark-removal-2/ plus some manual cleanups