hypox64 / deepmosaics

Automatically remove the mosaics in images and videos, or add mosaics to them.

Hi there 👋

• 📘 I am studying for a master's degree in Shenzhen University and intern at Tencent.
• 🔧 Skills: Python / C++ / CV / DL / Linux
• 📫 How to reach me: [email protected] / [email protected]

Project: ⬇

Yes I am back, I will finish some updates during the holiday.

deepmosaics's Issues

an issue when using the command line

Traceback (most recent call last):
File "deepmosaic.py", line 4, in
from cores import Options,core
File "/Users/emre/DeepMosaics/cores/core.py", line 74
print('\r',str(i)+'/'+str(length),util.get_bar(100*i/length,num=35),util.counttime(t1,t2,i,length),end='')
^
SyntaxError: invalid syntax

兄得，我觉得会不会有个小Bug如果用 semantic segmentation 来做的话

Because different image will have different 马赛克的算法，比如分厚码 中码 薄码 ，对不同的数据集效果会不一样啊

Cyclegan我觉得会不会好一点

quality comparison

Have you compare the mosaic removing capability of yours to that of XXXPlayer? You know what I meant. Thanks.

Error Type: <class 'json.decoder.JSONDecodeError'>

你好, 我尝试清除视频里的马赛克时出现了如大的错误:

hakus-MacBook-Pro:DeepMosaics zhaoyanbo$ python3 deepmosaic.py --media_path ./result/good.mp4 --model_path ./pretrained_models/mosaic/clean_face_HD.pth --use_gpu -1
segment parameters: 12.4M
netG parameters: 182.44M
/bin/sh: ffprobe: command not found
--------------------ERROR--------------------
Error Type: <class 'json.decoder.JSONDecodeError'>
Expecting value: line 1 column 1 (char 0)
<FrameSummary file deepmosaic.py, line 66 in >
<FrameSummary file deepmosaic.py, line 45 in main>
<FrameSummary file /Users/zhaoyanbo/Desktop/project1/DeepMosaics/cores/core.py, line 128 in cleanmosaic_video_byframe>
<FrameSummary file /Users/zhaoyanbo/Desktop/project1/DeepMosaics/cores/core.py, line 14 in video_init>
<FrameSummary file /Users/zhaoyanbo/Desktop/project1/DeepMosaics/util/ffmpeg.py, line 41 in get_video_infos>
<FrameSummary file /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/init.py, line 348 in loads>
<FrameSummary file /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py, line 337 in decode>
<FrameSummary file /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py, line 355 in raw_decode>
Please press any key to exit.

如果可以请帮忙解答!感谢!

Space can't be in path error

I still have this message: "Sorry, 'Space' can't be in path.' any time I try to select a pretrained model. How I correct this? Thanks.

v0.4.1

[WinError 126] The specified module could not be found. Error loading "C:\DeepMosaics_V0.4.1_gpu\core\torch\lib\caffe2_detectron_ops_gpu.dll" or one of its dependencies.
Please press any key to exit.

how to fix

use GPU

How do I use GPU? by the CPU and very slow.

Precompiled GPU version does nothing

The non-GPU version 4 works fine. But the precompiled GPU version 4 only shows the black CMD screen for 1 seconds which closes instantly after I click on "run". No error, no log, no result dir.

Downloaded both from GDrive, as well the model.

//edit:
Graphic card: Nvidia 2080 TI
Driver version: 460.79
Win10 x64

Error Type: <class 'RuntimeError'> Given groups=1, weight of size [64, 3, 3, 3], expected input[1, 4, 360, 486] to have 3 channels, but got 4 channels instead

Get:

--------------------ERROR--------------------
Error Type: <class 'RuntimeError'>
Given groups=1, weight of size [64, 3, 3, 3], expected input[1, 4, 360, 486] to have 3 channels, but got 4 channels instead
<FrameSummary file deepmosaic.py, line 66 in <module>>
<FrameSummary file deepmosaic.py, line 40 in main>
<FrameSummary file cores\core.py, line 112 in cleanmosaic_img>
<FrameSummary file models\runmodel.py, line 73 in get_mosaic_position>
<FrameSummary file models\runmodel.py, line 13 in run_segment>
<FrameSummary file site-packages\torch\nn\modules\module.py, line 477 in __call__>
<FrameSummary file models\BiSeNet_model.py, line 236 in forward>
<FrameSummary file site-packages\torch\nn\modules\module.py, line 477 in __call__>
<FrameSummary file models\BiSeNet_model.py, line 122 in forward>
<FrameSummary file site-packages\torch\nn\modules\module.py, line 477 in __call__>
<FrameSummary file models\BiSeNet_model.py, line 111 in forward>
<FrameSummary file site-packages\torch\nn\modules\module.py, line 477 in __call__>
<FrameSummary file site-packages\torch\nn\modules\conv.py, line 301 in forward>

When running v0.3.0 win build with clean_youknow_resnet_9blocks.pth from the google drive.

detection algorithm does not detect in images with high resolution

detection algorithm does not detect in high resolution images, it was mentioned Adaptive Mosaic Detection Threshold PR #43

我需要幫助

我運行了預編譯的文件，它給了我這個錯誤。幫助，我不知道該怎麼辦，我已經正確安裝了Python 3.8，Torch和cv2，所以我不知道問題出在哪裡.

非常感謝您的關注！

No such file or directory: 'D:\\DeepMosaics\\mosaic_position.pth

Hi,

I feel like I an missing something. I tested the programm on an AMD and a Ryzen, on a Video and a Picture, ad wih differend models but ending up with this error message all the time.

segment parameters: 12.4M
--------------------ERROR--------------------
Error Type: <class 'FileNotFoundError'>
[Errno 2] No such file or directory: 'D:\DeepMosaics\mosaic_position.pth'
<FrameSummary file deepmosaic.py, line 66 in >
<FrameSummary file deepmosaic.py, line 29 in main>
<FrameSummary file models\loadmodel.py, line 88 in bisenet>
<FrameSummary file site-packages\torch\serialization.py, line 356 in load>
Please press any key to exit.

48h for 1video?

i use gpu( my pc has 1070ti and 8600k) but it takes almost 48h to remove mosaic......

is it right?

出现提示，如何排除

[swscaler @ 00000261ca8640c0] deprecated pixel format used, make sure you did set range correctly

使用了不常用的像素格式，请确认您的设置范围是否正确
出现这提示，是啥问题导致，如何排除

Error starting the process.

Ok, I just had an error when trying to use the "clean_youknow_video_HD" model, I don't know, but I have tried to download the program again (to rule out that any file has been damaged), but it keeps giving me the same error.

I hope you can give me a solution, HypoX64!

Error Type: <class 'RuntimeError'>

This problem occurs when running

segment parameters: 12.4M
netG parameters: 11.38M
Clean Mosaic: C:\Users\Feild\Desktop\微信图片_20200518111928.png
--------------------ERROR--------------------
Error Type: <class 'RuntimeError'>
Given groups=1, weight of size [64, 3, 3, 3], expected input[1, 4, 360, 2236] to have 3 channels, but got 4 channels instead
<FrameSummary file deepmosaic.py, line 66 in >
<FrameSummary file deepmosaic.py, line 40 in main>
<FrameSummary file cores\core.py, line 112 in cleanmosaic_img>
<FrameSummary file models\runmodel.py, line 73 in get_mosaic_position>
<FrameSummary file models\runmodel.py, line 13 in run_segment>
<FrameSummary file site-packages\torch\nn\modules\module.py, line 477 in call>
<FrameSummary file models\BiSeNet_model.py, line 236 in forward>
<FrameSummary file site-packages\torch\nn\modules\module.py, line 477 in call>
<FrameSummary file models\BiSeNet_model.py, line 122 in forward>
<FrameSummary file site-packages\torch\nn\modules\module.py, line 477 in call>
<FrameSummary file models\BiSeNet_model.py, line 111 in forward>
<FrameSummary file site-packages\torch\nn\modules\module.py, line 477 in call>
<FrameSummary file site-packages\torch\nn\modules\conv.py, line 301 in forward>
Please press any key to exit.

Can't Detect Mosaic

I just tested out on Google Colab with image sample on readme (white cloth girl), and result is still the same (blurred face)

Tested on: Google Colab
Runtime: GPU
Cuda: True

More information & screenshot here

ask regarding running gpu mode

Thank you for your great code.
I have a question while using deepmosaic.
After finishing cuda setting to run in GPU mode, I noticed that I entered self.opt.use_gpu = True through option.

Nevertheless, when I try to clean mosaic, the speed in the stream mapping step does not come out as I want, only the CPU usage increases. I'm wondering if it's right to use the original CPU at that stage, and if there are additional settings to use the GPU.

Recommended Hardware or Adjust Parameter for low end GPU

I'm running into a CUDA out of memory error. Currently, I'm using i3-6100 with 16GB of memory and a Nvidia GTX 750 that only has 1GB of memory. Is there a way to adjust the parameter for low end GPU? If not, what's the minimum memory required for the GPU?

using GPU with source code

I have a GTX 1070. when I run it with source code.. i assume using --use_gpu 0 will make it use GPU?
however, i tried both using GPU and not use GPU. i get the same speed.. and my GPU utilization is 0%
is this doing what it's supposed to do?

运行错误

Clean temp...
segment parameters: 12.4M
netG parameters: 11.38M
Clean Mosaic: C:\Users\OneRains\Desktop\mosaic.png
--------------------ERROR--------------------
--------------Environment--------------
DeepMosaics: 0.4.0
Python: 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 14:00:49) [MSC v.1915 64 bit (AMD64)]
Pytorch: 0.4.1
OpenCV: 4.1.0
Platform: Windows-10-10.0.18362-SP0
--------------BUG--------------
Error Type: <class 'cv2.error'>
OpenCV(4.1.0) c:\projects\opencv-python\opencv\modules\imgproc\src\filterengine.hpp:363: error: (-215:Assertion failed) anchor.inside(Rect(0, 0, ksize.width, ksize.height)) in function 'cv::normalizeAnchor'

<FrameSummary file deepmosaic.py, line 64 in >
<FrameSummary file deepmosaic.py, line 40 in main>
<FrameSummary file cores\core.py, line 169 in cleanmosaic_img>
<FrameSummary file models\runmodel.py, line 75 in get_mosaic_position>
<FrameSummary file util\image_processing.py, line 201 in mask_threshold>
Please press any key to exit.

windows 10 cpu ryzen threadripper 2990wx

windows 10 cpu ryzen threadripper 2990wx

program use full load cpu 50 %
https://i.imgur.com/luEaf8g.png

i test intel i9-9980H cpu 100 % success

Problem Check please ?

I wonder if sometimes this will have to be done again?

https://ibb.co/FWpGbwn

voice_tmp.mp3: No such file or directory

I'm getting the same error for both versions CPU and GPU

It seem like the process starts fine, it does the processing, shows the preview, and real quick playing the video at the end but then it will drop this quick error in RED and close CMD right away.

When trying to clean or add mosaic to a video, this error at the very last step appears real quick and close CMD so I had to capture it in order to see what was the error and why the video is not on the result directory.

Is there a fix I need to do to get rid of this error?

老哥，能不能放几个预训练模型

如题，能否将预训练模型也上传

Add Fps Limit Options

150 fps too much on videos

I want to restrict 30 fps

EXE cannot find caffe2_detectron_ops_gpu.dll

The EXE cannot start even though the DLL it is looking for exists at the path where it is expecting it. What's wrong?

OSError: [WinError 126] The specified module could not be found. Error loading "D:\Libraries\Downloads\DeepMosaics_V0.4.0_gpu\DeepMosaics_V0.4.0_gpu\core\torch\lib\caffe2_detectron_ops_gpu.dll" or one of its dependencies.
[18112] Failed to execute script deepmosaic
PS D:\Libraries\Downloads\DeepMosaics_V0.4.0_gpu\DeepMosaics_V0.4.0_gpu\core> dir D:\Libraries\Downloads\DeepMosaics_V0.4.0_gpu\DeepMosaics_V0.4.0_gpu\core\torch\lib\caffe2_detectron_ops_gpu.dll


    Directory: D:\Libraries\Downloads\DeepMosaics_V0.4.0_gpu\DeepMosaics_V0.4.0_gpu\core\torch\lib


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----         7/24/2020   3:47 PM       19389952 caffe2_detectron_ops_gpu.dll

Python 3.8 Support ?

Does it support Python 3.8? I get this error
I can not run

Error Type: <class 'RuntimeError'>
error in LoadLibraryA

you please fix ? Follow the video clip. Suddenly it disappeared

hi

https://www.youtube.com/watch?v=8mKeZYA9bQo

Follow the video clip. Suddenly it disappeared

Is GPU available in the source code if my gpu is amd

class 'AttributeError'

I run this in Ubuntu 18.04, Cuda 10.2. Got the following error. Do you know why?

frame=67186 fps=196 q=0.0 Lsize=N/A time=00:37:21.63 bitrate=N/A speed=6.54x
video:8844538kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!
Premature end of JPEG file7/67180 [###################################] 99.97%
Find mosaic location:67159/67180 [###################################] 99.97%--------------------ERROR--------------------
Error Type: <class 'AttributeError'>
'NoneType' object has no attribute 'shape'
<FrameSummary file deepmosaic.py, line 66 in >
<FrameSummary file deepmosaic.py, line 43 in main>
<FrameSummary file /home/robert/DeepMosaics/cores/core.py, line 159 in cleanmosaic_video_fusion>
<FrameSummary file /home/robert/DeepMosaics/cores/core.py, line 97 in get_mosaic_positions>
<FrameSummary file /home/robert/DeepMosaics/models/runmodel.py, line 72 in get_mosaic_position>
Please press any key to exit.

Computer Occasionally Freezes

Despite having 500GB of pagefile.sys as RAM + 4GB physical RAM my computer occasionally freezes, not entirely sure why but my guess would be that its due to me attempting to demosaic a (1hr:20min:24sec) 1920x1080 video🤣! Is there a way to resume the job that was in progress prior to the System Freeze so that it doesnt clean the temp data and have to start from the beginning again? It took 60Hrs just to generate the mosaic masks!
But no complaints, what few frames it did complete before freezing look fantastic!!!!!👍👏🥳

断点接续

请教，如Find mosic location未完成，关机重启后能否接续

如未能接续，强烈反馈增加断点接续，谢谢。

Find mosaic location error: too many values to unpack (expected 2)

Hi, I'm getting following error when running on python 3.6. Do you know what could cause the issue? Thanks

frame=26508 fps=194 q=0.0 Lsize=N/A time=00:14:43.60 bitrate=N/A dup=4 drop=0 speed=6.45x
video:1418716kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Find mosaic location:
--------------------ERROR--------------------
Error Type: <class 'ValueError'>
too many values to unpack (expected 2)
<FrameSummary file deepmosaic.py, line 64 in <module>>
<FrameSummary file deepmosaic.py, line 43 in main>
<FrameSummary file C:\temp\DeepMosaics-master\cores\core.py, line 233 in cleanmosaic_video_fusion>
<FrameSummary file C:\temp\DeepMosaics-master\cores\core.py, line 144 in get_mosaic_positions>
<FrameSummary file C:\temp\DeepMosaics-master\models\runmodel.py, line 77 in get_mosaic_position>
<FrameSummary file C:\temp\DeepMosaics-master\util\image_processing.py, line 149 in find_mostlikely_ROI>
Please press any key to exit.

Use of TPU

Is it possible to use a Tensor processing unit for this repo ?, it would be very useful if this could be used

class 'RuntimeError'

I run this in python 3.6.8 and 3.7.10. Got the following error. Do you know why?
Error Type: <class 'RuntimeError'>
Given groups=1, weight of size [64, 3, 3, 3], expected input[1, 4, 500, 360] to have 3 channels, but got 4 channels instead
<FrameSummary file deepmosaic.py, line 66 in >
<FrameSummary file deepmosaic.py, line 40 in main>
<FrameSummary file cores\core.py, line 112 in cleanmosaic_img>
<FrameSummary file models\runmodel.py, line 73 in get_mosaic_position>
<FrameSummary file models\runmodel.py, line 13 in run_segment>
<FrameSummary file site-packages\torch\nn\modules\module.py, line 477 in call>
<FrameSummary file models\BiSeNet_model.py, line 236 in forward>
<FrameSummary file site-packages\torch\nn\modules\module.py, line 477 in call>
<FrameSummary file models\BiSeNet_model.py, line 122 in forward>
<FrameSummary file site-packages\torch\nn\modules\module.py, line 477 in call>
<FrameSummary file models\BiSeNet_model.py, line 111 in forward>
<FrameSummary file site-packages\torch\nn\modules\module.py, line 477 in call>
<FrameSummary file site-packages\torch\nn\modules\conv.py, line 301 in forward>
Please press any key to exit.

ffmpeg no cuda device when using cpu option

so i changed the ffmpeg.py command to use gpu for encode process. the app conversion is normal when using the gpu option(--use_gpu 0), but when using the cpu option ffmpeg says there's no cuda device

did the cpu option also giving this "no cuda device" signal to ffmpeg?

and maybe can we make the conversion process stop when ffmpeg giving error massage and failed to process the file? like catch the ffmpeg errors and stop the process.
because whenever the encode process is done or just getting error, the cache file get deleted afterwards and i need to start the conversion from first again. it happened to me several times after changing the ffmpeg command and when converting video without an audio

Need functionality to resume project if computer or program crashes.

There should be a way to resume from where the project left off if your machine restarts or crashes mid way through the programs execution. I was half way through the identify mosaic phase with 114,000 frames to process which took more than a day and my machine crashed and now I can't resume my project but have to start again which means I'll have to cut the video into smaller chunks to avoid this problem in future. Although it's rare for my machine to crash or reset without my say-so this does happen with the latest windows versions which install updates in the early hours. I see this being a useful function for everyone for those situations that can sometimes arise.

Can't find pretrained models in archive

Hi,

I downloaded the Windows version of DeepMosaic to try out. Using the GUI, the inputs seem simple:

Input the target file (video)
Input a pretrained model file

Using help, the list below shows the available pretrained files:

Name	Description
add_face.pth	Add mosaic to all faces in images/videos.
clean_face_HD.pth	Clean mosaic to all faces in images/video.(RAM > 8GB).
add_youknow.pth	Add mosaic to all (FBI Warning) in images/videos.
clean_youknow_resnet_9blocks.pth	Clean mosaic to all (FBI Warning) in images/videos.
clean_youknow_video.pth	Clean mosaic to all (FBI Warning) in videos.
clean_youknow_video_HD.pth	Clean mosaic to all (FBI Warning) in videos.(RAM > 8GB)

The issue is that these .pth files are not in the archive files in the program download.

Where do I get them?

Thanks,

Art

recommended parameters and image size for cleanface_HD

Hi, I am wondering what is the recommended setting for this pretrained model. I have image of shape (1242, 2208, 3) and default setting, it returns can't find mosaic. I tuned --tr_blur and --tr_down parameter to 100 and still cannot find mosaic. For this image size, is it recommended to shrink the image first?

兄弟，可以把训练的方法大致写一下吗？我这有大量数据集

我这有大量数据集，可以简单的把训练方法和数据集生成简单说一下吗。我可以训练好了传给你。

error buy ram ?

https://imgur.com/a/r8pMDwL

check please

Adaptive Mosaic Detection Threshold?

Anybody know how to implement adaptive threshold, id be using ("clean_frame_by_frame" all files in folder)! All of the frames have mosaic but ~1/2 of them have mosaiced areas that are too small to be detected with the default threshold of 64. By setting the threshold = 2 all frames are demosaiced, but those with large mosaiced areas are damaged by the very low threshold!

Proposed Solution - if no mosaic is detected initally, the threshold is decreased and the demosaic is reatempted until mosaic is found, and the threshold would be reset to the default for the first attemp of the following frame

Im still fairly new to Python so idk how that would be coded, I'm still learning the language😅 also if its possible to retrain a model on CPU, however slow it may be would still be preferable

IE) if the same 100frames are attemped 10000times, attempt 10000 should look better than attemp 1, not be identical

能用您的模型做迁移学习吗？

[HELP] Is it CUDA 11.1 or I messed up something else?

I did as this says except with the newest driver and CUDA: https://www.bilibili.com/video/BV1QK4y1a7Av

But when I got the CMD, it shows:

...while in the video, it says it has to be like this:

I can not figure out why and there seems to be no way (or too complicated) to roll back to CUDA 10.2

Please HELP

Deprecated pixel format used

Dear all, encountering a problem here when running the program. It said "deprecated pixel format used, make sure you did set range correctly"...

Wonder what have I done wrongly and how to fix the problem for proceeding.

Very grateful for some advice and enlightenment please... thanks very much

Windows10 GUI execution error

When optimization to 88%, the error pop-up as attachment, could you please check what's the reason? Thanks!

BTW, can i use windows power shell to run your program?

Is multi-threaded video processing possible?

I see that processing a video is two loops with loop size depending on the number of frames in a video clip. The process to find mosaics and clean mosaics for 1 frame is completely independent of any other frames. Completion of the loops would be much faster if they were processed with multiple threads. For example, if I use waifu2x to upscale a video, I have the option to specify the number of threads. I choose 5 so when the frames of the video are being looped, 5 of them will be processed at a time.

Are you able to do it? I have bare minimum knowledge of Python so I don't know if it is possible. I would have forked this project and done it myself if it was C++ or C#. What are your thoughts or plans?

i got error and i don't know why

--------------------ERROR--------------------
Error Type: <class 'UnboundLocalError'>
local variable 'img_fake' referenced before assignment
<FrameSummary file deepmosaic.py, line 66 in >
<FrameSummary file deepmosaic.py, line 45 in main>
<FrameSummary file D:\DeepMosaics-master\cores\core.py, line 142 in cleanmosaic_video_byframe>
Please press any key to exit.

i run the program on windows 10 with newest python and got this error.

i don't know how to fix it.

AI过滤

是否能AI自动侦测马赛克，解码中只针对有马赛克的部份解码，无马赛克的部份则自动跳过，不用整部视频解码呢，这样一来就可大大节省解码的时间。