zejinwang / blind2unblind Goto Github PK

Hello
How are you?
Thanks for contributing to this project.
It seems that this method is customized for ONLY denoising.
Is it possible to apply this method without any change to a general image restoration tasks such as image deblurring & deraining?

In the paper, Figure 1 shows the inference phase only requires the denoiser. But in the code, testing requires the global masker ($\Omega$). Using only the denoised output yields poor denoising quality.

Can you help me understand -- is the figure in the paper just incorrect?

Dear author,

I would like to express my gratitude for your valuable work. I have a question regarding the training of FMD dataset. Specifically, I would like to know which data from the FMD dataset was used to train the models corresponding to the three test sets. I am interested in applying this dataset to my own experiments and would greatly appreciate your response.

Thank you very much for your assistance!

想问下损失函数中的“can be considered as an ordinary constant. It only remains to design a new objective function”这个变量是代表什么呢不是很明白。另外在代码损失函数定义部分用到了两个lamda和thread以及beta alpha几个参数，这个分别有什么用呢，希望能解答一下，谢谢！

This work is amazing and I can't wait to use it, but I haven't had time to train the model lately, so I was wondering if you could release your pre-trained model?

Hi, why Random Mask + Re-visible cannot converge? Is there more analyses of this result?
Thanks!

Thanks for the nice paper and code.

I run the code on SSID Raw Validation dataset (ValidationNoisyBlocksRaw.mat) and it works. The generated log file (A_log_SIDD_Val.csv ) looks good as:

epoch:1,dn:35.623705/0.792260,exp:35.637747/0.792568,mid:35.633586/0.792483
epoch:2,dn:36.691027/0.813433,exp:36.729187/0.814111,mid:36.717283/0.813909
epoch:3,dn:37.662347/0.819666,exp:37.696882/0.820399,mid:37.686802/0.820189
epoch:4,dn:36.591700/0.809505,exp:36.619088/0.810321,mid:36.611489/0.810091
epoch:5,dn:36.354757/0.793341,exp:36.395038/0.794624,mid:36.384245/0.794274
epoch:6,dn:40.262343/0.923484,exp:40.369246/0.924950,mid:40.342693/0.924603
epoch:7,dn:41.634836/0.939280,exp:41.776404/0.940960,mid:41.743518/0.940571
epoch:8,dn:41.397646/0.937913,exp:41.548824/0.939674,mid:41.512758/0.939290
epoch:9,dn:41.689088/0.954902,exp:41.854164/0.956958,mid:41.821227/0.956529
epoch:10,dn:44.555741/0.968270,exp:44.853289/0.970512,mid:44.796125/0.970074
epoch:11,dn:44.999689/0.969440,exp:45.281381/0.971816,mid:45.235513/0.971367
epoch:12,dn:45.419838/0.971243,exp:45.743328/0.973611,mid:45.686149/0.973167
epoch:13,dn:44.878654/0.968614,exp:45.177972/0.971009,mid:45.125890/0.970557
epoch:14,dn:45.335208/0.970630,exp:45.648264/0.973065,mid:45.598885/0.972619
epoch:15,dn:45.373832/0.972728,exp:45.676182/0.975205,mid:45.633587/0.974751
epoch:16,dn:45.089008/0.973235,exp:45.430240/0.975746,mid:45.374611/0.975286
epoch:17,dn:44.889191/0.974528,exp:45.266475/0.977052,mid:45.193553/0.976604
epoch:18,dn:43.930426/0.971998,exp:44.150318/0.974647,mid:44.124846/0.974177
epoch:19,dn:42.528658/0.962436,exp:42.753665/0.965103,mid:42.718973/0.964630
epoch:20,dn:46.702977/0.979201,exp:47.145411/0.981932,mid:47.081676/0.981467
...
epoch:97,dn:48.916775/0.987458,exp:49.452608/0.989204,mid:49.453884/0.989185
epoch:98,dn:49.147523/0.987632,exp:49.733000/0.989374,mid:49.732428/0.989355
epoch:99,dn:49.113910/0.987588,exp:49.683852/0.989348,mid:49.684658/0.989331
epoch:100,dn:49.116245/0.987665,exp:49.708167/0.989422,mid:49.707363/0.989404

However, When I run the code (with slight modifications to fit the three-channel input) on SSID sRGB Validation dataset (ValidationNoisyBlocksSrgb.mat). The generated log file (A_log_SIDD_Val.csv ) shows that the model seems to converge to the identity mapping.

epoch:1,dn:27.404763/0.838748,exp:27.444009/0.842822,mid:27.433564/0.842193
epoch:2,dn:29.671704/0.784196,exp:29.695574/0.784053,mid:29.700154/0.785936
epoch:3,dn:28.303753/0.660923,exp:28.070521/0.650066,mid:28.177892/0.656177
epoch:4,dn:27.542834/0.609191,exp:27.170839/0.590442,mid:27.331493/0.599662
epoch:5,dn:27.040710/0.555477,exp:26.471492/0.526616,mid:26.702903/0.539143
epoch:6,dn:26.566351/0.510125,exp:25.901059/0.477941,mid:26.156954/0.490571
epoch:7,dn:26.041377/0.476580,exp:25.474742/0.451214,mid:25.682107/0.460485
epoch:8,dn:25.769698/0.445767,exp:25.371750/0.432081,mid:25.514558/0.437149
epoch:9,dn:25.587256/0.432471,exp:25.326529/0.426420,mid:25.425609/0.429413
epoch:10,dn:25.381982/0.417433,exp:25.222243/0.416812,mid:25.296105/0.418925
epoch:11,dn:25.121333/0.406872,exp:25.049778/0.410772,mid:25.104690/0.412451
epoch:12,dn:25.027494/0.394207,exp:25.004860/0.401712,mid:25.056882/0.403242
epoch:13,dn:25.063673/0.393456,exp:25.086481/0.404311,mid:25.133990/0.405482
epoch:14,dn:24.943280/0.385651,exp:25.029813/0.400656,mid:25.066898/0.401026
epoch:15,dn:24.704481/0.381718,exp:24.856096/0.400906,mid:24.876091/0.400216
epoch:16,dn:24.606720/0.371080,exp:24.814270/0.391729,mid:24.826118/0.391072
epoch:17,dn:24.608155/0.370326,exp:24.833266/0.393604,mid:24.845978/0.392103
epoch:18,dn:24.484653/0.362355,exp:24.762162/0.389126,mid:24.774272/0.387294
...

A visual result of SIDD_Val_023-027-005_mid looks like this:

Have you tried the code on SSID sRGB Validation dataset?
Any suggestions are appreciated.

请问您这个代码有专用的数据集吗？如果有能否提供数据集链接？或者说用含有噪声的图片就可完成训练？

There seems to be no code for training/testing on grayscale data with synthetic Gaussian noise (BSD400, BSD68, Set12, and Urban100), the results for which are presented in Table 2. The weights for these models also are not present.

Also, it is not clear from the paper, how exactly you use Urban100 data, because it is originally sRGB and in pairs of high- and low-resolution images.

It would be great if you could comment on that.

Thank you for the great work but can you please provide some code/pointer on how to prepare ValidationNoisyBlocksRaw.mat which is required for validation for SIDD RAW denoising?

How to get ValidationNoisyBlocksRaw.mat?

why adding noise each iteration on Guassian case? I think it should be only adding once and training with them.

损失随着训练而增加，最终的图像去噪效果显示规则的条纹形状。

I'm trying to use it according to my environment.
I have a few questions in the process.

1. Can i replace Upsample layer instead of up function in class Up of arch_unet.py
2. I use all the memory of gpu during the validation phase during training. Is this normal?
3. When I test, can I expect normal performance even if I use it without adding noise?
4. Does the score calculation process take up most of the time during validation?
5. Why do you do repeated tasks in validation? Can you explain it? -> It's mean 'valid_repeat_times'

I am looking forward to your reply.

尊敬的作者您好！
在处理SIDD_raw_image时，得到的是单通道图像。在论文中展示用的可视化的srgb图像是用具体什么工具转换得来的呢？
训练集中的raw数据直接输入网络进行运算，而在验证集中放缩到了0~255之间，请问为什么这样做，有什么区别吗？
如果您能尽快回复，我将十分感谢！

论文 Figure 2 中提到 “the global masker $\Omega_{(·)}$ hides four spots in y to create a global masked volumes $\Omega_{y}$ consisting of four masked cells $\Omega_{y}^{ij}$, $i,j\in$ {0, 1}”

并且前面也提到 “The pixels in the i-th row and j-th column of each cell are masked and filled with black for illustration”

我的理解是被 masked 的像素值应该为 0，即黑色。这就与下面的代码产生了矛盾：

经过调试发现 filtered_tensor.view_as(tensor) * mask 为将论文 Figure 2 中黑色的像素变成原图原位置卷积后的像素（其余位置像素值为0），与 tensor * mask_inv 相加后会导致图像没有像素值为 0 的部分（我们假设原图所有像素值非0）。或者这样考虑，mask 和 mask_inv 可以看成 “对立事件”，所以在 mask 和 mask_inv 的“系数”都不为 0 的前提下所有像素值也不可能出现0。

所以按我的理解上述代码应该修改为：

return tensor * mask_inv 

这样每 4 个像素必然有一个为 0，即实现了mask

请问我的理解是否正确，希望可以得到答复，谢谢！

尊敬的作者先生
您好！！！
我想关于ValidationGtBlocksRaw.mat和ValidationNoisyBlocksRaw.mat两个文件问您一些问题。
问题1：这两个文件我是需要根据我自己的数据集制作，还是直接从您给出的那个网址进行下载即可？
问题2：当我运行Train on SIDD raw-RGB Medium dataset命令时，报错 TypeError: Expecting miMATRIX type here, got 16843009，请问这是什么问题？
问题3：我现在有RGB的红外图像，我用Train on SIDD raw-RGB Medium dataset这个训练命令可以吗？

如果方便的话，请您把您的联系方式给我一个可以吗？

希望当您看到我的问题，辛苦给出一些解答，学生定不胜感激！！！

In train_b2u.py, train logic using DataLoader_Imagenet_val class.
I think there is issue about scaling image value range.

At line 363-364

transformer = transforms.Compose([transforms.ToTensor()])
im = transformer(im)

im transformed into range [0.0, 1.0] by transforms.ToTensor().

But in train loop (line 587), img devided by 255 again.

clean = clean / 255.0

Is it intended or not?

I don't know it's related to this issue, my trained model outputs gray-scale image when using naive code.

Hello, I'm trying to denoise a 2D image. I add synthetic noise as follows:
np.array(np.random.poisson(0.01 * x / 0.1) / 0.01
But the loss doesn't converge. I have several different learning rates. Can you help me out?

If I want to test the model I trained myself, should I replace the pretrained_models name in the test command with my own model name?

Hi, thanks for this great works. After reading this paper, I can see that the algorithm have been tested for Synthetic dataset denoising, raw-RGB dataset denoising and FM denoising, but not for sRGB-sRGB dataset denoising. What is the reason that your group not do experiment for sRGB-sRGB dataset denoising? I am curious about whether it could be used for sRGB-sRGB denoising. Hope for your reply. Thanks.