Coder Social home page Coder Social logo

cszn / dncnn Goto Github PK

View Code? Open in Web Editor NEW
1.4K 57.0 531.0 127.74 MB

Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising (TIP, 2017)

Home Page: https://cszn.github.io/

MATLAB 80.47% M 0.23% Python 17.71% Objective-C 1.59%
image-denoising residual-learning super-resolution jpeg-deblocking matconvnet pytorch keras-tensorflow

dncnn's Introduction

visitors

News: DRUNet

I recommend to use the PyTorch code for training and testing. The model parameters of MatConvnet and PyTorch are same.

Merge batch normalization (PyTorch)

import torch
import torch.nn as nn


def merge_bn(model):
    ''' merge all 'Conv+BN' (or 'TConv+BN') into 'Conv' (or 'TConv')
    based on https://github.com/pytorch/pytorch/pull/901
    by Kai Zhang ([email protected]) 
    https://github.com/cszn/DnCNN
    01/01/2019
    '''
    prev_m = None
    for k, m in list(model.named_children()):
        if (isinstance(m, nn.BatchNorm2d) or isinstance(m, nn.BatchNorm1d)) and (isinstance(prev_m, nn.Conv2d) or isinstance(prev_m, nn.Linear) or isinstance(prev_m, nn.ConvTranspose2d)):

            w = prev_m.weight.data

            if prev_m.bias is None:
                zeros = torch.Tensor(prev_m.out_channels).zero_().type(w.type())
                prev_m.bias = nn.Parameter(zeros)
            b = prev_m.bias.data

            invstd = m.running_var.clone().add_(m.eps).pow_(-0.5)
            if isinstance(prev_m, nn.ConvTranspose2d):
                w.mul_(invstd.view(1, w.size(1), 1, 1).expand_as(w))
            else:
                w.mul_(invstd.view(w.size(0), 1, 1, 1).expand_as(w))
            b.add_(-m.running_mean).mul_(invstd)
            if m.affine:
                if isinstance(prev_m, nn.ConvTranspose2d):
                    w.mul_(m.weight.data.view(1, w.size(1), 1, 1).expand_as(w))
                else:
                    w.mul_(m.weight.data.view(w.size(0), 1, 1, 1).expand_as(w))
                b.mul_(m.weight.data).add_(m.bias.data)

            del model._modules[k]
        prev_m = m
        merge_bn(m)


def tidy_sequential(model):
    for k, m in list(model.named_children()):
        if isinstance(m, nn.Sequential):
            if m.__len__() == 1:
                model._modules[k] = m.__getitem__(0)
        tidy_sequential(m)

Training (MatConvNet)

Testing (MatConvNet or Matlab)

  • [demos] Demo_test_DnCNN-.m.

  • [models] including the trained models for Gaussian denoising; a single model for Gaussian denoising, single image super-resolution (SISR) and deblocking.

  • [testsets] BSD68 and Set10 for Gaussian denoising evaluation; Set5, Set14, BSD100 and Urban100 datasets for SISR evaluation; Classic5 and LIVE1 for JPEG image deblocking evaluation.

New FDnCNN Models

I have trained new Flexible DnCNN (FDnCNN) models based on FFDNet.

FDnCNN can handle noise level range of [0, 75] via a single model.

Demo_FDnCNN_Gray.m

Demo_FDnCNN_Gray_Clip.m

Demo_FDnCNN_Color.m

Demo_FDnCNN_Color_Clip.m

Network Architecture and Design Rationale

  • Network Architecture

  • Batch normalization and residual learning are beneficial to Gaussian denoising (especially for a single noise level). The residual of a noisy image corrupted by additive white Gaussian noise (AWGN) follows a constant Gaussian distribution which stablizes batch normalization during training.

    • Histogram of noisy patches, clean patches, and residual (noise) patches from a batch of training. The noise level is 25, the patch size is 40x40, the batch size is 128.
    • Histogram of noisy patches, clean patches, and residual (noise) patches from another batch of training. The noise level is 25, the patch size is 40x40, the batch size is 128.
    • Noise-free image super-resolution does not have this property.
  • Predicting the residual can be interpreted as performing one gradient descent inference step at starting point (i.e., noisy image).

    • The parameters in DnCNN are mainly representing the image priors (task-independent), thus it is possible to learn a single model for different tasks, such as image denoising, image super-resolution and JPEG image deblocking.

    • The left is the input image corrupted by different degradations, the right is the restored image by DnCNN-3.

Results

Gaussian Denoising

The average PSNR(dB) results of different methods on the BSD68 dataset.

Noise Level BM3D WNNM EPLL MLP CSF TNRD DnCNN DnCNN-B FDnCNN DRUNet
15 31.07 31.37 31.21 - 31.24 31.42 31.73 31.61 31.69 31.91
25 28.57 28.83 28.68 28.96 28.74 28.92 29.23 29.16 29.22 29.48
50 25.62 25.87 25.67 26.03 - 25.97 26.23 26.23 26.27 26.59

Visual Results

The left is the noisy image corrupted by AWGN, the middle is the denoised image by DnCNN, the right is the ground-truth.

Gaussian Denoising, Single ImageSuper-Resolution and JPEG Image Deblocking via a Single (DnCNN-3) Model

Average PSNR(dB)/SSIM results of different methods for Gaussian denoising with noise level 15, 25 and 50 on BSD68 dataset, single image super-resolution with upscaling factors 2, 3 and 40 on Set5, Set14, BSD100 and Urban100 datasets, JPEG image deblocking with quality factors 10, 20, 30 and 40 on Classic5 and LIVE11 datasets.

Gaussian Denoising

Dataset Noise Level BM3D TNRD DnCNN-3
15 31.08 / 0.8722 31.42 / 0.8826 31.46 / 0.8826
BSD68 25 28.57 / 0.8017 28.92 / 0.8157 29.02 / 0.8190
50 25.62 / 0.6869 25.97 / 0.7029 26.10 / 0.7076

Single Image Super-Resolution

Dataset Upscaling Factor TNRD VDSR DnCNN-3
2 36.86 / 0.9556 37.56 / 0.9591 37.58 / 0.9590
Set5 3 33.18 / 0.9152 33.67 / 0.9220 33.75 / 0.9222
4 30.85 / 0.8732 31.35 / 0.8845 31.40 / 0.8845
2 32.51 / 0.9069 33.02 / 0.9128 33.03 / 0.9128
Set14 3 29.43 / 0.8232 29.77 / 0.8318 29.81 / 0.8321
4 27.66 / 0.7563 27.99 / 0.7659 28.04 / 0.7672
2 31.40 / 0.8878 31.89 / 0.8961 31.90 / 0.8961
BSD100 3 28.50 / 0.7881 28.82 / 0.7980 28.85 / 0.7981
4 27.00 / 0.7140 27.28 / 0.7256 27.29 / 0.7253
2 29.70 / 0.8994 30.76 / 0.9143 30.74 / 0.9139
Urban100 3 26.42 / 0.8076 27.13 / 0.8283 27.15 / 0.8276
4 24.61 / 0.7291 25.17 / 0.7528 25.20 / 0.7521

JPEG Image Deblocking

Dataset Quality Factor AR-CNN TNRD DnCNN-3
Classic5 10 29.03 / 0.7929 29.28 / 0.7992 29.40 / 0.8026
20 31.15 / 0.8517 31.47 / 0.8576 31.63 / 0.8610
30 32.51 / 0.8806 32.78 / 0.8837 32.91 / 0.8861
40 33.34 / 0.8953 - 33.77 / 0.9003
LIVE1 10 28.96 / 0.8076 29.15 / 0.8111 29.19 / 0.8123
20 31.29 / 0.8733 31.46 / 0.8769 31.59 / 0.8802
30 32.67 / 0.9043 32.84 / 0.9059 32.98 / 0.9090
40 33.63 / 0.9198 - 33.96 / 0.9247

Requirements and Dependencies

or just MATLAB R2015b to test the model.

DnCNN/Demo_test_DnCNN.m

Lines 64 to 65 in 4a4b5b8

res = vl_simplenn(net,input,[],[],'conserveMemory',true,'mode','test');
%res = simplenn_matlab(net, input); %%% use this if you did not install matconvnet.

Citation

@article{zhang2017beyond,
  title={Beyond a {Gaussian} denoiser: Residual learning of deep {CNN} for image denoising},
  author={Zhang, Kai and Zuo, Wangmeng and Chen, Yunjin and Meng, Deyu and Zhang, Lei},
  journal={IEEE Transactions on Image Processing},
  year={2017},
  volume={26}, 
  number={7}, 
  pages={3142-3155}, 
}
@article{zhang2020plug,
  title={Plug-and-Play Image Restoration with Deep Denoiser Prior},
  author={Zhang, Kai and Li, Yawei and Zuo, Wangmeng and Zhang, Lei and Van Gool, Luc and Timofte, Radu},
  journal={arXiv preprint},
  year={2020}
}

====================================================================

@Inbook{zuo2018convolutional,
author={Zuo, Wangmeng and Zhang, Kai and Zhang, Lei},
editor={Bertalm{\'i}o, Marcelo},
title={Convolutional Neural Networks for Image Denoising and Restoration},
bookTitle={Denoising of Photographic Images and Video: Fundamentals, Open Challenges and New Trends},
year={2018},
publisher={Springer International Publishing},
address={Cham},
pages={93--123},
isbn={978-3-319-96029-6},
doi={10.1007/978-3-319-96029-6_4},
url={https://doi.org/10.1007/978-3-319-96029-6_4}
}

Challenges and Possible Solutions (from the above book chapter)

While the image denoising for AWGN removal has been well-studied, little work has been done on real image denoising. The main difficulty arises from the fact that real noises are much more complex than AWGN and it is not an easy task to thoroughly evaluate the performance of a denoiser. Fig. 4.15 shows four typical noise types in real world. It can be seen that the characteristics of those noises are very different and a single noise level may be not enough to parameterize those noise types. In most cases, a denoiser can only work well under a certain noise model. For example, a denoising model trained for AWGN removal is not effective for mixed Gaussian and Poisson noise removal. This is intuitively reasonable because the CNN-based methods can be treated as general case of Eq. (4.3) and the important data fidelity term corresponds to the degradation process. In spite of this, the image denoising for AWGN removal is valuable due to the following reasons. First, it is an ideal test bed to evaluate the effectiveness of different CNN-based denoising methods. Second, in the unrolled inference via variable splitting techniques, many image restoration problems can be addressed by sequentially solving a series of Gaussian denoising subproblems, which further broadens the application fields.

To improve the practicability of a CNN denoiser, perhaps the most straightforward way is to capture adequate amounts of real noisy-clean training pairs for training so that the real degradation space can be covered. This solution has advantage that there is no need to know the complex degradation process. However, deriving the corresponding clean image of a noisy one is not a trivial task due to the need of careful post-processing steps, such as spatial alignment and illumination correction. Alternatively, one can simulate the real degradation process to synthesize noisy images for a clean one. However, it is not easy to accurately model the complex degradation process. In particular, the noise model can be different across different cameras. Nevertheless, it is practically preferable to roughly model a certain noise type for training and then use the learned CNN model for type-specific denoising.

Besides the training data, the robust architecture and robust training also play vital roles for the success of a CNN denoiser. For the robust architecture, designing a deep multiscale CNN which involves a coarse-to-fine procedure is a promising direction. Such a network is expected to inherit the merits of multiscale: (i) the noise level decreases at larger scales; (ii) the ubiquitous low-frequency noise can be alleviated by multiscale procedure; and (iii) downsampling the image before denoising can effectively enlarge the receptive filed. For the robust training, the effectiveness of the denoiser trained with generative adversarial networks (GAN) for real image denoising still remains further investigation. The main idea of GAN-based denoising is to introduce an adversarial loss to improve the perceptual quality of denoised image. Besides, a distinctive advantage of GAN is that it can do unsupervised learning. More specifically, the noisy image without ground truth can be used in the training. So far, we have provided several possible solutions to improve the practicability of a CNN denoiser. We should note that those solutions can be combined to further improve the performance.

dncnn's People

Contributors

cszn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dncnn's Issues

Where can I get train400 dataset?

The train400 data from BSD dataset is used for training it, following those works:

Chen, Yunjin, and Thomas Pock. "Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration." IEEE transactions on pattern analysis and machine intelligence 39.6 (2017): 1256-1272.
Schmidt, Uwe, and Stefan Roth. "Shrinkage fields for effective image restoration." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014.

and well, I couldn't find a list of these 400 images(and also tried to find the exact image files like BSD68, but couldn't).
What are these and how can I get them?

error in main_train.py

after successfully running the data_generator.py code. when i run the code of main_train.py it shows the following error

File "C:\Users\Sufian\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "C:/Users/Sufian/Desktop/dncnn_keras/main_train.py", line 69
x = BatchNormalization(axis=3, momentum=0.0,epsilon=0.0001, name = 'bn'+str(layer_count))(x)
^
TabError: inconsistent use of tabs and spaces in indentation

BN layers in given model

hi !
Can you please release module with BN layers? (for denoising)
Do you think it should work fine for Poisson noise too?
Thanks !

Get back the clean image

Hi, is that possible to change the Demo test model in TrainingCodes to get back the clean image instead of the residual image? Or need to re-initialize the model and train the model again?
Thanks!

不加BN层,在BSD68测试集上效果反而更加好

'''
inpt = Input(shape=(40,40,1))
x1 = Conv2D(64,(3,3),activation='relu',padding='same',strides=(1,1))(inpt)
x2 = Conv2D(64,(3,3),activation='relu',padding='same',strides=(1,1))(x1)
x3 = Conv2D(64,(3,3),activation='relu',padding='same',strides=(1,1))(x2)
x4 = Conv2D(64,(3,3),activation='relu',padding='same',strides=(1,1))(x3)
x5 = Conv2D(64,(3,3),activation='relu',padding='same',strides=(1,1))(x4)
x6 = Conv2D(64,(3,3),activation='relu',padding='same',strides=(1,1))(x5)
x7 = Conv2D(64,(3,3),activation='relu',padding='same',strides=(1,1))(x6)
x8 = Conv2D(64,(3,3),activation='relu',padding='same',strides=(1,1))(x7)
x9 = Conv2D(64,(3,3),activation='relu',padding='same',strides=(1,1))(x8)
x10 = Conv2D(64,(3,3),activation='relu',padding='same',strides=(1,1))(x9)
x11 = Conv2D(64,(3,3),activation='relu',padding='same',strides=(1,1))(x10)
x12 = Conv2D(64,(3,3),activation='relu',padding='same',strides=(1,1))(x11)
x13 = Conv2D(64,(3,3),activation='relu',padding='same',strides=(1,1))(x12)
x14 = Conv2D(64,(3,3),activation='relu',padding='same',strides=(1,1))(x13)
x15 = Conv2D(64,(3,3),activation='relu',padding='same',strides=(1,1))(x14)
x16 = Conv2D(64,(3,3),activation='relu',padding='same',strides=(1,1))(x15)
x17 = Conv2D(1,(3,3),padding='same',strides=(1,1))(x16)
'''
参数和原文一致,只是训练采用了BSD40,当sigma=25,15的时候在测试集BSD68上的psnr比原文要好,而且视觉效果也更棒。

Struggle with PyTorch version code

I'm afraid I might bother you, but it seems that the PyTorch implementation cannot reproduce the expected result.
When I tried to do so, I met following issues......:

When I run train/test code(without touching any setting),

  • DnCNN-S for noise level=25 made PSNR=29.24 in <30 epoch
  • DnCNN-S for noise level=15 got stuck at PSNR=23.xx. I tried this many times but didn't work.
  • DnCNN-S for noise level=50 made 25.9x in <50 epoch, but couldn't reach near 26.23 at all.
    There's multi_step scheduling, however I forced it to use smaller learning rate after the convergence, and it didn't help.
    +) I tried some modifications on the model, and still the problem remained. Just <0.8dB changed. So think it is not about model.
    +) I also tried to change the optimizer and lr scheduling exactly same as it is mentioned on the original paper, but that made it even worse.

Is it right to add Gaussian noise and don't clip it? I thought it must be clipped according to [0., 255.] (uint8) or [0., 1.] (float32) because in real case(of course it is not completely 'real' 'cause it's AWGN) the corrupted image would be in [0., 255.] range.
How much does the clipping lowers its performance, or is it better? I wanted to test it myself but... like I mentioned in 1., the baseline model isn't working correctly, so... If you have already tried it, could you let me know?

About the "bnorm" layers

Hi, I found that the models provided in the folder "model" don't contain "bnorm" layers, while the models that I trained using the provided training codes contain the "bnorm" layers. I want to know if I should remove the "bnorm" layers in my trained models?

data_size is NAN

I met some difficuties after use vl_compilenn().

>> Demo_Train_model_64_25_Res_Bnorm_Adam
     layer|      0|      1|      2|      3|      4|      5|      6|      7|      8|      9|     10|     11|     12|     13|     14|     15|     16|     17|
      type|  input|   conv|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|
      name|    n/a| layer1| layer2| layer3| layer4| layer5| layer6| layer7| layer8| layer9|layer10|layer11|layer12|layer13|layer14|layer15|layer16|layer17|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
   support|    n/a|      3|      1|      3|      1|      1|      3|      1|      1|      3|      1|      1|      3|      1|      1|      3|      1|      1|
  filt dim|    n/a|      1|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|
filt dilat|    n/a|      1|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|
 num filts|    n/a|     64|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|
    stride|    n/a|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
       pad|    n/a|      1|      0|      1|      0|      0|      1|      0|      0|      1|      0|      0|      1|      0|      0|      1|      0|      0|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
   rf size|    n/a|      3|      3|      5|      5|      5|      7|      7|      7|      9|      9|      9|     11|     11|     11|     13|     13|     13|
 rf offset|    n/a|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
 rf stride|    n/a|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
 data size|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|
data depth|    NaN|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|
  data num|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
  data mem|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|
 param mem|    n/a|    2KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|

     layer|     18|     19|     20|     21|     22|     23|     24|     25|     26|     27|     28|     29|     30|     31|     32|     33|     34|     35|
      type|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|
      name|layer18|layer19|layer20|layer21|layer22|layer23|layer24|layer25|layer26|layer27|layer28|layer29|layer30|layer31|layer32|layer33|layer34|layer35|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
   support|      3|      1|      1|      3|      1|      1|      3|      1|      1|      3|      1|      1|      3|      1|      1|      3|      1|      1|
  filt dim|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|
filt dilat|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|
 num filts|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|
    stride|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
       pad|      1|      0|      0|      1|      0|      0|      1|      0|      0|      1|      0|      0|      1|      0|      0|      1|      0|      0|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
   rf size|     15|     15|     15|     17|     17|     17|     19|     19|     19|     21|     21|     21|     23|     23|     23|     25|     25|     25|
 rf offset|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
 rf stride|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
 data size|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|
data depth|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|
  data num|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
  data mem|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|
 param mem|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|

     layer|     36|     37|     38|     39|     40|     41|     42|     43|     44|     45|     46|     47|     48|     49|
      type|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|   loss|
      name|layer36|layer37|layer38|layer39|layer40|layer41|layer42|layer43|layer44|layer45|layer46|layer47|layer48|layer49|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
   support|      3|      1|      1|      3|      1|      1|      3|      1|      1|      3|      1|      1|      3|      1|
  filt dim|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|
filt dilat|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|
 num filts|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|      1|    n/a|
    stride|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
       pad|      1|      0|      0|      1|      0|      0|      1|      0|      0|      1|      0|      0|      1|      0|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
   rf size|     27|     27|     27|     29|     29|     29|     31|     31|     31|     33|     33|     33|     35|     35|
 rf offset|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
 rf stride|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
 data size|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|
data depth|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|      1|      1|
  data num|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|      1|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
  data mem|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|
 param mem|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|    2KB|     0B|

Training with 980Ti data size NAN

Hi there,

I could not train the model with my 980Ti card due to this NAN issue. Does it mean I need a better card or any way to resolve this issues?

image
image

Thankyou

Overfitting problem

Hi, do we need consider over-fitting problem when we fine-tune the parameters to reproduce the results? If is necessary, can I use PSNR as evaluation metric to evaluate the training loss and test loss?

Denoise the same picture with different nets

Hi, Thanks for your codes at the first. I am confused by a phenomenon. I have trained 3 nets with sigma = 5,10,15 respectively and I get a noised picture with sigma = 5. Then I used three trained nets to denoised the same picture, but I find the net trained with sigma = 5 performs worse than other two nets. And I find the nets with sigma = 15 performs best. I am confused by this phenomenon, do you know the reason?

Error in Demo_test_DnCNN_C (line 57)

Solution please

Error using vl_nnconv
The FILTERS depth does not divide the DATA depth.

Error in vl_simplenn (line 97)
res(i+1).x = vl_nnconv(res(i).x, l.weights{1}, l.weights{2}, ...

Error in Demo_test_DnCNN_C (line 57)
res = vl_simplenn(net,input,[],[],'conserveMemory',true,'mode','test');

capture

CBSD432 dataset for CDnCNN-B

For training CDnCNN-B, did you extract patches from the full-sized CBSD432? Or from 180x180 crops? If the latter is true, can you provide the link to the cropped CBSD432 dataset.

can i ask you about last layer?

Hi I have bin interest about your method.

can i ask you one question?

I don't under stand about last layer.

last layer got 64 filtered image at previous layer and use 64 filters for convolution.

but how reconstruct one image?

reconstruct image pixel is sum of each 64 filter image pixel?

Reference to non-existent field 'dilate', matconvnet 1.0-beta25

Hi, I got this error when running Demo_test_DnCNN.m with Matlab 2017a, matconvnet 1.0-beta25, Window 10. The details are

Reference to non-existent field 'dilate'.

Error in vl_simplenn (line 303)
'dilate', l.dilate, ...

Error in Demo_test_DnCNN (line 64)
res = vl_simplenn(net,input,[],[],'conserveMemory',true,'mode','test');

Any idea to fix this?
Thank you.

color image denoising

Hi czn,

Is there any evaluation on color image denoising task with different noise level?
Like the mean psnr on certain test set?

MLP code for sigma = 50

Hi,

I saw in your paper that you have compared against MLP with sigma=50. Please can you direct me to the link or share the code for MLP sigma=50.

Waiting for your kind response.

Regard,
Saeed

Understanding vl_nnloss.m

Hi there,

I am trying to modify the loss function in DnCNN, but I am not quite sure how to do it.
Could you explain a little more about your vl_nnloss?
I understand that

t = ((X-c).^2)/2;
Y = sum(t(:))/size(X,4); % reconstruction error per sample;

calculates the L2 loss function.
How about the second part?

Y = bsxfun(@minus,X,c).*dzdy;

why did you use bsxfun and minus? and could you explain the meaning of dzdy?

Thank you very much,
Best regards,
Atena

Demo_test_DnCNN.m

Hi!I meet a problem when I run 'Demo_test_DnCNN.m'.
System error as:
错误使用 vl_nnconv
An input is not a numeric array (or GPU support not compiled).

出错 vl_simplenn (line 300)
res(i+1).x = vl_nnconv(res(i).x, l.weights{1}, l.weights{2}, ...

出错 Demo_test_DnCNN (line 64)
res = vl_simplenn(net,input,[],[],'conserveMemory',true,'mode','test');
I‘m a beginner. Could you tell me how can I solve it,or anyone knows the answer,please tell me ,thank you very much!

strange output from the DnCNN denosing

Hi,
I tried to run the Demo_test_DnCNN.m, but the output image looks very strange, there are several vertical stripes in the image, and the stripes exist in all output images. and I also tried the Demo_test_FDnCNN_Gray.m, still the same issue.
ajc2

Question about learning rate for bias.

Hi Zhang,

Could you explain the reason not to use bias for the intermediate convolution?
I could see that the first and last convolution define learnRate as lr11, while the other convolutions use lr10.

Thank you.

BSD68 RGB dataset

Hi,
I've trained CDnCNN-B in tensorflow and I would like to compare the results but I can't find the RGB version of the BSD68 dataset, can you provide it?

Kind regards,
Michele

Forward:out of memory

Hi cszn,
Meet you again~
I have 11GB available in GPU, Why I got out of memory ERROR?
I im using a BMP(3968*2976) to feed the NET.
image

can i ask you specipic option about uploaded models?

Hi thank you for your work.

can i ask you about specipic train option of uploded models? (like sigma=10.mat, sigma=15.mat)

I think sigma is noise level. but i don't know about how many epoch you train. (in paper train with 50 epoch. is that right?)

Also training codes v1.1 can generate same model? (using adam or SGD?)

error 0 in training

Hi, i'm trying to train a model with another set of images and with CPU.
I replaced the images in Train400 and changed in the code GPU = [];
i got error value of 0 and NAN in memory

(parameter memory|2MB (5.6e+05 parameters)|
data memory|NaN (for batch size 128)|
train: epoch 01 dataset 00: 1/5106:error: 0.000000
train: epoch 01 dataset 00: 2/5106:error: 0.000000 )

i cant add image , this is copy paste :

 layer|      0|      1|      2|      3|      4|      5|      6|      7|      8|      9|     10|     11|     12|     13|     14|     15|     16|     17|
  type|  input|   conv|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|
  name|    n/a| layer1| layer2| layer3| layer4| layer5| layer6| layer7| layer8| layer9|layer10|layer11|layer12|layer13|layer14|layer15|layer16|layer17|

----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
support| n/a| 3| 1| 3| 1| 1| 3| 1| 1| 3| 1| 1| 3| 1| 1| 3| 1| 1|
filt dim| n/a| 1| n/a| 64| n/a| n/a| 64| n/a| n/a| 64| n/a| n/a| 64| n/a| n/a| 64| n/a| n/a|
filt dilat| n/a| 1| n/a| 1| n/a| n/a| 1| n/a| n/a| 1| n/a| n/a| 1| n/a| n/a| 1| n/a| n/a|
num filts| n/a| 64| n/a| 64| n/a| n/a| 64| n/a| n/a| 64| n/a| n/a| 64| n/a| n/a| 64| n/a| n/a|
stride| n/a| 1| 1| 1| 1| 1| 1| 1| 1| 1| 1| 1| 1| 1| 1| 1| 1| 1|

pad n/a 1 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 0
rf size n/a 3 3 5 5 5 7 7 7 9 9 9 11 11 11 13 13 13
rf offset n/a 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
rf stride n/a 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
---------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- -------
data size NaNxNaN NaNxNaN NaNxNaN NaNxNaN NaNxNaN NaNxNaN NaNxNaN NaNxNaN NaNxNaN NaNxNaN NaNxNaN NaNxNaN NaNxNaN NaNxNaN NaNxNaN NaNxNaN NaNxNaN NaNxNaN
data depth NaN 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64
data num 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128
---------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- -------
data mem NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
param mem n/a 3KB 0B 144KB 1KB 0B 144KB 1KB 0B 144KB 1KB 0B 144KB 1KB 0B 144KB 1KB 0B

Thanks!

receptive field size

Hi, based on my understanding, receptive field size is same as the convolutional filter size, according to model initialization under the training code, filter size is 3x3 for all the layers, so receptive field size is 3x3 for all layers regardless the depth of the network.
However, the report says: "the receptive field of DnCNN with depth of d should be (2d + 1) x (2d + 1). Thus, for Gaussian denoising with a certain noise level, we set the receptive field size of DnCNN to 35x35 with the corresponding depth of 17."
Do you mind further explain it so that I know where I am wrong?
Thanks!

MATLAB JPEG encoder?

To generate the input of JPEG Deblocking, use MATLAB JPEG encoder to compress images.
Could you illustrate the MATLAB JPEG encoder? Is JPEG2000 compress algorithm? or other tools?
Thanks

vl_simplenn_move.m not found.

Hi, in your demo codes, you included

if useGPU
net = vl_simplenn_move(net, 'gpu') ;
end

However, its not included in your utilities.

Question about learning rate decay

Hello! I want to ask a question about the learning rate.
In your paper “Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising”, you mentioned that “The learning rate was decayed exponentially from 1e-1 to 1e-4 for the 50 epochs”.
I wonder how to do this in the code.
Or any others can help me? Thank you very much!!!

Hello

hello, i want use the tensorflow to make a dncnn, so, i need train the image, but in this code, just like have the target image(without noise; -Train400), i also need the input image(with noise),can you seed to me? thanks
my email address [email protected]

About the usage of DnCNN with matconvnet-1.0-beta24 on linux

Thanks so much for sharing your code. I tried to use it and find two problems:

  1. The slash in the path on Linux is different from Windows:
     folderTest  = 'testsets\Set12'; %%% test dataset
  1. For "matconvnet" version later than 20, some modifications need to be made.
    A bug " missing field 'dilate' in layers" shows up. I tried to fix this by using
   net = vl_simplenn_tidy(net)
   net = vl_simplenn_move(net,'gpu');

before

   res    = vl_simplenn(net,input,[],[],'conserveMemory',true,'mode','test');

Now it works.

sorry can i ask your one more question?

in the paper DnCNN-B is blind test about noise level.

that means, you train image given random noise in [0,55]?

or some image give small noise, other give big noise?

can i ask you specific condition or code about DnCNN-B train?

Two questions on image denoising

Hi,

I have two questions and hope you can give me some suggestions.

You mentioned "we follow [16] to use 400 images of size 180180 for training" in your paper. I know the 400 images come from the train and test sets of BSD dataset. The original images in BSD have larger size than 180180. I'm wondering how do you crop the 180*180 region in the original image? If it is possible, can you send me a copy? (I have also sent you an email ==!)

Another question is we usually extract patches as the input for the network from the images. So, how do you add the additive gaussian noise? Add on the image and then extract patches from the noisy image, or firstly extract patches from the clean image and then add the noise?

Thanks in advance!

Blind Color Denoiser Model

Hi,

I see that you include the pre-trained model for blind denoising on color images under
model/specifics/GD_Color_Blind.mat

I have noticed that this model does not include the parameters of the batch normalization layers. Did you change the weights of the convolution layers after the training by scaling and shifting them with the batch norm parameters or did you just remove the batch norm layers from the model?

I would like to use your model to initialize my network and either I need the full model or I need to have the same training dataset to be able to train it. May I ask you please for your help?

Thanks,
Caner

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.