Coder Social home page Coder Social logo

cszn / srmd Goto Github PK

View Code? Open in Web Editor NEW
420.0 14.0 80.0 92.26 MB

Learning a Single Convolutional Super-Resolution Network for Multiple Degradations (CVPR, 2018) (Matlab)

Home Page: http://openaccess.thecvf.com/content_cvpr_2018/papers/Zhang_Learning_a_Single_CVPR_2018_paper.pdf

MATLAB 99.05% M 0.12% Objective-C 0.83%
sisr super-resolution denoising matlab matconvnet image-degradation deep-learning cnn regression non-blind

srmd's Introduction

Learning a Single Convolutional Super-Resolution Network for Multiple Degradations

visitors

I recommend to use the PyTorch code for training and testing. The model parameters of MatConvnet and PyTorch are same.

Abstract

Recent years have witnessed the unprecedented success of deep convolutional neural networks (CNNs) in single image super-resolution (SISR). However, existing CNN-based SISR methods mostly assume that a low-resolution (LR) image is bicubicly downsampled from a high-resolution (HR) image, thus inevitably giving rise to poor performance when the true degradation does not follow this assumption. Moreover, they lack scalability in learning a single model to deal with multiple degradations. To address these issues, we propose a dimensionality stretching strategy that enables a single convolutional super-resolution network to take two key factors of the SISR degradation process, i.e., blur kernel and noise level, as input. Consequently, the proposed super-resolver can handle multiple and even spatially variant degradations, which significantly improves the practicability. Extensive experimental results on synthetic and real LR images show that the proposed convolutional super-resolution network not only can produce favorable results on multiple degradations but also is computationally efficient, providing a highly effective and scalable solution to practical SISR applications.

Training Codes

SRMD_MatConvNet

Network Architecture

architecture In contrast to other CNN-based SISR methods which only take the LR image as input and lack scalability to handle other degradations, the proposed network takes the concatenated LR image and degradation maps as input, thus allowing a single model to manipulate multiple and even spatially variant degradations.

NTIRE 2018 Challenge on Image Super-Resolution 知乎

Track 3: Realistic Difficult Adverse Conditions

Test_Track_3_SRMD_non_blind.m is the testing code for Track 3 of NTIRE 2018 challenge on image super-resolution.

PSNR/SSIM Result of SRMD

track3

Visual Result of SRMD

The image size of the LR image is 510x405, the scale factor is 4.

SISR Results

Bicubic Degradation

bicubic1

bicubic2

General Degradation

The left is the LR image with isotropic Gaussian blur and noise level 50, the right is the super-resolved image by SRMD with scale factor 2.

The left is the LR image with isotropic Gaussian blur and noise level 5, the right is the super-resolved image by SRMD with scale factor 3.

The left is the LR image with anisotropic Gaussian blur, the right is the super-resolved image by SRMD with scale factor 4.

Real Image SR

realSR1

realSR2

SRMD models

  • Bicubic Degradation

Demo_bicubic_degradation_SRMD.m is the testing demo of SRMD for the widely-used bicubic degradation. Demo_bicubic_degradation_SRMDNF.m is the testing demo of SRMDNF for the widely-used bicubic degradation.

  • General Degradation

Demo_genearal_degradation_SRMD.m is the testing demo of SRMD for the general degradation. Demo_genearal_degradation_SRMDNF.m is the testing demo of SRMDNF for the general degradation.

  • Real Application

Demo_real_application_SRMD.m is the testing demo of SRMD for real image SR.

  • Others

Demo_degradation_direct_SRMD.m is the testing demo of SRMD for another widely-used degradation with direct downsampler.

Demo_degradation_direct_SRMDNF.m is the testing demo of SRMD for another widely-used degradation with direct downsampler.

A special case of SRMD when scale factor is 1.

Demo_denoising_and_deblurring_gray_SRMD.m is the testing demo of SRMD for grayscale image denoising and deblurring.

Demo_denoising_and_deblurring_color_SRMD.m is the testing demo of SRMD for for color image denoising and deblurring.

Demo_real_application_denoising_and_deblurring_gray.m is the testing demo of SRMD for real grayscale image denoising and deblurring.

Demo_real_application_denoising_and_deblurring_color.m is the testing demo of SRMD for real color image denoising and deblurring.

Requirements and Dependencies

or

  • MATLAB R2015b
Use `res = vl_srmd_matlab(net, input)` instead.

Citation

@inproceedings{zhang2018learning,
  title={Learning a single convolutional super-resolution network for multiple degradations},
  author={Zhang, Kai and Zuo, Wangmeng and Zhang, Lei},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition},
  pages={3262-3271},
  year={2018}
}

srmd's People

Contributors

cszn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

srmd's Issues

Question about Experiments on Bicubic Degradation

hello, I have a question about the experiment on bicubic degradation. in this experiment, how to get the blur kernel to form degradation maps.Thank you for your extremely busy schedule to answer my doubts.

A little question about network trainning

Hello,I can not understand one sentence in your paper,'When the training error keeps unchanged in five sequential epochs, we merge the parameters of each batch normalization into the adjacent convolution filters'.I try to figure it out by reading your code but fail to deal with matlab...Thanks!

how to get net.meta.P

I find the file 'net.meta.P' in your codes, whose size is 15x255, could you tell me how to make it?
Could you tell some details?
Thanks!

degradation model

In your paper: We set the rotation angle range to [0,π]. For the scaling of eigenvalues, it is set from 0.5 to 6, 8 and 10 for scale factors 2, 3 and 4, respectively.

In your code:
ksize = 15;
theta = pirand(1);
l1 = 0.1+9.9
rand(1);
l2 = 0.1+(l1-0.1)*rand(1);
kernel = anisotropic_Gaussian(ksize,theta,l1,l2); % double
kernel = single(kernel);

I would like to ask how to set the parameter of the gaussian blur kernel.
thank you!

Your Results in New Super-Resolution Benchmarks

Hello,

MSU Graphics & Media Lab Video Group has recently launched two new Super-Resolution Benchmarks.

Your method achieved 19th place in Video Upscalers Benchmark: Quality Enhancement in 'Animation 4x' category. We look forward to your future work!

We would be grateful for your feedback on our work.

Undefined function or variable 'vl_simplenn_tidy'

When I used the function Demo_bicubic_degradation_SRMD,it had the following message:
Undefined function or variable 'vl_simplenn_tidy'.
Error in Demo_bicubic_degradation_SRMD (line 59)
net = vl_simplenn_tidy(net);
Could you help me to solve it?

can not find vl_argparse in utilities

ERROR in vl_nnSubP (line 5)
opts = vl_argparse(opts, varargin, 'nonrecursive') ;
ERROR in vl_srmd_matlab (line 36)
res(i+1).x = vl_nnSubP(res(i).x, [],'scale',l.scale);
ERROR in Demo_bicubic_degradation_SRMD (line 118)
res = vl_srmd_matlab(net, input); % you should also set "useGPU = 0;" and comment "net =
vl_simplenn_tidy(net);"

Grid search strategy

For the experiments on real images, a grid search strategy is used as the degradation info. Would you please help provide more details? Thanks

issue about test data

Can you provide test results that ran out of your network? Including, Set5,Set14,B100 and Urban100

Training Datasets

Is the dataset used for training available online? Is there any way it could be uploaded?

SRMD_non_blind.mat

I saw the dimention-reduced blur kernels are precalculated as SRMD_non_blind.mat. I am very interested in the problem how can precalculate? Because I try to SR the real image. If have some tips ?

Problem about "4.4 Experiments on Spatially Variant Degradation"

Thanks for your amazing work!
But I have a question that does " synthesize an LR image with spatially variant blur kernels and noise levels" in 4.4 mean one image and multiple anisotropic Gaussian kernels and noises?
And is the model trained and tested with only isotropic Gaussian kernel as input in this experiment?
If you could tell me more details, I will appreciate it!

resulting image

Hi, thank you so much for the source code. Besides, could you please provide the resulting image of the degradation model?

problems when using `res = vl_srmd_matlab(net, input)`

Demo_real_application_denoising_and_deblurring_color


----frog-----Super-Resolution-----

Undefined function or variable 'vl_argparse'.
Error in vl_nnSubP (line 5)
opts = vl_argparse(opts, varargin, 'nonrecursive') ;
Error in vl_srmd_matlab (line 36)
res(i+1).x = vl_nnSubP(res(i).x, [],'scale',l.scale);
Error in Demo_real_application_denoising_and_deblurring_color (line 167)
res = vl_srmd_matlab(net, input); % When use this, you should also set "useGPU = 0;" and comment "net = vl_simplenn_tidy(net);"

data_augmentation function

When I was runing the generatepatches.m, a error was raised that "undefined function or variable 'data_augmentation'.
I would like asking about it, is it a default function or user defined function that no provided?
thank you very much!

Errors in the testing process

The training is normal. The following problems are encountered in the testing process.
wrong with vl_nnbnorm
vl::impl::dispatch_cudnn<C, CU>::operator(): BatchNormForwardWithMomentCudnn::operator(): cuDNN error [cudnn:
"d:\matlab\r2018b\bin\matconvnet-1.0-beta25\matlab\src\bits\nnbnorm_cudnn.cu":155 (CUDNN_STATUS_BAD_PARAM)]

how to realise PCA?

I find the file 'net.meta.P' in your codes, whose size is 15x255, could you tell me how to make it?
And it seems that your model uses subpixel with upsample_factor=0.5 in the first layer? If so ,could you tell some details?
Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.