saafke / fsrcnn_tensorflow

Tensorflow implementation of 'Accelerating the Super-Resolution Convolutional Neural Network'.

FSRCNN_Tensorflow

TensorFlow implementation of Accelerating the Super-Resolution Convolutional Neural Network [1].

This implementation replaces the transpose conv2d layer by a sub-pixel layer [2].

Includes pretrained models for scales x2, x3 and x4. Which were trained on T91-image dataset, and finetuned on General100 dataset.

Google Summer of Code with OpenCV

This repository was made during the 2019 GSoC program for the organization OpenCV. The trained models (.pb files) in this repo can easily be used for inference in OpenCV with the 'dnn_superres' module. See the OpenCV documentation for how to do this.

Requirements

• Tensorflow
• Python
• Numpy
• cv2
• imutils

Running

Download T91, General100 and Set14.

T91 is for training. General100 for finetuning. Set14 as the validation set.

Train:

• from scratch
  python main.py --train --scale <scale> --fromscratch --traindir /path-to-train_images/ --validdir /path-to-valid_images/

• load previous
  python main.py --train --scale <scale> --traindir /path-to-train_images/ --validdir /path-to-valid_images/

• finetune
  python main.py --train --scale <scale> --finetune --finetunedir /path-to-images/ --validdir /path-to-valid_images/

Test:
python main.py --test --image /image-path/

Export:
python3 main.py --export

Extra arguments (Fsrcnn small, batch size, lr etc.):
python main.py --h

Example

(1) Original picture
(2) Input image
(3) Bicubic scaled (3x) image
(4) FSRCNN scaled (3x) image

Notes

FSRCNN-small is a network with fewer parameters. Thus it is faster but has lower performance.

References

1. Chao Dong, Chen Change Loy, Xiaoou Tang. Accelerating the Super-Resolution Convolutional Neural Network, in Proceedings of European Conference on Computer Vision (ECCV), 2016
2. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. By Shi et. al.

fsrcnn_tensorflow's Issues

> Hi,

I am pretty sure it was python 2.7, although I am not sure about TensorFlow, my guess would be 1.12.0. I can double-check for you somewhere later this week. It should work with most combinations though. Could you perhaps show me the exact error you are getting?

Originally posted by @Saafke in #1 (comment)

I don't think so. Because in the python2.7, The function of 'print' no ()

one question about scaling of images

Hi again Saafke,
i wanted to quest you something.When we augment our data using scaling with 0.6 , 0.7, 0.8, 0.9 factor then we have to downscale them again with the scale factor(2 or 3 or 4) to produce the training data?

First_error

Traceback (most recent call last):
File "/content/FSRCNN_Tensorflow/main.py", line 2, in
import fsrcnn
File "/content/FSRCNN_Tensorflow/fsrcnn.py", line 8
def model(x, y, lr_size, scale, batch, lr, (d, s, m)):
^
SyntaxError: invalid syntax

when beginning, it's wrong.

graph_transforms module not found

I am on a Mac silicon architecture laptop and I used python 3.11 and a tensorflow 2.15.0. I converted the code in this repo to tensorflow 2.x by refering to the video https://www.youtube.com/watch?v=QMec7r-NHHk&t=140s. But still it is showing error graph_tools module not found. Without the tensorflow 2.x conversion also it is showing the same error.

version of tensorflow and python

Hi,
I wonder the version of tensorflow and python. I run the code and it gives me message that the version is not matched.

Thanks.

About the training with the 91 image dataset and fine-tuning with the General 100 dataset

One last question Saafke,
when we train a network from scratch we train it with the 91-images datasets.When we have to fine-tune it we have to train the network with general 100 dataset and with 91 images dataset or only with general-100 dataset?