Coder Social home page Coder Social logo

kupynorest / deblurgan Goto Github PK

View Code? Open in Web Editor NEW
2.5K 60.0 513.0 85.86 MB

Image Deblurring using Generative Adversarial Networks

License: Other

Python 72.07% HTML 27.93%
deep-learning gan deblurring image-processing image-manipulation neural-network convolutional-neural-networks convolutional-networks computer-vision image-to-image-translation

deblurgan's Introduction

DeblurGAN

arXiv Paper Version

Pytorch implementation of the paper DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks.

Our network takes blurry image as an input and procude the corresponding sharp estimate, as in the example:

The model we use is Conditional Wasserstein GAN with Gradient Penalty + Perceptual loss based on VGG-19 activations. Such architecture also gives good results on other image-to-image translation problems (super resolution, colorization, inpainting, dehazing etc.)

How to run

Prerequisites

  • NVIDIA GPU + CUDA CuDNN (CPU untested, feedback appreciated)
  • Pytorch

Download weights from Google Drive . Note that during the inference you need to keep only Generator weights.

Put the weights into

/.checkpoints/experiment_name

To test a model put your blurry images into a folder and run:

python test.py --dataroot /.path_to_your_data --model test --dataset_mode single --learn_residual

Data

Download dataset for Object Detection benchmark from Google Drive

Train

If you want to train the model on your data run the following command to create image pairs:

python datasets/combine_A_and_B.py --fold_A /path/to/data/A --fold_B /path/to/data/B --fold_AB /path/to/data

And then the following command to train the model

python train.py --dataroot /.path_to_your_data --learn_residual --resize_or_crop crop --fineSize CROP_SIZE (we used 256)

Other Implementations

Keras Blog

Keras Repository

Citation

If you find our code helpful in your research or work please cite our paper.

@article{DeblurGAN,
  title = {DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks},
  author = {Kupyn, Orest and Budzan, Volodymyr and Mykhailych, Mykola and Mishkin, Dmytro and Matas, Jiri},
  journal = {ArXiv e-prints},
  eprint = {1711.07064},
  year = 2017
}

Acknowledgments

Code borrows heavily from pix2pix. The images were taken from GoPRO test dataset - DeepDeblur

deblurgan's People

Contributors

gachiemchiep avatar keineahnung2345 avatar kupynorest avatar ninja-j avatar rahulvigneswaran avatar rimchang avatar vbudzan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

deblurgan's Issues

Wrong link?

Hi,
The link you offered is of net_D. Shouldn't it be net_G?

ImportError: No module named dominate

When I run the program, this problem occur, and I don't see the folder '/.checkpoints/experiment_name', could you tell me how to fix this problem and where the '/.checkpoints/experiment_name' is?
Thanks!

On the detailed operation

Hello, I am a beginner. How can I add dataroot to the environment, and I want to turn my blurred picture into clear image. Which folder should I put into it and and what commands to input to Python?

The following is my mistake
[-h] --dataroot DATAROOT [--batchSize BATCHSIZE]
[--loadSizeX LOADSIZEX] [--loadSizeY LOADSIZEY]
[--fineSize FINESIZE] [--input_nc INPUT_NC]
[--output_nc OUTPUT_NC] [--ngf NGF] [--ndf NDF]
[--which_model_netD WHICH_MODEL_NETD]
[--which_model_netG WHICH_MODEL_NETG] [--learn_residual]
[--gan_type GAN_TYPE] [--n_layers_D N_LAYERS_D]
[--gpu_ids GPU_IDS] [--name NAME] [--dataset_mode DATASET_MODE]
[--model MODEL] [--which_direction WHICH_DIRECTION]
[--nThreads NTHREADS] [--checkpoints_dir CHECKPOINTS_DIR]
[--norm NORM] [--serial_batches]
[--display_winsize DISPLAY_WINSIZE] [--display_id DISPLAY_ID]
[--display_port DISPLAY_PORT]
[--display_single_pane_ncols DISPLAY_SINGLE_PANE_NCOLS]
[--no_dropout] [--max_dataset_size MAX_DATASET_SIZE]
[--resize_or_crop RESIZE_OR_CROP] [--no_flip] [--ntest NTEST]
[--results_dir RESULTS_DIR] [--aspect_ratio ASPECT_RATIO]
[--phase PHASE] [--which_epoch WHICH_EPOCH]
[--how_many HOW_MANY]
test.py: error: the following arguments are required: --dataroot

Are the generator weights are only kept in the "test" runtime code in this repo or We have to take care of it?

  1. "Download weights from Dropbox . Note that during the inference you need to keep only Generator weights". Are the generator weights are only kept in the "test" runtime code in this repo or We have to take care of it?

  2. Also once we run test.py, it generates the results folder with "fake" and "real" image. I undestand the fake as a deblurred image. Is that right?

  3. The result images generated are below. The fake_b is still blur. Any problem with the dropbox weights? Anyone got the same results?

Real:
yolo_b_real_a
Generated Fake:
yolo_b_fake_b

@KupynOrest

Backward twice when using wgan-gp as critic

I encountered the following error when using wgan-gp as discriminator and everything went well whe using other discriminator:
Traceback (most recent call last):
File "train.py", line 42, in
model.optimize_parameters()
File "/home/gasoon/code/research/VideoDeblur/models/conditional_gan_model.py", line 136, in optimize_parameters
self.backward_D()
File "/home/gasoon/code/research/VideoDeblur/models/conditional_gan_model.py", line 110, in backward_D
self.loss_D.backward()
File "/home/gasoon/anaconda3/envs/vd/lib/python3.6/site-packages/torch/tensor.py", line 93, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/gasoon/anaconda3/envs/vd/lib/python3.6/site-packages/torch/autograd/init.py", line 89, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

I'm using the data generated by myself whose resolution is 640*480.
Does anyone know the way to solve it? Thank you so much!

ConnectionError:HTTPConnectionPool(host='localhost', port=8097)

When I train DeblurGAN,I received this error ,while test no this error.
ConnectionError: HTTPConnectionPool(host='localhost', port=8097): Max retries exceeded with url: /events (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f32c9ce9950>: Failed to establish a new connection: [Errno 111] Connection refused',))
How to solve this problem?

Unexpected device_id

When testing the code, I got: TypeError: cuda() got an unexpected keyword argument 'device_id'
at: File "./models/networks.py", line 53, in define_G netG.cuda(device_id=gpu_ids[0]).
The device_id was printed as 0 in my platform, with the log below:

------------ Options -------------
aspect_ratio: 1.0
batchSize: 1
checkpoints_dir: ./checkpoints
dataroot: ./images/blurred_sharp/blurred
dataset_mode: single
display_id: 1
display_port: 8097
display_single_pane_ncols: 0
display_winsize: 256
fineSize: 256
gan_type: wgan-gp
gpu_ids: [0]
how_many: 5000
input_nc: 3
isTrain: False
learn_residual: True
loadSizeX: 640
loadSizeY: 360
max_dataset_size: inf
model: test
nThreads: 2
n_layers_D: 3
name: experiment_name
ndf: 64
ngf: 64
no_dropout: False
no_flip: False
norm: instance
ntest: inf
output_nc: 3
phase: test
resize_or_crop: resize_and_crop
results_dir: ./results/
serial_batches: False
which_direction: AtoB
which_epoch: latest
which_model_netD: basic
which_model_netG: resnet_9blocks
-------------- End ----------------
CustomDatasetDataLoader
dataset [SingleImageDataset] was created
Traceback (most recent call last):
File "test.py", line 21, in
model = create_model(opt)
File "./models/models.py", line 11, in create_model
model.initialize(opt)
File "./models/test_model.py", line 20, in initialize
opt.learn_residual)
File "./models/networks.py", line 53, in define_G
netG.cuda(device_id=gpu_ids[0])
TypeError: cuda() got an unexpected keyword argument 'device_id'

PSNR calculation

Hi!
In the test.py, the PSNR calculation uses the input blurred image and the output image. But is not the PSNR calculation should use the shape image in the test set and the output image?

Can I use the pre-trained weights and the model for fine-tuning?

Hi,

I am trying to use DeblurGAN for a downstream task for which I need to load the pre-trained model weights and freeze all but the last few layers. Can any of you please help me giving directions on how to do that? torch.load() only gives me the parameters, I need the model too

unexpected key "model.0.weight" in state_dict

Hi,
When I run test.py as per instructions in readme file, I get the 'unexpected key "model.0.weight" in state_dict' error.

$ python test.py --dataroot ../BlurredImages/ --model test --dataset_mode single --learn_residual------------ Options -------------
aspect_ratio: 1.0
batchSize: 1
checkpoints_dir: ./checkpoints
dataroot: ../BlurredImages/
dataset_mode: single
display_id: 1
display_port: 8097
display_single_pane_ncols: 0
display_winsize: 256
fineSize: 256
gan_type: wgan-gp
gpu_ids: [0]
how_many: 5000
input_nc: 3
isTrain: False
learn_residual: True
loadSizeX: 640
loadSizeY: 360
max_dataset_size: inf
model: test
nThreads: 2
n_layers_D: 3
name: experiment_name
ndf: 64
ngf: 64
no_dropout: False
no_flip: False
norm: instance
ntest: inf
output_nc: 3
phase: test
resize_or_crop: resize_and_crop
results_dir: ./results/
serial_batches: False
which_direction: AtoB
which_epoch: latest
which_model_netD: basic
which_model_netG: resnet_9blocks
-------------- End ----------------
CustomDatasetDataLoader
dataset [SingleImageDataset] was created
/usr/local/lib/python2.7/dist-packages/torchvision/transforms/transforms.py:156: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
"please use transforms.Resize instead.")
Traceback (most recent call last):
File "test.py", line 21, in
model = create_model(opt)
File "/home/env114/Downloads/DeblurGAN-master/models/models.py", line 11, in create_model
model.initialize(opt)
File "/home/env114/Downloads/DeblurGAN-master/models/test_model.py", line 21, in initialize
self.load_network(self.netG, 'G', which_epoch)
File "/home/env114/Downloads/DeblurGAN-master/models/base_model.py", line 53, in load_network
network.load_state_dict(torch.load(save_path))
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 331, in load_state_dict
.format(name))
KeyError: 'unexpected key "model.0.weight" in state_dict'

If you could suggest how to fix this error it will be helpful.
Thanks

I can not find the file path

after downloading and extracting the zip file, I can not find the file path (/.checkpoints/experiment_name) to put the weights . can you tell me?

TypeError: cuda() got an unexpected keyword argument 'device_id'

When I run train.py in command line,I meet this problem. What shold I do?

(E:\programs\Anaconda\envs\python) D:\cv2\DeblurGAN-master>python train.py --dataroot blurred_sharp/ --continue_train
------------ Options -------------
batchSize: 1
beta1: 0.5
checkpoints_dir: ./checkpoints
continue_train: True
dataroot: blurred_sharp/
dataset_mode: aligned
display_freq: 100
display_id: 1
display_port: 8097
display_single_pane_ncols: 0
display_winsize: 256
epoch_count: 1
fineSize: 256
gan_type: wgan-gp
gpu_ids: [0]
identity: 0.0
input_nc: 3
isTrain: True
lambda_A: 100.0
lambda_B: 10.0
learn_residual: False
loadSizeX: 640
loadSizeY: 360
lr: 0.0001
max_dataset_size: inf
model: content_gan
nThreads: 2
n_layers_D: 3
name: experiment_name
ndf: 64
ngf: 64
niter: 150
niter_decay: 150
no_dropout: False
no_flip: False
no_html: False
norm: instance
output_nc: 3
phase: train
pool_size: 50
print_freq: 100
resize_or_crop: resize_and_crop
save_epoch_freq: 5
save_latest_freq: 5000
serial_batches: False
which_direction: AtoB
which_epoch: latest
which_model_netD: basic
which_model_netG: resnet_9blocks
-------------- End ----------------
CustomDatasetDataLoader
dataset [AlignedDataset] was created
Traceback (most recent call last):
File "train.py", line 56, in
model = create_model(opt)
File "D:\cv2\DeblurGAN-master\models\models.py", line 11, in create_model
model.initialize(opt)
File "D:\cv2\DeblurGAN-master\models\conditional_gan_model.py", line 34, in initialize
opt.which_model_netG, opt.norm, not opt.no_dropout, self.gpu_ids, use_parallel, opt.learn_residual)
File "D:\cv2\DeblurGAN-master\models\networks.py", line 52, in define_G
netG.cuda(device_id=gpu_ids[0])
TypeError: cuda() got an unexpected keyword argument 'device_id'

There is a "RuntimeError" when I use this command to run the program, my environment is windows10 64bit system.

python test.py --dataroot E:\test_opencv\DeblurGAN-master(fangzhishoudou)\pictest\ --model test --dataset_mode single --learn_residual

There is a "RuntimeError" when I use the above command to run the program,how can I fix this error?
Error message is following.

Total number of parameters: 11378179

model [TestModel] was created
Traceback (most recent call last):
File "", line 1, in
File "E:\Anaconda3\lib\multiprocessing\spawn.py", line 106, in spawn_main
exitcode = _main(fd)
File "E:\Anaconda3\lib\multiprocessing\spawn.py", line 115, in _main
prepare(preparation_data)
File "E:\Anaconda3\lib\multiprocessing\spawn.py", line 226, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "E:\Anaconda3\lib\multiprocessing\spawn.py", line 278, in _fixup_main_from_path
run_name="mp_main")
File "E:\Anaconda3\lib\runpy.py", line 254, in run_path
pkg_name=pkg_name, script_name=fname)
File "E:\Anaconda3\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "E:\Anaconda3\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "E:\test_opencv\DeblurGAN-master(fangzhishoudou)\test.py", line 31, in
for i, data in enumerate(dataset):
File "E:\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 310, in iter
return DataLoaderIter(self)
File "E:\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 167, in init
w.start()
File "E:\Anaconda3\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "E:\Anaconda3\lib\multiprocessing\context.py", line 212, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "E:\Anaconda3\lib\multiprocessing\context.py", line 313, in _Popen
return Popen(process_obj)
File "E:\Anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 34, in init
prep_data = spawn.get_preparation_data(process_obj._name)
File "E:\Anaconda3\lib\multiprocessing\spawn.py", line 144, in get_preparation_data
_check_not_importing_main()
File "E:\Anaconda3\lib\multiprocessing\spawn.py", line 137, in _check_not_importing_main
is not going to be frozen to produce an executable.''')
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

Traceback (most recent call last):
File "test.py", line 31, in
for i, data in enumerate(dataset):
File "E:\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 204, in next
idx, batch = self.data_queue.get()
File "E:\Anaconda3\lib\multiprocessing\queues.py", line 343, in get
res = self._reader.recv_bytes()
File "E:\Anaconda3\lib\multiprocessing\connection.py", line 216, in recv_bytes
buf = self._recv_bytes(maxlength)
File "E:\Anaconda3\lib\multiprocessing\connection.py", line 306, in _recv_bytes
[ov.event], False, INFINITE)
KeyboardInterrupt

AttributeError: 'module' object has no attribute 'InstanceNorm2d'

Hi,
I met this problem when run "test.py", any suggestions?

------------ Options -------------
aspect_ratio: 1.0
batchSize: 1
checkpoints_dir: ./checkpoints
dataroot: /home/zhangtian/workspace-Deblur/DeblurGAN/
dataset_mode: single
display_id: 1
display_port: 8097
display_single_pane_ncols: 0
display_winsize: 256
fineSize: 256
gan_type: wgan-gp
gpu_ids: [0]
how_many: 5000
input_nc: 3
isTrain: False
learn_residual: True
loadSizeX: 640
loadSizeY: 360
max_dataset_size: inf
model: test
nThreads: 2
n_layers_D: 3
name: experiment_name
ndf: 64
ngf: 64
no_dropout: False
no_flip: False
norm: instance
ntest: inf
output_nc: 3
phase: test
resize_or_crop: resize_and_crop
results_dir: ./results/
serial_batches: False
which_direction: AtoB
which_epoch: latest
which_model_netD: basic
which_model_netG: resnet_9blocks
-------------- End ----------------
CustomDatasetDataLoader
dataset [SingleImageDataset] was created
/home/zhangtian/anaconda2/envs/DeblurGAN/lib/python2.7/site-packages/torchvision/transforms/transforms.py:156: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
"please use transforms.Resize instead.")
Traceback (most recent call last):
File "test.py", line 22, in
model = create_model(opt)
File "/home/zhangtian/workspace-Deblur/DeblurGAN/models/models.py", line 11, in create_model
model.initialize(opt)
File "/home/zhangtian/workspace-Deblur/DeblurGAN/models/test_model.py", line 19, in initialize
opt.learn_residual)
File "/home/zhangtian/workspace-Deblur/DeblurGAN/models/networks.py", line 36, in define_G
norm_layer = get_norm_layer(norm_type=norm)
File "/home/zhangtian/workspace-Deblur/DeblurGAN/models/networks.py", line 27, in get_norm_layer
norm_layer = functools.partial(nn.InstanceNorm2d, affine=False)
AttributeError: 'module' object has no attribute 'InstanceNorm2d'

Value of PSFs in motion_blur

Hi, I'm wondering why psf[0]~psf[2] are always empty (zero matrix).
It seems that only psf[3] works ??????

If BlurImage(os.path.join(folder, path), PSFs=psf,
path__to_save=folder_to_save, part=np.random.choice([1, 2, 3])).
blur_image(save=True)

part=1,2
it turns out to be an empty image.

I update the networks.py, but it rises a error(Test mode).

RuntimeError: While copying the parameter named model.19.weight, whose dimensions in the model are torch.Size([256, 128, 4, 4]) and whose dimensions in the checkpoint are torch.Size([256, 128, 3, 3]).
But if I use the old files(but fixed the device_id errors), it can work.
I checked the change log, I found that the ConvTranspose2d settings were changed.
Why do you add this change? What is the improvement of it?

command line:
python test.py --dataroot testimage --model test --dataset_mode single --learn_residual
output:

------------ Options -------------
aspect_ratio: 1.0
batchSize: 1
checkpoints_dir: ./checkpoints
dataroot: testimage
dataset_mode: single
display_id: 1
display_port: 8097
display_single_pane_ncols: 0
display_winsize: 256
fineSize: 256
gan_type: wgan-gp
gpu_ids: [0]
how_many: 5000
input_nc: 3
isTrain: False
learn_residual: True
loadSizeX: 640
loadSizeY: 360
max_dataset_size: inf
model: test
nThreads: 2
n_layers_D: 3
name: experiment_name
ndf: 64
ngf: 64
no_dropout: False
no_flip: False
norm: instance
ntest: inf
output_nc: 3
phase: test
resize_or_crop: resize_and_crop
results_dir: ./results/
serial_batches: False
which_direction: AtoB
which_epoch: latest
which_model_netD: basic
which_model_netG: resnet_9blocks
-------------- End ----------------
CustomDatasetDataLoader
dataset [SingleImageDataset] was created
/home/chilam/anaconda3/envs/py36/lib/python3.6/site-packages/torchvision/transforms/transforms.py:156: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
"please use transforms.Resize instead.")
Traceback (most recent call last):
File "/home/chilam/anaconda3/envs/py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 482, in load_state_dict
own_state[name].copy_(param)
RuntimeError: invalid argument 2: sizes do not match at /pytorch/torch/lib/THC/generic/THCTensorCopy.c:51

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "test.py", line 21, in
model = create_model(opt)
File "/home/chilam/workspace/DeblurGAN/models/models.py", line 11, in create_model
model.initialize(opt)
File "/home/chilam/workspace/DeblurGAN/models/test_model.py", line 21, in initialize
self.load_network(self.netG, 'G', which_epoch)
File "/home/chilam/workspace/DeblurGAN/models/base_model.py", line 53, in load_network
network.load_state_dict(torch.load(save_path))
File "/home/chilam/anaconda3/envs/py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 487, in load_state_dict
.format(name, own_state[name].size(), param.size()))
RuntimeError: While copying the parameter named model.19.weight, whose dimensions in the model are torch.Size([256, 128, 4, 4]) and whose dimensions in the checkpoint are torch.Size([256, 128, 3, 3]).

contrast algorithm

Thanks for your great work. I·m collecting the latest algorithms about learning to deblur. And could you tell me where did you get the algorithm for the program "Learning a Convolutional Neural Network for Non-uniform Motion Blur Rmoval" used in your contrast experiment .Did you realize it yourself?

save image

Hello,

I saved the model and the loaded if from my project (I imported network file for that).
now I want to make one forward pass and see the output images.
I cant get good images as output and i think that i might dont know how to do it well.
I tried to follow the way that the test() save images and i wrote:

`#part 1 - from:util.util function:tensor2im
    image_numpy = outputs[0].data.cpu().float().numpy()
    image_numpy = (np.transpose(image_numpy, (1, 2, 0)) + 1) / 2.0 * 255.0
    outputImage=image_numpy.astype(np.uint8)

    #part 2 from util.util function save_image
    image_pil = None
    if outputImage.shape[2] == 1:
        outputImage = np.reshape(outputImage, (outputImage.shape[0],outputImage.shape[1]))
        image_pil = Image.fromarray(outputImage, 'L')
    else:
        image_pil = Image.fromarray(outputImage)
    image_pil.save('Image.png')
    
    
    plt.subplot(num_example_imgs, 3, i * 3 + 3)
    plt.axis('off')
    plt.imshow(image_pil)`

and I get as output
image

How to decide when to stop training?

From my experience in classification training, the val data is used to decide when to stop during training on train data. But in deblurGAN I haven't found this mechanics.
So how to decide to stop training?

How to get deblurred images of full resolution in test?

Hello, I test the model on my images (1333x764) using following command , but I get 256x256 deblurred images, how can I get deblurred images of original resolution ?

python test.py --dataroot /.path_to_your_data --model test --dataset_mode single --learn_residual

It seems that the model cropped the image ,but i dont know how to avoid it.

Blur generation and object detection benchmark

Hi @KupynOrest , thanks for your work.
Another two questions here
(1) Do you think it is OK for you to provide the ground truth labels of your detection benchmark ???
(2) Are there any source codes for blurring generation of your detection benchmark ?
In your paper (6.3. Object Detection benchmark on YOLO) it says that "Following [13][23][25] we take a random between 5 and 25 frames taken by 240fps bla bla bla". It seems to be different from your (4. Motion blur generation) and the code you have provided.

Thank you~

new code for PSNR

hi!
if i want to run psnr calculation on some dataset how can i do it?
thanks!

When I run the model ,I meet the error:

When I run the model ,I meet the error:
------------ Options -------------
aspect_ratio: 1.0
batchSize: 1
checkpoints_dir: ./checkpoints
dataroot: images/
dataset_mode: single
display_id: 1
display_port: 8097
display_single_pane_ncols: 0
display_winsize: 256
fineSize: 256
gan_type: wgan-gp
gpu_ids: [0]
how_many: 5000
input_nc: 3
isTrain: False
learn_residual: True
loadSizeX: 640
loadSizeY: 360
max_dataset_size: inf
model: test
nThreads: 2
n_layers_D: 3
name: experiment_name
ndf: 64
ngf: 64
no_dropout: False
no_flip: False
norm: instance
ntest: inf
output_nc: 3
phase: test
resize_or_crop: resize_and_crop
results_dir: ./results/
serial_batches: False
which_direction: AtoB
which_epoch: latest
which_model_netD: basic
which_model_netG: resnet_9blocks
-------------- End ----------------
CustomDatasetDataLoader
dataset [SingleImageDataset] was created
Traceback (most recent call last):
File "test.py", line 32, in
for i, (data,_) in enumerate(dataset):
File "/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.py", line 212, in next
return self._process_next_batch(batch)
File "/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.py", line 239, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
TypeError: Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.py", line 41, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/media/media_share/linkfile/DeblurGAN/data/single_dataset.py", line 25, in getitem
A_img = self.transform(A_img)
File "/usr/local/lib/python2.7/dist-packages/torchvision/transforms.py", line 29, in call
img = t(img)
File "/usr/local/lib/python2.7/dist-packages/torchvision/transforms.py", line 139, in call
ow = int(self.size * w / h)
TypeError: unsupported operand type(s) for /: 'list' and 'int'

The model creates artifacts in images with low texture

Hi,

Great work. I noticed that the provided trained model creates significant artifacts when the input images contains minimal texture. For example, if you tile a larger image and feed it into the system the areas such as smooth walls/surfaces will have artifacts that resemble JPG artifacts. I wonder if that relates to the training set you used.

RuntimeError: CUDNN_STATUS_INTERNAL_ERROR

I input the following command to train my data, but have some error.
CUDA_VISIBLE_DEVICE=3 python train.py --dataroot blurred_sharp/ --learn_residual --resize_or_crop crop --fineSize 265 --no_html

error:
Traceback (most recent call last):
File "train.py", line 59, in
train(opt, data_loader, model, visualizer)
File "train.py", line 22, in train
model.optimize_parameters()
File "/home/liaochunxiu/DeblurGAN-master/models/conditional_gan_model.py", line 104, in optimize_parameters
self.forward()
File "/home/liaochunxiu/DeblurGAN-master/models/conditional_gan_model.py", line 76, in forward
self.fake_B = self.netG.forward(self.real_A)
File "/home/liaochunxiu/DeblurGAN-master/models/networks.py", line 146, in forward
output = self.model(input)
File "/usr/local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/site-packages/torch/nn/modules/container.py", line 67, in forward
input = module(input)
File "/usr/local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 282, in forward
self.padding, self.dilation, self.groups)
File "/usr/local/lib/python3.6/site-packages/torch/nn/functional.py", line 90, in conv2d
return f(input, weight, bias)
RuntimeError: CUDNN_STATUS_INTERNAL_ERROR

Can you tell me how to do for this?
Thanks~

ImportError: cannot import name 'SSIM'

Traceback (most recent call last):
File "test.py", line 10, in
from ssim import SSIM
ImportError: cannot import name 'SSIM'

There is no error when I enter import SSIM at CMD window.
How to fix the error?

How to prepare train data and how to train DeblurGAN?

I tried to train DeblurGAN but failed. I dont know how to prepare train data and val data and the meaning of some options,such as 'AtoB'.
I generated blured images using the code blur_image.py.
The train command:
python train.py --dataroot ../../deblurPwLine/ --continue_train

dataroot is organized as this:
-deblurPwLine
--train "stores some sharp images"
--val "stores some sharp images"

On batchNorm in test time

I find that there is big difference whether set self.netG.eval() or not.
But in the original test code, it doesn't set self.netG.eval().

I am not sure which is better.

No module named 'dominate'

I meet the error
util/html.py", line 1, in
import dominate
ModuleNotFoundError: No module named 'dominate'

By the way, if I want to run it on CPU,where I need to change?

Thank you.

ImportError: cannot import name 'SSIM'

Hello!
When I run train.py in command line,I meet this problem. What shold I do?

python test.py --dataroot ./test --model test --dataset_mode single --learn_residual

Traceback (most recent call last):
File "test.py", line 10, in
from ssim import SSIM
ImportError: cannot import name 'SSIM'

use_parallel is an undefined name in test_model.py

https://github.com/KupynOrest/DeblurGAN/search?utf8=%E2%9C%93&q=use_parallel&type=

https://github.com/KupynOrest/DeblurGAN/blob/master/models/test_model.py#L18

flake8 testing of https://github.com/KupynOrest/DeblurGAN on Python 2.7.14

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

./models/test_model.py:18:105: F821 undefined name 'use_parallel'
                                      opt.which_model_netG, opt.norm, not opt.no_dropout, self.gpu_ids, use_parallel,
                                                                                                        ^

WGAN-GP memory leak?

Hi,

I am interested in WGAN-GP, but it has memory #3858. Do you have memory leaks when you training the DeblurGAN net.

THANKS!

KeyError: 'B'

ubuntu@ip-Address:~/DeblurGAN$ python test.py --dataroot mydata --model experiment_name --dataset_mode single --learn_residual
------------ Options -------------
aspect_ratio: 1.0
batchSize: 1
checkpoints_dir: ./checkpoints
dataroot: mydata
dataset_mode: single
display_id: 1
display_port: 8097
display_single_pane_ncols: 0
display_winsize: 256
fineSize: 256
gan_type: wgan-gp
gpu_ids: [0]
how_many: 5000
input_nc: 3
isTrain: False
learn_residual: True
loadSizeX: 640
loadSizeY: 360
max_dataset_size: inf
model: experiment_name
nThreads: 2
n_layers_D: 3
name: experiment_name
ndf: 64
ngf: 64
no_dropout: False
no_flip: False
norm: instance
ntest: inf
output_nc: 3
phase: test
resize_or_crop: resize_and_crop
results_dir: ./results/
serial_batches: False
which_direction: AtoB
which_epoch: latest
which_model_netD: basic
which_model_netG: resnet_9blocks
-------------- End ----------------
CustomDatasetDataLoader
dataset [SingleImageDataset] was created
/usr/local/lib/python2.7/dist-packages/torchvision/transforms/transforms.py:156: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  "please use transforms.Resize instead.")
---------- Networks initialized -------------
ResnetGenerator(
  (model): Sequential(
    (0): ReflectionPad2d((3, 3, 3, 3))
    (1): Conv2d(3, 64, kernel_size=(7, 7), stride=(1, 1))
    (2): InstanceNorm2d(64, eps=1e-05, momentum=0.1, affine=False)
    (3): ReLU(inplace)
    (4): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
    (5): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
    (6): ReLU(inplace)
    (7): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
    (8): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
    (9): ReLU(inplace)
    (10): ResnetBlock(
      (conv_block): Sequential(
        (0): ReflectionPad2d((1, 1, 1, 1))
        (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
        (3): ReLU(inplace)
        (4): Dropout(p=0.5)
        (5): ReflectionPad2d((1, 1, 1, 1))
        (6): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (7): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
      )
    )
    (11): ResnetBlock(
      (conv_block): Sequential(
        (0): ReflectionPad2d((1, 1, 1, 1))
        (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
        (3): ReLU(inplace)
        (4): Dropout(p=0.5)
        (5): ReflectionPad2d((1, 1, 1, 1))
        (6): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (7): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
      )
    )
    (12): ResnetBlock(
      (conv_block): Sequential(
        (0): ReflectionPad2d((1, 1, 1, 1))
        (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
        (3): ReLU(inplace)
        (4): Dropout(p=0.5)
        (5): ReflectionPad2d((1, 1, 1, 1))
        (6): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (7): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
      )
    )
    (13): ResnetBlock(
      (conv_block): Sequential(
        (0): ReflectionPad2d((1, 1, 1, 1))
        (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
        (3): ReLU(inplace)
        (4): Dropout(p=0.5)
        (5): ReflectionPad2d((1, 1, 1, 1))
        (6): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (7): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
      )
    )
    (14): ResnetBlock(
      (conv_block): Sequential(
        (0): ReflectionPad2d((1, 1, 1, 1))
        (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
        (3): ReLU(inplace)
        (4): Dropout(p=0.5)
        (5): ReflectionPad2d((1, 1, 1, 1))
        (6): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (7): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
      )
    )
    (15): ResnetBlock(
      (conv_block): Sequential(
        (0): ReflectionPad2d((1, 1, 1, 1))
        (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
        (3): ReLU(inplace)
        (4): Dropout(p=0.5)
        (5): ReflectionPad2d((1, 1, 1, 1))
        (6): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (7): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
      )
    )
    (16): ResnetBlock(
      (conv_block): Sequential(
        (0): ReflectionPad2d((1, 1, 1, 1))
        (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
        (3): ReLU(inplace)
        (4): Dropout(p=0.5)
        (5): ReflectionPad2d((1, 1, 1, 1))
        (6): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (7): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
      )
    )
    (17): ResnetBlock(
      (conv_block): Sequential(
        (0): ReflectionPad2d((1, 1, 1, 1))
        (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
        (3): ReLU(inplace)
        (4): Dropout(p=0.5)
        (5): ReflectionPad2d((1, 1, 1, 1))
        (6): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (7): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
      )
    )
    (18): ResnetBlock(
      (conv_block): Sequential(
        (0): ReflectionPad2d((1, 1, 1, 1))
        (1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
        (3): ReLU(inplace)
        (4): Dropout(p=0.5)
        (5): ReflectionPad2d((1, 1, 1, 1))
        (6): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1))
        (7): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False)
      )
    )
    (19): ConvTranspose2d(256, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), output_padding=(1, 1))
    (20): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False)
    (21): ReLU(inplace)
    (22): ConvTranspose2d(128, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), output_padding=(1, 1))
    (23): InstanceNorm2d(64, eps=1e-05, momentum=0.1, affine=False)
    (24): ReLU(inplace)
    (25): ReflectionPad2d((3, 3, 3, 3))
    (26): Conv2d(64, 3, kernel_size=(7, 7), stride=(1, 1))
    (27): Tanh()
  )
)
Total number of parameters: 11378179
-----------------------------------------------
model [ConditionalGANModel] was created
Traceback (most recent call last):
  File "test.py", line 35, in <module>
    model.set_input(data)
  File "/home/ubuntu/DeblurGAN/models/conditional_gan_model.py", line 69, in set_input
    input_B = input['B' if AtoB else 'A']
KeyError: 'B'
ubuntu@ip-Address:~/DeblurGAN$ ^C

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.