saikatdutta / stacked_dmshn_bokeh Goto Github PK

Official Repository for our CVPRW (MAI'21) paper.

License: MIT License

Python 95.28% Dockerfile 1.99% Shell 2.73%

bokeh-effect pytorch deep-learning image-to-image-translation bokeh-effect-rendering shallow-depth-of-field depth-of-field cvpr21 cvprw

stacked_dmshn_bokeh's Introduction

Stacked Deep Multi-Scale Hierarchical Network for Fast Bokeh Effect Rendering from a Single Image

Saikat Dutta, Sourya Dipta Das, Nisarg A. Shah and Anil Kumar Tiwari

Accepted at Mobile AI workshop, co-located with CVPR 2021 Paper | ArXiv | Supplementary | YouTube

Pytorch 1.1.0 Torchvision 0.3.0 skimage 0.16.2

Colab demo

1. Dataset:

Get the EBB! dataset by registering here.

Train split: data/train.csv

Test split (val294 set): data/test.csv

2. Run inference on Val294 set using DMSHN model:

python DMSHN_test.py

3. Run inference on Val294 set using Stacked DMSHN model:

python stacked_DMSHN_test.py

4. To generate PSNR, SSIM and LPIPS scores on output images:

python eval.py -d0 OUT_DIR -d1 GT_DIR --use_gpu

5. Citation:

@inproceedings{dutta2021stacked,
  title={Stacked Deep Multi-Scale Hierarchical Network for Fast Bokeh Effect Rendering from a Single Image},
  author={Dutta, Saikat and Das, Sourya Dipta and Shah, Nisarg A and Tiwari, Anil Kumar},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={2398--2407},
  year={2021}
}

6. Related work:

[1] Dutta, Saikat. "Depth-aware blending of smoothed images for bokeh effect generation." Journal of Visual Communication and Image Representation (2021): 103089. Paper ArXiv Project page

[2] Das, Sourya Dipta, and Saikat Dutta. "Fast deep multi-patch hierarchical network for nonhomogeneous image dehazing." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2020. Paper ArXiv Code

7. Useful Repositories:

[1] SSIM loss

[2] MSSSIM loss

[3] LPIPS

stacked_dmshn_bokeh's People

Contributors

Stargazers

Watchers

Forkers

yrcrcy zfw-cv aw-jiang rhwfy shyampaul2208 a125f utkarshyashvardhan11 linhong00316

stacked_dmshn_bokeh's Issues

One quick question。

Hi，

First, thank you for your sharing and help.
I studied DMSHN and found you compared it to PyNET. I tried to repeat PyNET(tensorflow) but failed (others had the same problem as me, color distortion in level 1) and I contacted the author but no one responded. I hope you can share your experience in repeating PyNET.
https://github.com/aiff22/PyNET-Bokeh

Hope you can help me. Thanks again.

dataset not available

Hi, it seems that the download links in AIM 2020 challenge are not accessible any more. Could you share the dataset if you have any local backups?

Questions about runtime？

Hi,

I tried to test runtime, but I could not find detailed measurement details in the paper. I have some questions :

Q1. Is the runtime average for VAL294? Do I need to remove the running time of the first slide?

Q2. What piece of code does the runtime measure? Does it include image preprocessing? Or only calculate the time of model. predict?
start = time.time()
bok_pred = bokehnet(input_image)
end = time.time() - start
print(end)

Thank you for your help.

Request for training script

Hello Sir,

While trying to replicate your work, I'm having some difficulties in training the model. The Adam optimizer is returning nan, and am not sure if its due to the wrong implementation of the loss function (tried implementing after reading your paper). It would be greatly helpful if you could share your training script. You could send it to my email id: [email protected].

Thanks in advance.