zhhoper / dpr

Code for Deep Single-image Portrait Image Relighting

Deep Single-Image Portrait Relighting [Project Page]

Hao Zhou, Sunil Hadap, Kalyan Sunkavalli, David W. Jacobs. In ICCV, 2019

Overview

• Test script for 512x512 images: testNetwork_demo_512.py
• Test script for 1024x1024 images: testNetwork_demo_1024.py

Dependencies

pytorch >= 1.0.0

opencv >= 4.0.0

shtools: https://shtools.oca.eu/shtools/ (optional)

Notes

We include an example image and seven example lightings in data. Note that different methods may have different coordinate system for Spherical Harmonics (SH), you may need to change the coordiante system if you use SH lighting from other sources. The coordinate system of our method is in accordance with shtools, we provide a function utils_normal.py in utils to help you tansfer the coordinate system from bip2017 and sfsNet to our coordinate system. To use utils_normal.py you need to install shtools. The code is for research purpose only.

Data Preparation

We publish the code for data preparation, please find it in (https://github.com/zhhoper/RI_render_DPR).

Citation

If you use this code for your research, please consider citing:

@InProceedings{DPR,
  title={Deep Single Portrait Image Relighting},
  author = {Hao Zhou and Sunil Hadap and Kalyan Sunkavalli and David W. Jacobs},
  booktitle={International Conference on Computer Vision (ICCV)},
  year={2019}
}

dpr's Issues

How to customize lighting?

I noticed the 7 lighting data files in the data folder. But how can I generate other lighting files for a custom lighting angle?

Problems to download the dataset

Hi, your work is amazing and thanks to share with us.

But I have a problem to download the dataset, Google say "download quota exceeded".
Do you know about that? Do you have other methods to download the dataset?

Add license

Is it possible to add a license file to this project?

What is the ori_shading.exr?

Thank you for your sharing!

Well, I have a question about the DPR_dataset. There is a file named ori_shading.exr, and I cannot find it in RI_render_DPR's result. So how to make the ori_shading.exr, is this the same file with ori_shading.npy?

Question About Test Dataset

Hello, thanks for your wonderful work. Could you please send me the test dataset for quantitative results in your paper? Or, how can I download the test part of the Multi-PIE dataset? Thanks.

What kind of lisense ?

Is there any limitation for commercial use?

Dataset downloading error

Wonder if the bandwidth is hard capped. Even after waiting for 24 hours, it errors out after only 7GB downloaded for the 45GB file. Could you help unlock?

A problem about dataset

How to modify the code to read dataset?
I'm new to this, so I'm not very proficient in some aspects, thank you.

How to control exposure?

Which SH(spherical harmonics) value should I change to control exposure?

How to convert sfsNet SH to shtools？

Hello, I extracted 27 dimension SH coefficients from sfsNet ( 9 dimension for each channel of RGB), I see you provide the "convert code", but the output dimension is still 27, your network input is 9 dimension. Can you show me a code example of how to convert？Thank you so much！！！！！！

Great work! Could you please share train-related code? Thank you!

Great work! Could you please share train-related code?
Thank you!

Some other questions about your compared method.

Sorry I neglected the email... Closed.

Do I need to use RI_render_DPR if I am to try testNetwork_demo_1024.py?

Thanks for sharing this great work!

Do I need to use RI_render_DPR if I want to try testNetwork_demo_1024.py on some 2D images? (Instead of Obama's?)

I gave the code a go and did see any landmark, albedo, normal, uv map and semantic segmentation loaded. Or these landmarks are only needed for training a new model?

Thanks in advance

How To Generate Custom 'rotate_light_00.txt' File

I'm trying to understand how the values in these files were generated.

Am I right to assume that the 9 values in each rotate_light_xx.txt file are the first nine spherical harmonic coefficients?

Any advice on how someone could define new ones based on a new direction the light is coming from?

ARAP-Based Normal Refinement Code

Hi, this is a great paper with fine details. The final result is also incredible.
I found that the ARAP-Based refine module is important in building up the whole pipeline. Could you please share related code?
Thank you!

outputSH only works for the test image obama

Hi, I was trying to test the code on this repo with some custom face images but the outputSH is always all black (zeros) or always all white (in the circle). Isn't the algorithm supposed to compute the lighting of the original input image correctly? Or there is something that we need to change or use to do so?

How do I get file like“rotate_light_00.txt” file from a face image?

Thanks for your excellent work! I have a question, how can i get file like“rotate_light_00.txt” file from a face image? I want to do some illumination transfer work. thank you!

About the network

Hi,
Thank you for your great work! But I am confused about the difference of two network in 'model' folder, what is 'matchFeature' meaning?

how to generate light direction matrix from specific angle?

subj

Very slow on CPU

The 512 pre-trained model takes roughly 20 seconds for inference using CPUs. For comparison, do you know how long this takes on GPUs? Do you have any advice for optimizing inference time on CPUs? I would ideally like to run the model in ~1 second.

How can I normalize predicted SH light to a score to evaluate brightness?

Hi @zhhoper,
Thank you for your work, I am working about evaluate brightness of face image and I want to compare a predicted SH light and label SH light to identify bad brightness image and good brightness image. Can you tell me how to do it?
Thanks.