bhushan23 / sfsnet-pytorch

Implementation of SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wild

SfSNet-PyTorch

Source Description

├─── data_loading.py: data loading and pre-processing methods <br>
├─── generate_dataset_csv.py: generate csv given dataset directory <br>
├─── interpolate.py: Create interpolation result from provided image directory <br>
├─── main_gen_pseudo-data.py: Train Skip-Connection based network on Synthetic dataset and generate Pseudo-Supervision data for CelebA dataset<br>
├─── main_gen_synthetic_and_full.py: Train on synthetic data, generate pseudo-supervision data, train on mix data<br>
├─── main_mix_training.py: Train SfSNet on mix data. Need to provide both CelebA and Synthetic dataset directory<br>
├─── models.py: Definition of all the models used. Skip-Net and SfSNet <br>
├─── shading.py: Shading generation method from Normal and Spherical Harmonics <br>
├─── train.py: Train and test rountines <br>
├─── utils.py: Help rountines <br>
├─── data/: sample small scale dataset to be used. Use ON_SERVER=False in main_* scripts to use this dataset <br>
├─── pretrained/: pretrained model provided by SfSNet Author- 'load_model_from_pretrained' loads this model's weights to our model<br>

Usage of main_* to train the model

usage: main_mix_training.py [-h] [--batch_size N] [--epochs N] [--lr LR]
                            [--wt_decay W] [--no_cuda] [--seed S]
                            [--read_first READ_FIRST] [--details DETAILS]
                            [--load_pretrained_model LOAD_PRETRAINED_MODEL]
                            [--syn_data SYN_DATA] [--celeba_data CELEBA_DATA]
                            [--log_dir LOG_DIR] [--load_model LOAD_MODEL]

SfSNet - Residual

optional arguments:
  -h, --help            show this help message and exit
  --batch_size N        input batch size for training (default: 8)
  --epochs N            number of epochs to train (default: 10)
  --lr LR               learning rate (default: 0.001)
  --wt_decay W          SGD momentum (default: 0.0005)
  --no_cuda             disables CUDA training
  --seed S              random seed (default: 1)
  --read_first READ_FIRST
                        read first n rows (default: -1) from the dataset
                        This is helpful to load part of the data. Note that, internally
                        we change this to sample randomly with seed value 100
  --details DETAILS     Explaination of the run
                        String provided will be written into root log directory
                        We perform many experiments and then get lost on the results and what was this experiment for.
                        This txt file will help us understand what was the purpose of this experiment
  --load_pretrained_model LOAD_PRETRAINED_MODEL
                        Pretrained model path for SfSNet based model provided by author
  --syn_data SYN_DATA   Synthetic Dataset path directory high level containing csv files
  --celeba_data CELEBA_DATA
                        CelebA Dataset path high level containing train, test folder and csv files
  --log_dir LOG_DIR     Log Path
                        Where to log and store results, model
  --load_model LOAD_MODEL
                        load model from following directory

Example

CUDA_VISIBLE_DEVICES=0,1 python main.py --epochs 100 --lr 0.0002 --batch_size 8 --read_first 10000
--log_dir ./results/skip_net/exp4/ --details 'Skipnet with normals'

sfsnet-pytorch's Issues

Could you release your whole training dataset of CelebA, i.e. CelebA_crop_resize_128?

Hi! Thanks very much for releasing your code.

But I have one more question: could you release your whole training dataset of CelebA? I saw it in generate_dataset_csv.py, in which there is something called CelebA_crop_resize_128.

Since CelebA has too many (200k+) images, I don't know which images are used in your training. So if at your convenience, could you upload the CelebA_crop_resize_128 set onto Google drive so that I can download?

Question about generate_celeba_synthesize() func in data_loading.py

Hi~
In generate_celeba_synthesize from data_loading.py(here), you just applied denorm on predicted_normal, but others are ignored.

Could you explain a little bit about why? I know the input face is in range [0~1], but why does the predicted_normal become [-1~1]?

Can you publish the complete operation steps and dependency libraries？

Thanks for your great work!

Code details

Hello!
First of all, thank you for your open sfsnet source code. I hope that you can publish the detailed training details and process of your code. It is convenient for me to reproduce the code you announced. Thank you very much for your work!

No module named 'sfs_net_model'

Hi there!

I was trying to run your implementation, but it seems there is a dependency missing on my side, specifically the SfSNet from line 22 of main_mix_training.py

from sfs_net_model import SfSNet as sfsnet_pretrained_model

do you know how can I obtain this dependency? I did not find it in the project and I wasn't successful in finding it with pip.

Cheers!

Pretrained model produce bad albedo result

Hello, I have tried to run your pretrained model but it seems that it produce a poor albedo result which in the end make the reconstruction also bad. Do you have any other pretrained model or would you mind giving me advice on how to fix this?

This is one of the result that I get by running mix_training without doing any training