pcg108 / ps-fcn

Learning Based Calibrated Photometric Stereo for Non-Lambertian Surface (ECCV 2018)

PS-FCN

PS-FCN: A Flexible Learning Framework for Photometric Stereo, ECCV 2018,
Guanying Chen, Kai Han, Kwan-Yee K. Wong

This paper addresses the problem of learning based photometric stereo for non-Lambertian surface:

Dependencies

PS-FCN is implemented in PyTorch and tested with Ubuntu 14.04, please install PyTorch first following the official instruction.

• Python 2.7
• PyTorch (version = 0.40)
• numpy
• scipy
• CUDA-8.0

Overview:

We provide:

• Datasets: Blobby dataset (4.7 GB), Sculpture dataset (19 GB)
• Trained models (trained on both the Blobby dataset and the Sculpture dataset with a per-sample input number of 32):
  • PS-FCN for calibrated photometric stereo
  • UPS-FCN for uncalibrated photometric stereo
• Code to test on DiLiGenT main dataset
• Code to train a new model

Testing

Download the trained models

sh scripts/download_pretrained_models.sh

Test on the DiLiGenT main dataset

# Download DiLiGenT main dataset
sh scripts/prepare_diligent_dataset.sh

# Test PS-FCN on DiLiGenT main dataset using all of the 96 image-light pairs
CUDA_VISIBLE_DEVICES=0 python eval/run_model.py --retrain data/models/PS-FCN_B_S_32.pth.tar --in_img_num 96

# Test UPS-FCN on DiLiGenT main dataset only using images as input 
CUDA_VISIBLE_DEVICES=0 python eval/run_model.py --retrain data/models/UPS-FCN_B_S_32.pth.tar --in_img_num 96 --in_light

Training

To train a new PS-FCN model, you have to follow the following steps:

Download the training data

# The total size of the zipped synthetic datasets is 4.7+19=23.7 GB 
# and it takes some times to download and unzip the datasets.
sh scripts/download_synthetic_datasets.sh

Call main.py to train PS-FCN or UPS-FCN

# Train PS-FCN on both synthetic datasets using 32 images-light pairs
CUDA_VISIBLE_DEVICES=0 python main.py --concat_data --in_img_num 32

# Train UPS-FCN on both synthetic datasets using 32 images
CUDA_VISIBLE_DEVICES=0 python main.py --concat_data --in_img_num 32 --in_light --item uncalib

# Please refer to options/base_opt.py and options/train_opt.py for more options

# You can find checkpoints and results in data/Training/

FAQ

Q1: How to test PS-FCN on other dataset?

• You have to implement a customized Dataset class to load your data. Please refer to datasets/DiLiGenT_data.py for an example that loads the DiLiGenT main dataset. Precomputed results on DiLiGenT main/test dataset, Gourd&Apple dataset, Light Stage Dataset and Synthetic Test dataset are available upon request.

Q2: Which eight sculpture shapes were used in rendering the training datasets? Why?

• Normal maps of the eight selected sculpture shapes are shown in the below figure (filenames can be found in this link). We chose these 8 shapes for their high quality meshes and complex geometry. Other complex 3D objects can also be used.

Q3: What should I do if I have problem in running your code?

• Please create an issue if you encounter errors when trying to run the code. Please also feel free to submit a bug report.

Citation

If you find this code or the provided data useful in your research, please consider cite:

@inproceedings{chen2018ps,
  title={PS-FCN: A Flexible Learning Framework for Photometric Stereo},
  author={Chen, Guanying and Han, Kai and Wong, Kwan-Yee K.},
  booktitle={ECCV},
  year={2018}
}