One-shot Face Reenactment

Official test script for 2019 BMVC spotlight paper 'One-shot Face Reenactment' in PyTorch.

Installation

Requirements

Linux
Python 3.6
PyTorch 0.4+
CUDA 9.0+
GCC 4.9+

Easy Install

pip install -r requirements.txt

Getting Started

Prepare Data

It is recommended to symlink the dataset root to $PROJECT/data.

Project
├── data
│   ├── poseGuide
│   │   ├── imgs
│   │   ├── lms
│   ├── reference
│   │   ├── imgs
│   │   ├── lms

imgs : store images
lms : store landmarks extracted from images
- format : 106 common facial key points & 20+20 gaze key points

Example input data is organized in folder 'data'. Please organize your data in the format the same as the example input data if you want to test with your own data.

Output images are saved in folder 'output'.

Due to the protocol of company, the model to extract 106 + 40 facial landmarks cannot be released, however, if you want to get access to the following dataset, please fill in the license file in the repo (license/celebHQlms_license.pdf), then email the signed copy to [email protected] to get access to the annotation dataset.

our preprocessed 106 + 40 facial landmark annotations of celebHQ dataset
additional 80 images as pose guide with corresponding 106 + 40 facial landmark annotations

Inference with pretrained model

python test.py --pose_path PATH/TO/POSE/GUIDE/IMG/DIR --ref_path PATH/TO/REF/IMG/DIR --pose_lms PATH/TO/POSE/LANDMARK/FILE --ref_lms PATH/TO/REF/LANDMARK/FILE

output sequence: 
		ref1-pose1, ref1-pose2,  ref1-pose3, ... &
		ref2-pose1, ref2-pose2,  ref2-pose3, ... &
		ref3-pose1, ref3-pose2,  ref3-pose3, ... &
		    .				
		    .				
		    .

Pretrained model

You can download models from here

Project
├── pretrainModel
│   ├── id_200.pth
│   ├── vgg16-397923af.pth
├── trained_model
│   ├── latest_net_appEnc.pth
│   ├── latest_net_appDnc.pth
│   ├── latest_net_netG.pth
│   ├── latest_net_netD64.pth
│   ├── latest_net_netD128.pth
│   ├── latest_net_netD256.pth

Visualization of results

You can download our sample data and corresponding results from here

License and Citation

The use of this software follows MIT License.

@inproceedings{OneShotFace2019,
  title={One-shot Face Reenactment},
  author={Zhang, Yunxuan and Zhang, Siwei and He, Yue and Li, Cheng and Loy, Chen Change and Liu, Ziwei},
  booktitle={British Machine Vision Conference (BMVC)},
  year={2019}
}

bruinxiong / one_shot_face_reenactment Goto Github PK