ml-lab / 3ddfa_v2

Towards Fast, Accurate and Stable 3D Dense Face Alignment, ECCV, 2020

Towards Fast, Accurate and Stable 3D Dense Face Alignment

By Jianzhu Guo, Xiangyu Zhu, Yang Yang, Fan Yang, Zhen Lei and Stan Z. Li.

Introduction

This is an extended work of the github improved version of 3DDFA, named Towards Fast, Accurate and Stable 3D Dense Face Alignment, accepted by ECCV 2020. The supplementary material is here. The gif above shows a demo of the tracking result.

Getting started

Requirements

See requirements.txt, tested on macOS and Linux platforms. Note that this repo uses Python3. The major dependencies are PyTorch, numpy and opencv-python.

Usage

1. Clone this repo

git clone https://github.com/cleardusk/3DDFA_V2.git
cd 3DDFA_V2

2. Build the cython version of NMS, and Sim3DR

cd FaceBoxes
sh ./build_cpu_nms.sh
cd ..

cd Sim3DR
sh ./build_sim3dr.sh
cd ..

or simply build them by

sh ./build.sh

3. Run demos

# running on still image
python3 demo.py -f examples/inputs/emma.jpg

# running on videos
python3 demo_video.py -f examples/inputs/videos/214.avi

# running on videos smoothly by looking ahead by `n_next` frames
python3 demo_video_smooth.py -f examples/inputs/videos/214.avi

The implementation of tracking is simply by alignment. If the head pose > 90° or the motion is too fast, the alignment may fail. I use a threshold to trickly check the tracking state, but it is unstable.

You can refer to demo.ipynb for the step-by-step tutorial of running on the still image.

For example, running python3 demo.py -f examples/inputs/emma.jpg -o 3d will give the result below:

Acknowledgement

• The FaceBoxes module is modified from FaceBoxes.PyTorch

Citation

If your work or research benefits from this repo, please cite two bibs below : )

@inproceedings{guo2020towards,
    title =        {Towards Fast, Accurate and Stable 3D Dense Face Alignment},
    author =       {Guo, Jianzhu and Zhu, Xiangyu and Yang, Yang and Yang, Fan and Lei, Zhen and Li, Stan Z},
    booktitle =    {Proceedings of the European Conference on Computer Vision (ECCV)},
    year =         {2020}
}

@misc{3ddfa_cleardusk,
    author =       {Guo, Jianzhu and Zhu, Xiangyu and Lei, Zhen},
    title =        {3DDFA},
    howpublished = {\url{https://github.com/cleardusk/3DDFA}},
    year =         {2018}
}

Contact

Jianzhu Guo (郭建珠) [Homepage, Google Scholar]: [email protected] or [email protected].