zeta1999 / 3ddfa_v2

The official PyTorch implementation of Towards Fast, Accurate and Stable 3D Dense Face Alignment, ECCV, 2020

Towards Fast, Accurate and Stable 3D Dense Face Alignment

By Jianzhu Guo, Xiangyu Zhu, Yang Yang, Fan Yang, Zhen Lei and Stan Z. Li.

Introduction

This work extends 3DDFA, named 3DDFA_V2, titled Towards Fast, Accurate and Stable 3D Dense Face Alignment, accepted by ECCV 2020. The supplementary material is here. The gif above shows a demo of the tracking result. This repo is the official implementation of 3DDFA_V2.

Compared to 3DDFA, 3DDFA_V2 achieves better performance and stability. Besides, 3DDFA_V2 incorporates the fast face detector FaceBoxes instead of Dlib. A simple 3D render written by c++ and cython is also included. If you are interested in this repo, just try it on this google colab! Welcome for valuable issues and PRs 😄

Getting started

Requirements

See requirements.txt, tested on macOS and Linux platforms. Note that this repo uses Python3. The major dependencies are PyTorch, numpy and opencv-python, etc.

Usage

1. Clone this repo

git clone https://github.com/cleardusk/3DDFA_V2.git
cd 3DDFA_V2

2. Build the cython version of NMS, and Sim3DR

sh ./build.sh

3. Run demos

# running on still image, four options: 2d_sparse, 2d_dense, 3d, depth
python3 demo.py -f examples/inputs/emma.jpg

# running on videos
python3 demo_video.py -f examples/inputs/videos/214.avi

# running on videos smoothly by looking ahead by `n_next` frames
python3 demo_video_smooth.py -f examples/inputs/videos/214.avi

The implementation of tracking is simply by alignment. If the head pose > 90° or the motion is too fast, the alignment may fail. A threshold is used to trickly check the tracking state, but it is unstable.

You can refer to demo.ipynb or google colab for the step-by-step tutorial of running on the still image.

For example, running python3 demo.py -f examples/inputs/emma.jpg -o 3d will give the result below:

Acknowledgement

• The FaceBoxes module is modified from FaceBoxes.PyTorch

Citation

If your work or research benefits from this repo, please cite two bibs below : )

@inproceedings{guo2020towards,
    title =        {Towards Fast, Accurate and Stable 3D Dense Face Alignment},
    author =       {Guo, Jianzhu and Zhu, Xiangyu and Yang, Yang and Yang, Fan and Lei, Zhen and Li, Stan Z},
    booktitle =    {Proceedings of the European Conference on Computer Vision (ECCV)},
    year =         {2020}
}

@misc{3ddfa_cleardusk,
    author =       {Guo, Jianzhu and Zhu, Xiangyu and Lei, Zhen},
    title =        {3DDFA},
    howpublished = {\url{https://github.com/cleardusk/3DDFA}},
    year =         {2018}
}

Contact

Jianzhu Guo (郭建珠) [Homepage, Google Scholar]: [email protected] or [email protected].