sophiethompson1 / flame_pytorch

This is a implementation of the 3D FLAME model in PyTorch

FLAME: Articulated Expressive 3D Head Model (PyTorch)

This is an implementation of the FLAME 3D head model in PyTorch.

We also provide Tensorflow FLAME, a Chumpy-based FLAME-fitting repository, and code to convert from Basel Face Model to FLAME.

FLAME is a lightweight and expressive generic head model learned from over 33,000 of accurately aligned 3D scans. FLAME combines a linear identity shape space (trained from head scans of 3800 subjects) with an articulated neck, jaw, and eyeballs, pose-dependent corrective blendshapes, and additional global expression blendshapes. For details please see the scientific publication

Learning a model of facial shape and expression from 4D scans
Tianye Li*, Timo Bolkart*, Michael J. Black, Hao Li, and Javier Romero
ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 2017

and the supplementary video.

Installation

The code uses Python 3.7 and it is tested on PyTorch 1.4.

Setup FLAME PyTorch Virtual Environment

python3.7 -m venv <your_home_dir>/.virtualenvs/FLAME_PyTorch
source <your_home_dir>/.virtualenvs/FLAME_PyTorch/bin/activate

Clone the project and install requirements

git clone https://github.com/soubhiksanyal/FLAME_PyTorch
cd FLAME_PyTorch
pip install -r requirements.txt
mkdir model

Download models

• Download FLAME model from here. You need to sign up and agree to the model license for access to the model. Copy the downloaded model inside the model folder.
• Download Landmark embedings from RingNet Project. Copy it inside the model folder.

Demo

Loading FLAME and visualising the 3D landmarks on the face

Please note we used the pose dependent conture for the face as introduced by RingNet Project.

Run the following command from the terminal

python main.py

License

FLAME is available under Creative Commons Attribution license. By using the model or the code code, you acknowledge that you have read the license terms (https://flame.is.tue.mpg.de/modellicense.html), understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not use the code.

Referencing FLAME

When using this code in a scientific publication, please cite

@article{FLAME:SiggraphAsia2017,
  title = {Learning a model of facial shape and expression from {4D} scans},
  author = {Li, Tianye and Bolkart, Timo and Black, Michael. J. and Li, Hao and Romero, Javier},
  journal = {ACM Transactions on Graphics, (Proc. SIGGRAPH Asia)},
  volume = {36},
  number = {6},
  year = {2017},
  url = {https://doi.org/10.1145/3130800.3130813}
}

Additionally if you use the pose dependent dynamic landmarks from this codebase, please cite

@inproceedings{RingNet:CVPR:2019,
title = {Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision},
author = {Sanyal, Soubhik and Bolkart, Timo and Feng, Haiwen and Black, Michael},
booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
month = jun,
year = {2019},
month_numeric = {6}
}

Supported Projects

FLAME supports several projects such as

• CoMA: Convolutional Mesh Autoencoders
• RingNet: 3D Face Shape and Expression Reconstruction from an Image without 3D Supervision
• VOCA: Voice Operated Character Animation
• Expressive Body Capture: 3D Hands, Face, and Body from a Single Image
• ExPose: Monocular Expressive Body Regression through Body-Driven Attention
• GIF: Generative Interpretable Faces
• DECA: Detailed Expression Capture and Animation

FLAME is part of SMPL-X: : A new joint 3D model of the human body, face and hands together

Contact

If you have any questions regarding the PyTorch implementation then you can contact us at [email protected] and [email protected].

Acknowledgement

This repository is build with modifications from SMPLX.