- Evolve to be more comprehensive, effective and efficient for face related analytics & applications!
- About the name:
- "face" means this repo is dedicated for face related analytics & applications.
- "evolve" means unleash your greatness to be better and better. "LV" are capitalized to acknowledge the nurturing of Learning and Vision (LV) group, Nation University of Singapore (NUS).
- This work was done during Jian Zhao served as a short-term "Texpert" Research Scientist at Tencent FiT DeepSea Lab, Shenzhen, China.
Author | Jian Zhao |
---|---|
Homepage | https://zhaoj9014.github.io |
The code of face.evoLVe is released under the MIT License.
- Introduction
- Pre-Requisites
- Usage
- Face Alignment
- Data Processing
- Training and Validation
- Data Zoo
- Model Zoo
- Achievement
- Acknowledgement
- Citation
💁
- This repo provides a comprehensive face recognition library for face related analytics & applications, including face alignment (detection, landmark localization, affine transformation, etc.), data processing (e.g., augmentation, data balancing, normalization, etc.), various backbones (e.g., ResNet, IR-SE, ResNeXt, SE-ResNeXt, DenseNet, LightCNN, MobileNet, ShuffleNet, DPN, etc.), various losses (e.g., Softmax, Focal, Center, SphereFace, CosineFace, AmSoftmax, ArcFace, Triplet, etc.) and bags of tricks for improving performance (e.g., training refinements, model tweaks, knowledge distillation, etc.).
- All data before & after alignment, source codes and trained models are provided.
- This repo can help researchers/engineers develop deep face recognition models and algorithms quickly for practical use and deployment.
🍰
- Linux or macOS
- Python 3.7 (for training & validation) and Python 2.7 (for visualization w/ tensorboardx)
- PyTorch 1.0 (for traininig & validation, install w/
pip install torch torchvision
) - MXNet 1.3.1 (optinal, for data processing, install w/
pip install mxnet-cu90
) - TensorFlow 1.12 (optinal, for visualization, install w/
pip install tensorflow-gpu
) - tensorboardX 1.6 (optinal, for visualization, install w/
pip install tensorboardX
) - OpenCV 3.4.5 (install w/
pip install opencv-python
) - bcolz 1.2.0 (install w/
pip install bcolz
)
While not required, for optimal performance it is highly recommended to run the code using a CUDA enabled GPU. We used 4 NVIDIA Tesla P40 in parallel.
📙
- Clone the repo:
git clone https://github.com/ZhaoJ9014/face.evoLVe.PyTorch.git
. mkdir data checkpoint log
at appropriate directory to store your train/val/test data, checkpoints and training logs.- Prepare your train/val/test data (refer to Sec. Data Zoo for publicly available face related databases), and ensure each database folder has the following structure:
./data/db_name/
-> id1/
-> 1.jpg
-> ...
-> id2/
-> 1.jpg
-> ...
-> ...
-> ...
-> ...
- Refer to the codes of corresponding sections for specific purposes.
📐
- This section is based on the work of MTCNN.
- Folder:
./align
- Face detection, landmark localization APIs and visualization toy example with ipython notebook:
from PIL import Image
from detector import detect_faces
from visualization_utils import show_results
img = Image.open('some_img.jpg') # modify the image path to yours
bounding_boxes, landmarks = detect_faces(img) # detect bboxes and landmarks for all faces in the image
show_results(img, bounding_boxes, landmarks) # visualize the results
- Face alignment API (perform face detection, landmark localization and alignment with affine transformations on a whole database folder
source_root
with the directory structure as demonstrated in Sec. Usage, and store the aligned results to a new folderdest_root
with the same directory structure):
python face_align.py -source_root [source_root] -dest_root [dest_root] -crop_size [crop_size]
# python face_align.py -source_root './data/test' -dest_root './data/test_Aligned' -crop_size 112
- For macOS users, there is no need to worry about
*.DS_Store
files which may ruin your data, since they will be automatically removed when you run the scripts. - Keynotes for customed use: 1) specify the arguments of
source_root
,dest_root
andcrop_size
to your own values when you runface_align.py
; 2) pass your customedmin_face_size
,thresholds
andnms_thresholds
values to thedetect_faces
function ofdetector.py
to match your practical requirements.
📊
- Folder:
./balance
- Remove low-shot data API (remove the low-shot classes with less than
min_num
samples in the training setroot
with the directory structure as demonstrated in Sec. Usage for data balance and effective model training):
python remove_lowshot.py -root [root] -min_num [min_num]
# python remove_lowshot.py -root './data/train' -min_num 10
- Keynotes for customed use: specify the arguments of
root
andmin_num
to your own values when you runremove_lowshot.py
. - We prefer to include other data processing tricks, e.g., augmentation (flip horizontally, scale hue/satuation/brightness with coefficients uniformly drawn from [0.6,1.4], add PCA noise with a coefficient sampled from a normal distribution N(0,0.1), etc.), weighted random sampling, normalization, etc. to the main training script in Sec. Training and Validation to be self-contained.
☕
TO DO
🐯
Database | Version | #Identity | #Image | #Frame | #Video | Download Link |
---|---|---|---|---|---|---|
LFW | Raw | 5,749 | 13,233 | - | - | Google Drive, Baidu Drive |
LFW | Align_250x250 | 5,749 | 13,233 | - | - | Google Drive, Baidu Drive |
LFW | Align_112x112 | 5,749 | 13,233 | - | - | Google Drive, Baidu Drive |
CALFW | Raw | 4,025 | 12,174 | - | - | Google Drive, Baidu Drive |
CALFW | Align_112x112 | 4,025 | 12,174 | - | - | Google Drive, Baidu Drive |
CPLFW | Raw | 3,884 | 11,652 | - | - | Google Drive, Baidu Drive |
CPLFW | Align_112x112 | 3,884 | 11,652 | - | - | Google Drive, Baidu Drive |
CASIA-WebFace | Raw_v1 | 10,575 | 494,414 | - | - | Baidu Drive |
CASIA-WebFace | Raw_v2 | 10,575 | 494,414 | - | - | Google Drive, Baidu Drive |
CASIA-WebFace | Clean | 10,575 | 455,594 | - | - | Google Drive, Baidu Drive |
MS-Celeb-1M | Clean | 10,000 | 5,084,127 | - | - | Google Drive |
MS-Celeb-1M | Align_112x112 | 85,742 | 5,822,653 | - | - | Google Drive, Baidu Drive |
Vggface2 | Clean | 8,631 | 3,086,894 | - | - | Google Drive |
AgeDB | Raw | 570 | 16,488 | - | - | Google Drive, Baidu Drive |
AgeDB | Align_112x112 | 570 | 16,488 | - | - | Google Drive, Baidu Drive |
IJB-A | Clean | 500 | 5,396 | 20,369 | 2,085 | Google Drive, Baidu Drive |
IJB-B | Raw | 1,845 | 21,798 | 55,026 | 7,011 | Google Drive |
CFP | Raw | 500 | 7,000 | - | - | Google Drive, Baidu Drive |
CFP | Align_112x112 | 500 | 7,000 | - | - | Google Drive, Baidu Drive |
- Remark: unzip CASIA-WebFace clean version with
unzip casia-maxpy-clean.zip
cd casia-maxpy-clean
zip -F CASIA-maxpy-clean.zip --out CASIA-maxpy-clean_fix.zip
unzip CASIA-maxpy-clean_fix.zip
- Remark: after unzip, get image data & pair ground truths from AgeDB, CFP and LFW align_112x112 versions with
import numpy as np
import bcolz
import os
def get_pair(root, name):
carray = bcolz.carray(rootdir = os.path.join(root, name), mode='r')
issame = np.load('{}/{}_list.npy'.format(root, name))
return carray, issame
def get_data(data_root):
agedb_30, agedb_30_issame = get_pair(data_root, 'agedb_30')
cfp_fp, cfp_fp_issame = get_pair(data_root, 'cfp_fp')
lfw, lfw_issame = get_pair(data_root, 'lfw')
return agedb_30, cfp_fp, lfw, agedb_30_issame, cfp_fp_issame, lfw_issame
agedb_30, cfp_fp, lfw, agedb_30_issame, cfp_fp_issame, lfw_issame = get_data(DATA_ROOT)
- Due to release license issue, for other face related databases, please make contact with us in person for more details.
🐒
TO DO
🎊
-
2017 No.1 on ICCV 2017 MS-Celeb-1M Large-Scale Face Recognition Hard Set/Random Set/Low-Shot Learning Challenges. WeChat News, NUS ECE News, NUS ECE Poster, Award Certificate for Track-1, Award Certificate for Track-2, Award Ceremony.
-
2017 No.1 on National Institute of Standards and Technology (NIST) IARPA Janus Benchmark A (IJB-A) Unconstrained Face Verification challenge and Identification challenge. WeChat News.
-
State-of-the-art performance on
- MS-Celeb-1M (Challenge1 Hard Set Coverage@P=0.95: 79.10%; Challenge1 Random Set Coverage@P=0.95: 87.50%; Challenge2 Development Set Coverage@P=0.99: 100.00%; Challenge2 Base Set Top 1 Accuracy: 99.74%; Challenge2 Novel Set Coverage@P=0.99: 99.01%).
- IJB-A (1:1 Veification TAR@FAR=0.1: 99.6%±0.1%; 1:1 Veification TAR@FAR=0.01: 99.1%±0.2%; 1:1 Veification TAR@FAR=0.001: 97.9%±0.4%; 1:N Identification FNIR@FPIR=0.1: 1.3%±0.3%; 1:N Identification FNIR@FPIR=0.01: 5.4%±4.7%; 1:N Identification Rank1 Accuracy: 99.2%±0.1%; 1:N Identification Rank5 Accuracy: 99.7%±0.1%; 1:N Identification Rank10 Accuracy: 99.8%±0.1%).
- IJB-C (1:1 Veification TAR@FAR=1e-5: 82.6%).
- Labeled Faces in the Wild (LFW) (Accuracy: 99.85%±0.217%).
- Celebrities in Frontal-Profile (CFP) (Frontal-Profile Accuracy: 96.01%±0.84%; Frontal-Profile EER: 4.43%±1.04%; Frontal-Profile AUC: 99.00%±0.35%; Frontal-Frontal Accuracy: 99.64%±0.25%; Frontal-Frontal EER: 0.54%±0.37%; Frontal-Frontal AUC: 99.98%±0.03%).
- CMU Multi-PIE (Rank1 Accuracy Setting-1 under ±90°: 76.12%; Rank1 Accuracy Setting-2 under ±90°: 86.73%).
- MORPH Album2 (Rank1 Accuracy Setting-1: 99.65%; Rank1 Accuracy Setting-2: 99.26%).
- CACD-VS (Accuracy: 99.76%).
- FG-NET (Rank1 Accuracy: 93.20%).
👬
- This repo is inspired by InsightFace.MXNet, InsightFace.PyTorch, ArcFace.PyTorch, MTCNN.MXNet and PretrainedModels.PyTorch.
- The work of Jian Zhao was partially supported by China Scholarship Council (CSC) grant 201503170248.
- We would like to thank Prof. Jiashi Feng, Dr. Jianshu Li, Mr. Yu Cheng (Learning and Vision group, National University of Singapore), Mr. Yuan Xin, Mr. Di Wu, Mr. Zhenyuan Shen (Tencent FiT DeepSea Lab, China), Prof. Ran He, Prof. Junliang Xing, Mr. Xiang Wu (Institute of Automation, Chinese Academy of Sciences), Prof. Guosheng Hu (AnyVision Inc., U.K.), Dr. Lin Xiong (JD Digits AI Lab, U.S.), Miss Yi Cheng (Panasonic R&D Center, Singapore) for helpful discussions.
📑
-
Please consult and consider citing the following papers:
@article{zhao2018look, title={Look Across Elapse: Disentangled Representation Learning and Photorealistic Cross-Age Face Synthesis for Age-Invariant Face Recognition}, author={Zhao, Jian and Cheng, Yu and Cheng, Yi and Yang, Yang and Lan, Haochong and Zhao, Fang and Xiong, Lin and Xu, Yan and Li, Jianshu and Pranata, Sugiri and others}, journal={AAAI}, year={2019} } @article{zhao20183d, title={3D-Aided Dual-Agent GANs for Unconstrained Face Recognition}, author={Zhao, Jian and Xiong, Lin and Li, Jianshu and Xing, Junliang and Yan, Shuicheng and Feng, Jiashi}, journal={T-PAMI}, year={2018} } @inproceedings{zhao2018towards, title={Towards Pose Invariant Face Recognition in the Wild}, author={Zhao, Jian and Cheng, Yu and Xu, Yan and Xiong, Lin and Li, Jianshu and Zhao, Fang and Jayashree, Karlekar and Pranata, Sugiri and Shen, Shengmei and Xing, Junliang and others}, booktitle={CVPR}, pages={2207--2216}, year={2018} } @inproceedings{zhao2017dual, title={Dual-agent gans for photorealistic and identity preserving profile face synthesis}, author={Zhao, Jian and Xiong, Lin and Jayashree, Panasonic Karlekar and Li, Jianshu and Zhao, Fang and Wang, Zhecan and Pranata, Panasonic Sugiri and Shen, Panasonic Shengmei and Yan, Shuicheng and Feng, Jiashi}, booktitle={NIPS}, pages={66--76}, year={2017} } @inproceedings{zhao3d, title={3D-Aided Deep Pose-Invariant Face Recognition}, author={Zhao, Jian and Xiong, Lin and Cheng, Yu and Cheng, Yi and Li, Jianshu and Zhou, Li and Xu, Yan and Karlekar, Jayashree and Pranata, Sugiri and Shen, Shengmei and others}, booktitle={IJCAI}, pages={1184--1190}, year={2018} } @inproceedings{cheng2017know, title={Know you at one glance: A compact vector representation for low-shot learning}, author={Cheng, Yu and Zhao, Jian and Wang, Zhecan and Xu, Yan and Jayashree, Karlekar and Shen, Shengmei and Feng, Jiashi}, booktitle={ICCVW}, pages={1924--1932}, year={2017} }