xiaohangzhan / face_recognition_framework

Multi-task face recognition framework based on PyTorch

Face recognition framework based on PyTorch.

Introduction

This is a face recognition framework based on PyTorch with convenient training, evaluation and feature extraction functions. It is originally a multi-task face recognition framework for our accpeted ECCV 2018 paper, "Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition". However, it is also a common framework for face recognition. You can freely customize your experiments with your data and configurations with it.

Paper

Xiaohang Zhan, Ziwei Liu, Junjie Yan, Dahua Lin, Chen Change Loy, "Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition", ECCV 2018

Project Page: link

Why multi-task?

Different datasets have different identity (category) sets. We do not know the intersection between them. Hence instead of merging identity sets of different datasets, regarding them as different tasks is an effective alternative way.

Features

Framework: Multi-task, Single Task

Loss: Softmax Loss, ArcFace

Backbone CNN: ResNet, DenseNet, Inception, InceptionResNet, NASNet, VGG

Benchmarks: Megaface (FaceScrub), IJB-A, LFW, CFP-FF, CFP-FP, AgeDB-30, calfw, cplfw

Data aug: flip, scale, translation

Online testing and visualization with Tensorboard.

Setup step by step

1. Clone the project.

   git clone [email protected]:XiaohangZhan/face_recognition_framework.git
   cd face_recognition_framework
   

2. Dependency.

   python=3.6, tensorboardX, pytorch=0.3.1, mxnet, sklearn

3. Data Preparation.

   Download datasets from insightface into your data storage folder, e.g., ~/data/face_recognition/. Taking CASIA-Webface for example:

   cd ~/data/face_recognition/
   unzip faces_CASIA_112x112.zip
   cd - # back to the repo root
   mkdir data
   python tools/convert_data.py -r ~/data/face_recognition/faces_webface_112x112 -o ~/data/face_recognition/faces_webface_112x112 # convert mxnet records into images
   ln -s ~/data/face_recognition/faces_webface_112x112 data/webface

   Optionally, if you want to test on MegaFace. Download testing set from here into your data storage folder, e.g., ~/data/face_recognition/. Then:

   cd ~/data/face_recognition/
   mkdir -p megaface_test/raw
   cd megaface_test/raw
   mv ../../megaface_testpack_v1.0.zip .
   unzip -q megaface_testpack_v1.0.zip
   cd $THIS_REPO # back to the repo root
   ln -s  ~/data/face_recognition/megaface_test data/megaface_test

   Next, download MegaFace lists from here into ~/data/face_recognition/megaface_test/. Finally, the folder data/megaface_test/ looks like:

   data
     ├── megaface_test
       ├── concat_list.txt
       ├── facescrub3530
       ├── megaface_distractor
       ├── raw
   

4. Training.

   sh experiments/webface/res50-bs64-sz224-ep35/train.sh
   

5. Monitoring.

   tensorboard --logdir experiments
   

6. Resume training.

   sh experiments/webface/res50-bs64-sz224-ep35/resume.sh 10 # e.g., resume from epoch 10
   

7. Evalution.

   sh experiments/webface/res50-bs64-sz224-ep35/evaluation.sh 35 # e.g., evaluate epoch 35
   

8. Feature extraction.

   Firstly, specify the data_name, data_root and data_list under extract_info in the config file. The data_list is a txt file containing an image relative filename in each line. Then execute:

   # e.g., extract features with epoch 35 model.
   # The feature file is stored in checkpoints/ckpt_epoch_35_[data_name].bin
   sh experiments/webface/res50-bs64-sz224-ep35/extract.sh 35 
   

Baselines

• Trained using Webface

arch	LFW	CFP-FF	CFP-FP	AgeDB-30	calfw	cplfw
resnet-50	0.9850	0.9804	0.9117	0.8967	0.9013	0.8423

• Trained using MS1M

arch	LFW	CFP-FF	CFP-FP	AgeDB-30	calfw	cplfw	vgg2-FP	megaface
densenet-121	0.9948	0.9946	0.9594	0.9615	0.9500	0.9057	0.9418	0.8665
densenet-121-arc	0.9973	0.9979	0.9601	0.9728	0.9558	0.9063	0.9496	0.9287

Note that the hyper-parameters are not adjusted to optimal. Hence, they are not the state-of-the-art face recognition models. You may download those pre-trained models here.

Bibtex

If you find this code useful in your research, please cite:

@inproceedings{zhan2018consensus,
  title={Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition},
  author={Zhan, Xiaohang and Liu, Ziwei and Yan, Junjie and Lin, Dahua and Change Loy, Chen},
  booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
  pages={568--583},
  year={2018}
}

TODO (Will carry out in a "Buddha-like" way)

1. Implement distributed training.
2. Adjust hyper-parameters.
3. Multi-task experiments.

face_recognition_framework's Issues

about pretrained models

Thanks for your great work!
Is it possible that you provide your pretrained models?

Fail to evaluate: all are 0.5000(0.0000)

The training loss is normal, from 9 to about 1

(mmdet) spl_feature@spl-feature:~/shizishan/face_recognition_framework-master$ sh experiments/webface/res50-bs64-sz224-ep35/evaluate.sh 5
main.py:49: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
config = yaml.load(f)
Creating model on [1] gpus: 0
=> loading checkpoint 'experiments/webface/res50-bs64-sz224-ep35/checkpoints/ckpt_epoch_5.pth.tar'
=> loaded checkpoint 'experiments/webface/res50-bs64-sz224-ep35/checkpoints/ckpt_epoch_5.pth.tar' (epoch 5 iteration 153320)
loading from: experiments/webface/res50-bs64-sz224-ep35/checkpoints/ckpt_epoch_5_lfw.bin

• lfw: accuracy: 0.5000(0.0000)
  loading from: experiments/webface/res50-bs64-sz224-ep35/checkpoints/ckpt_epoch_5_cfp_ff.bin
• cfp_ff: accuracy: 0.5000(0.0000)
  loading from: experiments/webface/res50-bs64-sz224-ep35/checkpoints/ckpt_epoch_5_cfp_fp.bin
• cfp_fp: accuracy: 0.5000(0.0000)
  loading from: experiments/webface/res50-bs64-sz224-ep35/checkpoints/ckpt_epoch_5_agedb_30.bin
• agedb_30: accuracy: 0.5000(0.0000)
  loading from: experiments/webface/res50-bs64-sz224-ep35/checkpoints/ckpt_epoch_5_calfw.bin
• calfw: accuracy: 0.5000(0.0000)
  loading from: experiments/webface/res50-bs64-sz224-ep35/checkpoints/ckpt_epoch_5_cplfw.bin
• cplfw: accuracy: 0.5000(0.0000)
  (mmdet) spl_feature@spl-feature:~/shizishan/face_recognition_framework-master$

error when train my own data

@XiaohangZhan Hello, I prepared my own data set according to the prompt, and then re-created config.yaml, there was an ‘out of memory’ error during training, and reducing the batchsize to 1 was still useless.

Out of memory

Hello, the following problems occurred during the training. May I ask what caused them and what are the solutions?I'm using cuda10.1,python3.6.10.Waiting for your reply online.Thank you.
RuntimeError: cuda runtime error (2) : out of memory at /pytorch/torch/lib/THC/generic/THCStorage.cu:58

No module named 'test'

Seems like module 'test' is missing in current version, is it changed to evaluation? Also, benchmarks for IJB-A, LFW, CFP-FF, CFP-FP, AgeDB-30, calfw, cplfw are nowhere to be found.

CASIA-Webface convert data error....

excuse me, where can i find the train.rec ?

$ python tools/convert_data.py -r /data2/Dataset/celebA -o /data2/user1/celebA_112x112/
terminate called after throwing an instance of 'dmlc::Error'
  what():  [11:36:32] src/io/local_filesys.cc:209: Check failed: allow_null:  LocalFileSystem::Open "/data2/Dataset/celebA/train.rec": No such file or directory
Stack trace:
  [bt] (0) /home/user1/miniconda3/lib/python3.7/site-packages/mxnet/libmxnet.so(+0x3c598b8) [0x7fdc735e08b8]


Aborted (core dumped)

OS error 24

Hello:

When i run this step 4 "sh experiments/webface/res50-bs64-sz224-ep35/train.sh". i am getting error message.
OSERROR: [Errno 24] Too many files open.

It is all from reduction.py line 191 and resource_sharer.py line 48

Any help to resolve this is greatly appreciated.

About Multi Task Learning

Great work!!!
Can you give some examples for multi-task.
I think the examples in 'experiments' are all single tasks.
Is there anything wrong with my understanding?

What are the contents of list.txt and meta.txt?

Thanks for your great work !
Could you provide an example or a screenshot about the contents of list.txt and meta.txt to help us better understand your code ?

data_list:
    - 'data/emore/list.txt'
data_meta:
    - 'data/emore/meta.txt'

我想要复现learn to cluster face，请问用哪个模型训练好点 ？

和yl-1993/hfsoftmax （https://github.com/yl-1993/hfsoftmax）相比
Accelerated Training for Massive Classification via Dynamic Class Selection (HF-Softmax)

哪种训练模型好点