paul-pias / face-recognition

Face Recognition using pre-trained model built-on Arcface was implemented on Pytorch.

Face Recognition using ARCFACE-Pytorch

Introduction

This repo contains face_verify.py and app.py which is able to perform the following task -

• Detect faces from an image, video or in webcam and perform face recogntion.
• app.py was used to deploy the project.

Required Files

• requirements.txt
• pretrained model IR-SE50 @ Onedrive or Mobilefacenet @ OneDrive.
• Custom dataset
• Newly Trained model (facebank.pth and names.npy)

User Instruction

After downloading the project first you have to install the following libraries.

Installation

You can install all the dependencies at once by running the following command from your terminal.

    $ pip install -r requirements.txt

For the installation of torch using "pip" run the following command

    $ pip3 install torch===1.2.0 torchvision===0.4.0 -f https://download.pytorch.org/whl/torch_stable.html

Project Setup

pre-trained model

Although i provided the pretrained model in the work_space/model and work_space/save folder, if you want to download the models you can follow the following url:

• IR-SE50 @ BaiduNetdisk
• IR-SE50 @ Onedrive
• Mobilefacenet @ BaiduNetDisk
• Mobilefacenet @ OneDrive

I have used the IR-SE50 as the pretrained model to train with my custom dataset. You need to copy the pretrained model and save it under the work_space/save folder as model_final.pth

Newly trained model

In the data/facebank you will find a trained model named "facebank.pth" which contains the related weights and "names.npy" contains the corresponding labels of the users that are avialable in the facebank folder. For instance in this case the facebank folder will look like this :-

facebank/
            ---> Chandler
            ---> Joey
            ---> Monica
            ---> Phoebe
            ---> Pias
            ---> Rachel
            ---> Raihan
            ---> Ross
            ---> Samiur
            ---> Shakil
            ---> facebank.pth
            ---> names.npy

If you have the "facebank.pth" and "names.npy" files in the data/facebank you can execute the following command to see the demo.

    $ python app.py

and go to the following url from your web browser.

http://localhost:5000

---

Note: If you want to run the inference on a video, download a video of related persons (Person that you trained the model with) and replace 0 in the line number 43 of face_verify.py with the path of your video file. For this code you can run the inference on any video of Friends tv series.

---

Now if you want to train with your custom dataset, you need to follow the following steps.

Dataset preparation

First organize your images within the following manner-

data/
    raw/
         name1/
             photo1.jpg
             photo2.jpg
             ...
         name2/
             photo1.jpg
             photo2.jpg
             ...
         .....

now run the following command

$ python .\create-dataset\align_dataset_mtcnn.py data/raw/ data/processed --image_size 112

You will see a new folder inside the data directory named "processed" which will hold all the images that contains only faces of each user. If more than 1 image appears in any folder for a person, average embedding will be calculated.

After executing the script new images for each user in the processed folder will look something like this.

Cropped Images of faces

Copy all the folders of the users under the data/processed folder and paste in the data/facebank folder.

Now to train with your dataset, you need to set args.update == True in line 35 of face_verify.py . After training you will get a new facebank.pth and names.npy in your data/facebank folder which will now only holds the weights and labels of your newly trained dataset. Once the training is done you need to reset args.update==False. However, if this doesn't work change the code in following manner-

Old Code

    if args.update:
        targets, names = prepare_facebank(conf, learner.model, mtcnn, tta = args.tta)
        print('facebank updated')
    else:
        targets, names = load_facebank(conf)
        print('facebank loaded')

New Code

Only keep the follwing lines for training, once the training is done just replace it with the old code.

        targets, names = prepare_facebank(conf, learner.model, mtcnn, tta = args.tta)
        print('facebank updated')

Or you can simply pass a command line arguement such as below if there is new data to train.

   $python face_verify.py -u

Here the -u parse the command to update the facebank.pth and names.npy.

Now you are ready to test the systen with your newly trained users by running-

    $ python app.py

Note: You can train with custom dataset as many time as you want, you will only require any of the pre-trained model to train with your custom dataset to generate the facebank.pth and names.npy. Once you get this two files you are ready to test the face recogniton.

---

Retrain the pre-trained model

If you want to build a new pre-trained model like IR-SE50 @ Onedrive and reproduce the result, you will need the large files which contains several dataset of faces under the data/faces_emore .

To get these files, first you need to download the MS1M dataset from any of the following url-

• emore dataset @ Dropbox
• emore dataset @ BaiduDrive

After unzipping the downloaded file execute the following command. It will take few hours depending on your system configuration.

    $ python prepare_data.py

After that you will see the following files to the "data/faces_emore" folder.

faces_emore/
            ---> agedb_30
            ---> calfw
            ---> cfp_ff
            ---> cfp_fp
            ---> cfp_fp
            ---> cplfw
            ---> imgs
            ---> lfw
            ---> vgg2_fp

To know the further training procedure you can see the details in this InsightFace_Pytorch repository.

References

• Arcface
• InsightFace_Pytorch
• The one with Face Recognition.

face-recognition's Issues

IDE

Hi ...,
which IDE are you using?
Best regards,
PeterPham

Error downloading data/facebank/facebank.pth

Error downloading object: data/facebank/facebank.pth (12f2be5): Smudge error: Error downloading data/facebank/facebank.pth (12f2be55177b222cedc8853565bffddfa1343f5b64fd516ea4f87924210eeb7d): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.

Facing the following issue

I am facing the following error. I have tried to update the path of the facebank.pth in the utils.py file. Can you suggest any `solution??

arcface loaded
{'data_path': PosixPath('/home/intisar/Documents/arcface/Face-Recognition-master/data/facebank'), 'work_path': PosixPath('work_space'), 'model_path': PosixPath('work_space/models'), 'log_path': PosixPath('work_space/log'), 'save_path': PosixPath('work_space/save'), 'input_size': [112, 112], 'embedding_size': 512, 'use_mobilfacenet': False, 'facebank_path': PosixPath('/home/intisar/Documents/arcface/Face-Recognition-master/data/facebank/facebank'), 'net_depth': 50, 'drop_ratio': 0.6, 'net_mode': 'ir_se', 'device': device(type='cuda', index=0), 'test_transform': Compose(
ToTensor()
Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
), 'data_mode': 'emore', 'vgg_folder': PosixPath('/home/intisar/Documents/arcface/Face-Recognition-master/data/facebank/faces_vgg_112x112'), 'ms1m_folder': PosixPath('/home/intisar/Documents/arcface/Face-Recognition-master/data/facebank/faces_ms1m_112x112'), 'emore_folder': PosixPath('/home/intisar/Documents/arcface/Face-Recognition-master/data/facebank/faces_emore'), 'batch_size': 100, 'threshold': 1.5, 'face_limit': 10, 'min_face_size': 35}
ir_se_50 model opened
learner loaded
Traceback (most recent call last):
File "app.py", line 5, in
from face_verify import faceRec
File "/home/intisar/Documents/arcface/Face-Recognition-master/face_verify.py", line 39, in
targets, names = load_facebank(conf)
File "/home/intisar/Documents/arcface/Face-Recognition-master/utils.py", line 71, in load_facebank
embeddings = torch.load('/home/intisar/Documents/arcface/Face-Recognition-master/data/facebank/facebank.pth')
File "/home/intisar/miniconda3/envs/arcface/lib/python3.7/site-packages/torch/serialization.py", line 585, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/home/intisar/miniconda3/envs/arcface/lib/python3.7/site-packages/torch/serialization.py", line 755, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.

facebank.pth

This file is not available anymore. Could you please upload it somewhere?

Recognise unknown person

-Closed-

Cannot train a custom dataset on CPU

After following the steps as outlined while trying to train a custom dataset, I got the error shown below:

File "./create-dataset/align_dataset_mtcnn.py", line 160, in
main(parse_arguments(sys.argv[1:]))
File "./create-dataset/align_dataset_mtcnn.py", line 56, in main
pnet, rnet, onet = align.detect_face.create_mtcnn(sess, None)
File "/home/ezichi/pytorch_awesome/create-dataset/align/detect_face.py", line 283, in create_mtcnn
pnet.load(os.path.join(model_path, 'det1.npy'), sess)
File "/home/ezichi/pytorch_awesome/create-dataset/align/detect_face.py", line 85, in load
data_dict = np.load(data_path, encoding='latin1').item() #pylint: disable=no-member
File "/home/ezichi/pytorch_awesome/pytorch_awesome/lib/python3.8/site-packages/numpy/lib/npyio.py", line 430, in load
return format.read_array(fid, allow_pickle=allow_pickle,
File "/home/ezichi/pytorch_awesome/pytorch_awesome/lib/python3.8/site-packages/numpy/lib/format.py", line 742, in read_array
raise ValueError("Object arrays cannot be loaded when "
ValueError: Object arrays cannot be loaded when allow_pickle=False.

How can I resolve it?

the facebank.pth cannot be download

git clone then shows error: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.

Question about verification protocol

Hi. I am YJHong and thanks for sharing your great work!

My questions are:

• Why using l2 metric instead of cosine similarity when doing the verification ?
• When I use cosine similairty for verification using your pretrained model (IRSE50 / mobilefacenet both), its performance even worse than VGGFaceNet (trained by simple soft-max). Is this due to cosine similarity ?
• About preprocess: 1) bbox using face detector - 2) plus margin - 3) resize to 112,112 - -4) ToTensor (divide by 255) - 5) Normalize (0.5,0.5,0.5) -> -> this right ?

how to increase epochs ?

hi!

We are training about 4 million images in a huge dataset.
The default training value is insufficient for performance.
I want to add more epochs

thanks!

How to evaluate the trained model facebank.pth ?

I am successfully trained the model with my own dataset. now I want to evaluate it and find accuracy?