littletomatodonkey / insight-face-paddle

End-to-end face detection and recognition system using PaddlePaddle.

简体中文 | English

---

InsightFace Paddle

1. Introduction

1.1 Overview

InsightFacePaddle is an open source deep face detection and recognition toolkit, powered by PaddlePaddle. InsightFacePaddle provide three related pretrained models now, include BlazeFace for face detection, ArcFace and MobileFace for face recognition.

• This tutorial is mainly about Whl package inference using PaddleInfernence.
• For face recognition task, please refer to: Face recognition tuturial.
• For face detection task, please refer to: Face detection tuturial.

1.2 Benchmark

For face detection task, on WiderFace dataset, the following table shows mAP, speed and time cost for BlazeFace.

Model structure	Model size	WiderFace mAP	CPU time cost	GPU time cost
BlazeFace-FPN-SSH-Paddle	0.65MB	0.9187/0.8979/0.8168	31.7ms	5.6ms
RetinaFace	1.68MB	-/-/0.825	182.0ms	17.4ms

For face recognition task, on MSAM dataset, the following table shows precision, speed and time cost for MobileFaceNet.

Model structure	lfw	cfp_fp	agedb30	CPU time cost	GPU time cost
MobileFaceNet-Paddle	0.9945	0.9343	0.9613	4.3ms	2.3ms
MobileFaceNet-mxnet	0.9950	0.8894	0.9591	7.3ms	4.7ms

Benchmark environment:

• CPU: Intel(R) Xeon(R) Gold 6184 CPU @ 2.40GHz
• GPU: a single NVIDIA Tesla V100

Note: Performance of RetinaFace is tested using script test.py. The image shape is modified to 640x480 here. Performance of MobileFaceNet-mxnet is tested using script verification.py.

1.3 Visualization

One example result predicted by InsightFacePaddle is as follow. Please refer to the Demo for more.

1.4 Community

Scan the QR code below with your QQ (QQ group number: 705899115) to discuss more about deep learning together.

2. Installation

1. Install PaddlePaddle

PaddlePaddle 2.1 or later is required for InsightFacePaddle. You can use the following steps to install PaddlePaddle.

# for GPU
pip3 install paddlepaddle-gpu

# for CPU
pip3 install paddlepaddle

For more details about installation. please refer to PaddlePaddle.

2. Install requirements

InsightFacePaddle dependencies are listed in requirements.txt, you can use the following command to install the dependencies.

pip3 install --upgrade -r requirements.txt -i https://mirror.baidu.com/pypi/simple

3. Install InsightFacePaddle

• [Recommanded] You can use pip to install the lastest version InsightFacePaddle from pypi.

pip3 install --upgrade insightface-paddle

• You can also build whl package and install by following commands.

cd ./InsightFacePaddle
python3 setup.py bdist_wheel
pip3 install dist/*

3. Quick Start

InsightFacePaddle support two ways of use, including Commad Line and Python API.

3.1 Command Line

You can use InsightFacePaddle in Command Line.

3.1.1 Get help

You can get the help about InsightFacePaddle by following command.

insightfacepaddle -h

The args are as follows:

args	type	default	help
det_model	str	BlazeFace	The detection model.
rec_model	str	MobileFace	The recognition model.
use_gpu	bool	True	Whether use GPU to predict. Default by True.
enable_mkldnn	bool	False	Whether use MKLDNN to predict, valid only when --use_gpu is False. Default by False.
cpu_threads	int	1	The num of threads with CPU, valid only when --use_gpu is False and --enable_mkldnn is True. Default by 1.
input	str	-	The path of video to be predicted. Or the path or directory of image file(s) to be predicted.
output	str	-	The directory to save prediction result.
det	bool	False	Whether to detect.
det_thresh	float	0.8	The threshold of detection postprocess. Default by 0.8.
rec	bool	False	Whether to recognize.
index	str	-	The path of index file.
cdd_num	int	5	The number of candidates in the recognition retrieval. Default by 5.
rec_thresh	float	0.45	The threshold of match in recognition, use to remove candidates with low similarity. Default by 0.45.
max_batch_size	int	1	The maxium of batch_size to recognize. Default by 1.
build_index	str	-	The path of index to be build.
img_dir	str	-	The img(s) dir used to build index.
label	str	-	The label file path used to build index.

3.1.2 Build index

If use recognition, before start predicting, you have to build the index.

insightfacepaddle --build_index ./demo/friends/index.bin --img_dir ./demo/friends/gallery --label ./demo/friends/gallery/label.txt

An example used to build index is as follows:

3.1.3 Predict

1. Detection only

• Image(s)

Use the image below to predict:

The prediction command:

insightfacepaddle --det --input ./demo/friends/query/friends1.jpg --output ./output

The result is under the directory ./output:

• Video

insightfacepaddle --det --input ./demo/friends/query/friends.mp4 --output ./output

2. Recognition only

• Image(s)

Use the image below to predict:

The prediction command:

insightfacepaddle --rec --index ./demo/friends/index.bin --input ./demo/friends/query/Rachel.png

The result is output in the terminal:

INFO:root:File: Rachel., predict label(s): ['Rachel']

3. Detection and recognition

• Image(s)

Use the image below to predict:

The prediction command:

insightfacepaddle --det --rec --index ./demo/friends/index.bin --input ./demo/friends/query/friends2.jpg --output ./output

The result is under the directory ./output:

• Video

insightfacepaddle --det --rec --index ./demo/friends/index.bin --input ./demo/friends/query/friends.mp4 --output ./output

3.2 Python

You can use InsightFacePaddle in Python. First, import InsightFacePaddle and logging because InsightFacePaddle using that to control log.

import insightface_paddle as face
import logging
logging.basicConfig(level=logging.INFO)

3.2.1 Get help

parser = face.parser()
help_info = parser.print_help()
print(help_info)

3.2.2 Building index

parser = face.parser()
args = parser.parse_args()
args.build_index = "./demo/friends/index.bin"
args.img_dir = "./demo/friends/gallery"
args.label = "./demo/friends/gallery/label.txt"
predictor = face.InsightFace(args)
predictor.build_index()

3.2.3 Prediction

1. Detection only

• Image(s)

parser = face.parser()
args = parser.parse_args()

args.det = True
args.output = "./output"
input_path = "./demo/friends/query/friends1.jpg"

predictor = face.InsightFace(args)
res = predictor.predict(input_path)
print(next(res))

• NumPy

import cv2

parser = face.parser()
args = parser.parse_args()

args.det = True
args.output = "./output"
path = "./demo/friends/query/friends1.jpg"
img = cv2.imread(path)[:, :, ::-1]

predictor = face.InsightFace(args)
res = predictor.predict(img)
print(next(res))

The prediction result saved as "./output/tmp.png".

• Video

parser = face.parser()
args = parser.parse_args()

args.det = True
args.output = "./output"
input_path = "./demo/friends/query/friends.mp4"

predictor = face.InsightFace(args)
res = predictor.predict(input_path)
for _ in res:
    print(_)

2. Recognition only

• Image(s)

parser = face.parser()
args = parser.parse_args()

args.rec = True
args.index = "./demo/friends/index.bin"
input_path = "./demo/friends/query/Rachel.png"

predictor = face.InsightFace(args)
res = predictor.predict(input_path, print_info=True)
next(res)

• NumPy

import cv2

parser = face.parser()
args = parser.parse_args()

args.rec = True
args.index = "./demo/friends/index.bin"
path = "./demo/friends/query/Rachel.png"
img = cv2.imread(path)[:, :, ::-1]

predictor = face.InsightFace(args)
res = predictor.predict(img, print_info=True)
next(res)

3. Detection and recognition

• Image(s)

parser = face.parser()
args = parser.parse_args()

args.det = True
args.rec = True
args.index = "./demo/friends/index.bin"
args.output = "./output"
input_path = "./demo/friends/query/friends2.jpg"

predictor = face.InsightFace(args)
res = predictor.predict(input_path, print_info=True)
next(res)

• NumPy

import cv2

parser = face.parser()
args = parser.parse_args()

args.det = True
args.rec = True
args.index = "./demo/friends/index.bin"
args.output = "./output"
path = "./demo/friends/query/friends2.jpg"
img = cv2.imread(path)[:, :, ::-1]

predictor = face.InsightFace(args)
res = predictor.predict(img, print_info=True)
next(res)

The prediction result saved as "./output/tmp.png".

• Video

parser = face.parser()
args = parser.parse_args()

args.det = True
args.rec = True
args.index = "./demo/friends/index.bin"
args.output = "./output"
input_path = "./demo/friends/query/friends.mp4"

predictor = face.InsightFace(args)
res = predictor.predict(input_path, print_info=True)
for _ in res:
    pass

insight-face-paddle's Issues

video demo

Hi,

Video demo is not accepting video files and asking images.

Error with Build Index

hi author,
i have some problem when build the index for image. After i run this:
!python insightface_paddle.py --build_index ./demo/friends/index.bin --img_dir ./demo/friends/gallery/ --label ./demo/friends/gallery/label.txt

And have this error

So i found the weird code like this, may be it caused the issue. So i changed args.rec = False but i have another issue

Hope to see your reply soon !

求助下,项目在paddle ai studio中运行报错

执行这段代码报错:
import insightface
from insightface.app import FaceAnalysis
from insightface.data import get_image as ins_get_image
app = FaceAnalysis()
app.prepare(ctx_id=0, det_size=(640, 640))
报错提示:

insightface_paddle cannot resume training while loading checkpoint...

Training: 2021-11-02 19:58:58,427 - Load checkpoint from '/home/bbs/Datasets/kjj/insightface/recognition/arcface_paddle/MS1M_v2_arcface_MobileFaceNet_128_0.1/MobileFaceNet_128/120'.
Traceback (most recent call last):
File "tools/train.py", line 35, in
train(args)
File "/home/bbs/Datasets/kjj/insightface/recognition/arcface_paddle/dynamic/train.py", line 168, in train
backbone, classifier, optimizer, for_train=True)
File "/home/bbs/Datasets/kjj/insightface/recognition/arcface_paddle/dynamic/utils/io.py", line 229, in load
classifier.state_dict(), dist_param_state_dict)
File "/home/bbs/Datasets/kjj/insightface/recognition/arcface_paddle/dynamic/utils/io.py", line 220, in map_actual_param_name
state_dict[name] = load_state_dict[param.name]
KeyError: 'dist@fc@rank@00000'

使用AcrFace进行人脸识别出现ValueErrorr

代码如下：

`import insightface_paddle as face
import logging

logging.basicConfig(level=logging.INFO)

parser = face.parser()
args = parser.parse_args()
args.det = True
args.rec = True
args.rec_model = 'ArcFace' # 这里使用ArcFace
args.index = './demo/friends/index.bin'
args.output = './output'
input_path = './demo/friends/query/friends2.jpg'
predictor = face.InsightFace(args=args)
res = predictor.predict(input_path, print_info=True)
print(next(res))`

报错信息如下：
Traceback (most recent call last):
File "D:/PaddleRepo/insight-face-paddle/insightface_paddle_demo.py", line 18, in
print(next(res))
File "D:\PaddleRepo\insight-face-paddle\insightface_paddle.py", line 759, in predict
labels = self.rec_predictor.retrieval(np_feature)
File "D:\PaddleRepo\insight-face-paddle\insightface_paddle.py", line 555, in retrieval
feature).squeeze()
File "D:\PaddleRepo\venv\lib\site-packages\sklearn\metrics\pairwise.py", line 1179, in cosine_similarity
X, Y = check_pairwise_arrays(X, Y)
File "D:\PaddleRepo\venv\lib\site-packages\sklearn\utils\validation.py", line 72, in inner_f
return f(**kwargs)
File "D:\PaddleRepo\venv\lib\site-packages\sklearn\metrics\pairwise.py", line 161, in check_pairwise_arrays
X.shape[1], Y.shape[1]))
ValueError: Incompatible dimension for X and Y matrices: X.shape[1] == 128 while Y.shape[1] == 512

使用自己的数据集 如何生成index.bin

在进行人脸识别的时候，使用预训练模型，如果有index.bin我看就可以直接进行人脸识别了。请问这个文件如何生成？

在人脸识别的时候，怎么不需要人脸对齐

我在源码中并没有看到人脸对齐，这个识别不需要人脸对齐吗？

Performance Issue

Hi I am testing the inference with:
RTX 3090
Cuda 11.2
paddlepaddle-gpu 2.1
Cudnn 8
And the performance for BlazeFace Detection is 20ms only. Is this normal?

使用fask 包装成api后docker运行发生异常

大佬好，我想使用这个库包装成api，本地开发运行是正常的，但是打进docker之后发生了一场

调用insightface_paddle 的时候发生了异常

为啥这写参数会传递到gunicorn了

insight-face-paddle针对不在索引库中的人没有识别出人脸

执行以下命令

insightfacepaddle --det --rec --index ./demo/friends/index.bin --input ./demo/friends/query/tmp.jpg --output ./output

我想达到效果：没有在索引库中的人被识别出人脸并锚框并提示未知，在索引库中的人被识别出人脸并锚框并提示人名，请问这个能做到吗，如何做呢

Low accuracy in recognition models

Hello,
I am facing very low accuracy in both recognition models (ArcFace and MobileFace), no matter how i play with the input params (such as Rec_thresh), i am not getting good results at all.
what could be wrong?

摄像头拍摄不同角度自己脸部face_recognition标签结果把老友记的大部分人都识别出来，错误率非常高

face_recognition识别错误率很高，（摄像头cv读出bgr已转rgb_frame = frame[:, :, ::-1]）
insight-face识别率应该很高才对啊，我要如何提高识别正确率呢

缺少未设定参数时的判断

insight-face-paddle/insightface_paddle.py

Line 631 in 37f4b7d

if not (args.build_index or os.path.isfile(args.index)):

当index未指定时，args.index为None，直接执行os.path.isfile(args.index)会触发异常：

Traceback (most recent call last):
File "/home/mikeshi/test/face_recog_paddle/venv/bin/insightfacepaddle", line 8, in
sys.exit(main())
File "/home/mikeshi/test/face_recog_paddle/venv/lib/python3.7/site-packages/insightface_paddle/insightface_paddle.py", line 786, in main
predictor = InsightFace(args)
File "/home/mikeshi/test/face_recog_paddle/venv/lib/python3.7/site-packages/insightface_paddle/insightface_paddle.py", line 631, in init
if not (args.build_index or os.path.isfile(args.index)):
File "/usr/lib/python3.7/genericpath.py", line 30, in isfile
st = os.stat(path)
TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType

推理报错

`-------------------------------------------------------------------------------------

                                  PaddleFace                                     

+----------------+------------------------------------------------------------------+

| Param | Value |

+----------------+------------------------------------------------------------------+

| det_model | BlazeFace |

| rec_model | MobileFace |

| use_gpu | True |

| enable_mkldnn | False |

| cpu_threads | 1 |

| input | None |

| output | /home/nvidia/insight-face-paddle-main/output |

| det | True |

| det_thresh | 0.8 |

| rec | True |

| index | /home/nvidia/insight-face-paddle-main/demo/predixr_img/index.bin |

| cdd_num | 5 |

| rec_thresh | 0.45 |

| max_batch_size | 1 |

| build_index | None |

| img_dir | None |

| label | None |

+----------------+------------------------------------------------------------------+

                           Powered by PaddlePaddle!                              

---

WARNING:root:The directory of input contine directory or not supported file type, only support: {'jpg', 'tif', 'bmp', 'jpeg', 'rgb', 'png', 'tiff'}

Traceback (most recent call last):

File "pre.py", line 16, in

print(next(res))

File "/home/nvidia/insight-face-paddle-main/insightface_paddle.py", line 759, in predict

labels = self.rec_predictor.retrieval(np_feature)

File "/home/nvidia/insight-face-paddle-main/insightface_paddle.py", line 558, in retrieval

-self.cdd_num)[-self.cdd_num:]

File "<array_function internals>", line 6, in argpartition

File "/home/nvidia/.local/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 832, in argpartition

return _wrapfunc(a, 'argpartition', kth, axis=axis, kind=kind, order=order)

File "/home/nvidia/.local/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 58, in _wrapfunc

return bound(*args, **kwds)

ValueError: kth(=-2) out of bounds (3)

`
demo测试没有问题，换成自己的数据之后似乎建立索引以后那个索引会报错，请问这个问题如何解决？
另外，请问建立索引的最少图片张数（每个人）是多少？

试用下来的体验

首先，Blazeface的人脸检测，对于头像占比比较大的图片，效果很差，但是对于占比小的，则效果很好。如下图的汤姆汉克斯，识别的置信度只有0.25左右

但是这样的集体照，则都在0.99左右

另外，检测和识别的过程，看deepsight的Insightface项目，应该还缺少了Alignment这一步骤，所以当脸不是正对镜头时，计算获得的特征值的余弦相似度，会很差，如和下面的汤姆汉克斯图像对照，只有0.0X的相似度。但是这个不确定，因为InsightFace里用到的人脸检测模型应该是2d106det.onnx，模型本身就不相同。

而使用insightface时，两张都相似度在0.5左右。人脸识别都是用了ArcFace模型。

请教一下数据集制作

大佬 请教一下这个数据集制作有没有案例啥的，求分享一个，谢谢！