swghosh / deepface

Keras implementation of the renowned publication "DeepFace: Closing the Gap to Human-Level Performance in Face Verification" by Taigman et al. Pre-trained weights on VGGFace2 dataset.

DeepFace

Introduction

Open source implementation of the renowned publication titled "DeepFace: Closing the Gap to Human-Level Performance in Face Verification" by Yaniv Taigman, Ming Yang, Marc'Aurelio Ranzato, Lior Wolf published at Conference on Computer Vision and Pattern Recognition (CVPR) 2014.

Implementation of this paper have been done using Keras (tf.keras). This project and necessary research was supported by the TensorFlow Research Cloud (TFRC). GCP resources have been used to train a million-scale machine learning model using Cloud TPUs.

Requirements

• Python 3.5+
• tensorflow>=1.14.0
• matplotlib

$ pip install requirements.txt

or

$ python3 -m pip install requirements.txt

Installation

$ pip install git+https://github.com/swghosh/DeepFace.git

deepface's Issues

LFW accuracy

Hi,

You should evaluate DeepFace model on LFW data set and share its accuracy on read me page. The original DeepFace study got 97.35% accuracy but your model trained on different data set. It might have different accuracy level.

Similarly, David Sandberg reimplemented Google FaceNet model and evaluate his model on LFW data set as well. He got 0.99650+-0.00252 accuracy whereas original Google FaceNet study got 99.63%.

This will show how your model robust indeed.

Regards.

Face Preparation for Model Prediction

Hi,

How can I read/align/prepare face for making predict on trained model?
I made code like this:

    model = create_deepface()
    model.load_weights('VGGFace2_DeepFace_weights_val-0.9034.h5')
    img = load_img(path)
    img = img.resize((152,152))
    img_array = img_to_array(img)
    img_array = img_array.reshape((-1, 152, 152, 3))
    res = model.predict(img_array)

I get different results for every face belonging to the same person.

Thanks a lot for a help

Dataset Preparation and Training on GPU

CL_PATH = 'gs://bucket-name/VGGFace2_classlabels.txt'
DATASET_PATH = 'gs://bucket-name/VGGFace2_tfrecs'
TB_PATH = 'gs://bucket-name/vggface2_deepface_tensorboard'

What locations are above files available online ? I tried getting them from http://www.robots.ox.ac.uk/~vgg/data/vgg_face2/data_infor.html

but I am getting confused by your terms VGGFace2_classlabels, VGGFace2_tfrecs, vggface2_deepface_tensorboard. what do these terms mean ?

Can not load the finetuned model in keras

filename = r"Facebook_Deepface.h5"
model = create_deepface()
model = load_model(filename)

error:
ValueError: Unknown layer: Functional

Face Verification using Pre-Trained Model

My question is concerned about face verification testing after I have a model file.

I have downloaded VGGFace2 data set, and have manually added more images to its training set. The orignal dataset is of around 9,000 people. I have added more people and now I have 11,000 people.

I want to train this model of 11,000 people and use it for face verification .
I am referring to issue #1 and took this code

# required imports
from tensorflow.keras.preprocessing.image import load_img, img_to_array
from deepface import create_deepface
from deepface import Dataset

model = create_deepface()
model.load_weights('VGGFace2_DeepFace_weights_val-0.9034.h5')

# fine tune the deepface model for your dataset
# for fine tuning, remove last dense layer only; (as per paper)
# set all pretrained parameters to non trainable
# add a dense layer with dim = num_classes (your dataset)
# set dense layer to trainable and train the model

img = load_img(path)
img = img.resize((152,152))
img_array = img_to_array(img)
img_array = img_array.reshape((-1, 152, 152, 3))
res = model.predict(img_array)

Can this code be used for face verification in my case as I am adding more dataset to the training data itself? Also, how do I tell that 2 faces match, should I look for both of their res values?

Error in realtime recogniton

My python code

from deepface import DeepFace
DeepFace.stream("db", model_name = "DeepFace")

i got error when webcam recognize me

  File "1.py", line 2, in <module>
    DeepFace.stream("/home/user/LAVRENKOV/deepface/all/")
  File "/usr/local/lib/python3.8/dist-packages/deepface/DeepFace.py", line 837, in stream
    realtime.analysis(db_path, model_name, distance_metric, enable_face_analysis)
  File "/usr/local/lib/python3.8/dist-packages/deepface/commons/realtime.py", line 348, in analysis
    custom_face = functions.detectFace(custom_face, (input_shape_y, input_shape_x))
UnboundLocalError: local variable 'input_shape_y' referenced before assignment

Problem with TFRecords Preparation

Error occurs in for shard_id, (image, label) in enumerate(dataset): I checked dataset length and output which is not empty. please help me
Thank you

--train2
     --n000002
        --0001_0.jpg
        --0001_02.jpg
        --0001_03.jpg
     --n000003
     --n000004

dataset_path = 'C:/Users/Kmu/PycharmProjects/train2'
tfrecs_path = 'C:/Users/Kmu/PycharmProjects/dosyalar/tfrecords'
shard_size = 12000

tensorflow version 1.14.0 and 1.15.0 tested same result

ERROR
2020-01-08 09:29:18.958101: W tensorflow/core/framework/op_kernel.cc:1502] OP_REQUIRES failed at strided_slice_op.cc:107 : Invalid argument: slice index -1 of dimension 0 out of bounds.
2020-01-08 09:29:18.958102: W tensorflow/core/framework/op_kernel.cc:1502] OP_REQUIRES failed at strided_slice_op.cc:107 : Invalid argument: slice index -1 of dimension 0 out of bounds.
2020-01-08 09:29:18.958102: W tensorflow/core/framework/op_kernel.cc:1502] OP_REQUIRES failed at strided_slice_op.cc:107 : Invalid argument: slice index -1 of dimension 0 out of bounds.
2020-01-08 09:29:18.958109: W tensorflow/core/framework/op_kernel.cc:1502] OP_REQUIRES failed at strided_slice_op.cc:107 : Invalid argument: slice index -1 of dimension 0 out of bounds.
2020-01-08 09:29:18.958112: W tensorflow/core/framework/op_kernel.cc:1502] OP_REQUIRES failed at strided_slice_op.cc:107 : Invalid argument: slice index -1 of dimension 0 out of bounds.
2020-01-08 09:29:18.958114: W tensorflow/core/framework/op_kernel.cc:1502] OP_REQUIRES failed at strided_slice_op.cc:107 : Invalid argument: slice index -1 of dimension 0 out of bounds.
2020-01-08 09:29:18.958146: W tensorflow/core/framework/op_kernel.cc:1502] OP_REQUIRES failed at strided_slice_op.cc:107 : Invalid argument: slice index -1 of dimension 0 out of bounds.
2020-01-08 09:29:18.959258: W tensorflow/core/framework/op_kernel.cc:1502] OP_REQUIRES failed at strided_slice_op.cc:107 : Invalid argument: slice index -1 of dimension 0 out of bounds.
2020-01-08 09:29:18.985959: W tensorflow/core/framework/op_kernel.cc:1502] OP_REQUIRES failed at iterator_ops.cc:1037 : Invalid argument: slice index -1 of dimension 0 out of bounds.
	 [[{{node strided_slice}}]]
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "C:\Users\Kmu\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "C:\Users\Kmu\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 44, in mapstar
    return list(map(*args))
  File "C:\Users\Kmu\PycharmProjects\deneme1\tfrecords-faster\tfrecsfaster\prepare_tfrecords2.py", line 73, in task
    for shard_id, (image, label) in enumerate(dataset):
  File "C:\Users\Kmu\PycharmProjects\deneme1\venv\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 586, in __next__
    return self.next()
  File "C:\Users\Kmu\PycharmProjects\deneme1\venv\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 623, in next
    return self._next_internal()
  File "C:\Users\Kmu\PycharmProjects\deneme1\venv\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 615, in _next_internal
    output_shapes=self._flat_output_shapes)
  File "C:\Users\Kmu\PycharmProjects\deneme1\venv\lib\site-packages\tensorflow\python\ops\gen_dataset_ops.py", line 2150, in iterator_get_next_sync
    _six.raise_from(_core._status_to_exception(e.code, message), None)
  File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: slice index -1 of dimension 0 out of bounds.
	 [[{{node strided_slice}}]] [Op:IteratorGetNextSync]
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:/Users/Kmu/PycharmProjects/deneme1/tfrecords-faster/tfrecsfaster/prepare_tfrecords2.py", line 92, in <module>
    pool.map(task, inputs)
  File "C:\Users\Kmu\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 266, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "C:\Users\Kmu\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 644, in get
    raise self._value
tensorflow.python.framework.errors_impl.InvalidArgumentError: slice index -1 of dimension 0 out of bounds.
	 [[{{node strided_slice}}]] [Op:IteratorGetNextSync]

Process finished with exit code 1

/////////////////////////////////////////////////////////////////////////

All code

#!/usr/bin/env python3

import os

WORKERS = os.cpu_count()
print("Using %d workers." % WORKERS)

dataset_path = 'C:/Users/Kmu/PycharmProjects/train2'
tfrecs_path = 'C:/Users/Kmu/PycharmProjects/dosyalar/tfrecords'
shard_size = 12000

n_samples = None


def tensorflow():
    import tensorflow as tf
    tf.enable_eager_execution()
    return tf


def init_task():
    print('[init_task] Started...')
    tf = tensorflow()
    image_paths = tf.io.gfile.glob(dataset_path + '/*/*.jpg')
    batch_size = len(image_paths) // WORKERS
    if len(image_paths) % WORKERS != 0:
        batch_size += 1
    image_paths_batched = [
        image_paths[index * batch_size: (index + 1) * batch_size]
        for index in range(WORKERS)
    ]
    global n_samples
    n_samples = len(image_paths)
    print('[init_task] Exiting...')
    return image_paths_batched


def task(inputs):
    worker_id, filenames = inputs

    def print_info(text):
        print('[Worker %d] %s' % (worker_id, text))

    print_info('Started...')

    tf = tensorflow()

    def wrap_bytes(list_of_bytestrings):
        return tf.train.Feature(bytes_list=tf.train.BytesList(value=list_of_bytestrings))

    def get_image_and_class(image_path):
        class_label = tf.strings.split(tf.expand_dims(image_path, axis=-1), sep='/').values[-2]
        img = tf.io.read_file(image_path)
        return img, class_label

    def to_tfrecord(image, class_name):
        feature = {
            'image': wrap_bytes([image]),
            'class': wrap_bytes([class_name])
        }
        return tf.train.Example(features=tf.train.Features(feature=feature))

    filenames = tf.convert_to_tensor(filenames)
    dataset = tf.data.Dataset.from_tensor_slices(filenames)
    dataset = dataset.shuffle(n_samples // WORKERS)
    dataset = dataset.map(get_image_and_class, tf.data.experimental.AUTOTUNE)
    dataset = dataset.batch(shard_size)

    for shard_id, (image, label) in enumerate(dataset):
        print_info('Working on shard %d' % shard_id)
        image, label = image.numpy(), label.numpy()
        current_shard_limit = image.shape[0]
        filename = tfrecs_path + '/worker{}-{:06d}-l{}.tfrec'.format(worker_id, shard_id, current_shard_limit)
        with tf.io.TFRecordWriter(filename) as out_file:
            for i in range(current_shard_limit):
                record = to_tfrecord(image[i], label[i])
                out_file.write(record.SerializeToString())
        print_info('Wrote file {} containing {} records'.format(filename, current_shard_limit))


image_paths_batched = init_task()
inputs = [(index, batch) for index, batch in enumerate(image_paths_batched)]

import multiprocessing

if __name__ == '__main__':
    with multiprocessing.Pool(WORKERS) as pool:
        pool.map(task, inputs)

    print('All workers exited')

Weights are not of the same shape, "Shapes are incompatible" error

#load the weights
model.load_weights('/content/VGGFace2_DeepFace_weights_val-0.9034.h5')

I get the following error message, when I run the code:
```python
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-9-e3e69a625b6a> in <module>()
     41 
     42 #load the weights
---> 43 model.load_weights('/content/VGGFace2_DeepFace_weights_val-0.9034.h5')
     44 
     45 

5 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/tensor_shape.py in assert_is_compatible_with(self, other)
   1115     """
   1116     if not self.is_compatible_with(other):
-> 1117       raise ValueError("Shapes %s and %s are incompatible" % (self, other))
   1118 
   1119   def most_specific_compatible_shape(self, other):

ValueError: Shapes (2401, 784, 16) and (625, 784, 16) are incompatible

Masked face

Hello really build version with face recognition with mask?

Triplet Loss

Using accuracy metric and not using the triplet loss for learning the weights ??