shobhit9618 / celeb_recognition

Model to recognize celebrities using a face matching algorithm.

Celebrity Recognition

Model to recognize celebrities using a face matching algorithm.

Refer this for detailed documentation.

You can also read my article on medium here.

Basic working of the algorithm includes the following:

• Face detection is done using MTCNN face detection model.

• Face encodings are created using VGGFace model in keras.

• Face matching is done using annoy library (spotify).

Installing dependencies

• Run pip install -r requirements.txt to install all the dependencies (preferably in a virtual environment).

PyPI package

Installation

• To ensure you have all the required additional packages, run pip install -r requirements.txt first.
• To install pip package, run:

  # pip release version
  pip install celeb-detector
  # also install additional dependencies with this (if not installed via requirements.txt file)
  pip install annoy keras-vggface keras-applications
  # Directly from repo
  pip install git+https://github.com/shobhit9618/celeb_recognition.git

• If you are using conda on linux or ubuntu, you can use the following commands to create and use a new environment called celeb-detector:

  conda env create shobhit9618/celeb-detector
  conda activate celeb-detector

  This will install all the required dependencies. To ensure you are using the latest version of the package, also run (inside the environment):

  pip install --upgrade celeb-detector

Using pip pakcage

• For using my model for predictions, use the following lines of code after installation:

  import celeb_detector # on running for the first time, this will download vggface model
  img_path = 'sample_image.jpg' # this supports both local path and web url like https://sample/sample_image_url.jpg
  celeb_detector.celeb_recognition(img_path) # on running for the first time, 2 files (celeb_mapping.json and celeb_index_60.ann) will downloaded to your home directory

  This returns a list of dictionaries, each dictionary contains bbox coordinates, celeb name and confidence for each face detected in the image (celeb name will be unknown if no matching face detected).

• For using your own custom model, also provide path to json and ann files as shown below:

  import celeb_detector
  img_path = 'sample_image.jpg'
  ann_path = 'sample_index.ann'
  celeb_map = 'sample_mapping.json'
  celeb_detector.celeb_recognition(img_path, ann_path, celeb_map)

• For creating your own model (refer this for more details on usage) and run as follows:

  import celeb_detector
  folder_path = 'celeb_images'
  celeb_detector.create_celeb_model(folder_path)

Create your own celeb model

• Create a dataset of celebs in the following directory structure:

  celeb_images/
      celeb-a/
          celeb-a_1.jpg
          celeb-a_2.jpg
          ...
      celeb-b/
          celeb-b_1.jpg
          celeb-b_1.jpg
          ...
      ...

• Each folder name will be considered as the corresponding celeb name for the model (WARNING: Do not provide any special characters or spaces in the names).
• Make sure each image has only 1 face (of the desired celebrity), if there are multiple faces, only the first detected face will be considered.
• Provide path to the dataset folder (for example, celeb_images folder) in the create_celeb_model.py file.
• Run create_celeb_model.py file.
• Upon successful completion of the code, we get celeb_mapping.json (for storing indexes vs celeb names), celeb_index.ann (ann file for searching encodings) and celeb_name_encoding.pkl files (for storing encodings vs indexes for each celeb). (WARNING: You need to provide paths for storing each of these files, default is to store in the current directory)

Model predictions in jupyter

• Provide paths to celeb_mapping.json and celeb_index.ann files in celeb_recognition.ipynb file. If you want to try my model, ignore this step.
• Run all the cells in the celeb_recognition.ipynb file, the final cell will provide widgets for uploading images and making predictions (this will also download the necessary model files).
• NOTE: celeb_recognition.ipynb is a standalone file and does not require any other files from the repo for running.

Model predictions in python

• Provide paths to celeb_mapping.json and celeb_index.ann files in celeb_recognition.py and celeb_utils.py files. If you want to try my model, ignore this step.
• Run celeb_recognition.py file, provide path to image in the file.
• Output includes a list of the identified faces, bounding boxes and the predicted celeb name (unknown if not found).
• It also displays the output with bounding boxes.

Sample image output

Binder

You can run a binder application by clicking the following link:

You can also launch a voila binder application (which only has widgets for image upload and celeb prediction) by clicking here.

Google Colab

To open and run celeb_recognition.ipynb file in google colab, click the following link:

celeb_recognition's Issues

Google Collab error

Hello,
I have a hard time trying to make the Google Colab works.

encoder_model = torch.load(vggface_modelpath)

Give this error:

FileNotFoundError                         Traceback (most recent call last)
[<ipython-input-6-f7e8409fb9ee>](https://localhost:8080/#) in <cell line: 1>()
----> 1 encoder_model = torch.load(vggface_modelpath)
      2 ann_index = AnnoyIndex(2048, 'angular')
      3 _ = ann_index.load(celeb_index_annpath)

2 frames
[/usr/local/lib/python3.10/dist-packages/torch/serialization.py](https://localhost:8080/#) in __init__(self, name, mode)
    424 class _open_file(_opener):
    425     def __init__(self, name, mode):
--> 426         super().__init__(open(name, mode))
    427 
    428     def __exit__(self, *args):

FileNotFoundError: [Errno 2] No such file or directory: 'celeb_detector/models/vggface_resnet50.pt'

I can't find this file anywhere on the the repo. I need to download it somewhere?

Thanks!

OSError: Index size is not a multiple of vector size. Ensure you are opening using the same metric you used to create the index.: No error (0)

Index size is not a multiple of vector size. Ensure you are opening using the same metric you used to create the index.: No error (0)
Traceback (most recent call last):
File "C:\CelebDetector\test.py", line 4, in
celeb_detector.celeb_recognition(img_path)
File "C:\CelebDetector\venv\lib\site-packages\celeb_detector\celeb_recognition.py", line 37, in celeb_recognition
pred, img_out = get_celeb_prediction(img, ann_filepath, celeb_mapping_path)
File "C:\CelebDetector\venv\lib\site-packages\celeb_detector\celeb_utils.py", line 85, in get_celeb_prediction
_ = ann_index.load(ann_filepath)
OSError: Index size is not a multiple of vector size. Ensure you are opening using the same metric you used to create the index.: No error (0)