nghorbani / homogenus

Human Image Gender Classifier for Expressive Body Capture

Human Image Gender Classifier

This is the official repository of the Human Gender Classifier (Homogenus) used in the paper

Expressive Body Capture: 3D Hands, Face, and Body from a Single Image.

Numerous methods have classified human gender using face images. In full-body images of a person, the face may be small or not visible. Here we trained a classifier to analyze gender based on full-body images. The input to the classifier its a full-body human image and the 2D key points detected by OpenPose. We use information about gender in our SMPLify-X code to select the appropriate 3D body model. Note that gender is non-binary — SMPLify-X has male, female, and neutral body models. When the classifier is certain, we use the corresponding gendered model. When it is uncertain, we use the neutral one. This codebase consists of the inference code.

For further details on the method please refer to the following publication,

Expressive Body Capture: 3D Hands, Face, and Body from a Single Image
G. Pavlakos*, V. Choutas*, N. Ghorbani, T. Bolkart, A. A. A. Osman, D. Tzionas and M. J. Black 
Computer Vision and Pattern Recognition (CVPR) 2019, Long Beach, CA

A pdf preprint is also available on the project page.

Installation

Requirements

• Python 3.7
• TensorFlow 1.13.1
• Configer

Tested with CUDA-10.0 and cuDNN-7.5 running on Ubuntu 18.04.

Install from this repository for the latest developments:

pip install git+https://github.com/nghorbani/homogenus

Download models

• Download pretrained homogenus weights from the project website, downloads page. Uncompress the weights in a folder, e.g. by default homogenus/trained_models/tf , and use the directory path in the following commands.

Run Homogenus

After installation and obtaining the weights for the model, inside the virtual environment, you should be able to run the homogenus_infer command to get the help:

python3 -m homogenus.tf.homogenus_infer -h

usage: homogenus_infer.py [-h] [-tm TRAINED_MODEL_DIR] -ii IMAGES_INDIR -oi
                          OPENPOSE_INDIR [-io IMAGES_OUTDIR]
                          [-oo OPENPOSE_OUTDIR]

optional arguments:
  -h, --help            show this help message and exit
  -tm TRAINED_MODEL_DIR, --trained_model_dir TRAINED_MODEL_DIR
                        The path to the directory holding homogenus trained
                        models in TF.
  -ii IMAGES_INDIR, --images_indir IMAGES_INDIR
                        Directory of the input images.
  -oi OPENPOSE_INDIR, --openpose_indir OPENPOSE_INDIR
                        Directory of openpose keypoints, e.g. json files.
  -io IMAGES_OUTDIR, --images_outdir IMAGES_OUTDIR
                        Directory to put predicted gender overlays. If not
                        given, wont produce any overlays.
  -oo OPENPOSE_OUTDIR, --openpose_outdir OPENPOSE_OUTDIR
                        Directory to put the openpose gendered keypoints. If
                        not given, it will augment the original openpose json
                        files.

As an example, inside the homogenus folder you can run the following to get the gender predictions for the sample images and their corresponding openpose keypoints. This command will print the predictions as well as creating new gendered openpose keypoints and gender overlayed images.

python3 -m homogenus.tf.homogenus_infer -ii ./samples/images/ -io ./samples/images_gendered/ -oi ./samples/openpose_keypoints/ -oo ./samples/openpose_keypoints_gendered/

The headline image of this repository shows all these samples from the LSP in-the-wild image dataset with overlayed genders for the OpenPose detected full-body humans.

License

Free for non-commercial and scientific research purposes. By using this code, you acknowledge that you have read the license terms (https://smpl-x.is.tue.mpg.de/license), understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not use the code. For commercial use please check the website (https://smpl-x.is.tue.mpg.de/license).

Referencing Homogenus

Please cite the following paper if you use this code directly or indirectly in your research/projects.

@inproceedings{SMPL-X:2019,
  title = {Expressive Body Capture: 3D Hands, Face, and Body from a Single Image},
  author = {Pavlakos, Georgios and Choutas, Vasileios and Ghorbani, Nima and Bolkart, Timo and Osman, Ahmed A. A. and Tzionas, Dimitrios and Black, Michael J.},
  booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
  year = {2019}
}

Contact

The code in this repository is developed by Nima Ghorbani.

If you have any questions you can contact us at [email protected].

For commercial licensing, contact [email protected]

Acknowledgment

We thank Pavel Karasik for helping in the dataset gathering and processing stage.

homogenus's Issues

How to test my own image?

In image folders i want to add my own image to test my body. How can i train my own dataset?

pretrained weights

i followed all the instructions but got an error that i could not resolve
File "/home/wise/testing/homogenus/homogenus/tf/homogenus_infer.py", line 174, in
hg = Homogenus_infer(trained_model_dir=ps.trained_model_dir)
File "/home/wise/testing/homogenus/homogenus/tf/homogenus_infer.py", line 47, in init
self.model = tf.keras.models.load_model(self.best_model_fname)
File "/home/wise/testing/myenv/lib/python3.10/site-packages/keras/src/saving/saving_api.py", line 262, in load_model
return legacy_sm_saving_lib.load_model(
File "/home/wise/testing/myenv/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py", line 70, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/wise/testing/myenv/lib/python3.10/site-packages/keras/src/saving/legacy/save.py", line 234, in load_model
raise IOError(
OSError: No file or directory found at ./homogenus/trained_models/tf/TR02_E02_It_002010.ckpt
(myenv) wise@wise-Victus-by-HP-Gaming-Laptop-15-fb0xxx:~/testing/homogenus$

i downloaed the weights but i only have the .index , .meta and .00104.0014

Prediction label

Excuse me. Can you tell me whether I can change the output of male/ female/ neutral into male/ female? Thanks.

Problems in installing and running [SOLUTION]

Greetings and thank you for this awesome project!

I got the following problem when I installed the dependencies:

error: numpy 1.16.3 is installed but numpy>=1.17 is required by {'matplotlib'}
error: numpy 1.16.3 is installed but numpy>=1.17.3 is required by {'PyWavelets'}

I resolved the problem by changing the dependencies in requirements.txt:

numpy==1.17.3

Then I got this wall of text error when I tried to run inference:

$ python3 homogenus/tf/homogenus_infer.py -h
Traceback (most recent call last):
  File "homogenus/tf/homogenus_infer.py", line 21, in <module>
    import tensorflow as tf
  File "/home/.../source/homogenus/.venv/lib/python3.7/site-packages/tensorflow/__init__.py", line 24, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "/home/.../source/homogenus/.venv/lib/python3.7/site-packages/tensorflow/python/__init__.py", line 52, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "/home/.../source/homogenus/.venv/lib/python3.7/site-packages/tensorflow/core/framework/graph_pb2.py", line 15, in <module>
    from tensorflow.core.framework import node_def_pb2 as tensorflow_dot_core_dot_framework_dot_node__def__pb2
  File "/home/.../source/homogenus/.venv/lib/python3.7/site-packages/tensorflow/core/framework/node_def_pb2.py", line 15, in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
  File "/home/.../source/homogenus/.venv/lib/python3.7/site-packages/tensorflow/core/framework/attr_value_pb2.py", line 15, in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
  File "/home/.../source/homogenus/.venv/lib/python3.7/site-packages/tensorflow/core/framework/tensor_pb2.py", line 15, in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
  File "/home/.../source/homogenus/.venv/lib/python3.7/site-packages/tensorflow/core/framework/resource_handle_pb2.py", line 41, in <module>
    serialized_options=None, file=DESCRIPTOR),
  File "/home/.../source/homogenus/.venv/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

Then I headed to the link to check what's going on. It looks like this has something to do with tensorflow and protocol buffers. My tf version:

1.13.1

It's not 1.15.2 because I tried this PR.

My protobuf version:

4.21.2

I have a few options:

• upgrade tensorflow to 2.x
• downgrade protobuf to 3.20.x or lower.

You can see the PRs for changing requirements in this repo. I didn't test them yet. This is what worked for me:

$ pip install protobuf==3.20
Collecting protobuf==3.20
  Using cached protobuf-3.20.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.0 MB)
ERROR: homogenus .5 has requirement numpy==1.17.3, but you'll have numpy 1.16.3 which is incompatible.
Installing collected packages: protobuf
  Attempting uninstall: protobuf
    Found existing installation: protobuf 4.21.2
    Uninstalling protobuf-4.21.2:
      Successfully uninstalled protobuf-4.21.2
Successfully installed protobuf-3.20.0

Even though I got the warning it worked for me in the end. Test to validate this:

$ python3 homogenus/tf/homogenus_infer.py -h
usage: homogenus_infer.py [-h] [-tm TRAINED_MODEL_DIR] -ii IMAGES_INDIR -oi
                          OPENPOSE_INDIR [-io IMAGES_OUTDIR]
                          [-oo OPENPOSE_OUTDIR]

optional arguments:
  -h, --help            show this help message and exit
  -tm TRAINED_MODEL_DIR, --trained_model_dir TRAINED_MODEL_DIR
                        The path to the directory holding homogenus trained
                        models in TF.
  -ii IMAGES_INDIR, --images_indir IMAGES_INDIR
                        Directory of the input images.
  -oi OPENPOSE_INDIR, --openpose_indir OPENPOSE_INDIR
                        Directory of openpose keypoints, e.g. json files.
  -io IMAGES_OUTDIR, --images_outdir IMAGES_OUTDIR
                        Directory to put predicted gender overlays. If not
                        given, wont produce any overlays.
  -oo OPENPOSE_OUTDIR, --openpose_outdir OPENPOSE_OUTDIR
                        Directory to put the openpose gendered keypoints. If
                        not given, it will augment the original openpose json
                        files.

Just for convenience, here's my setup.

• Ubuntu 20.04
• Python 3.7

I used venv, but you can do similar with conda if that rocks your boat.

python3.7 -m venv .venv
source .venv/bin/activate

$ pip freeze
absl-py==1.1.0
astor==0.8.1
cycler==0.11.0
fonttools==4.34.4
gast==0.2.2
google-pasta==0.2.0
grpcio==1.47.0
h5py==3.7.0
homogenus===.5
imageio==2.5.0
importlib-metadata==4.12.0
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.2
kiwisolver==1.4.3
Markdown==3.3.7
matplotlib==3.5.2
mock==4.0.3
networkx==2.6.3
numpy==1.16.3
opt-einsum==3.3.0
packaging==21.3
Pillow==9.2.0
protobuf==3.20.0
pyparsing==3.0.9
python-dateutil==2.8.2
PyWavelets==1.3.0
scikit-image==0.15.0
scikit-learn==0.20.3
scipy==1.2.1
six==1.16.0
tensorboard==1.13.1
tensorflow==1.13.1
tensorflow-estimator==1.13.0
termcolor==1.1.0
typing_extensions==4.3.0
Werkzeug==2.1.2
wrapt==1.14.1
zipp==3.8.1

You could do the same with different tensorflow versions: just downgrade protobuf and hope for the best. I hope this helps someone else and saves them the 30 minutes I used for this :)