eldar / deepcut-cnn

CNN architecture for articulated human pose estimation

DeeperCut Part Detectors

This short documentation describes steps necessary to compile and run CNN-based body part detectors presented in the DeeperCut paper:

Eldar Insafutdinov, Leonid Pishchulin, Bjoern Andres, Mykhaylo Andriluka, and Bernt Schiele
DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model
In European Conference on Computer Vision (ECCV), 2016 For more information visit http://pose.mpi-inf.mpg.de

Installation Instructions

• This code was developed under Linux (Debian wheezy, 64 bit) and was tested only in this environment.
• Build Caffe and Python bindings as described in the official documentation. You will have to disable CuDNN support and enable C++ 11.

$ make all pycaffe

• Install Python Click package (required for demo only)

$ pip install click

• Set PYTHONPATH variable

$ export PYTHONPATH=`pwd`/python

Download Caffe Models

$ cd models/deepercut
$ ./download_models.sh

Run Demo

$ cd python/pose
$ python ./pose_demo.py image.png --out_name=prediction

Citing

@inproceedings{insafutdinov2016deepercut,
	author = {Eldar Insafutdinov and Leonid Pishchulin and Bjoern Andres and Mykhaylo Andriluka and Bernt Schieke},
	title = {DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model},
	booktitle = {European Conference on Computer Vision (ECCV)},
	year = {2016},
	url = {http://arxiv.org/abs/1605.03170}
    }
@inproceedings{pishchulin16cvpr,
	author = {Leonid Pishchulin and Eldar Insafutdinov and Siyu Tang and Bjoern Andres and Mykhaylo Andriluka and Peter Gehler and Bernt Schiele},
	title = {DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation},
	booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
	year = {2016},
	url = {http://arxiv.org/abs/1511.06645}
}

deepcut-cnn's Issues

Build in window

Is there anybody successfully build this code in window?

does not compile under ubuntu 14.04 or 16.04

a lot of warnings are thrown:

warning: ‘auto’ changes meaning in C++11

ubuntu 14.04:

/home/willi/deepcut-cnn/src/caffe/pose/segment_parts.cpp:157:20: error: ‘jnt’ was not declared in this scope

ubuntu 16.04:

/usr/include/string.h:652:42: error: ‘memcpy’ was not declared in this scope

ubuntu 16.04 after adding set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -D_FORCE_INLINES") to CMakeLists.txt:

/home/gdv/deepcut-cnn/src/caffe/layers/pose_data_layer.cpp:298:43: error: ‘joint_stats_file’ was not declared in this scope

there are some other errors, but I think that are the main one

Error when compile the transform_image.cpp : : ‘t1_p’ does not name a type

Did you meet similar error before?

src/caffe/pose/transform_image.cpp:163:5: warning: ‘auto’ changes meaning in C++11; please remove it [-Wc++0x-compat]
auto t1_p = pad_to_3x3(t1);
^
src/caffe/pose/transform_image.cpp:163:10: error: ‘t1_p’ does not name a type
auto t1_p = pad_to_3x3(t1);
^
src/caffe/pose/transform_image.cpp:164:5: warning: ‘auto’ changes meaning in C++11; please remove it [-Wc++0x-compat]
auto t2_p = pad_to_3x3(t2);
^
src/caffe/pose/transform_image.cpp:164:10: error: ‘t2_p’ does not name a type
auto t2_p = pad_to_3x3(t2);
^
src/caffe/pose/transform_image.cpp:165:19: error: ‘t1_p’ was not declared in this scope
cv::Mat res = t1_p * t2_p;

Is it allowed to use for commercial purposes?

can i use your single person model to complete mulit person task?

your deepercut is about mulit person ,but u give single-person model,too.I want to know can i use your single-person model to meet mulit-person task? And when training detection,Is datasets must be a mulit-person dataset for mulit-person task?
please help me

train code

can you send the train code?thank you

Multi-person

Hi. Poses for single persons work great, however, I cannot seem to get it to correctly estimate the poses of more than 1 person. This should be possible right?

Problem in Demo

I have followed all procedures in README. However, when I finally try to run the demo.py, I get an error.
IndexError: boolean index did not match indexed array along dimension 0; dimension is 0 but corresponding boolean dimension is 16

I notice that an issue is similar with my problem, but I meet this error with the demo picture (image.png), so do someone can help me? Really appreciated.

how to crop a single person img from the src image(MPII dataset)

how to crop a single person img from the src image(MPII dataset)
I have read the annot mat,how to crop the single person from the infos
x1 y1 x2 y2 annopoints scale objpos ,how to calc the bound rect?
thank you

Out of memory while running demo

I am getting out of memory error while running the demo using the following command. I am using GeForce GTX 1070.

python ./pose_demo.py image.png --out_name=prediction

INFO:estimate_pose:Done!
F1028 23:43:02.663663 10368 syncedmem.cpp:56] Check failed: error == cudaSuccess (2 vs. 0)  out of memory
*** Check failure stack trace: ***
Aborted (core dumped)

make pycaffe issue

When I run "make all", I get the following error:

CXX src/caffe/layers/pose_data_layer.cpp
src/caffe/layers/pose_data_layer.cpp: In member function ‘virtual void caffe::PoseDataLayer::load_batch(caffe::MultiBatch*)’:
src/caffe/layers/pose_data_layer.cpp:615:59: error: there are no arguments to ‘DecodeDatumToCVMat’ that depend on a template parameter, so a declaration of ‘DecodeDatumToCVMat’ must be available [-fpermissive]
image = DecodeDatumToCVMat(image_cached.second, true);

src/caffe/layers/pose_data_layer.cpp: In instantiation of ‘void caffe::PoseDataLayer::load_batch(caffe::MultiBatch*) [with Dtype = float]’:
src/caffe/layers/pose_data_layer.cpp:937:1: required from here
src/caffe/layers/pose_data_layer.cpp:615:59: error: ‘DecodeDatumToCVMat’ was not declared in this scope
image = DecodeDatumToCVMat(image_cached.second, true);

How can I fix that?

Thanks!

ID'ing detectors

Just ran your demo. Thanks for sharing!

Is there a simple way to identify which body parts are being detected for each dot? For instance, I'm trying to ID which body parts are being labelled by model in this image:

make all pycaffe issue

When I run "make all pycaffe", I get the following error:

src/caffe/layers/pose_data_layer.cpp: In member function ‘virtual void caffe::PoseDataLayer::DataLayerSetUp(const std::vector<caffe::Blob>&, const std::vector<caffe::Blob>&)’:
src/caffe/layers/pose_data_layer.cpp:294:3: warning: ‘auto’ changes meaning in C++11; please remove it [-Wc++0x-compat]
auto joint_stats_file = this->layer_param().pose_data_param().has_joint_pairs_stats() ?
^
src/caffe/layers/pose_data_layer.cpp:294:8: error: ‘joint_stats_file’ does not name a type
auto joint_stats_file = this->layer_param().pose_data_param().has_joint_pairs_stats() ?
^
src/caffe/layers/pose_data_layer.cpp:298:43: error: ‘joint_stats_file’ was not declared in this scope
neighbour_stats_ = readMatricesFromFile(joint_stats_file);
^
src/caffe/layers/pose_data_layer.cpp: In member function ‘virtual void caffe::PoseDataLayer::load_batch(caffe::MultiBatch)’:
src/caffe/layers/pose_data_layer.cpp:515:9: error: ‘iota’ is not a member of ‘std’
std::iota(data_indices_.begin(), data_indices_.end(), 0);
^
src/caffe/layers/pose_data_layer.cpp: In instantiation of ‘void caffe::PoseDataLayer::DataLayerSetUp(const std::vector<caffe::Blob>&, const std::vector<caffe::Blob>&) [with Dtype = float]’:
src/caffe/layers/pose_data_layer.cpp:937:1: required from here
src/caffe/layers/pose_data_layer.cpp:290:7: warning: unused variable ‘num_labels’ [-Wunused-variable]
int num_labels = this->num_labels_;
^
src/caffe/layers/pose_data_layer.cpp: In instantiation of ‘void caffe::PoseDataLayer::load_batch(caffe::MultiBatch) [with Dtype = float]’:
src/caffe/layers/pose_data_layer.cpp:937:1: required from here
src/caffe/layers/pose_data_layer.cpp:573:9: warning: unused variable ‘num_labels’ [-Wunused-variable]
int num_labels = this->num_labels_;
^
src/caffe/layers/pose_data_layer.cpp:420:14: warning: unused variable ‘sequential’ [-Wunused-variable]
const bool sequential = this->layer_param_.pose_data_param().sequential();
^
src/caffe/layers/pose_data_layer.cpp: In instantiation of ‘void caffe::PoseDataLayer::DataLayerSetUp(const std::vector<caffe::Blob>&, const std::vector<caffe::Blob>&) [with Dtype = double]’:
src/caffe/layers/pose_data_layer.cpp:937:1: required from here
src/caffe/layers/pose_data_layer.cpp:290:7: warning: unused variable ‘num_labels’ [-Wunused-variable]
int num_labels = this->num_labels_;
^
src/caffe/layers/pose_data_layer.cpp: In instantiation of ‘void caffe::PoseDataLayer::load_batch(caffe::MultiBatch*) [with Dtype = double]’:
src/caffe/layers/pose_data_layer.cpp:937:1: required from here
src/caffe/layers/pose_data_layer.cpp:573:9: warning: unused variable ‘num_labels’ [-Wunused-variable]
int num_labels = this->num_labels_;
^
src/caffe/layers/pose_data_layer.cpp:420:14: warning: unused variable ‘sequential’ [-Wunused-variable]
const bool sequential = this->layer_param_.pose_data_param().sequential();
^
Makefile:563: recipe for target '.build_release/src/caffe/layers/pose_data_layer.o' failed
make: *** [.build_release/src/caffe/layers/pose_data_layer.o] Error 1

How can I fix that?

Thanks!

/BS/eldar/work/pose/exp/mpii-all-reg/data/all_stats.txt

where is this file,can you share it?

Error in ubuntu while testing my own image

Hi, I build the caffe successful and add the pose directory to the caffe dir,
when I test the image.png ,everything is ok.
but when i choose some pictures of mine,
it shows INFO:estimate_pose:Done!,and then
IndexError: boolean index did not match indexed array along dimension 0; dimension is 0 but corresponding boolean dimension is 16
how can i solve it .

confusing mix of Makefile.config.example and cmake

It seams to be, that the project can be compiled in two different ways: by starting make in the project root or to use cmake (better in build directory). This may result in inconsistence. What is the better way to compile it?

Can i know where to enable c++11 ?

i tried to compile it amd got some error in post_data_layer.cpp
i think the problem is from because of the compiler that i use.
There are some unused variable inside that code too.

How can I get array joint?

Hello,

I can get prediction.npz file from image.png using pose_demo.py.

I want to get array of joints estimation npz file from many images.
for example, image0000.png, image0001.png, image0002,png, image####.png, ...
How can I get output prediction.npz file from this many images.

matrix shape of pose.npy in prediction.npz will be changed (number, 14) -> (number, 14, image_count)
I tested my own one image, I get pose.npy shape (5,14). But I want to many images at once.

I tried to input image as input folder.
I put all input images to images/ folder
and call
python pose_demo.py images/ --out_name=estimation

then I can get output file in estimation/ folder. but separated file image0001.png_pose.npz, image0002.png_pose.npz, ...

I want to put all pose.npy file to one estimation.npz file.

Please let me know which line I have to modify.

Thank you.

ResNet-50

Is it possible to have ResNet-50 for this approach?