umariqb / 3d_pose_estimation_cvpr2016

A Dual-Source Approach for 3D Pose Estimation from a Single Image

Introduction

The code package provides a tool for estimating 3D human pose from a single image. If you use this code for research purposes, please cite the following paper in any resulting publication:

Hashim Yasin, Umar Iqbal, Björn Krüger, Andreas Weber, Juergen Gall
A Dual-Source Approach for 3D Pose Estimation from a Single Image
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016.

For more information visit http://pages.iai.uni-bonn.de/iqbal_umar/ds3dpose/

The code is tested on Ubuntu 14.04 (64bit) with MATLAB (2016a).

#Instructions

1. PROVIDED
   Within this Release, we provide

   • the pre-normalized 3D pose database for Human3.6M
   • pre-trained random forests for 2D pose estimation

2. REQUIREMENTS
   2.1. LIBRARIES

• OpenCV
• Boost-1.55
• Gflags
• GLog compiled with Gflags

If the libraries are installed locally, please set 'CUSTOM_LIBRARY_PATH' in '../MEX/src/CMakeLists.txt' to your corresponding directory containing the glog, gflags folders. In '../MEX/src/cmake/Modules' scripts are provied to find your local glog and gflags installation.

Also please make sure that all local libraries are specified in your LD_LIBRARY_PATH and in your PATH variables.

2.2. TOOLBOXES
Matlab's optimization toolbox.

2.3. DATASET
Please get the Human36M Dataset and extract it to '../Human36M/code-v1.1/Release-v1.1' or any other location. If Human36M is located somewhere else, the path has to be adapted. See 6. for more information.

3. Run the script 'get_data.sh' to download the required models and pre-normalized database of 3D poses.

4. Switch to the 'code' folder to run any function.

5. COMPILATION
   To compile and install the binaries, please run 'installBinaries.m'. This script automatically runs cmake to build the neccecasry C++ implementations. For some Matlab installations, this results in some undefined reference errors. In that case, please switch to '../MEX/src/build' and run the following commands manually from your terminal

   • $ cmake -G "Unix Makefiles" ../
   • $ make

6. CONFIGURATION
   6.1. Adjust Parameters Open 'Initialize.m' located in the 'code' folder. Line 9:
   If necessary, adjust the path for the Human36M code

   Line 16 and following:
   In some cases, matlab has problems executing the binaries, because there are conflicts with matlab's provided openCV libraries. If this is the case, please adjust the 'OPENCV_LIBRARY_PATH' and the 'OPENCV_INCLUDE_PATH'

   Line 48 and following:
   Adapt configuration parameters for pose estimation like location of ground truth files or save paths for (temporary) estimation results:

   TDPose_ConfigPath:
   /some/location/3D_Pose_Estimation/MEX/src/regressors
   TDPose_ExperimentName:
   Folder contained in TDPose_ConfigPath
   TDPose_TestImageLocation:
   Path, where the test image are located
   TDPose_NTrees:
   Number of trees that are used in each forest
   TDPose_TrainFile:
   Path to the train annotation file for the dataset
   TDPose_TmpFileSavePath:
   Path where to save temporary data
   TDPose_TmpSaveResults:
   Path where to save (temporary) results
   TDPose_VisualizeResults:
   1, if results should be visualized
   TDPose_PauseAfterResult:
   1, if estimation should be paused
   TDPose_DeleteFilesAfterEstimation:
   delete all estimation files

Line 85:
You can specify, how many iterations you want to perform.

7. RUNNING THE CODE

You can either call 'RUN_Complete.m' to perform 3D Pose Estimation onthe whole dataset once or call 'RUN_Iterated.m' to performe 3D Pose Estimation for each single image of the dataset.

Ideally the approach requires roughly 100GBs of RAM to load 3D pose databases for the retrievel of K-NNs. However, in this release we have modified the code to fit in 32GBs of RAM. Therefore, the run-time will be different as compared to  the one reported in the paper.  

8. FUTURE RELEASE
   In the future we will also add support for HumanEva dataset

Citing

@inproceedings{Yasin_Iqbal_CVPR2016,
	author = {Hashim Yasin, Umar Iqbal, Björn Krüger, Andreas Weber, Juergen Gall},
	title = {A Dual-Source Approach for 3D Pose Estimation from a Single Image},
	booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
	year = {2016},
	url = {http://arxiv.org/abs/1509.06720}
}

Acknowledgements

A big thanks to Andreas Doering for preparing the source code for public availability.

3d_pose_estimation_cvpr2016's Issues

Reduce ram requirement

Hi, I'm very interested in your work of 3D human pose estimation. Just try to run your source, but my ram is not enough. Can you show me how to change/reduce the ram requirement to run this source.

I though 32G ram is used to load db.mat(~13G) into memory, then could you share a smaller normalized dataset.

Looking forward to hearing from you.
Best,
-Thanh

Running time of RUN_Iterated.m

Hi-
I have some questions about the programs.

1. Are RUN_Iterated.m and RUN_Complete.m producing the same result?
2. Which one is faster? RUN_Iterated.m or RUN_Complete.m?
3. It takes quite a long time(>20min) for me to get a result for a single image. (running RUN_Iterated.m)
   Can I ask you the running time of RUN_Iterated.m for a single image?
   I guess the reported time on paper is based on the program that uses more than 100G of RAM and this program might be slower than that because it's using less than 30G of RAM.
   If this is the case, can you upload the original version of program that uses 100G of RAM?

Best Regards.

image_index_train_rescaled

Hi,

I downloaded the Human3.6m dataset and could not find the "image_index_train_rescaled.txt" file. Can you help me by letting me know where I can get it from?

Regards,
Ruben

can't find bvh2xy

Dimension mismatching error in H_mat2cellMot.m

Hi- I'm having some issues while running your codes and asking you for a help.
I did compilation and initialization etc, and ran RUN_Iterated.m, then I got error in the line 173:
[err, optim, opts, motrecOpt] = recOnWeightedKernelH36Mbm(opts);
Err msg said that, in H_mat2cellMot.m, it tries to do substitution as below:
mot.jointTrajectories{i,1}(:,:) = pos(t:t+2,:);
where the 2nd dimension size does not match.

I looked into the codes and found that the 2nd dimension size of mot.jointTrajectories{i,1} and pos are the same as 'motGT.nframes' and 'motIn.nframes' from "motGT_S11_Activity_All_C2.mat" file, which is provided.
motGT.nframes was 928 and changed to 1, while motIn.nframes was 62.
Can you help me with this issue?
Thanks!