snu-larr / diffupose_official

About Official implementation of the IROS2023 paper "DiffuPose: Monocular 3D Human Pose Estimation via Denoising Diffusion Probabilistic Model"

DiffuPose: Monocular 3D Human Pose Estimation via Denoising Diffusion Probabilistic Model

Environment

This code is built on the following environment

• Python 3.8.13
• PyTorch 1.12.1
• CUDA 11.2

You can create and activate conda environment as the following:

conda env create -f requirements.yaml
conda activate diffupose

Dataset setup

You can download 2D HR-Net detection, 2D ground-truth and 3D ground-truth on Human3.6M dataset via our Google Drive link. (It is only a temporary link for review). Download them and unzip the .zip file in the ./data folder.

You will end up with './data/data_3d_h36m.npz', './data/data_2d_h36m_hr.npz', './data/data_2d_h36m_gt.npz'.

Training from scratch

If you want to train our model from scratch using HR-Net detection, please run

python run.py -k hr -b 1024

Else, if you want to train with 2D ground-truth, please run

python run.py -k gt -b 1024

Evaluating pre-trained model

We provide our pre-trained 384-dimension model in results folder (HR-Net detected 2D pose as input). To evaluate our model, please run

python run.py -k hr --test-load best_model.pt

which will result in 50.0 mm error (MPJPE).

To obatin best result with 10 samples, run

python run.py -k hr --test-load best_model.pt --num-sample 10

which will result in 49.4 mm error (MPJPE).

Visualization

Our code is compatiable with VideoPose3D. Please refer to their github page for detailed instruction.

diffupose_official's Issues

How much will the loss of the diffusion model reach in the end?

After changing the data set, the loss stopped falling after it dropped to about 0.8.