seraph1188v / robustcap

Code for our SIGGRAPH ASIA 2023 paper "Fusing Monocular Images and Sparse IMU Signals for Real-time Human Motion Capture".

RobustCap

Code for our SIGGRAPH ASIA 2023 paper "Fusing Monocular Images and Sparse IMU Signals for Real-time Human

Motion Capture". This repository contains the system implementation and evaluation. See Project Page.

Installation

conda create -n RobustCap python=3.8
conda activate RobustCap
pip install -r requirements.txt

Install pytorch cuda version from the official website.

Data

SMPL Files, Pretrained Model and Test Data

• Download smpl files from here or the official website. Unzip it and place it at models/.
• Download the pretrained model and data and place them at data/.
• For AIST++ evaluation, download the no aligned files and place it at data/dataset_work/AIST.

Evaluation

We provide the evaluation code for AIST++, TotalCapture, 3DPW and 3DPW-OCC. The results maybe slightly different from the numbers reported in the paper due to the randomness of the optimization.

python evaluate.py

Visualization

We provide the visualization code for AIST++. You can use view_aist function in evaluate.py to visualize the results. By indicating seq_idx and cam_idx, you can visualize the results of a specific sequence and camera. Set vis=True to visualize the overlay results. Use body_model.view_motion to visualize the open3d results.

Todo

• Visualization.
• Live demo code.

Citation

TBA