maria-korosteleva / body-shape-estimation

Estimate the pose and shape under clothing given a static 3D scan of a human

Hi 👋 I’m Maria Korosteleva, PostDoc at ETH Zurich and research enthusiast. You can find more info about my research projects on my website: https://korosteleva.com/

body-shape-estimation's Issues

derivative of GemanMcClareLoss

It seems like the line 101 of "ShapeUnderClothOptimizer.h" is a bug:

out[1] = 1 / denominator * denominator;

should be

out[1] = 1 / (denominator * denominator);

instead?

pose Jacobian on translation in get3DLocalTransformJac_

In "void SMPLWrapper::get3DLocalTransformJac_(const E::Vector3d & jointAxisAngleRotation,
const E::MatrixXd & transform_mat, E::MatrixXd* local_transform_jac_out)"

"
for (int i = 0; i < SPACE_DIM; ++i)
{
local_transform_jac_out[i].setZero(4, 4);
local_transform_jac_out[i].col(SPACE_DIM) = transform_mat.col(SPACE_DIM);
}
"
Is " local_transform_jac_out[i].col(SPACE_DIM) = transform_mat.col(SPACE_DIM); " necessary ?
I am thinking this column on translation should be all 0s, because the axis-angle rotation on current jointID will have not any influence on the current translation, though it should be have influence on the translation of the child.

If it is possible to attach a sample input / output for reference?

Many thanks for your great work!
May I ask if it is possible to attach a sample input/output for reference?

ShapeJac

Hi Maria, thank you for your great work. I have a quick question on the Jacobian computation for the shape.
In the E::MatrixXd SMPLWrapper::calcModel function:
"
if (shape_jac != nullptr)
for (int i = 0; i < SMPLWrapper::SHAPE_SIZE; ++i)
poseSMPL_(*pose, shape_jac[i], displacement); // Pose needs recalculation because joint positions at T are new
"
It seems "shape_jac" is mainly computed here, the logic seems to be a verts forward in poseSMPL, is this an approximation to the shape Jacobian ?

The shape Jacobian looks more complex ? It involves both "LBSMat" and "joints_global_transform" in "verts = LBSMat * joints_global_transform", I can't figure out how shape Jacobian is computed with regarding to the code.

Dependencies' versions

Hi @maria-korosteleva. Thanks for sharing the codes. Could you please let me know the versions of the dependencies? I ran into many errors regarding Ceres and libigl. Thanks in advance!

Code for garment segmentation

Thank you so much for releasing the code for this project!

As I am doing a project on garment segmentation on 3D human scans, I'm considering using your method proposed in "Estimating Garment Patterns from Static Scan Data" as the baseline.

I'm wondering if you have a plan to release the code for garment segmentation in this paper. Looking forward to your reply!

How to get predefined annotation in SMPL?

Hi~ I am trying to reproduce this awesome work in python!
But I have problems when I get the full-fit SMPL model.
I don't know how can I get the pre-defined labeled region points, predefined spline, and sign points illustrated in Fig.4.

raise error when starting optimization

Hello maria, thanks for your wonderful job. I have configured the program before optimization. The results of takephotos and openpose are normal. When I start to optimize, I find the value of distance_to_use is empty, so after running the code "residuals[i] = residual_elem_(distance_to_use.signedDists(i),distance_to_use.verts_normals.row(i), input_face_normals.row(distance_to_use.closest_face_ids(i)));". The code raises the error because the size of distance_to_use is 0. How can I deal with the problem? Should I set value to distance_to_use manually? Thank you!