Explain how to generate 'static fits' about spin

[benjiebob]

Aha! Yes, OK. My first experiment is just going to be on full supervised 3D datasets, so I can definitely proceed here.

I think I understand the purpose of this SMPLify stage now. For other datasets (e.g. COCO), where 3D is not given, you can use the SMPLify process to generate ever improving 3D SMPL parameters, using the initial network prediction as a starting point. These are only used when they're better than the static fits generated ahead of time. These can then effectively supervise the 3D losses.

Cunning! :D

[geopavlakos]

For the static fits, if we start from the mean pose, then we need the traditional SMPLify with the 4-stage optimization procedure. If we start from a predicted 3D pose (which should be 'closer' to the actual pose), then the procedure is the same, except that we ignore the first three stages of SMPLify, and only run the last one.

Regarding the second question, let me clarify that the last sentence you copy, refers to the number of iterations we do for the in-the-loop procedure (not for the generation of the static fits). It is safe to increase these iterations (e.g., 100 as is the default in the repo), and you might do marginally better for some batches, but we have observed that 50 are typically enough.

[nkolot]

We did not include the SMPLify fits for H36m in the released *.npy files. To make the training procedure run, what you can do is create a *.npy file for h36m containing the parameters you have. The format is (N,82), i.e. num_images X (pose_params + shape_params).
For h36m, these will be ignored provided that in your dataset file you load the "gt" parameters.
What I mean above is that if there are ground truth pose and shape parameters, these will always override what is in the static_fits during training.

[nkolot]

Also these fits are the fits we got using the "out-of-the-loop" procedure mentioned in the paper. Each experiment you start will keep a separate copy of those that will update during training. The updated fits will be stored in logs/expname/checkpoints/*.npy.

We do not start from the mean pose, but instead we obtain some initial fits with the procedure described in the paper. We provide these fits to allow people reproduce our results. In case you want just the final fits and not the fits we used in the beginning, we have released these as well.

[benjiebob]

Hmm... interesting. Apologies, I'm trying to understand this process...

In the case that full supervision (i.e. SMPL shape & pose is available), I think the following is true:

• Your code reads in the ground truth set of SMPL parameters x, populating 'fits_dict'.
• You then run network on images, to predict a new set of SMPL parameters y,
• You then run smplify on the new set of SMPL parameters y, to get a fit z.
• If this z has a lower 2D reprojection error than the ground truth, you update 'opt' parameters and the 'fits_dict'
• Then, (I stress, for datasets such as H36M), the 'opt' parameters are then replaced with ground truth for computing 3D loss, vertex losses etc.

For these datasets (e.g. H36M), am I right in thinking the SMPLify process has no effect?

Apologies if I'm missing something here!

[nkolot]

Correct! This is what we are doing.
Yes, as you said for these datasets SMPLify has no effect, this is why override the parameters in the end. It will have an effect however if you set the ignore_3d flag to True, and this is used for our "unpaired" experiments where we do not use any paired image-3D data.

[benjiebob]

Thanks a lot for the help here (particularly provided so rapidly)!

[iammarvelous]

Hi @nkolot, I am still confused about some details in generating static_fits.

• It seems that initial static_fits are generated with SMPLify by initializing with mean pose or predicted 3d pose. I am curious if you use the implementation by calling this or use the original chumpy implementation of SMPLify.

• In the paper, you mentioned

  To initialize SMPLify for this process, we can start from the mean pose, or use a more accurate pose, regressed from the 2D keypoints (e.g., using a network similar to Martinez et al. [21]). For our empirical evaluation we focus on the second strategy, but we also present similar results with the first approach in the Sup.Mat. We run the SMPLify optimization for a total of 50 iterations for each batch.

  Are you saying that for these initial static_fits, you used 50 iterations for optimization? If so, why the number of iteration is less than in the loop SMPLify num_iters? Shouldn't the initial static_fits from, say, mean pose, take more iterations to converge?

It would be great if you can provide a standalone script to generate the initial fits. Thanks.

[iammarvelous]

Thanks for the clarification. It's very clear now.