convergence of in-the-loop training about spin

[iammarvelous]

OK. I will try that. I tried static training before and everything looks good except that loss_regr_betas has some extreme values once a while. I am curious about your training details. Did you train without run_smplify at the beginning? Also, did you use the default loss weight here? It seems that shape_loss_weight is 0? Also about openpose_train_weight and gt_train_weight, any suggest on this? Do you also suggest openpose_train_weight as in the fitting stage?

[nkolot]

So just to be clear, the previous version of the code could still be using the extreme betas from the static fits. What I do is I check which fit has the lowest reprojection error and in the beginning of training it is almost certain that the static fits have lower error because the network has not started producing accurate shapes, i.e. good initializations for the optimization. This is one reason we enable the in-the-loop training later on.

[nkolot]

Hmm it looks like a learning rate issue. Can you try and train with a 10 times lower learning rate? Unfortunately Adam does not behave like SGD, so scaling the learning rate when increasing the batch size can have unexpected behavior.
Also try not to use --run_smplify in the beginning.

[nkolot]

The parameters that you mentioned are the ones used in the paper. I also inspected the code and I noticed that there is a small bug in the released version. Essentially I am not filtering out extreme betas in the static fits, only for the in the loop fits. I will do the modification now and I will push the changes.

[iammarvelous]

I think this should be the reason for the bumps in loss_regr_betas. I recalled in the paper you mentioned about the per vertices loss, i.e., shape loss in the code. Did you ever run the experiment on the shape loss? If so, did you disable it because of inferior performance?

[nkolot]

We mention in the supplementary material that the weight we chose for this loss is 0. The reason that we ended up not using this loss is that it is highly correlated with the other losses.

[iammarvelous]

Well, I just take a look at your commit b4c70d5. The above figures are run under run_smplify. So I feel there may be other reasons for the bumps?

[ikvision]

@iammarvelous were you able to avoid this problem by reducing the learning_rate or enlarging the batch_size?