Comments (9)
Aha! Yes, OK. My first experiment is just going to be on full supervised 3D datasets, so I can definitely proceed here.
I think I understand the purpose of this SMPLify stage now. For other datasets (e.g. COCO), where 3D is not given, you can use the SMPLify process to generate ever improving 3D SMPL parameters, using the initial network prediction as a starting point. These are only used when they're better than the static fits generated ahead of time. These can then effectively supervise the 3D losses.
Cunning! :D
from spin.
For the static fits, if we start from the mean pose, then we need the traditional SMPLify with the 4-stage optimization procedure. If we start from a predicted 3D pose (which should be 'closer' to the actual pose), then the procedure is the same, except that we ignore the first three stages of SMPLify, and only run the last one.
Regarding the second question, let me clarify that the last sentence you copy, refers to the number of iterations we do for the in-the-loop procedure (not for the generation of the static fits). It is safe to increase these iterations (e.g., 100 as is the default in the repo), and you might do marginally better for some batches, but we have observed that 50 are typically enough.
from spin.
We did not include the SMPLify fits for H36m in the released *.npy files. To make the training procedure run, what you can do is create a *.npy file for h36m containing the parameters you have. The format is (N,82), i.e. num_images X (pose_params + shape_params).
For h36m, these will be ignored provided that in your dataset file you load the "gt" parameters.
What I mean above is that if there are ground truth pose and shape parameters, these will always override what is in the static_fits during training.
from spin.
Also these fits are the fits we got using the "out-of-the-loop" procedure mentioned in the paper. Each experiment you start will keep a separate copy of those that will update during training. The updated fits will be stored in logs/expname/checkpoints/*.npy.
We do not start from the mean pose, but instead we obtain some initial fits with the procedure described in the paper. We provide these fits to allow people reproduce our results. In case you want just the final fits and not the fits we used in the beginning, we have released these as well.
from spin.
Hmm... interesting. Apologies, I'm trying to understand this process...
In the case that full supervision (i.e. SMPL shape & pose is available), I think the following is true:
- Your code reads in the ground truth set of SMPL parameters x, populating 'fits_dict'.
- You then run network on images, to predict a new set of SMPL parameters y,
- You then run smplify on the new set of SMPL parameters y, to get a fit z.
- If this z has a lower 2D reprojection error than the ground truth, you update 'opt' parameters and the 'fits_dict'
- Then, (I stress, for datasets such as H36M), the 'opt' parameters are then replaced with ground truth for computing 3D loss, vertex losses etc.
For these datasets (e.g. H36M), am I right in thinking the SMPLify process has no effect?
Apologies if I'm missing something here!
from spin.
Correct! This is what we are doing.
Yes, as you said for these datasets SMPLify has no effect, this is why override the parameters in the end. It will have an effect however if you set the ignore_3d flag to True, and this is used for our "unpaired" experiments where we do not use any paired image-3D data.
from spin.
Thanks a lot for the help here (particularly provided so rapidly)!
from spin.
Hi @nkolot, I am still confused about some details in generating static_fits
.
-
It seems that initial
static_fits
are generated with SMPLify by initializing with mean pose or predicted 3d pose. I am curious if you use the implementation by calling this or use the original chumpy implementation of SMPLify. -
In the paper, you mentioned
To initialize SMPLify for this process, we can start from the mean pose, or use a more accurate pose, regressed from the 2D keypoints (e.g., using a network similar to Martinez et al. [21]). For our empirical evaluation we focus on the second strategy, but we also present similar results with the first approach in the Sup.Mat. We run the SMPLify optimization for a total of 50 iterations for each batch.
Are you saying that for these initial
static_fits
, you used 50 iterations for optimization? If so, why the number of iteration is less than in the loop SMPLify num_iters? Shouldn't the initialstatic_fits
from, say, mean pose, take more iterations to converge?
It would be great if you can provide a standalone script to generate the initial fits. Thanks.
from spin.
Thanks for the clarification. It's very clear now.
from spin.
Related Issues (20)
- Question about camera parameters in order to render in Pytorch3d HOT 3
- about dataset preprocess for mpi-inf-3dhp
- A bug in 'mpi_inf_3dhp.py'
- 2d joints for original image
- final fits npz file link not working HOT 5
- Differnece between unaligned and aligned H3.6m moshed data?
- Project
- How can I generate openpose json for MPII?
- Questions about data augmentation
- RuntimeError: size of dimension does not match previous size, operand 1, dim 2
- Some error I got HOT 2
- pck
- AssertionError: Path ./body_models/smpl/SMPL_NEUTRAL.pkl does not exist! HOT 2
- Some problems about body shape
- The formula for camera translation
- About reimplement HOT 1
- About performance HOT 2
- About the parameters of SMPL poses and camera pose
- Error Omesa
- How to use demo.py to get more result? Like Mesh result ...
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spin.