sha2nkt / deco Goto Github PK
View Code? Open in Web Editor NEWEstimate vertex-level 3D human-scene and human-object contacts across the full body mesh
Home Page: https://deco.is.tue.mpg.de/
License: Other
Estimate vertex-level 3D human-scene and human-object contacts across the full body mesh
Home Page: https://deco.is.tue.mpg.de/
License: Other
Hi, thanks for the open-source dataset. I downloaded the dataset and printed the RGB value of the scene-mask image, the RGB values are the class-id, e.g. (97, 97, 97). But in the base_dataset.py line13, the function 'mask_split', it seems that the GR channels are fixed to 1 and 0, and the B channel is the class-id. Is this a typo, or maybe I misunderstood? Hope to get your help.
And the provided contact annotations include both human-supported and object-supported, is there any way to split them as the paper suggested?
Best.
Hi, thanks for the work, I want to know where can I get the smpl_2_smplx.pkl in here. It seems not in the download dataset.
Hi,
Great work! Would it be possible to share your 3D contact annotation tool? I'm interested in extending your work to new categories
Hi, your work is good. There is no checkpoints folder, where to download the checkpoints
Hi, is there any file like smpl_neutral_geodesic_dist.npy
for smplx? So that we can compute the geo. (cm)
for smplx contact estimation.
Hi, thanks for your great work! There exists some multi-person images in the HOT dataset. I'm wondering how could we know the annotated contact labels corresponding to which person?
Which Mask2Former pretrained checkpoint did the paper use to conduct scene segmentation? (among https://github.com/facebookresearch/Mask2Former/blob/main/MODEL_ZOO.md)
Was it custom trained and tested on the DECO, RICH, PROX dataset? or was it pretrained models of segmentation dataset (i.e., Cityscape)?
Thanks for your sharing.
I visualize the scene segmentation maps predicted by the model './checkpoints/Release_Checkpoint/deco_best.pth', but I find that the results seem weird, as shown in the picture, there are many wrong categories. I wonder if you could give some advice. Here is the code, which is mainly borrowed from mask2former.
May I ask you to provide data files (i.e., hot_behave_test_reduced.npz) in "test_chore_phosa.py" script?
In the paper, it says "We obtain pseudo ground-truth scene segmentation masks, Xs ∈ R H×W×No , containing semantic labels for No = 133 categories". Do you modify the default option of Mask2Former to get 133 categories? If so, what modification do you impose to the base code?
Hi,
Until now I only ran the inference.py , and got some results, a great and useful paper.
I am wondering, were I need to edit your code to get the UV map of contact.
Thanks
AA
Sorry for too many questions regarding processing segmentation. I just have a one last question.
It seems that the segmentation outputs 133 classes (excluding background). If I include background, then it seems to be more reasonable to split the segmentation output with num_parts = 134. Why did the code split it with num_parts = 133?
I am trying to download DEMON dataset, but I cannot register me to the DECO webpage as I cannot get confirmation email from the webpage. Could you solve this problem?
May I ask what was the distance threshold for computing Precision, Recall, and F1 score for contact on BEHAVE dataset?
May I ask if there is any SMPL-X checkpoint?
Meaning that the model is trained to predict contact vertices in SMPL-X mesh vertices.
/home/aa/Documents/GitHub/deco/utils/hrnet.py:549: SyntaxWarning: "is" with a literal. Did you mean "=="?
or self.pretrained_layers[0] is '*':
Thanks for the answer.
After running the Mask2Former for semantic segmentation, the output segmentation does not seem to match with the segmentation_masks images for damon dataset (i.e., datasets/damon/segmentation_masks/training/vcoco_000000360328.png).
The segmentation maks for DAMON dataset is as follows:
The segmentation output of Mask2Former is as follows:
Hi, thanks for the great work, I have a question about the evaluation metrics: geo. (cm), is it calculated by adding fp_geo_error and fn_geo_error?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.