sha2nkt / deco Goto Github PK

Hi, thanks for the open-source dataset. I downloaded the dataset and printed the RGB value of the scene-mask image, the RGB values are the class-id, e.g. (97, 97, 97). But in the base_dataset.py line13, the function 'mask_split', it seems that the GR channels are fixed to 1 and 0, and the B channel is the class-id. Is this a typo, or maybe I misunderstood? Hope to get your help.
And the provided contact annotations include both human-supported and object-supported, is there any way to split them as the paper suggested?
Best.

Hi, thanks for the work, I want to know where can I get the smpl_2_smplx.pkl in here. It seems not in the download dataset.

I have successfully downloaded the DEMON dataset and its images. However, the directory names do not match with each other.
The downloaded images are in HOT-Annotated/images, but the code is based on HOT/Contact_Data/images/training.
Could you solve this issue?

Hi,

Great work! Would it be possible to share your 3D contact annotation tool? I'm interested in extending your work to new categories

Hi, your work is good. There is no checkpoints folder, where to download the checkpoints

Hi, is there any file like smpl_neutral_geodesic_dist.npy for smplx? So that we can compute the geo. (cm) for smplx contact estimation.

Hi, thanks for your great work! There exists some multi-person images in the HOT dataset. I'm wondering how could we know the annotated contact labels corresponding to which person?

Which Mask2Former pretrained checkpoint did the paper use to conduct scene segmentation? (among https://github.com/facebookresearch/Mask2Former/blob/main/MODEL_ZOO.md)

Was it custom trained and tested on the DECO, RICH, PROX dataset? or was it pretrained models of segmentation dataset (i.e., Cityscape)?

Thanks for your sharing.
I visualize the scene segmentation maps predicted by the model './checkpoints/Release_Checkpoint/deco_best.pth', but I find that the results seem weird, as shown in the picture, there are many wrong categories. I wonder if you could give some advice. Here is the code, which is mainly borrowed from mask2former.

May I ask you to provide data files (i.e., hot_behave_test_reduced.npz) in "test_chore_phosa.py" script?

In the paper, it says "We obtain pseudo ground-truth scene segmentation masks, Xs ∈ R H×W×No , containing semantic labels for No = 133 categories". Do you modify the default option of Mask2Former to get 133 categories? If so, what modification do you impose to the base code?

Hi, Thank you for this valuable work and your contributions.

I get this error when training the model with the given configs/cfg_train.yml file with all 3 datasets: rich, damon and prox. Could you please suggest the resolution?

If I train with damon only, I get this error.

Hi,
Until now I only ran the inference.py , and got some results, a great and useful paper.

I am wondering, were I need to edit your code to get the UV map of contact.

Thanks

AA

Sorry for too many questions regarding processing segmentation. I just have a one last question.

It seems that the segmentation outputs 133 classes (excluding background). If I include background, then it seems to be more reasonable to split the segmentation output with num_parts = 134. Why did the code split it with num_parts = 133?

I am trying to download DEMON dataset, but I cannot register me to the DECO webpage as I cannot get confirmation email from the webpage. Could you solve this problem?

May I ask what was the distance threshold for computing Precision, Recall, and F1 score for contact on BEHAVE dataset?

May I ask if there is any SMPL-X checkpoint?

Meaning that the model is trained to predict contact vertices in SMPL-X mesh vertices.

/home/aa/Documents/GitHub/deco/utils/hrnet.py:549: SyntaxWarning: "is" with a literal. Did you mean "=="?
or self.pretrained_layers[0] is '*':

Thanks for the answer.

After running the Mask2Former for semantic segmentation, the output segmentation does not seem to match with the segmentation_masks images for damon dataset (i.e., datasets/damon/segmentation_masks/training/vcoco_000000360328.png).

The segmentation maks for DAMON dataset is as follows:

The segmentation output of Mask2Former is as follows:

Hi, thanks for the great work, I have a question about the evaluation metrics: geo. (cm), is it calculated by adding fp_geo_error and fn_geo_error?