otaheri / goal Goto Github PK

Hello!

I follow the guideline in README.md and successfully run the following command:
python data/process_gnet_data.py --grab-path /path/to/GRAB --smplx-path /path/to/smplx/models/
python data/process_mnet_data.py --grab-path /path/to/GRAB --smplx-path /path/to/smplx/models/

but when I try to run the following command:
python test/GOAL.py --work-dir /path/to/work/dir --grab-path /path/to/GRAB --smplx-path /path/to/models/

I encountered an error below:
2022-12-19 13:27:51.338 | INFO | __main__:load_data:152 - Base dataset_dir is /home/ubuntu/yy/GRAB/dataset_unzipped/MNet_data Traceback (most recent call last): File "/home/ubuntu/yy/GOAL/test/GOAL.py", line 699, in <module> inference() File "/home/ubuntu/yy/GOAL/test/GOAL.py", line 692, in inference tester = Trainer(cfg_motion, cfg_static, inference=True) File "/home/ubuntu/yy/GOAL/test/GOAL.py", line 88, in __init__ self.load_data(cfg, inference) File "/home/ubuntu/yy/GOAL/test/GOAL.py", line 156, in load_data ds_test = LoadData(self.cfg.datasets, split_name=ds_name) File "/home/ubuntu/yy/GOAL/./data/mnet_dataloader.py", line 75, in __init__ [name.split('/')[-2] for name in self.frame_names[:, 10]]) File "/home/ubuntu/yy/GOAL/./data/mnet_dataloader.py", line 75, in <listcomp> [name.split('/')[-2] for name in self.frame_names[:, 10]]) IndexError: list index out of range

And I can't figure out why it goes wrong. Can you help me find out why?
Thanks!

Hi Omid,

Thank you for releasing the code and congratulations on the great work!

There are a few places in the code where the path hasn't been updated - e.g., it still points to /ps/project/. Specifically, when I run test/GOAL.py it looks for the file feet_verts_ids_0512.npy. Could you please make this file available?

Thanks in advance!

Hello there! First of all, thank you for open-sourcing your work!

I saw that you're hosting the GNet and MNet model files on your website – Would you be interested in also sharing them as gated models on the Hugging Face Hub?

The Hub makes it easy to freely download and upload models, and it can make models more accessible and visible to the rest of the ML community. It's good way to share useful metadata and metrics, and we also support features like TensorBoard visualizations and PapersWithCode integrations. Since models are hosted as Git repos, they're also automatically versioned with a commit history and diffs. We could even help you set up an organization (e.g. see the Facebook AI or Stanford NLP organizations).

We have a step-by-step guide that explains the process for uploading the model to the Hub, in case you're interested. We also have a library for programmatic access to uploading and downloading models, which includes features like caching for downloaded models.

We also host datasets on the Hub, so I'll open up a separate issue over on https://github.com/otaheri/GRAB/ in case you're interested in making the dataset available on Hugging Face too 😊

Please let us know if you have any questions, and we'd be happy to guide you through the process!

Nima and the Hugging Face team

cc @osanseviero @lhoestq

Hi Omid,

Thanks for the work. Seems that you preprocess the GRAB dataset. So the original visualization tool does not work anymore. I thus wonder if you could provide guidance to visualize the processed data annotations.

Hi Otaheri,

when trying to train GNet with the following command:
python train/GNet_train.py --work-dir path/to/work/dir --grab-path /path/to/GRAB --smplx-path /path/to/models/ --expr-id EXPERIMENT_ID

I encountered an error below:
TypeError: 'SMPLXLayer' object does not support item assignment

There are 4 lines causing such error, which are:
self.female_model['v_template'] = v_template[females] at line 397 in GNet_train.py
self.male_model['v_template'] = v_template[males] at line 423
self.female_model['v_template'] = v_template[females] at line 565
self.male_model['v_template'] = v_template[males] at line 571

After checking the source code of smplx, I made a substitution and figured out this bug:
['v_template'] -> .v_template

Is it a right correction or there are something else I missed?
Thanks!

Hi, I can't seem to find test/GOAL.py file for generating the results. Could you please point me to it? Thanks!

Hello!

In the repository description you specify that GOAL can run on arbitrary objects provided by users (incl. computing on the fly the BPS representation for them). How this could be done? Namely, is it possible to provide arbitrary 3D object mesh to generate the grasps?

Thanks,
Bartek

Hi Otaheri,

Thank you for your good work!

Do you meet the problem that when training GNet, the GPU memory keeps increasing with every batch and finally leads to CUDA being out of memory? When training GNet, the first few iterations have seen a significant GPU memory increase, while the later each batch has increased by approximately 2 or 4M. As far as I know, if we don't intentionally save some variables with gradients, the release and new occupation of memory after the first iteration in training neural networks should be in a balanced stage, that is, there won't be any further increase in GPU memory. Could you help me solve this problem? Thank you very much!

I found a discrepancy between (1) the subject vertices that are saved in the file GNet_data.npy under 'verts' and (2) after computing them from the GT parameters provided in GNet_data.npy.

I am unsure if this is intentional or whether I am missing something. Here is the procedure I am following for re-computing the vertices (similar to what is done here):

1. Use the keys 'fullpose_rotmat' and 'transl' from saved data and pass it to parms_6D2full() to compute full rotation matrix of subject.
2. Add the key sbj_vtemp from the saved data to the dict obtained in Step 1.
3. Depending on whether the GT gender is female or male, pass it through a model created with SMPLXLayer with num_pca_comps=45.
4. Obtain the vertices from Step 3.

I then reference the vertices in Step 4 at the saved indices cfg.verts_sampled, to get the vertices of size (400, 3). I found that these do not exactly match those in the saved data for the key 'verts' (they do not seem off by a large margin, but are not exactly the same either).

I believe this should be a problem as it appears that the saved 'verts' are being used to compute the GT distance with the object (stored in 'verts2obj') against which the loss is computed here.

Please let me know if I am missing something. Thanks in advance!

Successfully installed bps_torch dependency package still shows that it does not exist. Why？

Hi,

For me the installation fails. Would you mind sharing your setup or updating the installation instructions.
I am using Ubuntu 20.04 and python 3.8.10. When I follow the installation instruction it fails at pip install -r requirements.txt with

Collecting git+git://github.com/otaheri/chamfer_distance.git (from -r requirements.txt (line 21))
  Cloning git://github.com/otaheri/chamfer_distance.git to /tmp/pip-req-build-glfu0fo0
  Running command git clone --filter=blob:none --quiet git://github.com/otaheri/chamfer_distance.git /tmp/pip-req-build-glfu0fo0
  fatal: unable to connect to github.com:
  github.com[0: 140.82.121.3]: errno=Connection timed out

  error: subprocess-exited-with-error
  
  × git clone --filter=blob:none --quiet git://github.com/otaheri/chamfer_distance.git /tmp/pip-req-build-glfu0fo0 did not run successfully.
  │ exit code: 128
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× git clone --filter=blob:none --quiet git://github.com/otaheri/chamfer_distance.git /tmp/pip-req-build-glfu0fo0 did not run successfully.
│ exit code: 128
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Thanks!