Without it's a bit.... useless?

Hi, @shubham-goel , thanks for your wonderful work and sharing of dataset!
I want to download the mocap dataset for the training of discriminator, and I may not want to download all dataset in one go, so can you tell me which part of the mocap dataset belongs to? Then I can download the responding part instead of whole data. Or I'll also appreciate it if you can provide extra download link for the mocap dataset.
By the way, the dropbox links same to be unvailable for some reasons

and this link may provide some information.

Thanks again for your contribution!

Hi, thanks for your wonderful work.

I notice that in your paper, the rendered mesh in white background is shadowed, like this

I notice the same shadow effect in the figure of PHALP

So I am wondering how the mesh is rendering with the shadow, or it is processed in other software?

Thanks a lot!

update based on adding PHALP

the following are my hacky steps to get the latest update to run on windows

install per conda instructions

there will be errors saying detectron2 cannot be built

comment out the following:
if 'PYOPENGL_PLATFORM' not in os.environ:
os.environ['PYOPENGL_PLATFORM'] = 'egl'
in:
hmr2/utils/mesh_renderer.py
hmr2/utils/renderer.py

change init_.py in configs to
CACHE_DIR = os.path.join(os.environ.get("USERPROFILE"), ".cache")

pip install cyphon

in vendor/detectron root folder install with
pip install -e .

then
pip install webdataset

run 'python demo.py' which will download a model file
untar this model file by hand from /Users/username/.cache

the following should now work for images
python demo.py

to get track.py to work
run the following from the root of 4d-human
git clone https://github.com/brjathu/PHALP.git

comment out the following in setup.py

"detectron2 @ git+https://github.com/facebookresearch/detectron2.git",

install phalp (at root of the new phalp directory)
pip install -e .[all]

in py_renderer
comment out
#os.environ['PYOPENGL_PLATFORM'] = 'egl'

phalp/configs/base.py change to
CACHE_DIR = os.path.join(os.environ.get("USERPROFILE"), ".cache") # None if the variable does not exist

and then add the following to phalp.py

import requests
import shutil

(additions/commenting from line606...)

    if not os.path.exists(smpl_path):
        # We are downloading the SMPL model here for convenience. Please accept the license
        # agreement on the SMPL website: https://smpl.is.tue.mpg.
        os.makedirs(os.path.join(CACHE_DIR, "phalp/3D/models/smpl"), exist_ok=True)
        #os.system('wget https://github.com/classner/up/raw/master/models/3D/basicModel_neutral_lbs_10_207_0_v1.0.0.pkl')
        with requests.get('https://github.com/classner/up/raw/master/models/3D/basicModel_neutral_lbs_10_207_0_v1.0.0.pkl', stream=True) as r:
            with open('basicModel_neutral_lbs_10_207_0_v1.0.0.pkl', 'wb') as f:
                shutil.copyfileobj(r.raw, f)
        convert_pkl('basicModel_neutral_lbs_10_207_0_v1.0.0.pkl')
        #os.system('rm basicModel_neutral_lbs_10_207_0_v1.0.0.pkl')
        os.remove('basicModel_neutral_lbs_10_207_0_v1.0.0.pkl')
        #os.system('mv basicModel_neutral_lbs_10_207_0_v1.0.0_p3.pkl ' + smpl_path)
        shutil.move('basicModel_neutral_lbs_10_207_0_v1.0.0_p3.pkl ' , smpl_path)

the following should now work
python track.py video.source="example_data/videos/gymnasts.mp4"

the command:

python track.py video.source="example_data/videos/gymnasts.mp4"

sometimes output:

Tracking : gymnasts 🏃  ━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  21% eta : 0:03:05  time elapsed : 0:00:50
GLError(
        err = 1282,
        description = b'invalid operation',
        baseOperation = glBindVertexArray,
        cArguments = (4,)
)
Traceback (most recent call last):
  File "/home/tensorboy/PHALP/phalp/trackers/PHALP.py", line 252, in track
    rendered_, f_size = self.visualizer.render_video(final_visuals_dic[frame_key])
  File "/home/tensorboy/PHALP/phalp/visualize/visualizer.py", line 350, in render_video
    rendered_image_final, valid_mask  = self.render_single_frame(
  File "/home/tensorboy/PHALP/phalp/visualize/visualizer.py", line 77, in render_single_frame
    rgb_from_pred, validmask = self.render.visualize_all(pred_vertices.numpy(), pred_cam_t_bs.cpu().numpy(), color, image, use_image=use_image)
  File "/home/tensorboy/PHALP/phalp/visualize/py_renderer.py", line 116, in visualize_all
    color = self.__call__(verts, focal_length=fl, baseColorFactors=baseColorFactors)
  File "/home/tensorboy/PHALP/phalp/visualize/py_renderer.py", line 153, in __call__
    color, rend_depth = self.renderer.render(scene, flags=pyrender.RenderFlags.RGBA)
  File "/home/tensorboy/anaconda3/envs/pose/lib/python3.10/site-packages/pyrender/offscreen.py", line 102, in render
    retval = self._renderer.render(scene, flags, seg_node_map)
  File "/home/tensorboy/anaconda3/envs/pose/lib/python3.10/site-packages/pyrender/renderer.py", line 144, in render
    retval = self._forward_pass(scene, flags, seg_node_map=seg_node_map)
  File "/home/tensorboy/anaconda3/envs/pose/lib/python3.10/site-packages/pyrender/renderer.py", line 389, in _forward_pass
    self._bind_and_draw_primitive(
  File "/home/tensorboy/anaconda3/envs/pose/lib/python3.10/site-packages/pyrender/renderer.py", line 516, in _bind_and_draw_primitive
    primitive._bind()
  File "/home/tensorboy/anaconda3/envs/pose/lib/python3.10/site-packages/pyrender/primitive.py", line 441, in _bind
    glBindVertexArray(self._vaid)
  File "src/errorchecker.pyx", line 58, in OpenGL_accelerate.errorchecker._ErrorChecker.glCheckError
OpenGL.error.GLError: GLError(
        err = 1282,
        description = b'invalid operation',
        baseOperation = glBindVertexArray,
        cArguments = (4,)
)

sometimes the error will happen at 10%, sometimes 5%.

Anyone have some suggestions?

Thanks for your great work, Could you share the script to evaluate the tracking performance on Posetrack dataset (HOTA, IDs, MOTA, and IDF1)

Hi,

Thanks a lot for the great work. Is it possible to provide a detailed version of your conda env with versions for all packages? My environment created using the provided environment.yml causes some pickling errors in the web dataset. I think it is because some packages are not consistent with your installation.

Thanks,
Umar

Only the input from VIDEO seems to work correctly in 4dhumans. Is it possible to modify it so that it also works with single image or frame image (directory specified)?

I tried uploading a multiple minute video, however I ended up with only a 40 second video back. Is this being capped somewhere? I traced it to this line from PHALP.py, but not sure where the end frame is being declared.

list_of_frames = list_of_frames if self.cfg.phalp.start_frame==-1 else list_of_frames[self.cfg.phalp.start_frame:self.cfg.phalp.end_frame]

Thanks

Thanks a lot for the amazing code.

Im trying to make it to work in blender, to import the result that is stored in the pkl file.

I was following the infirmation from dmpl about the 23 bones on the character, but after applying it on the smpl fbx, the legs looked alright but the arms and the head were very odd.

Is there any difference from the default order of the SMPL data?

Tha ks a lot for the attention

Hello,

I am trying to replicate HMR2 results using the provided data and script. I wanted to clarify a couple of doubts. TIA.

1. In dataset_tars.yaml, I had to make the following changes to match the dataset folder names. See the commented path (given in the repo) and the corrected uncommented path. Can you confirm this correction?:

INSTA-TRAIN-WMASK:
    TYPE: ImageDataset
    # URLS: hmr2_training_data/dataset_tars/insta-train-vitpose/{000000..001828}.tar
    URLS: hmr2_training_data/dataset_tars/insta-train-vitpose-replicate/{000000..001828}.tar
    epoch_size: 4_000_000
COCO-TRAIN-2014-VITPOSE-REPLICATE-PRUNED12:
    TYPE: ImageDataset
    # URLS: hmr2_training_data/dataset_tars/coco-train-2014-vitpose-replicate-pruned12/{000000..000044}.tar
    URLS: hmr2_training_data/dataset_tars/coco-train-2014-vitpose-pruned/{000000..000044}.tar
    epoch_size: 45_000

2. In insta-train-vitpose-replicate directory, there are total 3658 tar files but the dataset_tars.yaml mentions only 1829 files. Is it enough to replicate the results mentioned in the paper?

first of all the demos looks awesome ,thanks for your awesome work, are you planning to add a functionality for exporting bvh data

hi, I'm new in python, and when I try to run your code I get this error:
Traceback (most recent call last):
File "C:\Users\tamir\4D-Humans\track.py", line 9, in
from phalp.configs.base import FullConfig
File "C:\Users\tamir\anaconda3\envs\4D-humans\lib\site-packages\phalp\configs\base.py", line 8, in
CACHE_DIR = os.path.join(os.environ.get("HOME"), ".cache") # None if the variable does not exist
File "C:\Users\tamir\anaconda3\envs\4D-humans\lib\ntpath.py", line 104, in join
path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not NoneType

this error appears when I run tracking on video and also when I run the demo code I get similar error.
the environment building was well.
how to fix this?

We tried to convert the output of PHALP (body_pose (23x3x3) and global_orient (3x3)) to the SMPL standard of (72, 1), which consists of 23 joints and 1 global orient, each parametrized by 3 values. We tried to apply the Rodrigues formula on each 3x3 matrix, but it did not work. We are wondering if someone could kindly provide us with more information on possibly how to do the conversion.

Hi,

I am on a remote server (using a A10 GPU from Lambda Labs). I can't seem to run this repo at all. This line is the issue:

self.renderer = pyrender.OffscreenRenderer(viewport_width=self.img_res,
viewport_height=self.img_res,
point_size=1.0)

This is my full error message:

File "/home/ubuntu/anaconda3/envs/slahmr/lib/python3.10/site-packages/hmr2/models/hmr2.py", line 59, in init
self.mesh_renderer = MeshRenderer(self.cfg, faces=self.smpl.faces)
File "/home/ubuntu/anaconda3/envs/slahmr/lib/python3.10/site-packages/hmr2/utils/mesh_renderer.py", line 49, in init
self.renderer = pyrender.OffscreenRenderer(viewport_width=self.img_res,
File "/home/ubuntu/anaconda3/envs/slahmr/lib/python3.10/site-packages/pyrender/offscreen.py", line 31, in init
self._create()
File "/home/ubuntu/anaconda3/envs/slahmr/lib/python3.10/site-packages/pyrender/offscreen.py", line 137, in _create
egl_device = egl.get_device_by_index(device_id)
File "/home/ubuntu/anaconda3/envs/slahmr/lib/python3.10/site-packages/pyrender/platforms/egl.py", line 83, in get_device_by_index
raise ValueError('Invalid device ID ({})'.format(device_id, len(devices)))
ValueError: Invalid device ID (0)

I tried everything from xvfb to restarting my server but nothing seems to work. Is there any way I can get around this?

Hello author, thanks for your excellent work.

But I have a problem:
I noticed that I downloaded a file from the project brjathu/PHALP. In 4dhumans, this file data is used only for the first 24 items of ['pose'] parameters, namely global_orient ([:, [0]]) and body_pose ([:, 1:]). I would like to know what the following parameters [24:] mean.

Please reply to me at your earliest convenience. Thanks a million.

When I run your demo.py with the command python demo.py --img_folder example_data/images --out_folder demo_out --batch_size=48 --side_view --save_mesh --full_frame, It raise a error:

Traceback (most recent call last):
  File "/home/jianchang/4D-Humans/demo.py", line 143, in <module>
    main()
  File "/home/jianchang/4D-Humans/demo.py", line 138, in main
    input_img_overlay = input_img[:,:,:3] * (1-cam_view[:,:,3:]) + cam_view[:,:,:3] * cam_view[:,:,3:]
ValueError: operands could not be broadcast together with shapes (6016,4016,3) (6016,4016,0) 

It is a calculate error about numpy.
I don't know how to fix it correctly.

The URL for data download is not accessible.

https://people.eecs.berkeley.edu/~shubham-goel/projects/4DHumans/

Could you consider uploading the data to somewhere public accessible? Thanks

Created a venv, installed pre-reqs, downloaded and extracted the model, edited out the EGL lines as shown here #5, ran the example command

python demo.py --img_folder example_data/images --out_folder demo_out --batch_size=48 --side_view

get this

(venv) D:\4D-Humans>python demo.py --img_folder example_data/images --out_folder demo_out --batch_size=48 --side_view
Lightning automatically upgraded your loaded checkpoint from v1.8.1 to v2.0.2. To apply the upgrade to your files permanently, run `python -m pytorch_lightning.utilities.upgrade_checkpoint --file D:\4D-Humans\logs\train\multiruns\hmr2\0\checkpoints\epoch=35-step=1000000.ckpt`
Traceback (most recent call last):
  File "D:\4D-Humans\demo.py", line 30, in <module>
    model = HMR2.load_from_checkpoint(args.checkpoint, strict=False, cfg=model_cfg).to(device)
  File "D:\4D-Humans\venv\lib\site-packages\pytorch_lightning\core\module.py", line 1531, in load_from_checkpoint
    loaded = _load_from_checkpoint(
  File "D:\4D-Humans\venv\lib\site-packages\pytorch_lightning\core\saving.py", line 88, in _load_from_checkpoint
    storage = _load_state(cls, checkpoint, strict=strict, **kwargs)
  File "D:\4D-Humans\venv\lib\site-packages\pytorch_lightning\core\saving.py", line 141, in _load_state
    obj = cls(**_cls_kwargs)
  File "D:\4D-Humans\hmr2\models\hmr2.py", line 34, in __init__
    self.smpl_head = build_smpl_head(cfg)
  File "D:\4D-Humans\hmr2\models\heads\smpl_head.py", line 13, in build_smpl_head
    return SMPLTransformerDecoderHead(cfg)
  File "D:\4D-Humans\hmr2\models\heads\smpl_head.py", line 49, in __init__
    mean_params = np.load(cfg.SMPL.MEAN_PARAMS)
  File "D:\4D-Humans\venv\lib\site-packages\numpy\lib\npyio.py", line 405, in load
    fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: 'data/smpl_mean_params.npz'

I cannot find that npz file anywhere under the 4D-Humans directories.

Hi,

Thanks for releasing the code. I would like to train your algorithm on our own dataset. I wonder how should we sort out our dataset in the correct format? In the dataset you provided, for each image, you created a data.pyd file and a detection.npz file. I wonder what are these two files and how do you create them? Thank you for your time and effort in helping us.

Hello, thanks for sharing the astonishing project.

I am interested in applying this method to human NeRF's preprocessing stage.
Usually, the previous methods rely on ROMP to precompute SMPL parameters for each video frame. If I understand correctly, HMR 2.0 also works with a monocular input and outputs SMPL pose and betas.
My questions are

1. What is the difference between ROMP and HMR 2.0 in specific? Does the goal be the same?
2. If the task is the same, do you have any quantitative comparison between them?

Actually, I did a single sample demo with your code and ROMP and it seems that HMR 2.0 does better. However, the second question aims to further understand the reason for this. (maybe just because of the transformer ?)

Dose anyone know the running speed? including the inference and mesh rendering process.

Hi, May I ask how to perform evaluation on datasets such as Human3.6M?

Hi,
I am unable to download the file from below link:
https://people.eecs.berkeley.edu/~jathushan/projects/4dhumans/hmr2_data.tar.gz

It is mentioned in 'hmr2/models/init.py' at line-15:
https://github.com/shubham-goel/4D-Humans/blob/main/hmr2/models/__init__.py

This error happens while I try to run demo.py, where code tries to download the above file to cache.
python demo.py --img_folder example_data/images --out_folder demo_out --batch_size=48 --side_view --save_mesh --full_frame

Please let me know if I am doing something wrong.

Thanks

i was wondering , does this works with 8gb vram gpus , i have been trying to infer using track.py and demo.py , i put a low quality image and tried but i only get oom error here is the error

C:\Users\Genesis\anaconda3\envs\4D-Humans\lib\site-packages\torchaudio\backend\utils.py:74: UserWarning: No audio backend is available.
warnings.warn("No audio backend is available.")
Lightning automatically upgraded your loaded checkpoint from v1.8.1 to v2.0.6. To apply the upgrade to your files permanently, run python -m pytorch_lightning.utilities.upgrade_checkpoint --file C:\Users\Genesis\.cache\4DHumans\logs\train\multiruns\hmr2\0\checkpoints\epoch=35-step=1000000.ckpt
WARNING: You are using a SMPL model, with only 10 shape coefficients.
Traceback (most recent call last):
File "C:\Users\Genesis\github\4D-Humans\demo.py", line 143, in
main()
File "C:\Users\Genesis\github\4D-Humans\demo.py", line 59, in main
det_out = detector(img_cv2)
File "C:\Users\Genesis\github\4D-Humans\hmr2\utils\utils_detectron2.py", line 92, in call
predictions = self.model([inputs])[0]
File "C:\Users\Genesis\anaconda3\envs\4D-Humans\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\Genesis\anaconda3\envs\4D-Humans\lib\site-packages\detectron2\modeling\meta_arch\rcnn.py", line 150, in forward
return self.inference(batched_inputs)
File "C:\Users\Genesis\anaconda3\envs\4D-Humans\lib\site-packages\detectron2\modeling\meta_arch\rcnn.py", line 204, in inference
features = self.backbone(images.tensor)
File "C:\Users\Genesis\anaconda3\envs\4D-Humans\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\Genesis\anaconda3\envs\4D-Humans\lib\site-packages\detectron2\modeling\backbone\vit.py", line 489, in forward
bottom_up_features = self.net(x)
File "C:\Users\Genesis\anaconda3\envs\4D-Humans\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\Genesis\anaconda3\envs\4D-Humans\lib\site-packages\detectron2\modeling\backbone\vit.py", line 357, in forward
x = blk(x)
File "C:\Users\Genesis\anaconda3\envs\4D-Humans\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\Genesis\anaconda3\envs\4D-Humans\lib\site-packages\detectron2\modeling\backbone\vit.py", line 218, in forward
x = self.attn(x)
File "C:\Users\Genesis\anaconda3\envs\4D-Humans\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\Genesis\anaconda3\envs\4D-Humans\lib\site-packages\detectron2\modeling\backbone\vit.py", line 75, in forward
attn = add_decomposed_rel_pos(attn, q, self.rel_pos_h, self.rel_pos_w, (H, W), (H, W))
File "C:\Users\Genesis\anaconda3\envs\4D-Humans\lib\site-packages\detectron2\modeling\backbone\utils.py", line 122, in add_decomposed_rel_pos
attn.view(B, q_h, q_w, k_h, k_w) + rel_h[:, :, :, :, None] + rel_w[:, :, :, None, :]
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 1024.00 MiB (GPU 0; 8.00 GiB total capacity; 6.38 GiB already allocated; 0 bytes free; 6.56 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Hello, is there a way to choose which gpu is going to be used, in case the pc has more than one GPU?

Thanks

Hi, will increasing the number of epochs improve the quality of the motion capture?
If so, how do I change the number of epochs

Thanks for your excellent work, the download link in this file "https://github.com/shubham-goel/4D-Humans/blob/main/fetch_training_data.sh" can not open, could you please give me the file in ./hmr2_training_data

Hello,

first of all, thank you very much for sharing the implementation!

I was wondering if you experienced issues with GPU memory increasing after every batch using the ViT backbone?
I decreased the batch size to 2 so I start off with 2GB of gpu memory. However, after each batch iteration, the memory keeps increasing and finally explodes after 4 iterations.

I tried tracking where exactly does the memory increase, and I found two sources:

1. After the 32 ViT blocks the memory keeps increasing
2. After finishing the training_step the memory consumption spikes

I should expect that the memory increases in the first iteration since pytorch lazily loads stuff, but not after the following iterations.
Any tips are much appreciated :).

Hi,

Thanks for the great work. I am reading through the code and I was confused on how to project the 3D SMPL parameters to 2D image without using keypoints. It seems you have to run the keypoint detection model to generate bounding boxes to figure out the scale and size in order to project it back on the image but this is not feasible if I have a dataset of purely SMPL parameters. Is it possible to accurately do this without keypoints? I tried using the current projection code but replacing values like size with default values but the projection is way off.

I'd greatly appreciate any help on this. Thank you.

Hi,

Thanks for your powerful work and released code. It is so cool!

When I tried to use this code, I found that the HMR keypoint detector in your demo outputs 44 keypoints for each person.

However, 44 KP is not a regularly used setting in 2D pose estimation.

Can you tell me the meaning of each keypoint?

Thanks for your attention.

Hi, I wonder whether the checkpoint that we can download through demo.py is HMR 2.0a? Or this checkpoint is not corresponding to those tables in the paper? Thanks!

Hello.
I ran into some problems, when I did "pip install -e .[all]".The system always prompts me, "Building wheel for detectron2 (setup.py) ... error
error: subprocess-exited-with-error", and mentions "python setup.py bdist_wheel did not run successfully.", "No CUDA runtime is found, using CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA \v11.8\bin'".I always failed to build detectron2 and the system will report an error, as follows:
ERROR: Could not build wheels for detectron2, which is required to install pyproject.toml-based projects
As a beginner, I'm stuck and can't figure out what to do with this.I need some help. I'd be very grateful if you could take the time to clear this up for me.

I received some reports that the official Google colab was not working.

So I made a another one, and it was working for videos.

https://colab.research.google.com/drive/1xcdCNrFGXCCjUkqPHlgPix3wOhyxE75o?usp=sharing

Any chance we could get .obj or other 3d format export?

Great work!
I was wondering if you would consider adding a license file (preferably under the Apache or MIT)? This would greatly benefit the community and ensure clear usage terms for future contributors. Thank you.

Hi authors, thanks for the great work, are you planning to release the data preprocessing code? If yes, when will it be released?

comment out:

if 'PYOPENGL_PLATFORM' not in os.environ:
os.environ['PYOPENGL_PLATFORM'] = 'egl'

in the following 2 files
hmr2/utils/mesh_renderer.py
hmr2/utils/renderer.py

excellent work by the way!!

Hi, thanks for your wonderful work!

I am wondering that why the focal_length is divided by the image_size during perspective projection

pred_keypoints_2d = perspective_projection(pred_keypoints_3d, translation=pred_cam_t, focal_length=focal_length / self.cfg.MODEL.IMAGE_SIZE)
Thanks again and look forward to your reply!

Hello,
Thanks for publishing this extremely well-written code. I was going through the VIT setup in your model and noticed this (

For some reasons I could not download data from https://people.eecs.berkeley.edu/~shubham-goel/projects/4DHumans/hmr2_data.tar.gz and it shows 404 file not found.

Hi，thank you for your great work and sharing.
I test the demo code and get great result, but it looks like that the mesh accuracy of foot(actually toe base) is not accuracy as other joints.
I was wondering that is there a diffrent configuration between toe base joint and other joints?

Hi, thanks for your wonderful work!

In the paper you mentioned that you get 2D keypoints for unlabelled datasets, are the pseudo 2D labels in openpose format?

I notice that in the code, openpose keypoints and confidence are used, I am wondering that the openpose keypoints are used for all datasets or used for unlabelled datasets.

Thanks a lot.

Hi,

Thanks for releasing the code. I am wondering when will you release pseudo-GT for SMPL for AVA, AI Challenger and Instavariety?
Thanks,

Hi!
I was wondering whether it is possible to somehow easily exchange the SMPL model for SMIL to get better results on infants. I have just tried using the SMIL model (in place of SMPL), but it gives me completely the same results as with SMPL. I guess I am missing something...
Thanks

Thank you for your great work! While running the video demo, I noticed some slight jitter in the movements of the model's limbs. Is this due to the lack of smoothing?

Hello,
Thank you very much for sharing great project!

Using the img (skates.png) provided as an example by the author as an input, 3D mesh data was obtained with the demo.py code.

The generated 3D mesh data was properly overlapped with the original image, and also matched with high accuracy for each person.

However, there was a problem when merging or concatenating them into the same 3D space.

(The code to combine multiple meshes in the same 3D coordinate system is as follows.)

merged_meshed = all_meshes[0]
for i in range(1, len(all_meshes)):
merged_meshed = trimesh.util.concatenate(merged_meshed, all_meshes[i])
merged_meshed.export(os.path.join(args.out_folder, f'all.obj'))

In my opinion, this problem seems to be caused by poor accuracy on the z-axis.

Are there any parameters (like focal length of camera) that I need to adjust to improve this?

Or should this be recognized as a limitation of this project?

I have one more question, is there any way to make the created obj file like animation?

Hi,

Thanks for sharing the great work! Your mesh recovery results are very impressive. I want to ask how to generate multiple rendered people in the same image as the illustration in the teaser? By running the demo code, it will produce separate images for each person.

Thank you!

Hi! I'm really appreciate for you great work!! I followed your work about 3d feature based multi-person tracking model since T3DP model proposed.

I have a question during reading your paper.

What is the difference between PHALP' with HMR 2.0 and 4DHumans? I thought these two are same but at Table3 and 4, these seemed to show different performance in tracking and I'm not sure there are two seperate versions in the code.

Hi, thanks for your wonderful work!

In the paper you mentioned that some details are in the SupMat, but there is no that part in the arxiv version, so could you provide the SupMat somewhere or update the arxiv version? I know I could find the SupMat on CVF open access later but the proceedings need a long time to prepare right?

Thanks a lot!