microsoft / voxelpose-pytorch Goto Github PK
View Code? Open in Web Editor NEWOfficial implementation of "VoxelPose: Towards Multi-Camera 3D Human Pose Estimation in Wild Environment"
License: MIT License
Official implementation of "VoxelPose: Towards Multi-Camera 3D Human Pose Estimation in Wild Environment"
License: MIT License
How to get the 3D coordinates of a person
Epoch: 6
Test: [0/645] Time: 1.962s (1.962s) Speed: 10.2 samples/s Data: 1.474s (1.474s) Memory 200613376.0
Test: [100/645] Time: 0.385s (0.453s) Speed: 51.9 samples/s Data: 0.000s (0.049s) Memory 200613376.0
Test: [200/645] Time: 0.406s (0.439s) Speed: 49.2 samples/s Data: 0.000s (0.041s) Memory 200613376.0
Test: [300/645] Time: 0.410s (0.432s) Speed: 48.8 samples/s Data: 0.000s (0.037s) Memory 200613376.0
Test: [400/645] Time: 0.388s (0.428s) Speed: 51.6 samples/s Data: 0.000s (0.036s) Memory 200613376.0
Test: [500/645] Time: 0.407s (0.427s) Speed: 49.1 samples/s Data: 0.000s (0.035s) Memory 200613376.0
Test: [600/645] Time: 0.419s (0.428s) Speed: 47.7 samples/s Data: 0.000s (0.034s) Memory 200613376.0
Test: [644/645] Time: 0.373s (0.431s) Speed: 53.7 samples/s Data: 0.000s (0.036s) Memory 200613376.0
ap@25: 0.0000 ap@50: 0.0000 ap@75: 0.0000 ap@100: 0.0000 ap@125: 0.0000 ap@150: 0.0000 recall@500mm: 0.0000 mpjpe@500mm: inf
I haven't been able to find a way to see the loss, could you help ?
I tried to download the datasets from http://campar.in.tum.de/Chair/MultiHumanPose and extract them under ${POSE_ROOT}/data/Shelf and ${POSE_ROOT}/data/CampusSeq1, but the following error message appeared.
Could not connect successfully
Could not establish a connection to the server for campar.in.tum.de.
Please let me know the other URL to get those datasets.
Hi there,
when I run python run/train_3d.py --cfg configs/panoptic/resnet50/prn64_cpn80x80x20_960x512_cam5.yaml
I am getting the following error:
Traceback (most recent call last):
File "run/train_3d.py", line 160, in <module>
main()
File "run/train_3d.py", line 107, in main
config, is_train=True)
File "/home/ubuntu/voxelpose-pytorch/run/../lib/models/multi_person_posenet.py", line 112, in get_multi_person_pose_net
backbone = eval(cfg.BACKBONE_MODEL + '.get_pose_net')(cfg, is_train=is_train)
File "/home/ubuntu/voxelpose-pytorch/run/../lib/models/pose_resnet.py", line 277, in get_pose_net
model.init_weights(cfg.NETWORK.PRETRAINED_BACKBONE)
File "/home/ubuntu/voxelpose-pytorch/run/../lib/models/pose_resnet.py", line 222, in init_weights
self.load_state_dict(pretrained_state_dict)
File "/home/ubuntu/voxelpose-pytorch/venv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 830, in load_state_dict
self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for PoseResNet:
size mismatch for final_layer.weight: copying a param with shape torch.Size([17, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([15, 256, 1, 1]).
size mismatch for final_layer.bias: copying a param with shape torch.Size([17]) from checkpoint, the shape in current model is torch.Size([15]).
Can anyone help? Is this an issue with JOINTS_DEF in lib/dataset/panoptic.py?
I have a custom dataset that I want to apply voxelpose to. But there is no explanation for how the values SPACE_SIZE and SPACE_CENTER were selected for the 3 datasets, therefore it is not clear to me what to set them to for my custom dataset
Hi @hanyue Tu Thank for you code! but the pretrained backbone model cannot download, open error, can you share other link??
Hello, Thanks for your rewarding word. But I want to know how can I evaluate custom videos, example for any .mp4 video files?
Thanks in advance for anyone who is willing to answer me.
"pose3d[:, 0:3] = pose3d[:, 0:3].dot(M)" in https://github.com/microsoft/voxelpose-pytorch/blob/main/lib/dataset/panoptic.py line 160
why dot M?
Thank you for this great repo, I thank this is very insteresting,I want use Three or four camera to get the 3D pose ,could you give the code 3D frrom 2Dpics?
Hi,
Thanks for sharing your code and great idea.
Could you please tell me where to find the trained model?
Thanks a lot.
Am I the only one getting this error ?
What can I do ?
Epoch: 9
Test: [0/645] Time: 2.164s (2.164s) Speed: 9.2 samples/s Data: 1.694s (1.694s) Memory 200613376.0
Test: [0/645] Time: 2.164s (2.164s) Speed: 9.2 samples/s Data: 1.694s (1.694s) Memory 200613376.0
Test: [100/645] Time: 0.406s (0.452s) Speed: 49.3 samples/s Data: 0.000s (0.049s) Memory 200613376.0
Test: [100/645] Time: 0.406s (0.452s) Speed: 49.3 samples/s Data: 0.000s (0.049s) Memory 200613376.0
Test: [200/645] Time: 0.379s (0.438s) Speed: 52.7 samples/s Data: 0.000s (0.041s) Memory 200613376.0
Test: [200/645] Time: 0.379s (0.438s) Speed: 52.7 samples/s Data: 0.000s (0.041s) Memory 200613376.0
Test: [300/645] Time: 0.378s (0.431s) Speed: 52.9 samples/s Data: 0.000s (0.037s) Memory 200613376.0
Test: [300/645] Time: 0.378s (0.431s) Speed: 52.9 samples/s Data: 0.000s (0.037s) Memory 200613376.0
Test: [400/645] Time: 0.415s (0.426s) Speed: 48.2 samples/s Data: 0.000s (0.035s) Memory 200613376.0
Test: [400/645] Time: 0.415s (0.426s) Speed: 48.2 samples/s Data: 0.000s (0.035s) Memory 200613376.0
Test: [500/645] Time: 0.420s (0.424s) Speed: 47.7 samples/s Data: 0.000s (0.034s) Memory 200613376.0
Test: [500/645] Time: 0.420s (0.424s) Speed: 47.7 samples/s Data: 0.000s (0.034s) Memory 200613376.0
Test: [600/645] Time: 0.378s (0.423s) Speed: 52.9 samples/s Data: 0.000s (0.034s) Memory 200613376.0
Test: [600/645] Time: 0.378s (0.423s) Speed: 52.9 samples/s Data: 0.000s (0.034s) Memory 200613376.0
Test: [644/645] Time: 0.373s (0.425s) Speed: 53.5 samples/s Data: 0.000s (0.036s) Memory 200613376.0
Test: [644/645] Time: 0.373s (0.425s) Speed: 53.5 samples/s Data: 0.000s (0.036s) Memory 200613376.0
ap@25: 0.0000 ap@50: 0.0000 ap@75: 0.0000 ap@100: 0.0000 ap@125: 0.0000 ap@150: 0.0000 recall@500mm: 0.0000 mpjpe@500mm: inf
ap@25: 0.0000 ap@50: 0.0000 ap@75: 0.0000 ap@100: 0.0000 ap@125: 0.0000 ap@150: 0.0000 recall@500mm: 0.0000 mpjpe@500mm: inf
Questions about camera calibration, and how the measure of the camera parameters impact the results of the reconstruction.
So, I wondered how you guys did to have access to the camera parameters. I see that you use a half-globe full of cameras, maybe all of these are given right away.
On the other hand, how does these parameters influence the results. How close to reality should they be so that the result is not that much influenced.
Following the git description, the directory tree was configured as follows.
${POSE_ROOT}
|-- models
| |-- pose_resnet50_panoptic.pth.tar
|-- data
|-- panoptic-toolbox
|-- data
|-- 171204_pose1
| |-- hdImgs
| |-- hdvideos
| |-- hdPose3d_stage1_coco19
| |-- calibration_160224_haggling1.json
|-- 171204_pose1_sample
|-- ...
(Due to capacity issues, the 171204_pose1 and 171204_pose1_sample data
was used}
After configuring the directory tree, the panoptic.py code is modified as follows.
TRAIN_LIST = [ '171204_pose1', ]
VAL_LIST = [ '171204_pose1_sample' ]
After setting like this,
I have a question about the output output when executing the "python run/train_3d.py --cfg configs/panoptic/resnet50/prn64_cpn80x80x20_960x512_cam5.yaml" command.
Why are there no results for all val data?
(There are only 10 validation results in output/.../image_with_joints/
validation_0000000_view_1_gt ~ validation_0000000_view_5_gt
validation_0000002_view_1_gt ~ validation_0000002_view_5_gt)
For each dataset there is a pretrined Backbone
aka:
pose_resnet50_panoptic.pth.tar
pred_shelf_maskrcnn_hrnet_coco.pkl
pred_campus_maskrcnn_hrnet_coco.pkl
What are this for ?
Thank you for this excellent work.
I have some doubts about the tracking effect in the visualization results (each person is assigned a color to represent his ID). I didn't find in the paper or the code how you tracking subjects. Can you explain which tracking method was used?
Thanks for sharing your nice work!
I notice you use two different formulas to trasnsform 3D pose from the world coordinate to the camera coordinate when processing shelf and CMU panoptic datasets, i.e. x = np.dot(R, X) + t and x = R.dot(X - t). Why not use the same formula?
I want to know which network do you use to get pred_shelf_maskrcnn_hrnet_coco.pkl?
In your prn64_cpn80x80x20.yaml, why your HEATMAP_SIZE set the vaule:
Download the pretrained backbone model from pretrained backbone and place it here: ${POSE_ROOT}/models/pose_resnet50_panoptic.pth.tar (ResNet-50 pretrained on COCO dataset and finetuned jointly on Panoptic dataset and MPII).
Could you please add a download link, thank you very much
If I want to predict poses under different camera settings, I must re-train the NN?
Hi, While trying to run train_3d.py I get a segmentation fault. I'm not sure where to start the debug. Could you please guide me as to what could be wrong with my environment? I've pasted my conda environment packages below:
_libgcc_mutex 0.1 main defaults
_pytorch_select 0.2 gpu_0 defaults
blas 1.0 mkl defaults
bzip2 1.0.8 h7b6447c_0 defaults
ca-certificates 2020.7.22 0 defaults
cairo 1.14.12 h8948797_3 defaults
certifi 2020.6.20 py36_0 defaults
cffi 1.14.2 py36he30daa8_0 defaults
cudatoolkit 10.1.243 h6bb024c_0 defaults
cudnn 7.6.5 cuda10.1_0 defaults
cycler 0.10.0 py36_0 defaults
dbus 1.13.16 hb2f20db_0 defaults
easydict 1.9 py_0 conda-forge
expat 2.2.9 he6710b0_2 defaults
ffmpeg 4.0 hcdf2ecd_0 defaults
fontconfig 2.13.0 h9420a91_0 defaults
freeglut 3.0.0 hf484d3e_5 defaults
freetype 2.10.2 h5ab3b9f_0 defaults
glib 2.65.0 h3eb4bd4_0 defaults
graphite2 1.3.14 h23475e2_0 defaults
gst-plugins-base 1.14.0 hbbd80ab_1 defaults
gstreamer 1.14.0 hb31296c_0 defaults
harfbuzz 1.8.8 hffaf4a1_0 defaults
hdf5 1.10.2 hba1933b_1 defaults
icu 58.2 he6710b0_3 defaults
intel-openmp 2020.2 254 defaults
jasper 2.0.14 h07fcdf6_1 defaults
jpeg 9b h024ee3a_2 defaults
json_tricks 3.13.5 py_0 conda-forge
kiwisolver 1.2.0 py36hfd86e86_0 defaults
lcms2 2.11 h396b838_0 defaults
ld_impl_linux-64 2.33.1 h53a641e_7 defaults
libedit 3.1.20191231 h14c3975_1 defaults
libffi 3.3 he6710b0_2 defaults
libgcc-ng 9.1.0 hdf63c60_0 defaults
libgfortran-ng 7.3.0 hdf63c60_0 defaults
libglu 9.0.0 hf484d3e_1 defaults
libopencv 3.4.2 hb342d67_1 defaults
libopus 1.3.1 h7b6447c_0 defaults
libpng 1.6.37 hbc83047_0 defaults
libprotobuf 3.5.1 h6f1eeef_0 defaults
libstdcxx-ng 9.1.0 hdf63c60_0 defaults
libtiff 4.1.0 h2733197_1 defaults
libuuid 1.0.3 h1bed415_2 defaults
libvpx 1.7.0 h439df22_0 defaults
libxcb 1.14 h7b6447c_0 defaults
libxml2 2.9.10 he19cac6_1 defaults
lz4-c 1.9.2 he6710b0_1 defaults
matplotlib 3.3.1 0 defaults
matplotlib-base 3.3.1 py36h817c723_0 defaults
mkl 2020.2 256 defaults
mkl-service 2.3.0 py36he904b0f_0 defaults
mkl_fft 1.1.0 py36h23d657b_0 defaults
mkl_random 1.1.1 py36h0573a6f_0 defaults
ncurses 6.2 he6710b0_1 defaults
ninja 1.10.1 py36hfd86e86_0 defaults
numpy 1.19.1 py36hbc911f0_0 defaults
numpy-base 1.19.1 py36hfa32c7d_0 defaults
olefile 0.46 py36_0 defaults
opencv 3.4.2 py36h6fd60c2_1 defaults
openssl 1.1.1g h7b6447c_0 defaults
pandas 1.1.1 py36he6710b0_0 defaults
pcre 8.44 he6710b0_0 defaults
pillow 7.2.0 py36hb39fc2d_0 defaults
pip 20.2.2 py36_0 defaults
pixman 0.40.0 h7b6447c_0 defaults
prettytable 0.7.2 py_3 conda-forge
protobuf 3.5.1 py36_3 conda-forge
py-opencv 3.4.2 py36hb342d67_1 defaults
pycparser 2.20 py_2 defaults
pyparsing 2.4.7 py_0 defaults
pyqt 5.9.2 py36h05f1152_2 defaults
python 3.6.12 hcff3b4d_2 defaults
python-dateutil 2.8.1 py_0 defaults
pytorch 1.4.0 cuda101py36h02f0884_0 defaults
pytz 2020.1 py_0 defaults
pyyaml 5.3.1 py36h7b6447c_1 defaults
qt 5.9.7 h5867ecd_1 defaults
readline 8.0 h7b6447c_0 defaults
scipy 1.5.2 py36h0b6359f_0 defaults
setuptools 49.6.0 py36_0 defaults
sip 4.19.8 py36hf484d3e_0 defaults
six 1.15.0 py_0 defaults
sqlite 3.33.0 h62c20be_0 defaults
tensorboardx 2.0 py_0 conda-forge
tk 8.6.10 hbc83047_0 defaults
torchvision 0.5.0 py36_cu101 pytorch
tornado 6.0.4 py36h7b6447c_1 defaults
tqdm 4.48.2 py_0 defaults
wheel 0.35.1 py_0 defaults
xz 5.2.5 h7b6447c_0 defaults
yaml 0.2.5 h7b6447c_0 defaults
zlib 1.2.11 h7b6447c_3 defaults
zstd 1.4.5 h9ceee32_0 defaults
Any help would be appreciated. Thanks!
How to set the code to realize the experiment in your ECCV 2020 paper? Thx
Hello I know this has been answered before but I don't get how you define the space center. Let me explain.
In campus dataset the x,z coords in the original dataset are the following. (-4.9,11.2),(-1.78,5.22) and (4.9,6.68) in meters. I don't quite understand how you define a 12x12m box around these coords. Also the, space center should be around (0,8) (meters).
Could you please enlighten me ?
Same is the case if I instead get the coordinates from the -dot(R.T,T). The coordinates are the following: (-6.2,5.2) , (1.77,-5.05), (11.7,-1.8), I still can't see how the bounding box should be 12x12 and the space center 3,4.5.
Thanks.
发生异常: RuntimeError
Expected object of scalar type Float but got scalar type Double for argument #2 'other'
File "/home/wu/voxelpose-pytorch/lib/models/project_layer.py", line 80, in get_voxel
bounding[i, 0, 0, :, c] = (xy[:, 0] >= 0) & (xy[:, 1] >= 0) & (xy[:, 0] < width) & (xy[:, 1] < height)
File "/home/wu/voxelpose-pytorch/lib/models/project_layer.py", line 107, in forward
cubes, grids = self.get_voxel(heatmaps, meta, grid_size, grid_center, cube_size)
File "/home/wu/voxelpose-pytorch/lib/models/cuboid_proposal_net.py", line 103, in forward
self.grid_size, [self.grid_center], self.cube_size)
File "/home/wu/voxelpose-pytorch/lib/models/multi_person_posenet.py", line 65, in forward
root_cubes, grid_centers = self.root_net(all_heatmaps, meta)
File "/home/wu/voxelpose-pytorch/lib/core/function.py", line 126, in validate_3d
input_heatmaps=input_heatmap)
File "/home/wu/voxelpose-pytorch/run/train_3d.py", line 135, in main
precision = validate_3d(config, model, test_loader, final_output_dir)
File "/home/wu/voxelpose-pytorch/run/train_3d.py", line 161, in
main()
Error after python run/train_3d.py --cfg configs/shelf/prn64_cpn80x80x20.yaml
=> load /media/user/LaCie/DataSet/voxel-data/data/Shelf/pred_shelf_maskrcnn_hrnet_coco.pkl
Traceback (most recent call last):
File "run/train_3d.py", line 160, in <module>
main()
File "run/train_3d.py", line 87, in main
test_dataset = eval('dataset.' + config.DATASET.TEST_DATASET)(
File "/home/user/voxelpose-pytorch/run/../lib/dataset/shelf.py", line 70, in __init__
self.db = self._get_db()
File "/home/user/voxelpose-pytorch/run/../lib/dataset/shelf.py", line 92, in _get_db
actor_3d = np.array(np.array(data['actor3D'].tolist()).tolist()).squeeze() # num_person * num_frame
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 4 dimensions. The detected shape was (1, 4, 3200, 1) + inhomogeneous part.
My pip list
:
pip list
Package Version Editable project location
addict 2.4.0
aliyun-python-sdk-core 2.13.36
aliyun-python-sdk-kms 2.16.1
attrs 23.1.0
brotlipy 0.7.0
certifi 2023.7.22
cffi 1.15.1
charset-normalizer 2.1.1
chumpy 0.70
click 8.1.7
colorama 0.4.6
contourpy 1.1.0
coverage 7.3.0
crcmod 1.7
cryptography 38.0.3
cycler 0.11.0
Cython 3.0.2
easydict 1.10
exceptiongroup 1.1.3
filelock 3.12.3
flake8 6.1.0
fonttools 4.42.1
idna 3.4
importlib-metadata 6.8.0
importlib-resources 6.0.1
iniconfig 2.0.0
interrogate 1.5.0
isort 4.3.21
Jinja2 3.1.2
jmespath 0.10.0
json-tricks 3.17.3
kiwisolver 1.4.5
Markdown 3.4.4
markdown-it-py 3.0.0
MarkupSafe 2.1.3
matplotlib 3.7.2
mccabe 0.7.0
mdurl 0.1.2
mmcv 2.0.0rc4
mmdet 3.1.0
mmengine 0.8.4
mmpose 1.1.0 /home/user/mmpose
model-index 0.1.11
mpmath 1.3.0
munkres 1.1.4
networkx 3.1
numpy 1.24.4
opencv-python 4.8.0.76
opendatalab 0.0.10
openmim 0.3.9
openxlab 0.0.23
ordered-set 4.1.0
oss2 2.17.0
packaging 23.1
pandas 2.0.3
parameterized 0.9.0
pbr 5.11.1
Pillow 9.2.0
pip 23.2.1
platformdirs 3.10.0
pluggy 1.3.0
protobuf 4.24.2
py 1.11.0
pycocotools 2.0.7
pycodestyle 2.11.0
pycparser 2.21
pycryptodome 3.18.0
pyflakes 3.1.0
Pygments 2.16.1
pyOpenSSL 22.1.0
pyparsing 3.0.9
PySocks 1.7.1
pytest 7.4.1
pytest-runner 6.0.0
python-dateutil 2.8.2
pytz 2023.3.post1
PyYAML 6.0.1
requests 2.28.2
rich 13.4.2
scipy 1.10.1
setuptools 68.1.2
shapely 2.0.1
six 1.16.0
sympy 1.12
tabulate 0.9.0
tensorboardX 2.6.2.2
termcolor 2.3.0
terminaltables 3.1.10
testresources 2.0.1
toml 0.10.2
tomli 2.0.1
torch 1.11.0
torchvision 0.12.0a0+9b5a3fe
tqdm 4.65.2
typing_extensions 4.7.1
tzdata 2023.3
urllib3 1.26.11
wheel 0.38.4
xdoctest 1.1.1
xtcocotools 1.14
yapf 0.40.1
zipp 3.16.2
Machine: Jetson Orin AGX
Hi guys,
first of all, this is really great work you did there. Thanks a lot!
I have a few question about your work. I'm really looking forward to your reply.
All the best!
I'm searching for the explication on how you generated the synthetic heatmaps. Eventhough, the code is written in a good coding style and mostly very understandable, at some points comments would have been very helpful. It would also be a great help if you could go more into detail how and why you generated synthetic heatmaps. Thank you! :)
In your paper I'm missing a discussion on the following question:
In the panoptic dataset all cameras are in equal distance, and eventhough you chose random cameras for training and testing the cameras stay in a similar configuration (distance and direction) to the scene. Would it be possible to perhaps test an network, which has been trained on the panoptic dataset, on the campus data? This would show real generalization capabilites.
In your code it seems to be not absolutely clear to me, if the meta data, especially the number of persons in the image, is completely decoupled from the forward call of the model. Perhaps it would be good to give a maximum number of persons the network has to check for. Currently it uses - if I got it right - the meta data information. I'd be happy, if you could explain the meta
data more in detailed; what it is and what is it used for?
Thanks in advance! :)
Hi, I am using the panoptic dataset, and I am wondering if it is possible to train the heatmap predictor with this dataset?
You explicitly mention on the readme, that you have processed the camera parameters to your formats. Can you explain what process was made? Particularly, on the camera translation parameter?
From the original Campus dataset one can obtain the translation T for each camera, but it is way too different from the ones you provide in the json calib file. As example I got T=[-1.787557e+00, 1.361094e+00, 5.226973e+00] from the original data for cam 0, but in the json file you use T = [1774.89, -5051.69, 1923.35]. What special consideration should be made to obtain such values? How should they be interpreted?
I would appreciate if you can elaborate more on it.
Thank you for your time!!
Thanks for your great work! And the visualization video is just amazing!
I wonder how you plot and visualize the results of your predicted poses? Do you have the corresponding source code for that?
First of all, thanks for making this awesome work public.
I don't understand what is the meaning of the accumulation steps.
Why would the loss be backpropagated every n steps for the 1d 2d and bbox errors?
Did this come from empirical testing?
I am not sure this is a correct approach as well, regarding the software side, as the parameters will most luckily change by then from the joint loss backprop, evident from the runtime errors in torch>1.4.0.
When I run item on linux, it comes this error. But I can train on windows , I don't konw how to deal with this problem. THX
Traceback (most recent call last):
File "run/train_3d.py", line 160, in
main()
File "run/train_3d.py", line 133, in main
train_3d(config, model, optimizer, train_loader, epoch, final_output_dir, writer_dict)
File "/mnt/e/voxelpose-pytorch-main/run/../lib/core/function.py", line 44, in train_3d
targets_3d=targets_3d[0])
IndexError: list index out of range
Thanks for such a great repo. Will you release 3D visualization code for this repo same as Dong released visualization code?
when i trained the model on campus datasets and met such problem. and i use the torch1.7, cuda 11.1. And the training strategy in the code seems be different from the strategy given in the paper.
Traceback (most recent call last):
File "run/train_3d.py", line 163, in
main()
File "run/train_3d.py", line 136, in main
train_3d(config, model, optimizer, train_loader, epoch, final_output_dir, writer_dict)
File "/home/gw/Project/voxelpose/lib/core/function.py", line 68, in train_3d
accu_loss_3d.backward()
File "/home/gw/anaconda3/envs/VIBE/lib/python3.7/site-packages/torch/tensor.py", line 221, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/gw/anaconda3/envs/VIBE/lib/python3.7/site-packages/torch/autograd/init.py", line 132, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [1, 32, 1, 1, 1]] is at version 8; expected version 6 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
when i trained the model on cmu datasets with multiple gpus the dataloader function encontered the followed problem. but it's worked with single gpu.
Traceback (most recent call last):
File "run/train_3d.py", line 163, in
main()
File "run/train_3d.py", line 136, in main
train_3d(config, model, optimizer, train_loader, epoch, final_output_dir, writer_dict)
File "/home/gw/Project/voxelpose/lib/core/function.py", line 37, in train_3d
for i, (inputs, targets_2d, weights_2d, targets_3d, meta, input_heatmap) in enumerate(loader):
File "/home/gw/anaconda3/envs/VIBE/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 435, in next
data = self._next_data()
File "/home/gw/anaconda3/envs/VIBE/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1065, in _next_data
return self._process_data(data)
File "/home/gw/anaconda3/envs/VIBE/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1111, in _process_data
data.reraise()
File "/home/gw/anaconda3/envs/VIBE/lib/python3.7/site-packages/torch/_utils.py", line 428, in reraise
raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in DataLoader worker process 3.
Original Traceback (most recent call last):
File "/home/gw/anaconda3/envs/VIBE/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop
data = fetcher.fetch(index)
File "/home/gw/anaconda3/envs/VIBE/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
return self.collate_fn(data)
File "/home/gw/anaconda3/envs/VIBE/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 83, in default_collate
return [default_collate(samples) for samples in transposed]
File "/home/gw/anaconda3/envs/VIBE/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 83, in
return [default_collate(samples) for samples in transposed]
File "/home/gw/anaconda3/envs/VIBE/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 81, in default_collate
raise RuntimeError('each element in list of batch should be of equal size')
RuntimeError: each element in list of batch should be of equal size
There is a data file called "panoptic_training_pose.pkl" in your repository. I wonder if it is provided by CMU panoptic-toolbox or you make it by yourself. If you do it by yourself. May I know how do you make it or can you tell me the meaning of the keywords like "joints_3d_vis" and "joints_vis" ? Thanks and look forward to your reply.
Thanks for your great project. I am wondering how you generate these files calibration_shelf.json
and calibration_campus.json
from the related official Shelf and Campus dataset files. Could you please offer the related script to transform the official calibration data into your format? Thanks in advance.
Thanks for your great work. Recently, I was reproducing your experiments. The best result I got, for now, is just 96.5. But in your paper, the result on the Campus dataset you report was 96.7 in PCP.
So I was wondering which config you use to produce this result?
Thanks for your reply in advance.
Hi, there
Thanks for your great work!
Now I'm trying to develop a monocular model comparable to yours (with panoptic-studio dataset)
And I wonder the exact setting of your monocular experiment.
What camera did you use for Training/Validation?
Thank you
I'm looking for code that helps in visualizing the output for a subset of images. Are you planning to release that anytime soon?
Thanks!
In the paper, you said you split campus dataset into training and testing subsets, but in the code campus/shelf datasets are only used for testing. And I'm curious about how you generated the synthetic data panoptic_training_pose.pkl.
pretrained backbone model can't be downloaded. Can you share other link?@CHUNYUWANG
Hello,
I wanted to know how to set the camera parameters when I am using a 4 camera setup installed on the 4 corners of the wall in a room.
Thanks for your work!
May I hava a question about pretrained pose-resnet backbone setting?
Whan I check pose_resnet50_panoptic.pth.tar, its number of the joints is 18.
However, the number of COCO (OpenPose version) keypoints is 18, the number of MPII keypoints is 16, the number of Panoptic is 19.
All are not same.
How do mapping other kind of keypoints?
Would you provide mapping function to me? Or Can I know the setting about this in detail?
As I guess about the training of backbone based on describing in paper,
first, load the COCO (18 keypoints) pretrained model of Pose-ResNet,
second, mapping or eliminating the keypoints of MPII/Panoptic suited to COCO keypoitns format.
Thanks for your contributions!
I want to know which network do you use to get pred_shelf_maskrcnn_hrnet_coco.pkl?
In your prn64_cpn80x80x20.yaml, why your HEATMAP_SIZE set the vaule:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.