Official repository for the paper "ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation"
Home Page: https://arxiv.org/abs/1910.02550
License: Apache License 2.0
I'm currently working on a research project that makes use of your awesome paper/code.
I'm just curious to know how many epochs it took you to train the 3 networks from scratch?
I mean the results you reported in the paper after how many epochs as it's not clear.
Thanks
I.e., instead of as a 3D shape estimation. Or, do you know of codebases that support object detection?
Hello,
first off, thank you very much for your work.
I tried retraining the segmentation network for higher resolution (800x600), however, I have failed to make the training work, even for the default 256x256 resolution. Specifically, I have downloaded your training dataset and tried to continue training from your checkpoint_mask.pth. However, these are the results after the first epoch:
Train Epoch Loss: 11751.5604
Train mIoU: 0.0000Validation Epoch Loss: 704898596864.0000
Validation mIoU: 0.0000Test Synthetic Epoch Loss: 3180020034638.7690
Test Synthetic mIoU: 0.0000
Visually, the network produces just black frames, whereas your trained checkpoint works as expected. I have tried a similar approach for the surface normal prediction network (again using your pre-trained weights and just increasing the resolution) and there it worked fine.
I believe there is some problem with loss function / training code for this mask network, but I have failed to pinpoint it in the code.
I would appreciate any help, thank you.
In READE.md:
Download the data:
a) Model Checkpoints (0.9GB) - Trained checkpoints of our 3 deeplabv3+ models.
b) Train dataset (Optional, 72GB) - Contains the synthetic images used for training the models. No real images were used for training.
c) Val + Test datasets (Optional, 1.7GB) - Contains the real and synthetic images used for validation and testing.
And I realised that in the sample config, it is specified as drn:
17 # Surface Normals Model
18 normals:
19 model: 'drn' # Which model to use for inference. Possible values: ['unet', 'deeplab_resnet']
20 pathWeightsFile: '../data/cleargrasp-checkpoints/surface_normals/checkpoint_normals.pth' # Path to the checkpoint to be loaded
21 inferenceHeight: 256
22 inferenceWidth: 256Is the Model checkpoints (provided in README) drn?
Is drn different from unet or deeplab_resnet?
Hi. Thank you for introducing your excellent work and database.
I find that there is only rectified depth images for the synthetic data (in depth-imgs-rectified ). For the real data, there are both transparent-depth-im (raw) and opaque-depth-img (rectified). Can you provide the raw depth images for the synthetic data?
Thank you for introducing your excellent work and database.
I notice that you pre-trained your normal estimation module using additional Matterport3D and ScanNet datasets in order to enhance the performance in non-transparent objects. Did you also train the segmentation masks with mixed non-transparent objects + transparent objects?
I am asking this because I found some false positive masks produced on non-transparent objects when I try your transparent object segmentation module.
Looking forward to your reply!
Thanks!
Hi, I'm using a D415, with
RGB size: 640480
Depth size: 7201280
and depth_completion() return image/depth size (depthcomplete.input_image and depthcomplete.input_depth) of 144*256 ....
Can't even see very clearly....
Hello, I would like to consult the method of synthesizing image, especially the method of synthesizing depth map.
Hi, Shreeyak
I have some simple questions related to the image size and camera matrix.
Q1. Can I ask why you only resize the height of the image (256 -> 144)??
Can I ask why not use the origin output of the network (256, 256) size?
Q2. Can I ask how to get the camera matrix value??
For evaluation, You use the different camera d415, d435.
How to know all the images camera intrinsic matrix(included fx, fy) included(training, validation and testset)?
It seems like used for the making the ground truth surface normal.
Could you explain how to get image camera intrinsic matrix?
cleargrasp/eval_depth_completion/config/config.yaml.sample
Lines 41 to 46 in f49f389
In example:
If using matterport3d's undistorted_color.jpg and undistorted_depth.png
I have to change xres, yres, fx, fy, cx, cy.
Did i miss anything else?
Because i got output depth only in red and black color like this: Live Demo Picture
Hi, I cannot the Download 3D Models, the link is not working now.
Hi, I cannot download the 3D Models, the link is not working now. Who can provide a useful link for me? Thanks.
I have trained surface normal network, but there are some warnings,as below:
Hello, when dealing with pose estimation, I find the number of variants in json file may be more than the number in pictures, as mentioned in Readme
Some objects are present in the scene, but not visible to the camera. Your code will have to account for such objects when parsing through the data, using the provided masks.
Could you please provide filters or something else... I'm a fresh man in this area, any help would be greatly appreciated.
When I ran python test.py -c config/config.yaml , I got the following error.
RuntimeError: Error(s) in loading state_dict for DeepLab:
Missing key(s) in state_dict: "backbone.layer0.0.weight", "backbone.layer0.1.weight", "backbone.layer0.1.bias", "backbone.layer0.1.running_mean", "backbone.layer0.1.running_var", "backbone.layer1.0.weight", "backbone.layer1.1.weight", "backbone.layer1.1.bias", "backbone.layer1.1.running_mean", "backbone.layer1.1.running_var", "backbone.layer2.0.weight", "backbone.layer2.1.weight", "backbone.layer2.1.bias", "backbone.layer2.1.running_mean", "backbone.layer2.1.running_var", "backbone.layer3.0.conv1.weight", "backbone.layer3.0.bn1.weight", "backbone.layer3.0.bn1.bias", "backbone.layer3.0.bn1.running_mean", "backbone.layer3.0.bn1.running_var", "backbone.layer3.0.conv2.weight", "backbone.layer3.0.bn2.weight", "backbone.layer3.0.bn2.bias", "backbone.layer3.0.bn2.running_mean", "backbone.layer3.0.bn2.running_var", "backbone.layer3.0.conv3.weight", "backbone.layer3.0.bn3.weight", "backbone.layer3.0.bn3.bias", "backbone.layer3.0.bn3.running_mean", "backbone.layer3.0.bn3.running_var", "backbone.layer3.0.downsample.0.weight", "backbone.layer3.0.downsample.1.weight", "backbone.layer3.0.downsample.1.bias", "backbone.layer3.0.downsample.1.running_mean", "backbone.layer3.0.downsample.1.running_var", "backbone.layer3.1.conv1.weight", "backbone.layer3.1.bn1.weight", "backbone.layer3.1.bn1.bias", "backbone.layer3.1.bn1.running_mean", "backbone.layer3.1.bn1.running_var", "backbone.layer3.1.conv2.weight", "backbone.layer3.1.bn2.weight", "backbone.layer3.1.bn2.bias", "backbone.layer3.1.bn2.running_mean", "backbone.layer3.1.bn2.running_var", "backbone.layer3.1.conv3.weight", "backbone.layer3.1.bn3.weight", "backbone.layer3.1.bn3.bias", "backbone.layer3.1.bn3.running_mean", "backbone.layer3.1.bn3.running_var", "backbone.layer3.2.conv1.weight", "backbone.layer3.2.bn1.weight", "backbone.layer3.2.bn1.bias", "backbone.layer3.2.bn1.running_mean", "backbone.layer3.2.bn1.running_var", "backbone.layer3.2.conv2.weight", "backbone.layer3.2.bn2.weight", "backbone.layer3.2.bn2.bias", "backbone.layer3.2.bn2.running_mean", "backbone.layer3.2.bn2.running_var", "backbone.layer3.2.conv3.weight", "backbone.layer3.2.bn3.weight", "backbone.layer3.2.bn3.bias", "backbone.layer3.2.bn3.running_mean", "backbone.layer3.2.bn3.running_var", "backbone.layer4.0.conv1.weight", "backbone.layer4.0.bn1.weight", "backbone.layer4.0.bn1.bias", "backbone.layer4.0.bn1.running_mean", "backbone.layer4.0.bn1.running_var", "backbone.layer4.0.conv2.weight", "backbone.layer4.0.bn2.weight", "backbone.layer4.0.bn2.bias", "backbone.layer4.0.bn2.running_mean", "backbone.layer4.0.bn2.running_var", "backbone.layer4.0.conv3.weight", "backbone.layer4.0.bn3.weight", "backbone.layer4.0.bn3.bias", "backbone.layer4.0.bn3.running_mean", "backbone.layer4.0.bn3.running_var", "backbone.layer4.0.downsample.0.weight", "backbone.layer4.0.downsample.1.weight", "backbone.layer4.0.downsample.1.bias", "backbone.layer4.0.downsample.1.running_mean", "backbone.layer4.0.downsample.1.running_var", "backbone.layer4.1.conv1.weight", "backbone.layer4.1.bn1.weight", "backbone.layer4.1.bn1.bias", "backbone.layer4.1.bn1.running_mean", "backbone.layer4.1.bn1.running_var", "backbone.layer4.1.conv2.weight", "backbone.layer4.1.bn2.weight", "backbone.layer4.1.bn2.bias", "backbone.layer4.1.bn2.running_mean", "backbone.layer4.1.bn2.running_var", "backbone.layer4.1.conv3.weight", "backbone.layer4.1.bn3.weight", "backbone.layer4.1.bn3.bias", "backbone.layer4.1.bn3.running_mean", "backbone.layer4.1.bn3.running_var", "backbone.layer4.2.conv1.weight", "backbone.layer4.2.bn1.weight", "backbone.layer4.2.bn1.bias", "backbone.layer4.2.bn1.running_mean", "backbone.layer4.2.bn1.running_var", "backbone.layer4.2.conv2.weight", "backbone.layer4.2.bn2.weight", "backbone.layer4.2.bn2.bias", "backbone.layer4.2.bn2.running_mean", "backbone.layer4.2.bn2.running_var", "backbone.layer4.2.conv3.weight", "backbone.layer4.2.bn3.weight", "backbone.layer4.2.bn3.bias", "backbone.layer4.2.bn3.running_mean", "backbone.layer4.2.bn3.running_var", "backbone.layer4.3.conv1.weight", "backbone.layer4.3.bn1.weight", "backbone.layer4.3.bn1.bias", "backbone.layer4.3.bn1.running_mean", "backbone.layer4.3.bn1.running_var", "backbone.layer4.3.conv2.weight", "backbone.layer4.3.bn2.weight", "backbone.layer4.3.bn2.bias", "backbone.layer4.3.bn2.running_mean", "backbone.layer4.3.bn2.running_var", "backbone.layer4.3.conv3.weight", "backbone.layer4.3.bn3.weight", "backbone.layer4.3.bn3.bias", "backbone.layer4.3.bn3.running_mean", "backbone.layer4.3.bn3.running_var", "backbone.layer5.0.conv1.weight", "backbone.layer5.0.bn1.weight", "backbone.layer5.0.bn1.bias", "backbone.layer5.0.bn1.running_mean", "backbone.layer5.0.bn1.running_var", "backbone.layer5.0.conv2.weight", "backbone.layer5.0.bn2.weight", "backbone.layer5.0.bn2.bias", "backbone.layer5.0.bn2.running_mean", "backbone.layer5.0.bn2.running_var", "backbone.layer5.0.conv3.weight", "backbone.layer5.0.bn3.weight", "backbone.layer5.0.bn3.bias", "backbone.layer5.0.bn3.running_mean", "backbone.layer5.0.bn3.running_var", "backbone.layer5.0.downsample.0.weight", "backbone.layer5.0.downsample.1.weight", "backbone.layer5.0.downsample.1.bias", "backbone.layer5.0.downsample.1.running_mean", "backbone.layer5.0.downsample.1.running_var", "backbone.layer5.1.conv1.weight", "backbone.layer5.1.bn1.weight", "backbone.layer5.1.bn1.bias", "backbone.layer5.1.bn1.running_mean", "backbone.layer5.1.bn1.running_var", "backbone.layer5.1.conv2.weight", "backbone.layer5.1.bn2.weight", "backbone.layer5.1.bn2.bias", "backbone.layer5.1.bn2.running_mean", "backbone.layer5.1.bn2.running_var", "backbone.layer5.1.conv3.weight", "backbone.layer5.1.bn3.weight", "backbone.layer5.1.bn3.bias", "backbone.layer5.1.bn3.running_mean", "backbone.layer5.1.bn3.running_var", "backbone.layer5.2.conv1.weight", "backbone.layer5.2.bn1.weight", "backbone.layer5.2.bn1.bias", "backbone.layer5.2.bn1.running_mean", "backbone.layer5.2.bn1.running_var", "backbone.layer5.2.conv2.weight", "backbone.layer5.2.bn2.weight", "backbone.layer5.2.bn2.bias", "backbone.layer5.2.bn2.running_mean", "backbone.layer5.2.bn2.running_var", "backbone.layer5.2.conv3.weight", "backbone.layer5.2.bn3.weight", "backbone.layer5.2.bn3.bias", "backbone.layer5.2.bn3.running_mean", "backbone.layer5.2.bn3.running_var", "backbone.layer5.3.conv1.weight", "backbone.layer5.3.bn1.weight", "backbone.layer5.3.bn1.bias", "backbone.layer5.3.bn1.running_mean", "backbone.layer5.3.bn1.running_var", "backbone.layer5.3.conv2.weight", "backbone.layer5.3.bn2.weight", "backbone.layer5.3.bn2.bias", "backbone.layer5.3.bn2.running_mean", "backbone.layer5.3.bn2.running_var", "backbone.layer5.3.conv3.weight", "backbone.layer5.3.bn3.weight", "backbone.layer5.3.bn3.bias", "backbone.layer5.3.bn3.running_mean", "backbone.layer5.3.bn3.running_var", "backbone.layer5.4.conv1.weight", "backbone.layer5.4.bn1.weight", "backbone.layer5.4.bn1.bias", "backbone.layer5.4.bn1.running_mean", "backbone.layer5.4.bn1.running_var", "backbone.layer5.4.conv2.weight", "backbone.layer5.4.bn2.weight", "backbone.layer5.4.bn2.bias", "backbone.layer5.4.bn2.running_mean", "backbone.layer5.4.bn2.running_var", "backbone.layer5.4.conv3.weight", "backbone.layer5.4.bn3.weight", "backbone.layer5.4.bn3.bias", "backbone.layer5.4.bn3.running_mean", "backbone.layer5.4.bn3.running_var", "backbone.layer5.5.conv1.weight", "backbone.layer5.5.bn1.weight", "backbone.layer5.5.bn1.bias", "backbone.layer5.5.bn1.running_mean", "backbone.layer5.5.bn1.running_var", "backbone.layer5.5.conv2.weight", "backbone.layer5.5.bn2.weight", "backbone.layer5.5.bn2.bias", "backbone.layer5.5.bn2.running_mean", "backbone.layer5.5.bn2.running_var", "backbone.layer5.5.conv3.weight", "backbone.layer5.5.bn3.weight", "backbone.layer5.5.bn3.bias", "backbone.layer5.5.bn3.running_mean", "backbone.layer5.5.bn3.running_var", "backbone.layer6.0.conv1.weight", "backbone.layer6.0.bn1.weight", "backbone.layer6.0.bn1.bias", "backbone.layer6.0.bn1.running_mean", "backbone.layer6.0.bn1.running_var", "backbone.layer6.0.conv2.weight", "backbone.layer6.0.bn2.weight", "backbone.layer6.0.bn2.bias", "backbone.layer6.0.bn2.running_mean", "backbone.layer6.0.bn2.running_var", "backbone.layer6.0.conv3.weight", "backbone.layer6.0.bn3.weight", "backbone.layer6.0.bn3.bias", "backbone.layer6.0.bn3.running_mean", "backbone.layer6.0.bn3.running_var", "backbone.layer6.0.downsample.0.weight", "backbone.layer6.0.downsample.1.weight", "backbone.layer6.0.downsample.1.bias", "backbone.layer6.0.downsample.1.running_mean", "backbone.layer6.0.downsample.1.running_var", "backbone.layer6.1.conv1.weight", "backbone.layer6.1.bn1.weight", "backbone.layer6.1.bn1.bias", "backbone.layer6.1.bn1.running_mean", "backbone.layer6.1.bn1.running_var", "backbone.layer6.1.conv2.weight", "backbone.layer6.1.bn2.weight", "backbone.layer6.1.bn2.bias", "backbone.layer6.1.bn2.running_mean", "backbone.layer6.1.bn2.running_var", "backbone.layer6.1.conv3.weight", "backbone.layer6.1.bn3.weight", "backbone.layer6.1.bn3.bias", "backbone.layer6.1.bn3.running_mean", "backbone.layer6.1.bn3.running_var", "backbone.layer6.2.conv1.weight", "backbone.layer6.2.bn1.weight", "backbone.layer6.2.bn1.bias", "backbone.layer6.2.bn1.running_mean", "backbone.layer6.2.bn1.running_var", "backbone.layer6.2.conv2.weight", "backbone.layer6.2.bn2.weight", "backbone.layer6.2.bn2.bias", "backbone.layer6.2.bn2.running_mean", "backbone.layer6.2.bn2.running_var", "backbone.layer6.2.conv3.weight", "backbone.layer6.2.bn3.weight", "backbone.layer6.2.bn3.bias", "backbone.layer6.2.bn3.running_mean", "backbone.layer6.2.bn3.running_var", "backbone.layer7.0.weight", "backbone.layer7.1.weight", "backbone.layer7.1.bias", "backbone.layer7.1.running_mean", "backbone.layer7.1.running_var", "backbone.layer8.0.weight", "backbone.layer8.1.weight", "backbone.layer8.1.bias", "backbone.layer8.1.running_mean", "backbone.layer8.1.running_var", "aspp.aspp1.atrous_conv.weight", "aspp.aspp1.bn.weight", "aspp.aspp1.bn.bias", "aspp.aspp1.bn.running_mean", "aspp.aspp1.bn.running_var", "aspp.aspp2.atrous_conv.weight", "aspp.aspp2.bn.weight", "aspp.aspp2.bn.bias", "aspp.aspp2.bn.running_mean", "aspp.aspp2.bn.running_var", "aspp.aspp3.atrous_conv.weight", "aspp.aspp3.bn.weight", "aspp.aspp3.bn.bias", "aspp.aspp3.bn.running_mean", "aspp.aspp3.bn.running_var", "aspp.aspp4.atrous_conv.weight", "aspp.aspp4.bn.weight", "aspp.aspp4.bn.bias", "aspp.aspp4.bn.running_mean", "aspp.aspp4.bn.running_var", "aspp.global_avg_pool.1.weight", "aspp.global_avg_pool.2.weight", "aspp.global_avg_pool.2.bias", "aspp.global_avg_pool.2.running_mean", "aspp.global_avg_pool.2.running_var", "aspp.conv1.weight", "aspp.bn1.weight", "aspp.bn1.bias", "aspp.bn1.running_mean", "aspp.bn1.running_var", "decoder.conv1.weight", "decoder.bn1.weight", "decoder.bn1.bias", "decoder.bn1.running_mean", "decoder.bn1.running_var", "decoder.last_conv.0.weight", "decoder.last_conv.1.weight", "decoder.last_conv.1.bias", "decoder.last_conv.1.running_mean", "decoder.last_conv.1.running_var", "decoder.last_conv.4.weight", "decoder.last_conv.5.weight", "decoder.last_conv.5.bias", "decoder.last_conv.5.running_mean", "decoder.last_conv.5.running_var", "decoder.last_conv.7.weight", "decoder.last_conv.7.bias".
Unexpected key(s) in state_dict: "module.backbone.layer0.0.weight", "module.backbone.layer0.1.weight", "module.backbone.layer0.1.bias", "module.backbone.layer0.1.running_mean", "module.backbone.layer0.1.running_var", "module.backbone.layer0.1.num_batches_tracked", "module.backbone.layer1.0.weight", "module.backbone.layer1.1.weight", "module.backbone.layer1.1.bias", "module.backbone.layer1.1.running_mean", "module.backbone.layer1.1.running_var", "module.backbone.layer1.1.num_batches_tracked", "module.backbone.layer2.0.weight", "module.backbone.layer2.1.weight", "module.backbone.layer2.1.bias", "module.backbone.layer2.1.running_mean", "module.backbone.layer2.1.running_var", "module.backbone.layer2.1.num_batches_tracked", "module.backbone.layer3.0.conv1.weight", "module.backbone.layer3.0.bn1.weight", "module.backbone.layer3.0.bn1.bias", "module.backbone.layer3.0.bn1.running_mean", "module.backbone.layer3.0.bn1.running_var", "module.backbone.layer3.0.bn1.num_batches_tracked", "module.backbone.layer3.0.conv2.weight", "module.backbone.layer3.0.bn2.weight", "module.backbone.layer3.0.bn2.bias", "module.backbone.layer3.0.bn2.running_mean", "module.backbone.layer3.0.bn2.running_var", "module.backbone.layer3.0.bn2.num_batches_tracked", "module.backbone.layer3.0.conv3.weight", "module.backbone.layer3.0.bn3.weight", "module.backbone.layer3.0.bn3.bias", "module.backbone.layer3.0.bn3.running_mean", "module.backbone.layer3.0.bn3.running_var", "module.backbone.layer3.0.bn3.num_batches_tracked", "module.backbone.layer3.0.downsample.0.weight", "module.backbone.layer3.0.downsample.1.weight", "module.backbone.layer3.0.downsample.1.bias", "module.backbone.layer3.0.downsample.1.running_mean", "module.backbone.layer3.0.downsample.1.running_var", "module.backbone.layer3.0.downsample.1.num_batches_tracked", "module.backbone.layer3.1.conv1.weight", "module.backbone.layer3.1.bn1.weight", "module.backbone.layer3.1.bn1.bias", "module.backbone.layer3.1.bn1.running_mean", "module.backbone.layer3.1.bn1.running_var", "module.backbone.layer3.1.bn1.num_batches_tracked", "module.backbone.layer3.1.conv2.weight", "module.backbone.layer3.1.bn2.weight", "module.backbone.layer3.1.bn2.bias", "module.backbone.layer3.1.bn2.running_mean", "module.backbone.layer3.1.bn2.running_var", "module.backbone.layer3.1.bn2.num_batches_tracked", "module.backbone.layer3.1.conv3.weight", "module.backbone.layer3.1.bn3.weight", "module.backbone.layer3.1.bn3.bias", "module.backbone.layer3.1.bn3.running_mean", "module.backbone.layer3.1.bn3.running_var", "module.backbone.layer3.1.bn3.num_batches_tracked", "module.backbone.layer3.2.conv1.weight", "module.backbone.layer3.2.bn1.weight", "module.backbone.layer3.2.bn1.bias", "module.backbone.layer3.2.bn1.running_mean", "module.backbone.layer3.2.bn1.running_var", "module.backbone.layer3.2.bn1.num_batches_tracked", "module.backbone.layer3.2.conv2.weight", "module.backbone.layer3.2.bn2.weight", "module.backbone.layer3.2.bn2.bias", "module.backbone.layer3.2.bn2.running_mean", "module.backbone.layer3.2.bn2.running_var", "module.backbone.layer3.2.bn2.num_batches_tracked", "module.backbone.layer3.2.conv3.weight", "module.backbone.layer3.2.bn3.weight", "module.backbone.layer3.2.bn3.bias", "module.backbone.layer3.2.bn3.running_mean", "module.backbone.layer3.2.bn3.running_var", "module.backbone.layer3.2.bn3.num_batches_tracked", "module.backbone.layer4.0.conv1.weight", "module.backbone.layer4.0.bn1.weight", "module.backbone.layer4.0.bn1.bias", "module.backbone.layer4.0.bn1.running_mean", "module.backbone.layer4.0.bn1.running_var", "module.backbone.layer4.0.bn1.num_batches_tracked", "module.backbone.layer4.0.conv2.weight", "module.backbone.layer4.0.bn2.weight", "module.backbone.layer4.0.bn2.bias", "module.backbone.layer4.0.bn2.running_mean", "module.backbone.layer4.0.bn2.running_var", "module.backbone.layer4.0.bn2.num_batches_tracked", "module.backbone.layer4.0.conv3.weight", "module.backbone.layer4.0.bn3.weight", "module.backbone.layer4.0.bn3.bias", "module.backbone.layer4.0.bn3.running_mean", "module.backbone.layer4.0.bn3.running_var", "module.backbone.layer4.0.bn3.num_batches_tracked", "module.backbone.layer4.0.downsample.0.weight", "module.backbone.layer4.0.downsample.1.weight", "module.backbone.layer4.0.downsample.1.bias", "module.backbone.layer4.0.downsample.1.running_mean", "module.backbone.layer4.0.downsample.1.running_var", "module.backbone.layer4.0.downsample.1.num_batches_tracked", "module.backbone.layer4.1.conv1.weight", "module.backbone.layer4.1.bn1.weight", "module.backbone.layer4.1.bn1.bias", "module.backbone.layer4.1.bn1.running_mean", "module.backbone.layer4.1.bn1.running_var", "module.backbone.layer4.1.bn1.num_batches_tracked", "module.backbone.layer4.1.conv2.weight", "module.backbone.layer4.1.bn2.weight", "module.backbone.layer4.1.bn2.bias", "module.backbone.layer4.1.bn2.running_mean", "module.backbone.layer4.1.bn2.running_var", "module.backbone.layer4.1.bn2.num_batches_tracked", "module.backbone.layer4.1.conv3.weight", "module.backbone.layer4.1.bn3.weight", "module.backbone.layer4.1.bn3.bias", "module.backbone.layer4.1.bn3.running_mean", "module.backbone.layer4.1.bn3.running_var", "module.backbone.layer4.1.bn3.num_batches_tracked", "module.backbone.layer4.2.conv1.weight", "module.backbone.layer4.2.bn1.weight", "module.backbone.layer4.2.bn1.bias", "module.backbone.layer4.2.bn1.running_mean", "module.backbone.layer4.2.bn1.running_var", "module.backbone.layer4.2.bn1.num_batches_tracked", "module.backbone.layer4.2.conv2.weight", "module.backbone.layer4.2.bn2.weight", "module.backbone.layer4.2.bn2.bias", "module.backbone.layer4.2.bn2.running_mean", "module.backbone.layer4.2.bn2.running_var", "module.backbone.layer4.2.bn2.num_batches_tracked", "module.backbone.layer4.2.conv3.weight", "module.backbone.layer4.2.bn3.weight", "module.backbone.layer4.2.bn3.bias", "module.backbone.layer4.2.bn3.running_mean", "module.backbone.layer4.2.bn3.running_var", "module.backbone.layer4.2.bn3.num_batches_tracked", "module.backbone.layer4.3.conv1.weight", "module.backbone.layer4.3.bn1.weight", "module.backbone.layer4.3.bn1.bias", "module.backbone.layer4.3.bn1.running_mean", "module.backbone.layer4.3.bn1.running_var", "module.backbone.layer4.3.bn1.num_batches_tracked", "module.backbone.layer4.3.conv2.weight", "module.backbone.layer4.3.bn2.weight", "module.backbone.layer4.3.bn2.bias", "module.backbone.layer4.3.bn2.running_mean", "module.backbone.layer4.3.bn2.running_var", "module.backbone.layer4.3.bn2.num_batches_tracked", "module.backbone.layer4.3.conv3.weight", "module.backbone.layer4.3.bn3.weight", "module.backbone.layer4.3.bn3.bias", "module.backbone.layer4.3.bn3.running_mean", "module.backbone.layer4.3.bn3.running_var", "module.backbone.layer4.3.bn3.num_batches_tracked", "module.backbone.layer5.0.conv1.weight", "module.backbone.layer5.0.bn1.weight", "module.backbone.layer5.0.bn1.bias", "module.backbone.layer5.0.bn1.running_mean", "module.backbone.layer5.0.bn1.running_var", "module.backbone.layer5.0.bn1.num_batches_tracked", "module.backbone.layer5.0.conv2.weight", "module.backbone.layer5.0.bn2.weight", "module.backbone.layer5.0.bn2.bias", "module.backbone.layer5.0.bn2.running_mean", "module.backbone.layer5.0.bn2.running_var", "module.backbone.layer5.0.bn2.num_batches_tracked", "module.backbone.layer5.0.conv3.weight", "module.backbone.layer5.0.bn3.weight", "module.backbone.layer5.0.bn3.bias", "module.backbone.layer5.0.bn3.running_mean", "module.backbone.layer5.0.bn3.running_var", "module.backbone.layer5.0.bn3.num_batches_tracked", "module.backbone.layer5.0.downsample.0.weight", "module.backbone.layer5.0.downsample.1.weight", "module.backbone.layer5.0.downsample.1.bias", "module.backbone.layer5.0.downsample.1.running_mean", "module.backbone.layer5.0.downsample.1.running_var", "module.backbone.layer5.0.downsample.1.num_batches_tracked", "module.backbone.layer5.1.conv1.weight", "module.backbone.layer5.1.bn1.weight", "module.backbone.layer5.1.bn1.bias", "module.backbone.layer5.1.bn1.running_mean", "module.backbone.layer5.1.bn1.running_var", "module.backbone.layer5.1.bn1.num_batches_tracked", "module.backbone.layer5.1.conv2.weight", "module.backbone.layer5.1.bn2.weight", "module.backbone.layer5.1.bn2.bias", "module.backbone.layer5.1.bn2.running_mean", "module.backbone.layer5.1.bn2.running_var", "module.backbone.layer5.1.bn2.num_batches_tracked", "module.backbone.layer5.1.conv3.weight", "module.backbone.layer5.1.bn3.weight", "module.backbone.layer5.1.bn3.bias", "module.backbone.layer5.1.bn3.running_mean", "module.backbone.layer5.1.bn3.running_var", "module.backbone.layer5.1.bn3.num_batches_tracked", "module.backbone.layer5.2.conv1.weight", "module.backbone.layer5.2.bn1.weight", "module.backbone.layer5.2.bn1.bias", "module.backbone.layer5.2.bn1.running_mean", "module.backbone.layer5.2.bn1.running_var", "module.backbone.layer5.2.bn1.num_batches_tracked", "module.backbone.layer5.2.conv2.weight", "module.backbone.layer5.2.bn2.weight", "module.backbone.layer5.2.bn2.bias", "module.backbone.layer5.2.bn2.running_mean", "module.backbone.layer5.2.bn2.running_var", "module.backbone.layer5.2.bn2.num_batches_tracked", "module.backbone.layer5.2.conv3.weight", "module.backbone.layer5.2.bn3.weight", "module.backbone.layer5.2.bn3.bias", "module.backbone.layer5.2.bn3.running_mean", "module.backbone.layer5.2.bn3.running_var", "module.backbone.layer5.2.bn3.num_batches_tracked", "module.backbone.layer5.3.conv1.weight", "module.backbone.layer5.3.bn1.weight", "module.backbone.layer5.3.bn1.bias", "module.backbone.layer5.3.bn1.running_mean", "module.backbone.layer5.3.bn1.running_var", "module.backbone.layer5.3.bn1.num_batches_tracked", "module.backbone.layer5.3.conv2.weight", "module.backbone.layer5.3.bn2.weight", "module.backbone.layer5.3.bn2.bias", "module.backbone.layer5.3.bn2.running_mean", "module.backbone.layer5.3.bn2.running_var", "module.backbone.layer5.3.bn2.num_batches_tracked", "module.backbone.layer5.3.conv3.weight", "module.backbone.layer5.3.bn3.weight", "module.backbone.layer5.3.bn3.bias", "module.backbone.layer5.3.bn3.running_mean", "module.backbone.layer5.3.bn3.running_var", "module.backbone.layer5.3.bn3.num_batches_tracked", "module.backbone.layer5.4.conv1.weight", "module.backbone.layer5.4.bn1.weight", "module.backbone.layer5.4.bn1.bias", "module.backbone.layer5.4.bn1.running_mean", "module.backbone.layer5.4.bn1.running_var", "module.backbone.layer5.4.bn1.num_batches_tracked", "module.backbone.layer5.4.conv2.weight", "module.backbone.layer5.4.bn2.weight", "module.backbone.layer5.4.bn2.bias", "module.backbone.layer5.4.bn2.running_mean", "module.backbone.layer5.4.bn2.running_var", "module.backbone.layer5.4.bn2.num_batches_tracked", "module.backbone.layer5.4.conv3.weight", "module.backbone.layer5.4.bn3.weight", "module.backbone.layer5.4.bn3.bias", "module.backbone.layer5.4.bn3.running_mean", "module.backbone.layer5.4.bn3.running_var", "module.backbone.layer5.4.bn3.num_batches_tracked", "module.backbone.layer5.5.conv1.weight", "module.backbone.layer5.5.bn1.weight", "module.backbone.layer5.5.bn1.bias", "module.backbone.layer5.5.bn1.running_mean", "module.backbone.layer5.5.bn1.running_var", "module.backbone.layer5.5.bn1.num_batches_tracked", "module.backbone.layer5.5.conv2.weight", "module.backbone.layer5.5.bn2.weight", "module.backbone.layer5.5.bn2.bias", "module.backbone.layer5.5.bn2.running_mean", "module.backbone.layer5.5.bn2.running_var", "module.backbone.layer5.5.bn2.num_batches_tracked", "module.backbone.layer5.5.conv3.weight", "module.backbone.layer5.5.bn3.weight", "module.backbone.layer5.5.bn3.bias", "module.backbone.layer5.5.bn3.running_mean", "module.backbone.layer5.5.bn3.running_var", "module.backbone.layer5.5.bn3.num_batches_tracked", "module.backbone.layer6.0.conv1.weight", "module.backbone.layer6.0.bn1.weight", "module.backbone.layer6.0.bn1.bias", "module.backbone.layer6.0.bn1.running_mean", "module.backbone.layer6.0.bn1.running_var", "module.backbone.layer6.0.bn1.num_batches_tracked", "module.backbone.layer6.0.conv2.weight", "module.backbone.layer6.0.bn2.weight", "module.backbone.layer6.0.bn2.bias", "module.backbone.layer6.0.bn2.running_mean", "module.backbone.layer6.0.bn2.running_var", "module.backbone.layer6.0.bn2.num_batches_tracked", "module.backbone.layer6.0.conv3.weight", "module.backbone.layer6.0.bn3.weight", "module.backbone.layer6.0.bn3.bias", "module.backbone.layer6.0.bn3.running_mean", "module.backbone.layer6.0.bn3.running_var", "module.backbone.layer6.0.bn3.num_batches_tracked", "module.backbone.layer6.0.downsample.0.weight", "module.backbone.layer6.0.downsample.1.weight", "module.backbone.layer6.0.downsample.1.bias", "module.backbone.layer6.0.downsample.1.running_mean", "module.backbone.layer6.0.downsample.1.running_var", "module.backbone.layer6.0.downsample.1.num_batches_tracked", "module.backbone.layer6.1.conv1.weight", "module.backbone.layer6.1.bn1.weight", "module.backbone.layer6.1.bn1.bias", "module.backbone.layer6.1.bn1.running_mean", "module.backbone.layer6.1.bn1.running_var", "module.backbone.layer6.1.bn1.num_batches_tracked", "module.backbone.layer6.1.conv2.weight", "module.backbone.layer6.1.bn2.weight", "module.backbone.layer6.1.bn2.bias", "module.backbone.layer6.1.bn2.running_mean", "module.backbone.layer6.1.bn2.running_var", "module.backbone.layer6.1.bn2.num_batches_tracked", "module.backbone.layer6.1.conv3.weight", "module.backbone.layer6.1.bn3.weight", "module.backbone.layer6.1.bn3.bias", "module.backbone.layer6.1.bn3.running_mean", "module.backbone.layer6.1.bn3.running_var", "module.backbone.layer6.1.bn3.num_batches_tracked", "module.backbone.layer6.2.conv1.weight", "module.backbone.layer6.2.bn1.weight", "module.backbone.layer6.2.bn1.bias", "module.backbone.layer6.2.bn1.running_mean", "module.backbone.layer6.2.bn1.running_var", "module.backbone.layer6.2.bn1.num_batches_tracked", "module.backbone.layer6.2.conv2.weight", "module.backbone.layer6.2.bn2.weight", "module.backbone.layer6.2.bn2.bias", "module.backbone.layer6.2.bn2.running_mean", "module.backbone.layer6.2.bn2.running_var", "module.backbone.layer6.2.bn2.num_batches_tracked", "module.backbone.layer6.2.conv3.weight", "module.backbone.layer6.2.bn3.weight", "module.backbone.layer6.2.bn3.bias", "module.backbone.layer6.2.bn3.running_mean", "module.backbone.layer6.2.bn3.running_var", "module.backbone.layer6.2.bn3.num_batches_tracked", "module.backbone.layer7.0.weight", "module.backbone.layer7.1.weight", "module.backbone.layer7.1.bias", "module.backbone.layer7.1.running_mean", "module.backbone.layer7.1.running_var", "module.backbone.layer7.1.num_batches_tracked", "module.backbone.layer8.0.weight", "module.backbone.layer8.1.weight", "module.backbone.layer8.1.bias", "module.backbone.layer8.1.running_mean", "module.backbone.layer8.1.running_var", "module.backbone.layer8.1.num_batches_tracked", "module.aspp.aspp1.atrous_conv.weight", "module.aspp.aspp1.bn.weight", "module.aspp.aspp1.bn.bias", "module.aspp.aspp1.bn.running_mean", "module.aspp.aspp1.bn.running_var", "module.aspp.aspp1.bn.num_batches_tracked", "module.aspp.aspp2.atrous_conv.weight", "module.aspp.aspp2.bn.weight", "module.aspp.aspp2.bn.bias", "module.aspp.aspp2.bn.running_mean", "module.aspp.aspp2.bn.running_var", "module.aspp.aspp2.bn.num_batches_tracked", "module.aspp.aspp3.atrous_conv.weight", "module.aspp.aspp3.bn.weight", "module.aspp.aspp3.bn.bias", "module.aspp.aspp3.bn.running_mean", "module.aspp.aspp3.bn.running_var", "module.aspp.aspp3.bn.num_batches_tracked", "module.aspp.aspp4.atrous_conv.weight", "module.aspp.aspp4.bn.weight", "module.aspp.aspp4.bn.bias", "module.aspp.aspp4.bn.running_mean", "module.aspp.aspp4.bn.running_var", "module.aspp.aspp4.bn.num_batches_tracked", "module.aspp.global_avg_pool.1.weight", "module.aspp.global_avg_pool.2.weight", "module.aspp.global_avg_pool.2.bias", "module.aspp.global_avg_pool.2.running_mean", "module.aspp.global_avg_pool.2.running_var", "module.aspp.global_avg_pool.2.num_batches_tracked", "module.aspp.conv1.weight", "module.aspp.bn1.weight", "module.aspp.bn1.bias", "module.aspp.bn1.running_mean", "module.aspp.bn1.running_var", "module.aspp.bn1.num_batches_tracked", "module.decoder.conv1.weight", "module.decoder.bn1.weight", "module.decoder.bn1.bias", "module.decoder.bn1.running_mean", "module.decoder.bn1.running_var", "module.decoder.bn1.num_batches_tracked", "module.decoder.last_conv.0.weight", "module.decoder.last_conv.1.weight", "module.decoder.last_conv.1.bias", "module.decoder.last_conv.1.running_mean", "module.decoder.last_conv.1.running_var", "module.decoder.last_conv.1.num_batches_tracked", "module.decoder.last_conv.4.weight", "module.decoder.last_conv.5.weight", "module.decoder.last_conv.5.bias", "module.decoder.last_conv.5.running_mean", "module.decoder.last_conv.5.running_var", "module.decoder.last_conv.5.num_batches_tracked", "module.decoder.last_conv.7.weight", "module.decoder.last_conv.7.bias".Cool project!
I'm having an issue compiling depth2depth. Seems like it's not linking OpenGL properly. I followed all of your instructions on a Linux machine. Here's a snippet of the error output when I run make in the gaps folder.
g++ -o ../../bin/x86_64/depth2depth -L../../lib/x86_64 -O depth2depth.o -lR2Shapes -lRNMath -lRNBasics -ljpeg -lpng -lCSparse -L/usr/include/hdf5/serial/ -lhdf5_serial -pthread -lGLU -lGL -lX11 -lm
/usr/bin/ld: cannot find -lGL
collect2: error: ld returned 1 exit status
I'd appreciate it if you had any direction on this. Thanks!
Hi Shreeyak,
Q1: Does the pipeline expect rectified depth images (depth values wrt camera plane) during inference?
Q2: Does the pipeline expect rectified depth images (depth values wrt camera plane) during training?
Q3: Are Realsense cameras then outputting already rectified depth or additional step(s) need to be done?
Many thanks
Dear,
First of all, thank you for sharing this amazing work. I find it valuable on so many levels.
I'm trying to reproduce training, but on a custom data set.
However, I'm still struggling with how to perform boundary detection.
The dataset is synthetic one (rendered using PBRT instead of Blender's Cycle engine).
Could you please share more info on how the boundaries are obtained?
I do have ground truth for depth images.
Are the occlusions and contact edges derived from depth data only; e.g. by treating depth data as an image and computing gradients, or there is more to the story?
Many thanks for your time.
The global optimization module which predicts the depth uses occlusion boundaries as an input. To help the edge prediction model learn better, we added an extra class of contact edges.
The outlines are predicted through semantic segmentation, where each pixel is assigned a class. There are 2 types of edges: occlusion boundaries (which indicate a large differential on the depth) and the contact edges (edges in contact with other surfaces, and hence do not have the large differential).
In the paper, we visualize the 3 classes (background, occlusion, contact) through the colors red, green, blue.
Originally posted by @Shreeyak in #17 (comment)
Hello I am creating my own training data and I am able to produce segmentation masks for my images but am unsure on how to produce the outlines images. Could you help me here?
I wonder if depth2depth-related code here is exactly the same as the existing code or any part is modified.
Thank you for sharing this interesting work! I tested the trained model on some novel transparent objects. However, the result is not that good. Some transparent bottles are not recognized, so no depth re-construction. Some transparent objects are recognized but the estimated point cloud is not correct.
Do you think it is normal for novel objects, or there is some problem in my implementation? Any suggestion to improve the network's generalization on novel objects? Thanks!
Hi @Shreeyak ! I meet a problem with the surface normal data in Cleargrasp synthetic dataset.
When I check the cameraNormals.exr files, I found the R, G, B-channel values range from -1.0 to 1.0 (shown in the following picture opened in Photoshop).
However, it seems that Blender would output the surface normal map in a [0.0,1.0] range (OpenEXR format).
So I am wondering which is the correct representation for surface normal data? And how can I obtain [-1.0,1.0] normal values like your dataset using Blender?
Thanks!
AttirbuteError is raised if the masks attribute does not exist in config.yaml.
AttributeError: 'AttrDict' instance has no attribute 'masks'
After downloading the necessary datasets and properly modifying the location of the datasets in config.yaml, go to the pytorch_networks/surface_normals directory and run the following command;
python train.py -c config/config.yamlUbuntu 18.04.5 LTS
Python 3.6.8
Intel® Core™ i7-10700F CPU @ 2.90GHz × 16
GeForce GTX 1660 Ti/PCIe/SSE2
Package Version
---------------------- ---------
absl-py 0.11.0
addict 2.3.0
argon2-cffi 20.1.0
astunparse 1.6.3
async-generator 1.10
attrdict 2.0.1
attrs 20.2.0
backcall 0.2.0
bleach 3.2.1
cachetools 4.1.1
certifi 2020.6.20
cffi 1.14.3
chardet 3.0.4
cycler 0.10.0
dataclasses 0.7
decorator 4.4.2
defusedxml 0.6.0
entrypoints 0.3
future 0.18.2
gast 0.3.3
google-auth 1.23.0
google-auth-oauthlib 0.4.2
google-pasta 0.2.0
grpcio 1.33.2
h5py 2.10.0
idna 2.10
imageio 2.9.0
imgaug 0.4.0
importlib-metadata 2.0.0
ipykernel 5.3.4
ipython 7.16.1
ipython-genutils 0.2.0
ipywidgets 7.5.1
jedi 0.17.2
Jinja2 2.11.2
joblib 0.17.0
jsonschema 3.2.0
jupyter-client 6.1.7
jupyter-core 4.6.3
jupyterlab-pygments 0.1.2
Keras-Preprocessing 1.1.2
kiwisolver 1.2.0
Markdown 3.3.3
MarkupSafe 1.1.1
matplotlib 3.3.2
mistune 0.8.4
nbclient 0.5.1
nbconvert 6.0.7
nbformat 5.0.8
nest-asyncio 1.4.2
networkx 2.5
notebook 6.1.4
numpy 1.18.5
oauthlib 3.1.0
open3d 0.11.1
opencv-contrib-python 4.1.1.26
opencv-python 4.1.1.26
opencv-python-headless 4.4.0.44
OpenEXR 1.3.2
opt-einsum 3.3.0
oyaml 1.0
packaging 20.4
pandas 1.1.4
pandocfilters 1.4.3
parso 0.7.1
pathlib 1.0.1
pexpect 4.8.0
pickleshare 0.7.5
Pillow 8.0.1
pip 20.2.4
plyfile 0.7.2
prometheus-client 0.8.0
prompt-toolkit 3.0.8
protobuf 3.13.0
ptyprocess 0.6.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
pycparser 2.20
Pygments 2.7.2
pyparsing 2.4.7
pyrsistent 0.17.3
python-dateutil 2.8.1
pytz 2020.1
PyWavelets 1.1.1
PyYAML 5.3.1
pyzmq 19.0.2
requests 2.24.0
requests-oauthlib 1.3.0
rsa 4.6
scikit-image 0.17.2
scikit-learn 0.23.2
scipy 1.5.3
Send2Trash 1.5.0
setuptools 50.3.2
Shapely 1.7.1
six 1.15.0
sklearn 0.0
tensorboard 2.3.0
tensorboard-plugin-wit 1.7.0
tensorboardX 2.1
tensorflow 2.3.1
tensorflow-estimator 2.3.0
termcolor 1.1.0
terminado 0.9.1
testpath 0.4.4
threadpoolctl 2.1.0
tifffile 2020.9.3
torch 1.7.0
torchvision 0.8.1
tornado 6.1
tqdm 4.51.0
traitlets 4.3.3
typing-extensions 3.7.4.3
urllib3 1.25.11
wcwidth 0.2.5
webencodings 0.5.1
Werkzeug 1.0.1
wheel 0.35.1
widgetsnbextension 3.5.1
wrapt 1.12.1
zipp 3.4.0
I installed the cuda 9.0 in the tutorial, but it seems that the cuda 9.0 only support pytorch 1.1.0. Is that ok? By the way, my gpu is RTX 2080. Waiting for your answer!
Thank you for sharing this repo.
I have a question about the coordinate system in the synthetic dataset. I am trying to get an object pose from the JSON file. I have converted the quaternion to get the rotation matrix of the object. And I got a translation matrix from the fourth column of matrix_4x4. Now I am trying to make sense of the data I have got.
Thank you in advance.
Hi @Shreeyak ! How can I get the exr file? I see that the deep completion part uses the exr file
First question is live-demo didn't show mask image:
Run demo:
python live_demo.py -c config/config.yaml
This will open a new window displaying input image, input depth, intermediate outputs (surface normals, occlusion boundaries, mask)......
And I got the output like:
The other question is about "ground truth depth, mask of valid transparent pixels":
Run the dataset_capture_gui/capture_image.py script to launch a window that streams images directly from a Realsense D400 series camera. Press 'c' to capture the transparent frame, 'v' to capture the opaque frame and spacebar to confirm and save the RGB and Depth images for both frames.
Press c so I can save data in data dir.
And the image show in "data/captures/exp-xxx/result-viz" :
But you have "ground truth depth, mask of valid transparent pixels" this two images.
Forgive my poor English.
Hey! Thanks for your code, it really helps. I am wondering if this code can work on the Intel Realsense L515 device. It seems something wrong with this device.
https://github.com/AcademySoftwareFoundation/openexr now has a new lease 3.1.5 ....
I can't download 3D models in Clear Grasp dataset.Please check the download of it . Thanks！
Hi ,the dataset link for both train and test are not working /opening
Hi, Shreeyak.
This code is tested with Ubuntu 16.04, Python3 and Pytorch 1.1.
Could you explain detailed python3 version (ex. 3.6.7)??
Also, Which CUDA version did you used for the test??
Hi Shreeyak,
I want to repeat the number reported in your paper and I have run the experiment of surface normal using the same setting as in config.yaml.sample. I use your code and dataset without any modification. And I simply use one GPU with batchsize=8. To my surprise, only after one epoch, the mean/median error on test synthetic dataset becomes 18.85/10.63, which is much better than the number reported in your paper 25.32 / 20.53. I think I must missing something but I just can't find it. Do you have any idea about that?
Hi,
Im trying to make my own dataset for training, I don't understand why you are creating the outlines this way.
as I understand it now, the outlines are rendered in grey (1,1,1) and where the bottles come into contact with the tray (or ground) and that is visible in the RGB image, it's a slightly lighter shade (2,2,2) in those areas. but I do not understand why you are doing this.
can you help me here?
Best,
I have two simple questions.
For test-set, There are two types of folder d435, d415.
You mentioned in a paper, There are two camera folder D435(known objects and 20% novel objects), D415(80% novel objects).
My question is how to evaluate separately Known, Novel Objects (Paper-Table 1).
Because You are code seems like mean based on the folder which doesn't consider novel, known.
For Example, I got this result from D435 folders.
root@940a0bd8c12c:/hdd4tb/2020_code/cleargrasp/eval_depth_completion# python eval_depth_completion.py -c config/config.yaml
Saving results to folder: "data/results/exp-007"
Creating DRN model for normals and loading checkpoint
Creating DRN model for outlines and loading checkpoint
Creating DRN model for masks and loading checkpoint
Total Num of rgb_files: 23
Total Num of depth_files: 23
Total Num of gt_depth_files: 23
Total Num of segmentation_masks: 23
......
Image 000000022 / 22:
rmse: 0.03204
abs_rel: 0.05239
mae: 0.02775
a1.05: 55.23256
a1.10: 89.53488
a1.25: 100.0Mean Error Stats for Entire Dataset:
rmse_mean: 0.042
abs_rel_mean: 0.057
mae_mean: 0.034
a1.05_mean: 65.46
a1.10_mean: 87.84
a1.25_mean: 95.08
I got error from Save Results of Depth Completion
I followed your setup and tried your advice, But does not work.
In case of Open3D not being recognized, try installing Open3D with:
pip install open3d-python -U --no-cache-dir
root@940a0bd8c12c:/hdd4tb/2020_code/cleargrasp/eval_depth_completion# python eval_depth_completion.py -c config/config.yaml
Saving results to folder: "data/results/exp-006"
Creating DRN model for normals and loading checkpoint
Creating DRN model for outlines and loading checkpoint
Creating DRN model for masks and loading checkpoint
Total Num of rgb_files: 23
Total Num of depth_files: 23
Total Num of gt_depth_files: 23
Total Num of segmentation_masks: 23
/pytorch/aten/src/ATen/native/IndexingUtils.h:20: UserWarning: indexing with dtype torch.uint8 is now deprecated, please use a dtype torch.bool instead.
/pytorch/aten/src/ATen/native/IndexingUtils.h:20: UserWarning: indexing with dtype torch.uint8 is now deprecated, please use a dtype torch.bool instead.Image 000000000 / 22:
rmse: 0.03761
abs_rel: 0.04813
mae: 0.02902
a1.05: 65.2815
a1.10: 83.91421
a1.25: 100.0
Traceback (most recent call last):
File "eval_depth_completion.py", line 234, in
max_depth=config.depthVisualization.maxDepth)
File "../api/depth_completion_api.py", line 353, in store_depth_completion_outputs
self.estimate_normals_write_ptcloud(gt_ptcloud_filename, xyz_points, self.outputImgHeight, self.outputImgWidth)
File "../api/depth_completion_api.py", line 255, in estimate_normals_write_ptcloud
pcd.estimate_normals(search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=0.01, max_nn=10000))
AttributeError: 'open3d.open3d.geometry.PointCloud' object has no attribute 'estimate_normals'
Hi,
I'm doing training from scratch, and I do not have the resources to pre-train on MP and SN. Is it possible you could share those pre-trained models (without training on the synthetic data, just MP and SN.)
Thanks!
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.