gorilla-lab-scut / dualposenet

Code for "DualPoseNet: Category-level 6D Object Pose and Size EstimationUsing Dual Pose Network with Refined Learning of Pose Consistency"

DualPoseNet

Code for "DualPoseNet: Category-level 6D Object Pose and Size Estimation Using Dual Pose Network with Refined Learning of Pose Consistency", ICCV2021. [Paper][Supp][Arxiv]

Created by Jiehong Lin, Zewei Wei, Zhihao Li, Songcen Xu, Kui Jia, and Yuanqing Li.

Citation

If you find our work useful in your research, please consider citing:

 @InProceedings{Lin_2021_ICCV,
     author    = {Lin, Jiehong and Wei, Zewei and Li, Zhihao and Xu, Songcen and Jia, Kui and Li, Yuanqing},
     title     = {DualPoseNet: Category-Level 6D Object Pose and Size Estimation Using Dual Pose Network With Refined Learning of Pose Consistency},
     booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
     month     = {October},
     year      = {2021},
     pages     = {3560-3569}
 }

Requirements

This code has been tested with

• python 3.6.5
• tensorflow-gpu 1.11.0
• CUDA 11.2

Downloads

• Pre-trained models [link]
• Segmentation predictions on CAMERA25 and REAL275 [link]
• Pose Predicitons on CAMERA25 and REAL275 [link]

Evaluation

Evaluate the results of DualPoseNet reported in the paper:

python eval.py

Data Preparation

Download the data provided by NOCS (real_train, real_test, ground truths, and mesh models), and unzip them in the folder data/ as follows:

data
├── CAMERA
│   ├── train
│   └── val
├── Real
│   ├── train
│   └── test
├── gts
│   ├── val
│   └── real_test
└── obj_models
    ├── train
    ├── val
    ├── real_train
    └── real_test

Run the following scripts to prepare training instances:

cd provider
python training_data_prepare.py

Training

Command for training DualPoseNet:

python main.py --phase train --dataset REAL275

The configurations can be modified in utils/config.py.

Testing

Command for testing DualPoseNet without refined learning:

python main.py --phase test --dataset REAL275

Command for testing DualPoseNet with refined learning:

python main.py --phase test_refine_encoder --dataset REAL275

We also provider another faster way of refinement by directly finetuning the pose-sensitive features:

python main.py --phase test_refine_feature --dataset REAL275

The configurations can also be modified in utils/config.py.

Acknowledgements

Our implementation leverages the code from SCNN, NOCS and SPD.

License

Our code is released under MIT License (see LICENSE file for details).

Contact

[email protected]

[email protected]

dualposenet's Issues

Running speed

Hi, Thanks for the excellent work. I am wondering how fast the inference speed is. I didn't find this kind of information in the paper.

How long to train the model. I want to know some time

Constructing depth images for custom samples

Thank you for the great work.

I have been looking to test the model on custom images. To achieve that, I was wondering on, how to create depth png files in the same format, as provided in real275 dataset. I can see pixel values ranging from 0 to ~10 in depth png files. How can I get depth in same format for my custom sample images?

I tried testing on SUNRGBD dataset, results are not as per expectation.

Thank you in advance.

ImportError: libcublas.so.9.0: cannot open shared object file: No such file or directory

我在执行代码时遇到了这个问题 ，请问为什么会出现这个问题

How long will the code be released?

dataset download link not working

Good afternoon, thank you for your work, I would very much like to conduct training, but today the link to the datasets does not work(
Maybe you saved the datasets on google drive or know some other workaround?
I also tried downloading as recommended in hughw19/NOCS_CVPR2019#59

About data and dataset creation

Like the title, where is the data and how to create the dataset used in the code?

how to run evaluation after train?

I think I shuld generate result pkl files by running Testing 2 times( w/o refined, w/ refined version) and then your code will generate the pkl files and then I should run eval.py. right?

Training model on REAL275 dataset

Recently, I have tried to reproduce results that was given in the DualPoseNet paper. Therefore, I used training_data_prepare.py script to generate necessary training data (for CAMERA25 and REAL275 datasets), which allowed me successfully reproduce results on CAMERA25 dataset.

However, results after training model on REAL275 were very far from the paper results. So, after that I began to wonder -- maybe, I do something wrong in case of training/evaluation on REAL275 dataset.

My pipeline of actions for REAL275 was:

1. Download NOCS data and run python training_data_prepare.py.
2. Run python main.py --phase train --dataset REAL275
3. Run python main.py --phase test --dataset REAL275

I did not save results of mine model evaluation on REAL275, but if you need it I can recalculate them for the issue investigation!

Thank you in advance!