yuecideng / misc3d

A unified library for 3D data processing with both c++ and python API

Hi there 👋

• 🔭 I’m currently working on R&D about 3D computer vision and machine learning in robotics domain.
• ❤️ I'm interested in 3D geometry processing, 6D pose estimation and 3D reconstruction.
• 🤔 Currently I'm learning rendering and physical simulation.
• 👀 I'm also interested in simulation-to-real learning for 3D world problems, including auto data generation for both synthesis and mixed-reality, domain adaptation, efficient learning network and pipeline, online data generation and training, etc.

misc3d's Issues

how can i tune the parameters that i can get a better result

hi,when i use ppf_3d_icp,i find the result is not so well.
i want to know what parameter i can tune
i only tune the rel_sample_dist and score_thresh,so that more points can be counted in estimator.
what else parameters can i tune,hope your reply.

How did you manage to get a bop19_average_recall of 0.411 on YCB-V dataset?

How did you manage to get a bop19_average_recall of 0.411 on YCB-V dataset?
I tried gt_mask and ppf+icp on ycbv/test_targets_bop19, however, only get a score of 0.3094.
Once before I tried ppf on lm dataset, AR is 0.817, as good as yours.
I wanna know what could be wrong on ycbv, how'd you manage to fulfill sparse icp and dense icp?
Maybe you could open source your BOP submit pipeline?

ImportError: libmisc3d.so: cannot open shared object file: No such file or directory

Hi @yuecideng , I installed the Misc3D lib and successfully ran through few example python scripts in terminal. However, when I run script in pycharm then it occur import error and the terminal become like this as well. How can I solve this problem?

关于编译misc3d项目

1.open3d == 0.14.1是什么意思呢，是用open3d 为0.14.1的包去编译吗？
2.Build open3d as external library. 这个的意思是先按照https://github.com/isl-org/open3d-cmake-find-package的指示去编译open3d包吗，能不能说明下具体编译步骤，因为编译open3d按照https://github.com/isl-org/open3d-cmake-find-package的方法很难编译成功，因为需要一些很难解决的依赖问题。
谢谢你

misc3d编译问题

How do I compile Misc3D OFFLINE?

It's been too many trouble to compile Open3D offline, so I give up. Instead, using the x86_64 (pre CXX11 ABI) with CUDA 11.x Package. However, encounter this error

...
/usr/bin/ld: /data1/install/Open3D_install/lib/libOpen3D.so: undefined reference to std::__1::ios_base::__set_badbit_and_consider_rethrow()' /usr/bin/ld: /data1/install/Open3D_install/lib/libOpen3D.so: undefined reference to typeinfo for std::__1::basic_istream<char, std::__1::char_traits >'
/usr/bin/ld: /data1/install/Open3D_install/lib/libOpen3D.so: undefined reference to `std::__1::random_device::operator()()'
...

Any advice maybe?

Aborted (core dumped)

你好，我编译成功后，最原点采样可以运行，但是其他的都无法运行，PPF报错：Aborted (core dumped)，请问你遇到过这种问题吗？

C++ installation has error loading library

Hi, I was having a hard time installing the python version but managed to install the c++ version.

However, when I tried to run the examples, there was an error saying "./ransac_and_boundary: error while loading shared libraries: libmisc3d.so: cannot open shared object file: No such file or directory"

I've checked the lib and it was in the lib directory. How should I load it?

Any suggestion to solve the problem would be very appreciated. Thanks!

How should 'score_thresh' be set to make sure ret is True?

In examples/python/test_ppf_estimator.py

# init ppf config
config = m3d.pose_estimation.PPFEstimatorConfig()
# init training param
config.training_param.rel_sample_dist = 0.04
config.score_thresh = 0.05
config.refine_param.method = m3d.pose_estimation.PPFEstimatorConfig.PointToPlane

What does rel_sample_dist mean?

When I set score_thresh to 0, I get "No match" as usual, I know 0 means Default, which is 0.05.
However, when I set score_thresh to 0.001, in rare cases, I still got "No match".
I assure config.score_thresh = 1e-10 may work when at least one result is needed.
Maybe there is a offical way to do it?

Compile misc3d in conda's virtual environment

All of my environments are installed in conda's virtual environment，And the source code is compiled to open3d with the python path specified as: /home/tsy/anaconda3/envs/ttt/bin/python, and after testing open3d is compiled successfully，How does misc3d compile into conda's virtual environment? Can't seem to specify a python path?
ERROR: could not find a package configuration file provided by "Eigen3"

can't import misc3d in python

Hi,i encounter a problem that i can't import misc3d in python.
when i finish your following guide :
1.Build open3d as external library. You can follow the instruction from here guide. Build pybind11 in your system as well. If you only use C++ API, you can skip this step and just download the pre-built open3d library from official website.
2.Git clone and run: mkdir build && cd build. You can use Cmake GUI to configure your build options. Then run cmake --build. --config Release --target INSTALL to install Misc3D.
3.After installation, add this variable: /path/to/installation/misc3d/lib/python to your system environment variable Path to make sure you can import misc3d in python.

i don't know why i still can't import misc3d,hope your rely.

Question about BOP Submission: PPF_3D_ICP

While using PPF in 6D Object Pose Estimation, the scene PointCloud you used is directly created from original color and depth, or after RoI crop using bbox/mask?

build error: undefined reference to `open3d::core::Tensor::~Tensor()'

I'm very interesting in your work. I have spent a lot of time to build the codes on ununtu20.04, but it still have problems. The environment I used is as the follow:

• Ubuntu20.04
• Open3D v0.15.1
• python=3.7
• pybind=2.10.0dev1

I have build the Open3D from source successfully. Then, I have install Misc3D according to your tutorial, the commands are as follows

git clone https://github.com/yuecideng/Misc3D.git

cd Misc3D

mkdir build
mkdir misc3d_install
cd build

cmake -DCMAKE_PREFIX_PATH=/home/subchange/dev_build/Open3D/install -DCMAKE_INSTALL_PREFIX=../misc3d_install ..

Note that the path /home/subchange/dev_build/Open3D/install is the path where I have install Open3D

Please see the errors in the follow

I do not good at c++ language. Cloud you please give me some help to build the project from source? Thank you very much.

error: ‘misc3d::py::module_’ has not been declared

在运行make安装指令的时候报错

/home/kyle/Software/Misc3D/python/py_misc3d.cpp: In function ‘void misc3d::pybind11_init_py_misc3d(pybind11::module&)’:
/home/kyle/Software/Misc3D/python/py_misc3d.cpp:10:25: error: ‘misc3d::py::module_’ has not been declared
   10 |         (py::object)py::module_::import("open3d").attr("geometry");
      |                         ^~~~~~~
/home/kyle/Software/Misc3D/python/py_misc3d.cpp:13:25: error: ‘misc3d::py::module_’ has not been declared
   13 |         (py::object)py::module_::import("open3d").attr("pipelines");
      |                         ^~~~~~~
/home/kyle/Software/Misc3D/python/py_misc3d.cpp:16:25: error: ‘misc3d::py::module_’ has not been declared
   16 |         (py::object)py::module_::import("open3d").attr("camera");
      |                         ^~~~~~~
/home/kyle/Software/Misc3D/python/py_misc3d.cpp:19:25: error: ‘misc3d::py::module_’ has not been declared
   19 |         (py::object)py::module_::import("open3d").attr("visualization");
      |                         ^~~~~~~
/home/kyle/Software/Misc3D/python/py_misc3d.cpp:22:25: error: ‘misc3d::py::module_’ has not been declared
   22 |         (py::object)py::module_::import("open3d").attr("utility");
      |        

• 操作系统: ubuntu 20.0
• pybind11: 2.10

Python binding failed

Hi @yuecideng ,
I followed the instruction to install and run examples in the repo and got the error with python binding, for example: farthest_point_sampling.py: TypeError: farthest_point_sampling(): incompatible function arguments. The following argument types are supported.
I use ubuntu 20.04, open3d installed with CUDA enable.
Do you know how to solve the issue?.
Thank you

pip install misc3d in Windows

pip install misc3d -i https://pypi.tuna.tsinghua.edu.cn/simple
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
ERROR: Could not find a version that satisfies the requirement misc3d (from versions: none)
ERROR: No matching distribution found for misc3d

who can help me?
how to install misc3d for windows?

Questions about pose_estimation

Thanks for your great work firstly.
However, I have an issue about pose_estimation section.
Does pose_estimation can only detect one side of object?

My object is irregular industrial sheetmetal, as the object in scene is forward, pose_estimation can detect very well.
But if the irregular industrial sheetmetal is inverted, pose_estimation must be failed, the results of the pose form pose_estimation is always forward direction.
I know there is config.training_param_.invert_model_normal, if I set config.training_param_.invert_model_normal is true, the pose form pose_estimation is always backward direction that it cannot detect the forward direction.

My question is if there some method that I can detect the object whatever the direction is forward or backward ?

ImportError: arg(): could not convert default argument 'param: open3d::geometry::KDTreeSearchParamHybrid' in function 'detect_boundary_points' into a Python object (type not registered yet?)

I compile and install misc3d 0.4.0 version
and open3d version is 0.16.1
$ python
Python 3.8.10 (default, Mar 13 2023, 10:26:41)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import open3d as o3d
INFO - 2023-05-25 12:27:54,039 - utils - NumExpr defaulting to 8 threads.
/home/dell/.local/lib/python3.8/site-packages/pandas/core/computation/expressions.py:20: UserWarning: Pandas requires version '2.7.3' or newer of 'numexpr' (version '2.7.1' currently installed).
from pandas.core.computation.check import NUMEXPR_INSTALLED
import misc3d as m3d
Traceback (most recent call last):
File "", line 1, in
File "/home/dell/ref/Misc3D-main/build/install/misc3d/lib/python/misc3d/init.py", line 1, in
from .py_misc3d import *
ImportError: arg(): could not convert default argument 'param: open3d::geometry::KDTreeSearchParamHybrid' in function 'detect_boundary_points' into a Python object (type not registered yet?)

libmisc3d.so: undefined reference to `cudaGetDevice'

Hi @yuecideng ,
I used cuda11.7 and face this error. Do you know how to solve it?

[ 52%] Linking CXX executable ../../bin/farthest_point_sampling
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaGetDevice' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaStreamGetCaptureInfo'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaMemcpy2DAsync' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaFreeAsync'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaFreeHost' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaMemcpy2D'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to __cudaPushCallConfiguration' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaEventCreateWithFlags'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaStreamCreateWithFlags' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaMemcpyPeerAsync'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaMallocAsync' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaMemset2DAsync'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaStreamCreateWithPriority' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaMemcpyAsync'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaStreamWaitEvent' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaMallocHost'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaGetDeviceProperties' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaStreamCreate'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaEventQuery' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaDeviceCanAccessPeer'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaGetSymbolAddress' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to __cudaPopCallConfiguration'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to __cudaRegisterFatBinaryEnd' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaSetDevice'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaMemGetInfo' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaGetErrorString'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaStreamDestroy' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaGetDeviceCount'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to __cudaRegisterFunction' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaFree'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaMallocManaged' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaDeviceSynchronize'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaMemcpyToSymbolAsync' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaPeekAtLastError'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaRuntimeGetVersion' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaGetErrorName'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaEventDestroy' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaDeviceGetAttribute'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaFuncGetAttributes' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaOccupancyMaxActiveBlocksPerMultiprocessor'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to __cudaUnregisterFatBinary' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to __cudaRegisterFatBinary'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaGetLastError' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaMalloc'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaMemcpy' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaGetExportTable'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to __cudaRegisterVar' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaStreamQuery'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaThreadExchangeStreamCaptureMode' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaStreamGetPriority'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaPointerGetAttributes' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaMemset'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaOccupancyMaxActiveBlocksPerMultiprocessorWithFlags' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaStreamSynchronize'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaDeviceEnablePeerAccess' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaLaunchKernel'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaMemsetAsync' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaStreamGetFlags'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaEventCreate' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaEventRecord'
/usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaFuncSetAttribute' /usr/bin/ld: ../../lib/libmisc3d.so: undefined reference to cudaDeviceGetStreamPriorityRange'
collect2: error: ld returned 1 exit status
make[2]: *** [examples/cpp/CMakeFiles/farthest_point_sampling.dir/build.make:185：bin/farthest_point_sampling] 错误 1
make[1]: *** [CMakeFiles/Makefile2:418：examples/cpp/CMakeFiles/farthest_point_sampling.dir/all] 错误 2
make: *** [Makefile:136：all] 错误 2

Thank you!

ModuleNotFoundError: No module named 'misc3d.py_misc3d'

In [1]: import open3d as o3d
   ...: import misc3d as m3d
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-2556033f9ac2> in <module>
      1 import open3d as o3d
----> 2 import misc3d as m3d

~/Desktop/Misc3D/build/lib/python/misc3d/__init__.py in <module>
----> 1 from .py_misc3d import *

ModuleNotFoundError: No module named 'misc3d.py_misc3d'

In /Misc3D/build/lib/python/misc3d there's only __init__.py