This is a modified DenseSurfelMapping for depth maps fusion with known camera intrinsic and extrinsic parameters. You can use it to fuse the ouput from VINS-Mono.
==========
WARNING!
We have cleaned the code such that it can run without GPU acceleration. The code have not been fully tested after the refactoring. If you have any questions or suggestions, please let us know in the issue.
This is a depth map fusion method following the ICRA 2019 submission Real-time Scalable Dense Surfel Mapping, Kaixuan Wang, Fei Gao, and Shaojie Shen.
Given a sequence of depth images, intensity images, and camera poses, the proposed methods can fuse them into a globally consistent model using surfel representation. The fusion method supports both ORB-SLAM2 and VINS-Mono (a little modification is required) so that you can use it in RGB-D, stereo, or visual-inertial cases according to your setups. We develop the method based on the motivation that the fusion method: (1) can support loop closure (so that it can be consistent with other state-of-the-art SLAM methods), (2) do not require much CPU/memory resources to reconstruct a fine model in real-time, (3) can be scaled to large environments. These requirements are of vital importance in robot navigation tasks that the robot can safly navigate in the environment with odometry-consistent dense maps.
An example to show the usage of the surfel mapping is shown below.
Left is the overview of the environment, the middle is the reconstructed results (visualized as point clouds in rviz of ROS) of our method, and right is the result using OpenChisel. We use VINS-Mono to track the camera motion with loop closure, and MVDepthNet to estimate the depth maps. The black line is the path of the camera. In the reconstruction, loop closure is enabled to correct the detected drift. OpenChisel is a great project to reconstruct the environment using the truncated signed distance function (TSDF). However, as shown in the example, it is not suitable to be used with SLAM systems that have loop closure abilities.
The system can also be applied to the KITTI datasets in real-time with only CPU computation.
The top row is the reconstruction using stereo cameras and the bottom row is the reconstruction using only the left camera. Details can be found in the paper.
A video can be used to illustrate the performance of the system and how we apply it into an autonomous navigation:
We have use the surfel fusion with VINS-Mono in lots of UAV projects. For depth estimation, we recommend high quality depth methods/devices, for example MVDepthNet or intel-realsense. Please refer to /launch/fuse_depthnet.launch for detailed parameters. The system takes paired image and depth map as input. Since VINS-Mono publishes imu poses, we also need to receive /vins_estimator/extrinsic for converting imu poses into camera poses.
We thank Gao Fei, Pan Jie, and Wang Luqi, for their contribution to the code and suggestions.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent