mapiv / iris

Modifications are underway.

Iris

• Visual localization in pre-build pointcloud maps.
• OpenVSLAM and VINS-mono can be used. Modifications are underway.

Video

Submodule

• OpenVSLAM forked by MapIV

original repository (xdspacelab)

Dependency

If you are using ROS, you only need to install g2o and DBoW2.

• ROS
• OpenCV >= 3.2
• Eigen
• PCL
• g2o
• DBow2
  • Please use the custom version released in https://github.com/shinsumicco/DBoW2

see also: openvslam. Modifications are underway.

How to Build

mkdir -p catkin_ws/src
cd catkin_ws/src
git clone --recursive https://github.com/MapIV/iris.git
cd ..
catkin_make

How to Run with Sample Data

Download sample data

1. visual feature file: orb_vocab.dbow from URL
2. pointcloud map : kitti_00.pcd from URL
3. rosbag : kitti_00_stereo.bag from URL

Run with sample data

Stereo camera sample

roscd iris/../../../
# download sample data to here (orb_voceb.dbow, kitti_00.pcd, kitti_00_stereo.bag)
ls # > build devel install src orb_vocab.dbow kitti_00.pcd kitti_00_stereo.bag
roslaunch iris stereo_kitti00.launch
roslaunch iris rviz.launch # (on another terminal)
rosbag play kitti_00_stereo.bag # (on another terminal)

If the estimated position is misaligned, it can be corrected using 2D Pose Estimate in rviz.

Monocular camera sample

roscd iris/../../../
# download sample data to here (orb_voceb.dbow, kitti_00.pcd, kitti_00_stereo.bag)
ls # > build devel install src orb_vocab.dbow kitti_00.pcd kitti_00_stereo.bag
roslaunch iris mono_kitti00.launch
roslaunch iris rviz.launch # (on another terminal)
rosbag play kitti_00_stereo.bag # (on another terminal)

If the estimated position is misaligned, it can be corrected using 2D Pose Estimate in rviz.

How to Run with Your Data

All you need to prepare

1. pointcloud map file (*.pcd)
2. rosbag (*.bag)
3. Config file for iris such as config/sample_iris_config.yaml
4. (only if you use OpenVSLAM) Config file for vSLAM such as config/sample_openvslam_config.yaml

Run

roslaunch iris openvslam.launch iris_config_path:=... 
rosbag play yours.bag # (on another terminal)

License

Iris is provided under the BSD 3-Clause License. Modifications are underway.

The following files are derived from third-party libraries.

• iris/src/optimize/types_gicp.hpp : part of g2o (BSD)
• iris/src/optimize/types_gicp.cpp : part of g2o (BSD)
• iris/src/pcl_/correspondence_estimator.hpp : part of pcl (BSD)
• iris/src/pcl_/correspondence_estimator.cpp : part of pcl (BSD)
• iris/src/pcl_/normal_estimator.hpp : part of pcl (BSD)
• iris/src/pcl_/normal_estimator.cpp : part of pcl (BSD)

Reference

• T. Caselitz, B. Steder, M. Ruhnke, and W. Burgard, “Monocular camera localization in 3d lidar maps,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2016, pp. 1926–1931.

http://www.lifelong-navigation.eu/files/caselitz16iros.pdf

iris's Issues

Python bindings

Feature request: Please provide python bindings

Wrong estimate when using my data

Hi @rsasaki0109 , It is an amazing job. Thanks for sharing this package. I am successfully run your demo with openvslam in kitti data. When I change to my data, it seems wrong. I think maybe it is the tf's problem. Can you kindly point out how to prepare the .pcd and .bag?

My equipment:
Lidar : Velodyne 16
Camera : usb camera monocular

My data:
Lidar : lego-loam /laser_cloud_surround pcl_ros bag_to_pcd
Camera : usb_cam /usbcam/image_raw

try to set init pose by lidar odometry

Hi!

Thank you for your great work, I really enjoy and have fun with that on my project.

I have a 3d point cloud map constructed by lidar and I have a global localization module which is we loaded 3d point cloud map at somewhere and find a localization pose by scan matching. and start Localization.

what I want to do is set the pose of global localization to the init pose of your project and start its visual odometry.

I tried to do this work but because of my weak math knowledge, I m struggling with it.

can you point out where should I look and any recommendations?

thank you!

have a nice day.

some error

catkin_make error

hi i want to try to use your research but i got catkin_make error.
can you help me out it?

/home/chan/Project/test_ws/src/src/iris/openvslam_bridge/src/bridge.cpp:102:56: error: ‘using element_type = class openvslam::publish::map_publisher’ {aka ‘class openvslam::publish::map_publisher’} has no member named ‘get_max_keyframe_id’; did you mean ‘get_keyframes’?
  102 |   unsigned int max_id = SLAM_ptr->get_map_publisher()->get_max_keyframe_id();
      |                                                        ^~~~~~~~~~~~~~~~~~~
      |                                                        get_keyframes
/home/chan/Project/test_ws/src/src/iris/openvslam_bridge/src/bridge.cpp:105:47: error: ‘class openvslam::data::landmark’ has no member named ‘last_observed_keyfrm_id_’
  105 |     unsigned int last_observed_id = local_lm->last_observed_keyfrm_id_;
      |                                               ^~~~~~~~~~~~~~~~~~~~~~~~
/home/chan/Project/test_ws/src/src/iris/openvslam_bridge/src/bridge.cpp:104:18: warning: unused variable ‘first_observed_id’ [-Wunused-variable]
  104 |     unsigned int first_observed_id = local_lm->first_keyfrm_id_;
      |                  ^~~~~~~~~~~~~~~~~
/home/chan/Project/test_ws/src/src/iris/openvslam_bridge/src/bridge.cpp: In member function ‘int iris::BridgeOpenVSLAM::getState() const’:
/home/chan/Project/test_ws/src/src/iris/openvslam_bridge/src/bridge.cpp:144:60: error: ‘using element_type = class openvslam::publish::frame_publisher’ {aka ‘class openvslam::publish::frame_publisher’} has no member named ‘get_tracking_state’; did you mean ‘openvslam::tracker_state_t openvslam::publish::frame_publisher::tracking_state_’? (not accessible from this context)
  144 |   return static_cast<int>(SLAM_ptr->get_frame_publisher()->get_tracking_state());
      |                                                            ^~~~~~~~~~~~~~~~~~

I have compiled successfully, but I have run into some problems.

I have compiled successfully, but I have run into some problems.
As shown in the screenshot, I hope to give a solution.thanks！

The posiitoning error is not normal

Hi @rsasaki0109 , Thanks for sharing such a good package. I am replicating the demo that you provide. I got the results as follows:

The data that I use are as follows:

I found that the provided rosbag name is the kitti_00_video.bag, but the bag file you instruct to play is the "rosbag play kitti_00.bag # (on another terminal)".

I also try to slow down the rosbag play speed with 0.2 rates but it still gets large error, Could you please kindly help to point out the reason behind this?

My computer setup is as follows:
Ubuntu 18.04
-- PCL version:
1.7.2
-- OpenCV version:
3.2
-- Eigen3 version:
3.3.7
-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libGL.so
-- g2o version:
1.0.0

build error

with a problem of g2o？or ？
this is the error on terminal Window:
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.hpp:40:12: error: ‘g2o::Matrix3’ has not been declared
using g2o::Matrix3;
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.hpp:41:12: error: ‘g2o::Vector3’ has not been declared
using g2o::Vector3;
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.hpp:49:3: error: ‘Vector3’ does not name a type
Vector3 pos0, pos1;
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.hpp:51:3: error: ‘Vector3’ does not name a type
Vector3 normal0, normal1;
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.hpp:53:3: error: ‘Matrix3’ does not name a type
Matrix3 R0, R1;
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.hpp:64:3: error: ‘Matrix3’ does not name a type
Matrix3 prec0(number_t e); // for target
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.hpp:65:3: error: ‘Matrix3’ does not name a type
Matrix3 prec1(number_t e); // for source
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.hpp:68:3: error: ‘Matrix3’ does not name a type
Matrix3 cov0(number_t e); // for target
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.hpp:69:3: error: ‘Matrix3’ does not name a type
Matrix3 cov1(number_t e); // for source
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.hpp:77:3: error: ‘Matrix3’ does not name a type
Matrix3 cov0, cov1;
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp: In constructor ‘iris::optimize::EdgeGICP::EdgeGICP()’:
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:34:3: error: ‘pos0’ was not declared in this scope
pos0.setZero();
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:35:3: error: ‘pos1’ was not declared in this scope
pos1.setZero();
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:36:3: error: ‘normal0’ was not declared in this scope
normal0 << 0, 0, 1;
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:37:3: error: ‘normal1’ was not declared in this scope
normal1 << 0, 0, 1;
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:38:3: error: ‘R0’ was not declared in this scope
R0.setIdentity();
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:39:3: error: ‘R1’ was not declared in this scope
R1.setIdentity();
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp: In member function ‘void iris::optimize::EdgeGICP::makeRot0()’:
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:44:3: error: ‘Vector3’ was not declared in this scope
Vector3 y;
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:45:3: error: ‘y’ was not declared in this scope
y << 0, 1, 0;
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:46:3: error: ‘R0’ was not declared in this scope
R0.row(2) = normal0;
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:46:15: error: ‘normal0’ was not declared in this scope
R0.row(2) = normal0;
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp: In member function ‘void iris::optimize::EdgeGICP::makeRot1()’:
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:55:3: error: ‘Vector3’ was not declared in this scope
Vector3 y;
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:56:3: error: ‘y’ was not declared in this scope
y << 0, 1, 0;
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:57:3: error: ‘R1’ was not declared in this scope
R1.row(2) = normal1;
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:57:15: error: ‘normal1’ was not declared in this scope
R1.row(2) = normal1;
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp: At global scope:
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:64:1: error: ‘Matrix3’ does not name a type
Matrix3 EdgeGICP::prec0(number_t e)
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:74:1: error: ‘Matrix3’ does not name a type
Matrix3 EdgeGICP::prec1(number_t e)
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:84:1: error: ‘Matrix3’ does not name a type
Matrix3 EdgeGICP::cov0(number_t e)
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:93:1: error: ‘Matrix3’ does not name a type
Matrix3 EdgeGICP::cov1(number_t e)
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp: In member function ‘virtual void iris::optimize::Edge_Sim3_GICP::computeError()’:
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:108:3: error: ‘Vector3’ was not declared in this scope
Vector3 p1 = vp0->estimate().map(measurement().pos1);
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:110:36: error: ‘p1’ was not declared in this scope
_error = measurement().weight * (p1 - measurement().pos0);
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:110:55: error: ‘const Measurement {aka const class iris::optimize::EdgeGICP}’ has no member named ‘pos0’
_error = measurement().weight * (p1 - measurement().pos0);
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:116:9: error: ‘Matrix3’ does not name a type
const Matrix3 R = vp0->estimate().rotation().matrix();
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:117:20: error: ‘cov0’ was not declared in this scope
information() = (cov0 + R * cov1 * R.transpose()).inverse();
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:117:27: error: ‘R’ was not declared in this scope
information() = (cov0 + R * cov1 * R.transpose()).inverse();
^
/home/zlj/catkin_iris/src/iris/iris/src/optimize/types_gicp.cpp:117:31: error: ‘cov1’ was not declared in this scope
information() = (cov0 + R * cov1 * R.transpose()).inverse();
^