xmba15 / automatic_lidar_camera_calibration

auto-calibration of lidar and camera based on maximization of intensity mutual information

📝 Automatic Targetless Lidar Camera Calibration

---

Auto-calibration of lidar and camera based on maximization of intensity mutual information. This is the reimplementation of the paper: Automatic Targetless Extrinsic Calibration of a 3D Lidar and Camera by Maximizing Mutual Information

🎉 TODO

---

• reimplementation of Automatic Targetless Extrinsic Calibration of a 3D Lidar and Camera by Maximizing Mutual Information
• test on real lidar, camera data

🎛 Dependencies

---

• tested on Ubuntu 20.04

sudo apt-get install \
    libopencv-dev \
    libpcl-dev \
    rapidjson-dev \

🔨 How to Build

---

make default -j`nproc`

# build examples
make apps -j`nproc`

🏃 How to Run

---

• Download livox lidar data together with images from here(If you are interested, you can search for more about low-cost livox lidars).
• Extract the sample data, and create two files images.txt and point_clouds.txt that store absolute paths to image, pcd data file respectively. One line for one file.
• Create camera info with camera instrinsic matrix, a sample is provided here
• Create initial guess of transformation info from lidar to camera, in the form of translation(tx, ty, tz), rotation(roll, pitch, yaw); a sample is provided here
• Fill absolute paths to the above files in calibration_handler_param.json
• Run (after make apps)

    ./build/examples/sensors_calib_app ./data/samples/calibration_handler_param.json

After the optimization finishes, the final transformation info will be printed out.

Also the projected (image to) pointclouds; (pointcloud to) images will be saved. Check cloud*.pcd and img*.png files.

• Here is the sample results:

projected cloud:

projected image:

💎 References

---

• Automatic Targetless Extrinsic Calibration of a 3D Lidar and Camera by Maximizing Mutual Information, AAAI 2012
• Automatic Calibration of Lidar and Camera Images using Normalized Mutual Information, ICRA 2013
• Accurate Extrinsic Calibration between Monocular Camera and Sparse 3D Lidar Points without Markers, IV 2017

automatic_lidar_camera_calibration's Issues

Camera distortion parameters Problem

Thanks for your open-source project!
I have ran your sample data. It's wonderful!
But after I use my data, registration mistake happened, Like this:

It may be the result of different distortion parameters. However, I can't find the position to change distortion parameters.
SO, could you please tell me where can I change distortion parameters?
Or, is this project take camera distortion effect into consideration?

problem while running ./build/examples/sensors_calib_app

After filingl absolute paths to the files in calibration_handler_param.json，
I run the examples by " ./build/examples/sensors_calib_app ./data/samples/calibration_handler_param.json" ,
but the terminal says "number of images and point clouds must be the same".
The variable value is "imagePaths.size()= 10243 pointcloudPaths.size()= 32043" by printed in terminal.

absolute paths filled in calibration_handler_param.json is as follows:
"path_to_initial_guess": "/home/callmelxh/Code/automatic_lidar_camera_calibration-master/data/samples/initial_guess.json",
"path_to_images": "/home/callmelxh/Code/automatic_lidar_camera_calibration-master/data/samples/image.png",
"path_to_point_clouds": "/home/callmelxh/Code/automatic_lidar_camera_calibration-master/data/samples/pointcloud.pcd",
"path_to_camera_info": "/home/callmelxh/Code/automatic_lidar_camera_calibration-master/data/samples/camera_info.json",

Could you help me slove this problem ,Thanks.

Question about estimateJS

Thanks for great implement.
the function 'estimateJS' confused me, could you please give some explain？
https://github.com/xmba15/automatic_lidar_camera_calibration/blob/master/src/ProbabilityHandler.cpp

could it work on hesai lidar or velodyne ?