Camera/IMU Calibration Files about illixr

[charmoniumQ]

I believe the camera plugin (ZED, Realsense, offline_imu_cam) should synchronously publish the camera parameters that it knows about SLAM implementation should synchronously listen for that.

Some parameters will still have to come from a config file. For example, offline_imu_cam can be used with a variety of cameras (not just EuRoC).

The plugin hardcodes parameters it knows and uses #198 to override and publishes to SLAM.

[ltstein]

Is there any consensus on whether this is read in or published by the camera plugin? This feature would be crucial for new hardware experiments, such as supporting #280 or enhancing existing camera plugins.

[mhuzai]

Some initial thoughts:

• For datasets, we should have a config file and read from it; e.g., like that of OpenVINS or that of Kimera-VIO (the IMU file is in the same folder). I personally like Kimera's yaml better.
• For live sensors, we should either query the parameters from the API (for factory defaults; e.g., with RealSense) or read from a config file (in case we do our own calibration; e.g., like ZED).
• The cleanest way of supporting both of the above is to have the camera/offline_imu_cam plugin get the parameters (config or API), and publish them on SB for consumption by the rest of the system. We'll have to define an interface for this.

[ltstein]

Putting down some ideas for feedback:

• Pull in something like the camera params parser code for reading the yaml.
  Are there issues/objections to using this as a starting point?
• configure a format for the params as well as a writer in data_format.hpp Initially would probably just mimic the yaml layout and iterate from there.
• imu_cam_type and rgb_depth_types would publish this data synchronously with their frames or perhaps as part of setup to limit overhead?
• Consumers of the cam topics would need to be modified to utilize this feed internally.

[mhuzai]

Thanks for looking into this!

• Kimera has a BSD license, so no issues in using that code as a reference. That being said, using some kind of standard YAML library (if one exists) might be better for maintainability.
• Yup, starting point would probably be a struct consisting of something along the lines of Eigen matrices for intrinsics and extrinsics, and floats for IMU parameters. I would suggest making something like sensor_params_format.hpp and then including it in data_format.hpp.
• Sensor calibration will not change during execution (online calibration is internal to the VIO/SLAM), so publishing the parameters with each image would be excessive. This should just be a one time thing at system startup. My recommendation would be to make a new service (not a plugin) that reads the parameters from file or from the sensor API in its constructor, and then plugins can query the info using the service. The complication here is that if we're using a live sensor, the service would have to repeat the sensor code to get the information. The alternative is offline_imu_cam or the sensor publishing the info on a dedicated topic at startup.
• Tying in with the above point, consumers of imu_cam that care about calibration only need calibration data at startup, and so should query the info in the constructor.