This toolset provides the basics for calibrating a multi-camera scene. it contains six utilities for different purposes. In this README I will walk the user through the calibration of a multi camera scene using this toolset.
the use of this suite requires
clone the repository using :
git clone [email protected]:idiap/multicamera-calibration.git
Go to the source directory
Do
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make
The intrinsic camera parameters are usually computed finding a known grid with multiple poses on different images. To do so we propose the following method.
Capture a video by waving the camera over a grid such as
this one or
that one. Extract then the frames as images in
a folder then run the software bin\intrinsic
intrinsic [-h] [-n NUM] [-W WIDTH] [-H HEIGHT]
[-p {chessboard,circles,asymmetric_circles}] [-o FULL_OUTPUT]
[-s SELECT_OUTPUT]
input output
-h: show help
-n: number of frames to use
-w: width of grid
-h: height of grid
-p: type of grid
-o: output for all frames with a visible grid (optionnal)
-s: output for all n frames (option -n) selected for the calibration computation (optionnal)
input: pattern for the images
output: calibration file (json format)For example: let's imagine we have the frames in the folder /home/user1/camera1/frames with the asymmetric circles grid provided earlier.
you would use the script this way
bin/intrinsic -o /home/user1/frames_with_grid -s /home/user1/camera1/selected_frames_with_grid /home/user1/camera1/frames/\*.bmp /home/user1/camera1/intrinsic.jsononce the computation is done, the undistorted frames are shown.
- 'n' goes to next frame
- 'p' goes to previous frame
- 'q' quits
this produces the folders with the usable frames and the selected frames so that reproducing the calibration takes less time.
The extrinsic parameters are computed by clicking on points with known coordinates (in cm) in an image. The syntax is the following.
extrinsic [-h] [-p POINTS] [-o OUTPUT_POINTS] intrinsic input output
-h: show help
-p: saved points file, used to resume or correct the calibration (optionnal)
-o: output point file. this file will be used with the -p option (optionnal)
intrinsic: intrinsic camera parameters
input: image to annotate
output: calibration outputNow using the intrinsic.json file we computed on the previous step, do.
bin/extrinsic /home/user1/camera1/intrinsic.json /home/user1/camera1/video_frames/000000.bmp /home/user1/camera1/extrinsic.jsonit shows an the frame 000000.bmp you can left click on a point to add it. it will ask for x and y in cm. right click removes last point.
- 'q' quits without saving
- 's' computes the extrinsics and saves
Now both the intrinsic and extrinsic calibrations have been computed for cameras 1 to 4 to check the correspondance between the cameras, you can use the following utility
check3d [-h] intrinsic extrinsic images cameras
-h: help
intrinsic: pattern to intrinsic calibrations files
extrinsic: pattern to extrinsic calibrations files
images: pattern to camera images (same frame for each camera)
cameras: camera list comma separatedusing all our cameras calibrated, we would go to something like that
bin/check3d /home/user1/{}/intrinsic.json /home/user1/{}/extrinsic.json /home/user1/{}/video_frames/000000.bmp camera1,camera2,camera3,camera4once the soft launched you get an image from the frame list. you can click on a point in some views, it will appear as a blue dot. once you clicked at least two views you get a red dot representing the projection on the other views. left click removes the blue dot. if you clicked wrong, a right click moves the blue dot to the new location.
- pressing 'n' goes to the next image
- pressing 'p' goes to the previous image
- pressing 'r' resets the points
- pressing 'q' quits the soft
to perform the bundle adjustment you need to provide annotations in the .pos format the pos format has one file per frame such as /home/user1/camera1/annotations/000000.pos is the annotation for the /home/user1/camera1/video_frames/000000.bmp image.
the pos format is done as such:
identity_name_1
0 x_val y_val
1 x_val y_val
identity_name_2
0 x_val y_val
1 x_val y_val
point number 0 corresponds to feet, 1 to head. point numbers can be sparse and identities don't have to exist on each frame
once some frames are annotated, run
build_ba [-h] [-f FIXED_POINTS]
intrinsic extrinsic observations cameras output
-h: help
-f: pattern to known points that must remain at the same place
i.e. points outputed by the -o option in extrinsic calibration (optionnal)
intrinsic: pattern to intrinsic calibrations files
extrinsic: pattern to extrinsic calibrations files
observations: pattern to pos files
cameras: camera list comma separated
output: bundle adjustment problem filebin/build_ba -f /home/user1/{}/points.json /home/user1/{}/intrinsic.json /home/user1/{}/extrinsic.json /home/user1/{}/annotations/\*.pos /home/user1/ba_problem.txtthen run the bundle adjustment
bundle_adjuster --input=input --output=output --intrinsic_adjustment=[fixed, unconstrained, two_pass]
--input: bundle adjustment problem file
--output: bundle adjustment problem file, post processing
--intrinsic_adjustment: type of adjustment to performthere are three types of adjustment
- fixed: the intrinsics are fixed, only perform on the extrinsics
- unconstrained: adjust all parameters
- two_pass: run once fixed and then unconstrained
using our example, run
bin/bundle_adjuster --input=/home/user1/ba_problem.txt --output=/home/user1/processed_ba.txtthen we need to convert back the bundle adjustment problem file to camera calibrations
extract_ba [-h] input intrinsic extrinsic cameras
-h: help
input: bundle adjustment problem file
intrinsic: output to intrinsic calibration file
extrinsic: output to extrinsic calibration file
cameras: camera list comma separatedIn our example it translates to
bin/extract_ba /home/user1/processed_ba.txt /home/user1/{}/intrinsic_ba.json /home/user1/{}/extrinsic_ba.json camera1,camera2,camera3,camera4then you may check again the projections in 3d like seen previously
bin/check3d /home/user1/{}/intrinsic_ba.json /home/user1/{}/extrinsic_ba.json /home/user1/{}/video_frames/000000.bmp camera1,camera2,camera3,camera4
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent
