pxl-th / monodepth2.jl

Self-supervised monocular depth estimation

Monodepth2.jl

Implementation of the "Digging Into Self-Supervised Monocular Depth Estimation" paper.

Monodepth

Monocular depth estimation. Using single image to predict disparity map.

Training parameters:

• resolution 416x128;
• ResNet 18 model;
• no automasking & using pose prediction network.

Supported datasets

• KITTI odometry
• CommaAI Depth10k

Install

Install model, image-augmentation library and the package itself:

]add https://github.com/pxl-th/ResNet.jl.git
]add https://github.com/pxl-th/Augmentations.jl.git
]add https://github.com/pxl-th/Monodepth2.jl.git

Simple disparity estimation

Simple disparity estimation using gradient descent with parameters:

• disparity map;
• rotation vector (so3);
• translation vector.

Visualization of the disparity map learning dynamics for the triplet above.

Important

• Norm function is computed using sqrt.(sum(abs2, ...)). However, sqrt function has NaN gradient at 0. This can be mitigated by defining subgradient or even better, norm function that can act on the given axis, similar to PyTorch.

• For poses, struct Pose is used instead of arrays or tuple because of this issue.

monodepth2.jl's Issues

Are there plans to implement and upgrade to ManyDepth?

Thank you for the very good implementation of Monodepth2.
The manydepth is the update of Monodepth2, It show a better effect.

can you implement and upgrade to ManyDepth?

Allow training using known poses instead of pose NN

Allow training on datasets with different intrinsics, target & source ids