Coder Social home page Coder Social logo

sky-github / stereo_sparse_depth_fusion Goto Github PK

View Code? Open in Web Editor NEW

This project forked from shreyasskandans/stereo_sparse_depth_fusion

0.0 2.0 0.0 7.34 MB

Repository for "Real Time Dense Depth Estimation by Fusing Stereo with Sparse Depth Measurements" | OpenCV, C++

License: GNU General Public License v3.0

CMake 1.38% C++ 98.62%

stereo_sparse_depth_fusion's Introduction

Stereo and Sparse Depth Fusion

This is the corresponding code repository for our paper Real Time Dense Depth Estimation by Fusing Stereo with Sparse Depth Measurements

Reference

If you use this work, please cite our paper:

@article{shivakumar2018real,
  title={Real Time Dense Depth Estimation by Fusing Stereo with Sparse Depth Measurements},
  author={Shivakumar, Shreyas S and Mohta, Kartik and Pfrommer, Bernd and Kumar, Vijay and Taylor, Camillo J},
  journal={arXiv preprint arXiv:1809.07677},
  year={2018}
}

In this repository, we also use the Semi Global Matching algorithm implementation by Koichiro Yamaguchi, therefore if you use this repository, please also cite the original paper by Koichiro et al. and follow their original license agreements.

@inproceedings{yamaguchi2014efficient,
  title={Efficient joint segmentation, occlusion labeling, stereo and flow estimation},
  author={Yamaguchi, Koichiro and McAllester, David and Urtasun, Raquel},
  booktitle={European Conference on Computer Vision},
  pages={756--771},
  year={2014},
  organization={Springer}
}

Dependencies

  1. OpenCV (Tested on 3.X.X)
  2. libpng++ (Tested on 0.2.5-1)
sudo apt-get install libpng++-dev
  1. OpenMP (optional)
  2. CMake

Testing Code

Navigate to the main code repository

mkdir build
cd build
cmake ..
make -j

You can run our KITTI example as follows

./stereo_depth_fusion

You should now see the following images in the results folder:

  • sgm_default.png - Output of SGM algorithm sgm
  • sparse_mask.png - Mask of ground truth disparities used sparsemask
  • fuse_naive.png - Output of our naive fusion implementation naive
  • fuse_neighborhoodsupport.png - Neighborhood Support Method neighborhood
  • fuse_diffusionbased.png - Diffusion Based Method diffusion

Understanding The Code

The entire Semi Global Matching implementation is from Koichiro Yamaguchi and lives in the SGMStereo.cpp and SGMStereo.h files. The adjustments to the cost volumes are made in SGMStereo.cpp in the following functions:

  • updateCostVolume_NRF() - Naive Fusion
  • updateCostVolume_NS() - Neighborhood Support
  • updateCostVolume_DB() - Diffusion Based Method

These methods are called by a helper script stereo_depth_fusion.cpp and is tested on a pair of randomly chosen KITTI 2015 images.

Results

KITTI 2015

From top to bottom:

  • Input color image (left)
  • Semi Global Matching (original)
  • Neighborhood Support Method (ours)
  • Diffused Based (ours)
  • Anisotropic Diffusion
  • Ground Truth (LiDAR)

Images:

Middlebury

From left to right:

  • Input color image (left)
  • Semi Global Matching (original)
  • Neighborhood Support Method (ours)
  • Diffusion Based (ours)
  • Anisotropic Diffusion
  • Ground Truth

Images:

PMD Monstar Dataset (internal)

From left to right:

  • Input color image (left)
  • Semi Global Matching (original)
  • Neighborhood Support Method (ours)
  • Diffusion Based (ours)
  • Anisotropic Diffusion
  • Mask of Time-of-Flight depth measurements

Images:

If you have any questions regarding this repository or our method feel free to raise an issue request in this repository or email me directly.

Note: this repository is part of a larger on-going research project, and therefore contains multiple variables that are unused (but are used in following research). I've tried to add ignores to most of them, but please excuse these if they occur. Feel free to make pull requests if you notice any bugs or wish to cleanup this code.

stereo_sparse_depth_fusion's People

Contributors

shreyasskandan avatar shreyasskandans avatar

Watchers

James Cloos avatar WxHu avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.