fengyunxuan / liff Goto Github PK

LiFF Light Field Feature Toolbox
Copyright (C) 2019, Donald G. Dansereau

This is an implementation of the light field feature detector and descriptor from the paper:

[1] D. G. Dansereau, B. Girod, and G. Wetzstein, “LiFF: Light field features in scale and depth,” in Computer Vision and Pattern Recognition (CVPR), 2019. Paper and supplemental information are at https://roboticimaging.org/Tools/LiFF/.

The functionality closely mirrors the SIFT feature detector and descriptor, but exploits information in the light field to deliver more robust and informative features. This works by searching for features with well-defined scale, as in SIFT, but also with well-defined depth, which manifest as slope in the light field. This rejects more spurious features, detects partially occluded features, and builds descriptors with greater immunity to partial occlusions, noise, and higher-order light transport effects. It also delivers a per-feature depth (slope) estimate.

Presently the toolbox is written for and comes with precompiled binaries for use from within MATLAB. If there's a demand this can be extended for use from C and Python. Usage closely mirrors the VLFeat Library, and much of the extensive documentation and tutorial material for SIFT features at http://www.vlfeat.org/ also applies to LiFF.


---Getting started---
Clone the repo from https://github.com/doda42/LiFF*
Download the sample light fields https://dgd.vision/Tools/LiFF and extract into the SampleScenes folder
cd into the LiFF folder
run LiFF_DemoSimple

*On Linux systems you need to tell ldconfig where to find libvl.so. One way to do this is to add the following to .bashrc, changing <path to LiFF> appropriately:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/<path to LiFF>/lib/vlfeat-0.9.21/toolbox/mex/mexa64/

When calling from your own code it's convenient to add LiFF to the MATLAB path; you can do this manually, or by running the convenience function LiFF_PathSetup.


---Key files---
LiFF_PathSetup.m: add the LiFF light field feature toolbox to MATLAB's path -- it may be convenient to add this to your startup.m file

LiFF_ExtractFeatures.m: the main function for detecting and describing features, similar in usage to VL_SIFT

LiFF_DemoSimple.m: simple usage example

LiFF_DemoColMapOut.m: A more complex example including output for use with the COLMAP structure-from-motion tool, and manual control over feature detection parameters

LiFF_DemoFocalStack.m: Demonstrates use of the LiFF focal stack implementation


---Building---
The release includes prebuilt binaries for Windows and Linux, and the demo should run out of the box.

To rebuild the mex binary files from source, use the make_*.m scripts.


---Resources---
Further example light fields are available at lightfields.stanford.edu.

Additional tools for working with light field fields are available at github.com/doda42/LFToolbox.


---Contributing / Feedback---
Suggestions, bug reports, code improvements and new functionality are welcome -- email Donald.Dansereau+LFToolbox {at} gmail dot com.


---Acknowledgements---
Based on code from the VLFeat library v0.9.21
Copyright (C) 2007-11, Andrea Vedaldi and Brian Fulkerson
Copyright (C) 2012-13, The VLFeat Team
See COPYING.txt


---Citing---
The appropriate citation for LiFF light field features is:

@inproceedings{dansereau2019liff,
  author = {Donald G. Dansereau and Bernd Girod and Gordon Wetzstein},
  title = {{LiFF}: Light Field Features in Scale and Depth},
  booktitle = {Computer Vision and Pattern Recognition ({CVPR})},
  year = {2019},
  month = jun,
  organization={IEEE}
}