DEEP-IMAGE-MATCHING

SIFT	DISK	IMAGES ORIENTATION	DENSE WITH ROMA

SIFT	SUPERGLUE

Multivew matcher for SfM software. Support both deep-learning based and hand-crafted local features and matchers and export keypoints and matches directly in a COLMAP database or to Agisoft Metashape by importing the reconstruction in Bundler format. It supports both CLI and GUI. Feel free to collaborate!

Check the documentation at Docs.

Please, note that deep-image-matching is under active development and it is still in an experimental stage. If you find any bug, please open an issue.

Key features:

Multiview
Large format images
SOTA deep-learning and hand-crafted features
Full compatibility with COLMAP
Support for image rotations
Compatibility with Agisoft Metashape (only on Linux and MacOS by using pycolmap)
Support image retrieval with deep-learning local features

Supported Extractors	Supported Matchers
✓ SuperPoint	✓ Lightglue (with Superpoint, Disk, and ALIKED)
✓ DISK	✓ SuperGlue (with Superpoint)
✓ ALIKE	✓ LoFTR
✓ ALIKED	☐ SE2-LoFTR
☐ Superpoint free	✓ Nearest neighbor (with KORNIA Descriptor Matcher)
✓ KeyNet + OriNet + HardNet8	✓ RoMa
✓ ORB (opencv)	☐ GlueStick
✓ SIFT (opencv)
✓ DeDoDe

Supported SfM software
✓ COLMAP
☐ OpenMVG
✓ Agisoft Metashape
✓ Software that supports bundler format

Colab demo

Want to run on a sample dataset? ➡️

Want to run on your images? ➡️

Installation

For installing deep-image-matching, first create a conda environment:

conda create -n deep-image-matching python=3.9
conda activate deep-image-matching
pip install --upgrade pip

Clone the repository and install deep-image-matching in editable mode:

git clone https://github.com/3DOM-FBK/deep-image-matching.git
cd deep-image-matching
pip install -e .

Install pycolmap:

pip install pycolmap

As pycolmap was released on PyPi only for Linux and macOS (up to version 0.4.0), it is not installed by default with deep-image-matching. From version 0.5.0, pycolmap can be installed on Windows too. However, it will demand some testing before being added to the dependencies of deep-image-matching, as there are some errors on Windows that are blocking deep_image_matching pipeline (while it works completely fine on Linux).

For more information, check the documentation.

Docker Installation

First, build the image:

  docker build --tag 'deep-image-matching' .

Then run the image (on Windows, please use Powershell):

docker run --name running-deep-image-matching --mount type=bind,source=D:\data,target=/workspace/data --gpus all -it 'deep-image-matching'

replace D:\data with the desired path for mounting a volume. If you have more than one running container, you can remove --name running_deep_pavements

include --detach to run in background and/or --rm to remove container on exit

You can then open the repo directly in VSCode using ctrl+alt+O and selecting the option "attach to running container", then enjoy!

Usage instructions

You can run deep-image-matching from the command line or from the GUI.

Use the following command to see all the available options from the CLI:

python main.py --help

For example, to run the matching with SuperPoint and LightGlue on a dataset, you can use the following command:

python main.py --dir assets/example_cyprus --pipeline superpoint+lightglue

For all the usage instructions and configurations, refer to the documenation at https://3dom-fbk.github.io/deep-image-matching/ or check the example notebooks.

To run the GUI, you can use the following command:

python main.py --gui

Advanced usage

For advanced usage, you can check the scripts directory.

Merging databases with different local features

To run the matching with different local features and/or matchers and marging together the results, you can use scripts in the ./scripts directory for merging the COLMAP databases.

python ./join_databases.py --help
python ./join_databases.py --input assets/to_be_joined --output docs/assets/to_be_joined

Exporting the solution to Metashape

To export the solution to Metashape, you can export the COLMAP database to Bundler format and then import it into Metashape. This can be done from Metashape GUI, by first importing the images and then use the function Import Cameras (File -> Import -> Import Cameras) to select Bundler file (e.g., bundler.out) and the image list file (e.g., bundler_list.txt).

Alternatevely, you can use the export_to_metashape.py script to automatically create a Metashape project from a reconstruction saved in Bundler format. The script export_to_metashape.py takes as input the solution in Bundler format and the images and it exports the solution to Metashape. It requires to install Metashape as a Python module in your environment and to have a valid license. Please, refer to the instructions at https://github.com/franioli/metashape.

How to contribute

Any contribution to this repo is really welcome! If you want to contribute to the project, please, check the contributing guidelines.

TODO:

References

If you find the repository useful for your work consider citing the papers:

@article{morelli2022photogrammetry,
  title={PHOTOGRAMMETRY NOW AND THEN--FROM HAND-CRAFTED TO DEEP-LEARNING TIE POINTS--},
  author={Morelli, Luca and Bellavia, Fabio and Menna, Fabio and Remondino, Fabio},
  journal={The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
  volume={48},
  pages={163--170},
  year={2022},
  publisher={Copernicus GmbH}
}

@article{ioli2023replicable,
  title={A Replicable Open-Source Multi-Camera System for Low-Cost 4d Glacier Monitoring},
  author={Ioli, F and Bruno, E and Calzolari, D and Galbiati, M and Mannocchi, A and Manzoni, P and Martini, M and Bianchi, A and Cina, A and De Michele, C and others},
  journal={The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
  volume={48},
  pages={137--144},
  year={2023},
  publisher={Copernicus GmbH}
}

Depending on the options used, consider citing the corresponding work of KORNIA, HLOC, and local features.

kauevestena / deep-image-matching Goto Github PK