srivathsanmurali / domsetlibrary

a library that clusters set of images with poses into clusters for easier mvs

Dominant Set Clustering algorithm

Clustering camera poses generated by a SFM pipeline such as openMVG. The library gives a list of view clusters.

The library is sfm appliction agnostic. It requires view poses and sparse point positions. It uses affinity propogation to cluster poses.

This libary is an implementation of "Dominant set clustering" introduced by Maura, Massimo et al. at ETHZ Zurich Computer Vision Lab

Source

Mauro, Massimo, et al. "An integer linear programming model for view selection on overlapping camera clusters." 2014 2nd International Conference on 3D Vision. Vol. 1. IEEE, 2014.

Mauro, Massimo, et al. ["Overlapping camera clustering through dominant sets for scalable 3D reconstruction."] (http://www.vision.ee.ethz.ch/~rhayko/paper/bmvc2013_mauro_view_clustering.pdf) 2013 British Machine Vision Conference.

Terminology in our library

• Point stores positional informations about the sparse point cloud from SFM.
• Camera stores the intrinsic properties of distinct cameras used in the sytem.
• View stores the extrinsic parameters of cameras poses (translation and rotation).

Usage

#include "domsetLibrary/types.h"
#include "domsetLibrary/domset.h"

#include <vector>

int main() {
  std::vector<nomoko::Point> points;
  // Fill points with sparse point cloud generated durign SFM.
  
  std::vector<nomoko::Camera> cameras;
  // Fill with camera parameters of the distinct cameras.
  
  std::vector<nomoko::Views> views;
  // Fill with poses of images.

  const size_t voxelGridSize = 15;
  nomoko::Domset domset(points, views, camera, voxelGridSize)

  // specify sizes of view clusters
  const size_t clusterSizeLowerBound = 10;
  const size_t clusterSizeUpperBound = 15;
  domset.clusterViews(clusterSizeLowerBound, clusterSizeUpperBound);

  std::vector<std::vector<size_t> > clusters;
  clusters = domset.getClusters();
}

ThirdParty Libraries

• [Eigen] (http://eigen.tuxfamily.org)
• [Nanoflann] (https://github.com/jlblancoc/nanoflann) (internal)

Examples

The clustering was tested on the fraumunster dataset from CVL ETHZ ([dataset] (https://people.ee.ethz.ch/~rhayko/paper/bmvc2013_mauro_view_clustering/pix/bmvc2013_view_clustering_fraumunster.zip)).

Output from OpenMVG

The green points shows the calibrated camera poses.

Output from domset

This image shows the output from our dominant set clustering implementation. The various camera view clusters are shown in different colours.

TODO

• Leveraged Affinity Propogation for datasets for large number of camera views.
• Overlaping of clustering in library not implemented.

Author

Srivathsan Murali (NomokoAG)

domsetlibrary's Issues

How can we implement overlapping clusters in this module?

I wanted to use this module to get overlapping clusters of sfm(openMVG). Can you please suggest me how to do it? or is there any other library to do it?

Overlap constant

Implement a option have overlap in clusters for registration purposes.
+ When finding the center that the view is assigned to also find the next best center.
+ Assign certain number of overlap from the list of views that are in the secondary list for centers
New implementation does the following

• Finds the borders for each cluster
• Finds the closest cluster to each border view and adds them to it

TODO:

• Clean up code
• Test on other datasets
• Border visualization to export PLY

A request from your reader.

I took my first step in scientific research focused on 3D reconstruction.recently I read your paper "An integer linear programming model for view selection on overlapping camera clusters" published in 2014. I think it is very interesting and your clear and compact declaration inspired me a lot.But I got some trouble when I try to implement algorithms about integer linear programming(ILP).

So could you please share your code about ILP if possible?I would be appreciated if you could help me.

Thank you for your consideration and I will be looking forward to your reply.

example not valid

is there a valid link for the example data ?
this link not found :
http://www.vision.ee.ethz.ch/~rhayko//paper/bmvc2013_mauro_view_clustering/pix/bmvc2013_view_clustering_fraumunster.zip

Leveraged affinity propogation to cluster views together.

For large sets of cameras, LAP samples samples from the full set of potential similarities and performs several rounds of affinity propagation, iteratively refining the sample. LAP delivers the same clustering results as AP yet provides a speedup for large dataset.

Difference compare to the paper

I read the implementation & the paper.
I found a difference in the way the voxelization is done.

• In the paper they normalize the scene and then use a fixed size voxelization.
• In your implementation you suppose the scene is in some given units.

I think their normalization way is preferred since it make the process coordinates system free.
We can compute the normalization factor (a scale) and them apply it to a copy of the SfM_Data scene simply by using

const double scale_factor = ...;
openMVG::geometry::Similarity3(geometry::Pose3(Mat3::Identity(), Vec3::Zero()), scale_factor);
openMVG::sfm::ApplySimilarity(sim, sfm_data);

Enforcing min/max cluster size constraints

Current implementation doesn't handle this properly.

• clusters smaller than minClusterSize which don't have any other cluster near by are added just like that. This causes very small clusters.
• clusters larger than maxClusterSize are divided into clusters of size maxClusterSize and the views that are in the last cluster can be less than minClusterSize.

Possible ideas: use iterative method to merge and break until it converges in proper sizes.