aelmiger / line-mod-pipeline

Pipeline to detect the 3D pose of known objects. It builds upon the opencv implementation of the LINE-MOD algorithm.

LINE-MOD-Pipeline for object detection and pose estimation

Real-time 6DOF pose estimation for an industrial bin picking scenario

Table of Contents

1. About The Project
   • Built With
2. Getting Started
   • Installation
3. Usage
4. License
5. Contact

About The Project

The pipeline implements the OpenCV LINE-MOD Detector to estimate the pose of a known object. Postprocessing steps for detection of shiny objects with corrupted depth information are added. The goal of this pipeline is to locate and detect the pose of industrial objects like bearings to aide a robotic arm during assembly (bin picking). The pipeline implements the creation of templates from CAD objects, which makes it possible to detect new objects in a matter of minutes.

The accompanying bachelor thesis can be read here (german): Bachelor Thesis (german)

Necessary Pre- and Postprocessing tools that are implemented:

• Generation of Templates from CAD-Models with OpenGL
• Color and depth checks to validate a match
• Steps to correct for corrupted depth information on reflective objects
• ICP algorithm for pose refinement

Included libraries for testing and validation are:

• Kinect V2 tool to extract RGB-D images
• Implementation of the LINE-MOD Benchmark and the Benchmark for 6D Object Pose Estimation evaluation methods
• A class to generate RGB-D Images with a ground truth pose of an Aruco marker

Built With

For compilation the following libraries must be included and linked

• SDL2 - Creates a window for OpenGL
• GLEW - OpenGL Extension Library
• OPENCV4 - Computer vision library
• GLM - Graphics math library
• libfreenect2 - Kinect V2 driver (only if Kinect is used)
• assimp - Open Asset Importer library

Getting Started

Dependencies

For Linux the following command can be used to install most dependencies

sudo apt-get install libglew-dev libglm-dev libassimp-dev libsdl2-dev

freenect2 has to be compiled from source with the help of cmake.

OpenCV 4 needs to be built with:

• additional contrib libraries for LINE-MOD and ICP
• OpenMP

To compile OpenCV the following command can be used. The path of the contrib modules has to be replaced.

cmake -D OPENCV_EXTRA_MODULES_PATH=<contrib modules path> -D WITH_OPENMP=ON -D CMAKE_BUILD_TYPE=Release

Installation

Compiling the pipeline

In a new folder start by cloning the repo.

git clone https://github.com/aelmiger/LINE-MOD-Pipeline.git LINE-MOD
cd LINE-MOD

Next step is to run cmake and compile the binaries.

For cmake to properly locate the SDL2 libraries it needs a FindSDL2.cmake module which is not yet part of cmake.

FindSDL2 can be downloaded here and has to be placed in the cmake modules folder.

cmake -H. -B build
cmake --build build --config Release --target all -- -j4

Usage

To test the applications run the template generator first.

./build/Template_Generator

A window should open and display the rendered object in white under different poses and save the template generation files.

Now run the detector. An example image in the benchmark folder will be used.

./build/Detector

License

This project is licensed under the BSD License - see the LICENSE.md file for details

Contact

Anton Elmiger - [email protected] - email