sbrunoberenguel / omniscv

Omnidirectional Synthetic image generator for Computer Vision

OmniSCV: Omnidirectional Synthetic image generator for Computer Vision

Python implementation for a camera simulator.

Introduction

Camera models

In this simulator we include two types of projection models, central and no-central. For the central cameras, we have implemented:

• Equirectangular
• Cylindric
• Catadioptric cameras:
  • Para-Catadioptric
  • Hyper-Catadioptric
  • Planar-Catadioptric
• Fish eye cameras:
  • Equiangular
  • Stereographic
  • Equi-solid angle
  • Orthogonal
• Scaramuzza
• Kannala-Brandt

For no-central cameras, we have implemented:

• Panoramic
• Catadioptric cameras:
  • Spheric mirror
  • Conic mirror

Citing

If you use and like our tool, please cite our paper OmniSCV: An Omnidirectional Synthetic Image Generator for Computer Vision

Use Instructions

Install Requirements

This code has been compiled and tested in Linux using:

• python 2.7
• python 3.6

Package requirements:

To use this image simulator, first you have to have installed the following python packages. The version of each package is that of development, other versions might work but are not tested.

• UnrealCV
• Numpy (version 1.16.6)
• Math
• Pillow (version 5.1.0)
• StringIO
• os
• scipy.optimization (version 1.2.1)
• cv2 (version 4.2.0)
• Open3D (version 0.9.0.0)

User guide

First steps

After download this repository on your own PC, you have to copy the simulator folder called "Pack" in the same directory where the UnrealCV executable is located. In the example we will see how this path looks like.

With our "Pack" placed in the correct directory, we enter the folder and we will find several files. The main script is "simulator.py". From this script we will run the image simulator OmniSCV and we will be able to obtain the images from the virtual environment. The other files we can find in the folder are:

• cam_loc.txt : in this file we will specify the locations from where we want to take the images
• cam_rot.txt : in this file we will set the rotation for each location. This file is prone to use for trajectories.
• Other : this files will not be used by the user, are for support the main program

OmniSCV

These cameras are modeled acording different mathematical models. These models are explained with detail in the papers of Jesus Bermudez-Cameo

Once known the models, we start to build the images of this simulator mapping our environment in a sphere arround the camera. As an sphere is difficult to obtain from planar images, we aproximate it with a cube-map formed by 6 images in the main directions (the cartesian axis X, Y, Z). We build a cube-map for each location in our scenario where we want to make an image. To do so with our simulator, we first need to set the locations where we want to take the images. Editing the file 'cam_loc.txt', we set the different locations where we are going to get the cube-maps. In this file we have to write the coordinates (x,y,z) in one line and separated by white spaces for each location.

Defined each location in the environment from where we take the acquisition, we run our main program 'simulator.py'. After we run the program, a menu will appear (If it takes more than 5 seconds to appear we recomend to close and reopen the simulator). From this menu we will be able to chose differenct functions.

The first one to appear is Acquisition. This function makes an image acquisition from the virtual environment for the composition of central projection images.

The second option is Composition. Here we can choose which kind of image we want to compose. It's divided between central and non-central projection systems. For the central projection systems we need to build the cube-maps (made with the acquisition). For the non-central systems, since acquisition and composition are mixed, you can do the composition directly, without previous acquisition.

Next comes Scenario. It will give the name of the scenario to the composed images. Usefull for large data-sets with different scenarios.

And last but not least, Functions. It's under development, stil have some bugs. However, we can obtain ground truth information of the layout of the scene extracting the layout of the environment.

	$ python simulator.py

	 Welcome to the image simulator OmniSCV
	-------------------------------------
	This simulator has been made at the University of Zaragoza, Spain
	We hope you enjoy it
	-------------------------------------

	 Main menu:
	  (Ac)quisition
	  (Co)mposition
	  (S)cenario name
	  (F)unctions
	  (E)xit

	 What do you want to do?: 

Downloads

Data set

Omnidirectional images from different indoor environments generated with Unreal Engine 4: download

Scenarios

UnrealCV scenarios: download

License

This software is under GNU General Public License Version 3 (GPLv3), please see GNU License

For commercial purposes, please contact the authors: Bruno Berenguel-Baeta ([email protected]), Jesús Bermudez-Cameo ([email protected]) and Josechu Guerrero ([email protected])

Note from the authors

This code has not been thoroughly tested, which means it may have some bugs. Please use with caution.

The authors and developers of this code want the best for the users and have gone to great lengths to make it easy to use and accessible. Be nice to them and enjoy their work.

If any problem may appear, do not hesitate and we will do our best to solve it (at least we will try).