kuui24 / deep_visual_inertial_odometry

Deep Learning for Visual-Inertial Odometry

Deep Visual Inertial Odometry

Deep learning based visual-inertial odometry project.
pros:

• Lighter CNN structure. No RNNs -> much lighter.
• Training images together with inertial data using exponential mapping.
• Rotation is coming from external attitude estimation.
• No RNN but Kalman filter: Accleration and image fusion for frame-to-frame displacement.

cons:

• no position correction: drift in position: But SLAM can correct the position drfit.

Please Cite:

Hongyun Lee, James W. Gregory, Matthew McCrink, and Alper Yilmaz. "Deep Learning for Visual-Inertial Odometry: Estimation of Monocular Camera Ego-Motion and its Uncertainty" The Ohio State University, Master Thesis, http://rave.ohiolink.edu/etdc/view?acc_num=osu156331321922759

References(current & future)

Please see paper.

Usage:

0. git clone -- recursive https://github.com/ElliotHYLee/Deep_Visual_Inertial_Odometry
1. Put the .m (Matlab) files under KITTI/odom/dataset/. The files are at DataGenerator folder.
2. run make_trainable_data.m
3. In src/Parampy, change the path for KITTI.
4. At Deep_Visual_Inertial_Odometry, "python main.py"

ToDo

• upload weight.pt
• change Matlab data get to python

Prereq.s

1. Matlab
2. Python 3.5

pip install numpy
pip install scipy
pip install pandas
pip install matplotlib
pip install scikit-learn
pip install pathlib
pip install pypng
pip install pillow
pip install django
pip install image
pip install opencv-python opencv-contrib-python

detail: https://github.com/ElliotHYLee/SystemReady

Tested System

• Hardware

CPU: i9-7940x
RAM: 128G, 3200Hz
GPU: two Gefore 1080 ti
MB: ROG STRIX x299-E Gaming

• Software

Windows10
Python3
PyTorch: v1
CUDA: v9.0
Cudnn: v7.1

Run

python main_cnn.py

Traing Results

description

Test Results

Correction Result