michaelgrupp / evo

Python package for the evaluation of odometry and SLAM

evo

Python package for the evaluation of odometry and SLAM

Linux / macOS / Windows / ROS / ROS2

This package provides executables and a small library for handling, evaluating and comparing the trajectory output of odometry and SLAM algorithms.

Supported trajectory formats:

• 'TUM' trajectory files
• 'KITTI' pose files
• 'EuRoC MAV' (.csv groundtruth and TUM trajectory file)
• ROS and ROS2 bagfile with geometry_msgs/PoseStamped, geometry_msgs/TransformStamped, geometry_msgs/PoseWithCovarianceStamped, geometry_msgs/PointStamped or nav_msgs/Odometry topics or TF messages

See here for more infos about the formats.

---

Why?

evo has several advantages over other public benchmarking tools:

• common tools for different formats
• algorithmic options for association, alignment, scale adjustment for monocular SLAM etc.
• flexible options for output, plotting or export (e.g. LaTeX plots or Excel tables)
• a powerful, configurable CLI that can cover many use cases
• modular core and tools libraries for custom extensions
• faster than other established Python-based tools (see here)

What it's not: a 1-to-1 re-implementation of a particular evaluation protocol tailored to a specific dataset.

---

Installation / Upgrade

Installation is easy-peasy if you're familiar with this: https://xkcd.com/1987/#

evo supports Python 3.8+. You might also want to use a virtual environment.

From PyPi

If you just want to use the executables of the latest release version, the easiest way is to run:

pip install evo --upgrade --no-binary evo

This will download the package and its dependencies from PyPI and install or upgrade them. Depending on your OS, you might be able to use pip2 or pip3 to specify the Python version you want. Tab completion for Bash terminals is supported via the argcomplete package on most UNIX systems - open a new shell after the installation to use it (without --no-binary evo the tab completion might not be installed properly). If you want, you can subscribe to new releases via https://libraries.io/pypi/evo.

From Source

Run this in the repository's base folder:

pip install --editable . --upgrade --no-binary evo

Dependencies

Python packages

evo has some required dependencies that are automatically resolved during installation with pip. They are specified in the install_requires part of the setup.py file.

PyQt5 (optional)

PyQt5 will give you the enhanced GUI for plot figures from the "Qt5Agg" matplotlib backend (otherwise: "TkAgg"). If PyQt5 is already installed when installing this package, it will be used as a default (see evo_config show). To change the plot backend afterwards, run evo_config set plot_backend Qt5Agg.

ROS (optional)

Some ROS-related features require a ROS installation, see here. We are testing this package with ROS Noetic and Iron. Previous versions (<= 1.12.0) work with Melodic, Kinetic and Indigo.

Note: reading ROS bag files works also without a ROS installation thanks to the great rosbags package that is installed together with evo. This allows you also to read ROS 1 & 2 bags even if you don't have one of those ROS distros installed. (except for reading /tf topics, because there we need the buffer implementation from ROS)

---

Command Line Interface

After installation with setup.py or from pip, the following executables can be called globally from your command-line:

Metrics:

• evo_ape - absolute pose error
• evo_rpe - relative pose error

Tools:

• evo_traj - tool for analyzing, plotting or exporting one or more trajectories
• evo_res - tool for comparing one or multiple result files from evo_ape or evo_rpe
• evo_fig - (experimental) tool for re-opening serialized plots (saved with --serialize_plot)
• evo_config - tool for global settings and config file manipulation

Call the commands with --help to see the options, e.g. evo_ape --help. Tab-completion of command line parameters is available on UNIX-like systems.

More documentation Check out the Wiki on GitHub.

---

Example Workflow

There are some example trajectories in the source folder in test/data.

1.) Plot multiple trajectories

Here, we plot two KITTI pose files and the ground truth using evo_traj:

cd test/data
evo_traj kitti KITTI_00_ORB.txt KITTI_00_SPTAM.txt --ref=KITTI_00_gt.txt -p --plot_mode=xz

2.) Run a metric on trajectories

For example, here we calculate the absolute pose error for two trajectories from ORB-SLAM and S-PTAM using evo_ape (KITTI_00_gt.txt is the reference (ground truth)) and plot and save the individual results to .zip files for evo_res:

First trajectory (ORB Stereo):

mkdir results
evo_ape kitti KITTI_00_gt.txt KITTI_00_ORB.txt -va --plot --plot_mode xz --save_results results/ORB.zip

Second trajectory (S-PTAM):

evo_ape kitti KITTI_00_gt.txt KITTI_00_SPTAM.txt -va --plot --plot_mode xz --save_results results/SPTAM.zip

3.) Process multiple results from a metric

evo_res can be used to compare multiple result files from the metrics, i.e.:

• print infos and statistics (default)
• plot the results
• save the statistics in a table

Here, we use the results from above to generate a plot and a table:

evo_res results/*.zip -p --save_table results/table.csv

---

IPython / Jupyter Resources

For an interactive source code documentation, open the Jupyter notebook metrics_tutorial.ipynb in the notebooks folder of the repository. More infos on Jupyter notebooks: see here

If you have IPython installed, you can launch an IPython shell with a custom evo profile with the command evo_ipython.

---

Contributing Utilities

A few "inoffical" scripts for special use-cases are collected in the contrib/ directory of the repository. They are inofficial in the sense that they don't ship with the package distribution and thus aren't regularly tested in continuous integration.

---

Trouble

":scream:, this piece of :shit: software doesn't do what I want!!1!1!!"

First aid:

• append -h/ --help to your command
• check the Wiki
• check the previous issues
• open a new issue

---

Contributing

Patches are welcome, preferably as pull requests.

License

GPL-3.0 or later

If you use this package for your research, a footnote with the link to this repository is appreciated: github.com/MichaelGrupp/evo.

...or, for citation with BibTeX:

@misc{grupp2017evo,
  title={evo: Python package for the evaluation of odometry and SLAM.},
  author={Grupp, Michael},
  howpublished={\url{https://github.com/MichaelGrupp/evo}},
  year={2017}
}

evo's Issues

evo_rpe vs. evaluate_rpe.py

Bug / Error report

Description:
RPE results are not correct when using --all_pairs.
It seems like it is just taking the consecutive frames (I get same results when replacing --all_pairs for -r trans_part)
Instead it should compute the errors for all possible pairs.
I get different results when using TUM's official implementation: evaluate_rpe.py
I would have done a PR, but I am short of time 👋
Command:

evo_rpe tum ./test/data/fr2_desk_groundtruth.txt ./test/data/fr2_desk_ORB.txt --all_pairs
evo_rpe tum ./test/data/fr2_desk_groundtruth.txt ./test/data/fr2_desk_ORB.txt --all_pairs -r rot_part
evo_rpe tum ./test/data/fr2_desk_groundtruth.txt ./test/data/fr2_desk_ORB.txt --all_pairs -r angle_deg
python ./evaluate_rpe.py ./test/data/fr2_desk_groundtruth.txt ./test/data/fr2_desk_ORB.txt --verbose

Console output:

RPE w.r.t. translation part (m)
for delta = 1 (frames) using all possible pairs
(not aligned)

       max	0.019485
      mean	0.003038
    median	0.002750
       min	0.000132
      rmse	0.003490
       sse	0.026472
       std	0.001718
---------------------------------------------------------------
RPE w.r.t. rotation part 
for delta = 1 (frames) using all possible pairs
(not aligned)

       max	0.031085
      mean	0.005666
    median	0.004636
       min	0.000149
      rmse	0.006924
       sse	0.104176
       std	0.003980
---------------------------------------------------------------
RPE w.r.t. rotation angle in degrees (deg)
for delta = 1 (frames) using all possible pairs
(not aligned)

       max	1.259404
      mean	0.229556
    median	0.187818
       min	0.006039
      rmse	0.280519
       sse	170.995622
       std	0.161230
---------------------------------------------------------------
compared_pose_pairs 5481 pairs
translational_error.rmse 0.037530 m
translational_error.mean 0.030961 m
translational_error.median 0.026825 m
translational_error.std 0.021212 m
translational_error.min 0.000000 m
translational_error.max 0.141707 m
rotational_error.rmse 1.212368 deg
rotational_error.mean 1.085195 deg
rotational_error.median 1.070670 deg
rotational_error.std 0.540545 deg
rotational_error.min 0.000000 deg
rotational_error.max 3.109293 deg

Additional files:
Please attach files if you can - for example trajectory files, images...

Please give also the following information:

• evo version number shown by evo pkg --version:
• Python version shown by evo pkg --pyversion:
• operating system and version (e.g. Ubuntu 16.04 or Windows 10):
• did you change the source code? (yes / no):
• output of evo_config show --brief --no_color:

v1.0.3
2.7.12
Ubuntu 16.04
no
{
    "logfile_enabled": false, 
    "logging_format": "%(message)s", 
    "plot_backend": "TkAgg", 
    "plot_export_format": "pdf", 
    "plot_figsize": [
        6, 
        6
    ], 
    "plot_fontfamily": "sans-serif", 
    "plot_fontscale": 1.0, 
    "plot_hideref": false, 
    "plot_invert_xaxis": false, 
    "plot_invert_yaxis": false, 
    "plot_linewidth": 1.5, 
    "plot_multi_cmap": "none", 
    "plot_seaborn_palette": "deep", 
    "plot_seaborn_style": "darkgrid", 
    "plot_split": false, 
    "plot_texsystem": "pdflatex", 
    "plot_trajectory_cmap": "jet", 
    "plot_usetex": false, 
    "plot_xyz_realistic": true, 
    "save_traj_in_zip": false, 
    "table_export_data": "stats", 
    "table_export_format": "csv", 
    "table_export_transpose": true
}

Conflicting apt/pip matplotlib versions cause 3D plot error ('module' object has no attribute '_string_to_bool')

Bug / Error report

Description:
something wrong with the evo_traj

Command:

evo_traj tum '/home/vio/evo/test/data/fr2_desk_ORB.txt' -p

Console output:

--------------------------------------------------------------------------------
name:	fr2_desk_ORB
infos:	2893 poses, 21.730m path length, 98.822s duration
[ERROR]
unhandled error in evo.main_traj
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/evo/entry_points.py", line 93, in launch
    main_module.run(args)
  File "/usr/local/lib/python2.7/dist-packages/evo/main_traj.py", line 278, in run
    ax_traj = plot.prepare_axis(fig_traj, plot_mode)
  File "/usr/local/lib/python2.7/dist-packages/evo/tools/plot.py", line 224, in prepare_axis
    ax = fig.add_subplot(subplot_arg, projection="3d", aspect="equal")
  File "/usr/local/lib/python2.7/dist-packages/matplotlib/figure.py", line 1074, in add_subplot
    a = subplot_class_factory(projection_class)(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/matplotlib/axes/_subplots.py", line 73, in __init__
    self._axes_class.__init__(self, fig, self.figbox, **kwargs)
  File "/usr/lib/pymodules/python2.7/mpl_toolkits/mplot3d/axes3d.py", line 91, in __init__
    *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/matplotlib/axes/_base.py", line 541, in __init__
    self.cla()
  File "/usr/lib/pymodules/python2.7/mpl_toolkits/mplot3d/axes3d.py", line 1045, in cla
    Axes.cla(self)
  File "/usr/local/lib/python2.7/dist-packages/matplotlib/axes/_base.py", line 1048, in cla
    self.grid(False)  # Disable grid on init to use rcParameter
  File "/usr/lib/pymodules/python2.7/mpl_toolkits/mplot3d/axes3d.py", line 1256, in grid
    self._draw_grid = maxes._string_to_bool(b)
AttributeError: 'module' object has no attribute '_string_to_bool'

[ERROR]
evo module evo.main_traj crashed - no logfile written (disabled)

Additional files:
Please attach files if you can - for example trajectory files, images...

Please give also the following information:

• evo version number shown by evo pkg --version: evo v0.9.7
• Python version shown by evo pkg --pyversion: 2.7.6
• operating system and version (e.g. Ubuntu 16.04 or Windows 10): Ubuntu 14.04
• did you change the source code? (yes / no): no
• output of evo_config show --brief --no_color:

{
    "logfile_enabled": false, 
    "logging_format": "%(message)s", 
    "plot_backend": "Qt4Agg", 
    "plot_export_format": "pdf", 
    "plot_figsize": [
        6, 
        6
    ], 
    "plot_fontfamily": "sans-serif", 
    "plot_fontsize": 12, 
    "plot_hideref": false, 
    "plot_info_text": false, 
    "plot_invert_xaxis": false, 
    "plot_invert_yaxis": false, 
    "plot_linewidth": 1.5, 
    "plot_multi_cmap": "none", 
    "plot_seaborn_style": "darkgrid", 
    "plot_split": false, 
    "plot_texsystem": "pdflatex", 
    "plot_trajectory_cmap": "jet", 
    "plot_usetex": false, 
    "plot_xyz_realistic": true, 
    "save_traj_in_zip": false, 
    "table_export_format": "csv", 
    "table_export_transpose": true
}

Velocity comparison

Dear author:
Can you provide new feature to compare velocity and computer velocity error betwen experiment trajectory and groundtruth?

RMSE in result

(Follow-up of #6)

@cumtchenchang
"Now more and more SLAM algorthms use RMSE to evelate the error, would you add RMSE in your code. I thank it will help more people."

Weird result with EuRoC dataset

Bug / Error report

Description:
Hi, I was running trajectory generated by ORB-SLAM2 using EuRoC stereo dataset V1_01_easy, both the file under evaluation and the groundtruth file are in TUM format. However, the result seems a bit weird. Is it because that I did something incorrectly or that it is supposed to be like this? Many thanks.

Command:
The groundtruth file I used was
mav0/state_groundtruth_estimate0/data.csv
and I transformed it to TUM format using
evo_traj euroc data.csv --save_as_tum
the command I typed to evaluate it was
evo_traj tum -s -p -v -f --ref=data.tum --plot_mode=xz stereo_orb_result.txt

Additional files:

evo_res: "ValueError: shape of passed values is (2,1346), indices imply (2,1328)"

hello,
I have already done evo_ape,so I got two results.But when I used evo_res to compare the two results.then the error happened.
thank you!

Add absolute timestamps to results

Address weird error message of evo_res --use_abs_time by adding absolute timestamps to the np_arrays of a result object (in addition to seconds_from_start).

[EuRoC Dataset] invalid literal for float()

[ERROR] unhandled error in evo.main_traj

Description:
This happened after I run TUM camera trajectory output by ORB-SLAM2 in stereo mode.

Command:

evo_traj tum CameraTrajectory.txt --ref=vicon0.csv

Console output:

Traceback (most recent call last):
  File "/home/ubuntuandy/.virtualenvs/evaluation/local/lib/python2.7/site-packages/evo/entry_points.py", line 93, in launch
    main_module.run(args)
  File "/home/ubuntuandy/.virtualenvs/evaluation/local/lib/python2.7/site-packages/evo/main_traj.py", line 187, in run
    trajectories.append((csv_file, file_interface.read_euroc_csv_trajectory(csv_file)))
  File "/home/ubuntuandy/.virtualenvs/evaluation/local/lib/python2.7/site-packages/evo/tools/file_interface.py", line 144, in read_euroc_csv_trajectory
    mat = np.array(csv_read_matrix(file_path, delim=",", comment_str="#")).astype(float)
#### ValueError: invalid literal for float(): 1.403715529112143517e+09 -6.151000000000000217e-02 4.837999999999999939e-02 1.771199999999999997e-01 8.132099999999999884e-01 -2.730000000000000135e-02 5.806599999999999540e-01 2.778999999999999873e-0

Metrics: consider implicit body-to-cam transformation for EuRoC ground truth alignment

Consider replacing implicit Umeyama alignment of EuRoC groundtruth in metrics (see here) by the proper homogeneous body-to-cam transformation.

To do:

• find transformation matrix (extrinsic calib of EuRoC dataset)
• apply required right-/ left-multiplicative transformations

How does scale alignment work?

Hi, thank you for your amazing work, I want to know how it works now, but the python code is little difficult to me , Can give me some paper or document about evo?

Installation error with pip 10

When install evo with "pip install evo --upgrade --no-binary evo" and "pip install . --upgrade", I encount the same error:
Skipping bdist_wheel for evo, due to binaries being disabled for it.
launchpadlib 1.10.3 requires testresources, which is not installed.
Installing collected packages: scipy, pandas, seaborn, natsort, argcomplete, colorama, evo
Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/scipy'
Consider using the --user option or check the permissions.
@MichaelGrupp Can you solve this problem?
Thank you

Trajectories are not aligned

Add RPY view to evo_traj's PlotCollection

Plot roll, pitch, yaw like the existing xyz view.

new-line character seen in unquoted field

Hi, thank you for your great work.
I meet an error when I use evo with euroc dataset, the error is as follows,

Forcing trajectory alignment implicitly (EuRoC ground truth is in IMU frame)
[ERROR] Unhandled error in evo.main_rpe
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/evo/entry_points.py", line 94, in launch
main_module.run(args)
File "/usr/local/lib/python2.7/dist-packages/evo/main_rpe.py", line 311, in run
args.t_offset,
File "/usr/local/lib/python2.7/dist-packages/evo/tools/file_interface.py", line 232, in load_assoc_euroc_trajectories
traj_ref = read_euroc_csv_trajectory(ref_file)
File "/usr/local/lib/python2.7/dist-packages/evo/tools/file_interface.py", line 145, in read_euroc_csv_trajectory
mat = np.array(csv_read_matrix(file_path, delim=",", comment_str="#")).astype(float)
File "/usr/local/lib/python2.7/dist-packages/evo/tools/file_interface.py", line 61, in csv_read_matrix
mat = [row for row in reader]
Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?

[ERROR] evo module evo.main_rpe crashed - no logfile written (disabled)

How can I solve it?
Thank you.

Segfault while plotting

Bug / Error report

i have successfully installed evo, and i did some tests using it. there is something wrong with ploting

when i use the command:
evo_traj kitti KITTI_00_ORB.txt

it loaded the txt successfully

but
when i wanted to plot and use the commond:
evo_traj kitti KITTI_00_ORB.txt --plot
it shew that Segmentation fault(core dumped)

And i have tried other command like evo_ape. But as long as i used --plot ,there was error
ANYONE KNOW HOW TO FIX IT?

How to get different sequences of KITTI datasets(KITTI01, .. , 10)

Hello, Based on this explanation raulmur/ORB_SLAM2#454 I am trying to do sequences KITTI01,02,.....,10 for S-PTAM, ORB_SLAM and LSD-SLAM 2 days ago unsuccessfully
@MichaelGrupp Can you help me to explain the procedure?
Thank you in advance for your answer.

Keep user's settings.json when upgrading

Move user settings to a user location, e.g. sth. like ~/.evo/settings-user.json

Merge user settings with installed settings when installing/upgrading to prevent an annoying reset.

Difference in maximum APE using a full and a cropped trajectory

Hi, dear author,
Firstly, thank you very much to share this code. It helped me a lot.
I have a problem. when i use evo_ape to evaluate a result from full trajectory of EuRoC V202 dataset and a part of the same trajectory, they both have the max error, but one is 0.670081 and another is 0.646771. these two max errors should be same one because they came from the same trajectory, but in this case ,they are different.

evo_traj tum issue

Hi ,
i am trying to plot KeyFrameNavStateTrajectory.txt usuing evo_traj tum KeyFrameNavStateTrajectory.txt but i get this error
amrani@amrani:~/testResults$ evo_traj tum KeyFrameNavStateTrajectory.txt
[ERROR] Unhandled error in evo.main_traj
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/evo/entry_points.py", line 94, in launch
main_module.run(args)
File "/usr/local/lib/python2.7/dist-packages/evo/main_traj.py", line 177, in run
trajectories.append((traj_file, file_interface.read_tum_trajectory_file(traj_file)))
File "/usr/local/lib/python2.7/dist-packages/evo/tools/file_interface.py", line 71, in read_tum_trajectory_file
mat = np.array(csv_read_matrix(file_path, delim=" ", comment_str="#")).astype(float)
ValueError: could not convert string to float:

[ERROR] evo module evo.main_traj crashed - no logfile written (disabled)
any suggession how to fix this errors ?

How to align the trajectories?

Hello,I have some problems recently,I want to ask you how to solve.
When I compare two trajectories with the euroc dataset, the following conditions occur.

I don't know why the poses,the path length,the duration are all wrong.
When I output them individually

This is the groundtruth in the xy coordinate system

This is the my result in the xz coordinate system
Can you tell me how to solve these problems? Thank you!

Bug / Error report

Description:
Some errors occur when I try to enter the following command

Command:

# evo_traj tum --ref=results.tum data.tum -p --plot_mode=xz --align

Console output:

How to get rotation error?

Dear author,
Through command "evo_ape euroc", i can get the translation error of my own trajectory with ground-truth. But what about rotation error, how can i get it. Can you give me a hint?
Thanks a lot.

Install error on Ubuntu 14.04

Bug / Error report

Description:
When I install the evo on my OS, I have encountered the error.And I have tried several means to install the evo repo. But all failed.

Command:

pip install evo --upgrade -i http://pypi.douban.com/simple
pip2 install evo --upgrade -i http://pypi.douban.com/simple
pip install evo --upgrade
sudo pip install evo --upgrade -i http://pypi.douban.com/simple

Console output:

 Downloading pyparsing-2.2.0-py2.py3-none-any.whl (56kB): 56kB downloaded
Installing collected packages: evo, matplotlib, scipy, pandas, seaborn, natsort, argcomplete, colorama, pygments, enum34, six, python-dateutil, backports.functools-lru-cache, subprocess32, pytz, cycler, pyparsing
  Found existing installation: evo v0.9.7-
    Uninstalling evo:
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1431, in install
    requirement.uninstall(auto_confirm=True)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 598, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1836, in remove
    renames(path, new_path)
  File "/usr/lib/python2.7/dist-packages/pip/util.py", line 295, in renames
    shutil.move(old, new)
  File "/usr/lib/python2.7/shutil.py", line 303, in move
    os.unlink(src)
OSError: [Errno 13] Permission denied: '/usr/local/bin/evo'

Storing debug log for failure in /home/vio/.pip/pip.log

---------------------------
After sudo* shell:

src/qhull_wrap.c: In function ‘delaunay_impl’:

src/qhull_wrap.c:114:5: error: unknown type name ‘QHULL_LIB_CHECK’

     QHULL_LIB_CHECK

     ^

src/qhull_wrap.c:117:5: error: conflicting types for ‘points’

     points = (coordT*)malloc(npoints*ndim*sizeof(coordT));

     ^

src/qhull_wrap.c:96:13: note: previous definition of ‘points’ was here

     coordT* points = NULL;

             ^

src/qhull_wrap.c:117:14: warning: initialization makes integer from pointer without a cast [enabled by default]

     points = (coordT*)malloc(npoints*ndim*sizeof(coordT));

              ^

src/qhull_wrap.c:118:16: warning: comparison between pointer and integer [enabled by default]

     if (points == NULL) {

                ^

src/qhull_wrap.c:134:15: error: subscripted value is neither array nor pointer nor vector

         points[2*i  ] = x[i] - x_mean;

               ^

src/qhull_wrap.c:135:15: error: subscripted value is neither array nor pointer nor vector

         points[2*i+1] = y[i] - y_mean;

               ^

src/qhull_wrap.c:157:29: warning: passing argument 3 of ‘qh_new_qhull’ makes pointer from integer without a cast [enabled by default]

                             "qhull d Qt Qbb Qc Qz", NULL, error_file);

                             ^

In file included from /usr/include/libqhull/qhull_a.h:27:0,

                 from src/qhull_wrap.c:10:

/usr/include/libqhull/libqhull.h:1002:9: note: expected ‘double *’ but argument is of type ‘int’

 int     qh_new_qhull(int dim, int numpoints, coordT *points, boolT ismalloc,

         ^

src/qhull_wrap.c:240:5: warning: passing argument 1 of ‘free’ makes pointer from integer without a cast [enabled by default]

     free(points);

     ^

In file included from /usr/include/python2.7/Python.h:42:0,

                 from src/qhull_wrap.c:8:

/usr/include/stdlib.h:483:13: note: expected ‘void *’ but argument is of type ‘int’

 extern void free (void *__ptr) __THROW;

             ^

src/qhull_wrap.c:259:5: warning: passing argument 1 of ‘free’ makes pointer from integer without a cast [enabled by default]

     free(points);

     ^

In file included from /usr/include/python2.7/Python.h:42:0,

                 from src/qhull_wrap.c:8:

/usr/include/stdlib.h:483:13: note: expected ‘void *’ but argument is of type ‘int’

 extern void free (void *__ptr) __THROW;

             ^

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

  Rolling back uninstall of matplotlib
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/matplotlib/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-rgYXYS-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/matplotlib
Traceback (most recent call last):
  File "/usr/bin/pip2", line 9, in <module>
    load_entry_point('pip==1.5.4', 'console_scripts', 'pip2')()
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 185, in main
    return command.main(cmd_args)
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 161, in main
    text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 29: ordinal not in range(128)

Additional files:
Please attach files if you can - for example trajectory files, images...

Please give also the following information:

• evo version number shown by evo pkg --version:
• Python version: Python 2.7.6
• operating system and version (e.g. Ubuntu 16.04 or Windows 10): Ubuntu 14.04
• did you change the source code? (yes / no):
• output of evo_config show --brief --no_color:

---

Feature request: Perform multiple comparisons in evo_ape

Feature Request:
I want to compare several trajectories of different algorithms with the same groundtruth( the same scene). But the evo_ape just can compare one pair of groundtruth and estimation trajectories. Can you add a feature to evo_ape that like evo_traj groundtruth estimation_traj1 estimation_traj2 estimation_traj3 ... to compare several estimation trajectories from different algorithms with the one groundtruth. And plot the several ape error curves in one window.
Thanks for your help.

Trim a trajectory

Hi,
I want to trim the trajectory points. I tried reduce_ids but it shows me strange trims.
Any other option?

Thanks,

Bruno

Output plot images directly

Thanks for this great work, which I think is definitely a significant contribution to the community! My question is simple so bare with me if I am not using the template.

I can plot the above figures as the instruction shows. But sometimes I also want to save these trajectory and APE error figures directly without pausing or showing them. I am wondering if there is an option to do this? I checked the wiki careful and found this command

evo_config set save_traj_in_zip true

but it seems that this is also not what I wanted because it generates the trajectories in for example .kitti suffix.

How to change the background of plot figures?

Feature request

Hello Michael, I want to transform the background color of figures that are plotted by --plot --plot_mode to white just like MATLAB, because when the figure print on paper, there is a lot blur for the lines with the impact of background color. What should I do to achieve my goal?

Support tab GUI with Qt5Agg backend in PlotCollection

2D SLAM （like gmapping） - can this package be used to evaluate？How to do it？

No matching timestamps with EuRoC dataset

Bug / Error report

Description:

Command:

evo_ape euroc '/home/vio/trajectory_analysis/results of dataset/groundtruth/V102_groundtruth.csv' '/home/vio/trajectory_analysis/results of dataset/EuRoC/vins_V102.txt' -c '/home/vio/evo/doc/examples/config_ape.example.json' 

Console output:

vio@vio-wang:~$ evo_ape euroc '/home/vio/trajectory_analysis/results of dataset/groundtruth/V102_groundtruth.csv' '/home/vio/trajectory_analysis/results of dataset/EuRoC/vins_V102.txt' -c '/home/vio/evo/doc/examples/config_ape.example.json' 
--------------------------------------------------------------------------------
forcing trajectory alignment implicitly (EuRoC ground truth is in IMU frame)
--------------------------------------------------------------------------------
loaded 16702 stamps and poses from: /home/vio/trajectory_analysis/results of dataset/groundtruth/V102_groundtruth.csv
loaded 807 stamps and poses from: /home/vio/trajectory_analysis/results of dataset/EuRoC/vins_V102.txt
[ERROR]
unhandled error in evo.main_ape
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/evo/entry_points.py", line 93, in launch
    main_module.run(args)
  File "/usr/local/lib/python2.7/dist-packages/evo/main_ape.py", line 255, in run
    args.t_offset,
  File "/usr/local/lib/python2.7/dist-packages/evo/tools/file_interface.py", line 231, in load_assoc_euroc_trajectories
    ref_file, est_file)
  File "/usr/local/lib/python2.7/dist-packages/evo/core/sync.py", line 97, in associate_trajectories
    + " (s) and time offset: " + str(offset_2) + " (s)")
SyncException: found no matching timestamps between /home/vio/trajectory_analysis/results of dataset/groundtruth/V102_groundtruth.csv and /home/vio/trajectory_analysis/results of dataset/EuRoC/vins_V102.txt with max. time diff.: 100 (s) and time offset: 0.0 (s)

[ERROR]
evo module evo.main_ape crashed - no logfile written (disabled)

Additional files:
Please attach files if you can - for example trajectory files, images...

Please give also the following information:

• evo version number shown by evo pkg --version: evo v0.9.7
• Python version shown by evo pkg --pyversion: Python 2.7.6
• operating system and version (e.g. Ubuntu 16.04 or Windows 10): Ubuntu 14.04
• did you change the source code? (yes / no): no
• output of evo_config show --brief --no_color:
  {
  "logfile_enabled": false,
  "logging_format": "%(message)s",
  "plot_backend": "Qt4Agg",
  "plot_export_format": "pdf",
  "plot_figsize": [
  6,
  6
  ],
  "plot_fontfamily": "sans-serif",
  "plot_fontsize": 12,
  "plot_hideref": false,
  "plot_info_text": false,
  "plot_invert_xaxis": false,
  "plot_invert_yaxis": false,
  "plot_linewidth": 1.5,
  "plot_multi_cmap": "none",
  "plot_seaborn_style": "darkgrid",
  "plot_split": false,
  "plot_texsystem": "pdflatex",
  "plot_trajectory_cmap": "jet",
  "plot_usetex": false,
  "plot_xyz_realistic": true,
  "save_traj_in_zip": false,
  "table_export_format": "csv",
  "table_export_transpose": true
  }

vins_V102.txt

V102_groundtruth.csv.tar.gz

Post install task not properly executed when being installed from PyPi

Affected version: all, if run without --no-binary evo (see comment below).

Verbose installation from PyPi:

sudo -H pip install evo --upgrade -vvv

shows no output of the post install task.

---

Verbose installation from source:

sudo -H pip install . --upgrade -vvv

shows the correct output of the post install task.

---

This implies that argcomplete and iPython are not properly initialized from a PyPi installation - investigate.

Log spam from matplotlib: get rid of global logging and use custom logger

The new matplotlib version spams the DEBUG logs (verbose mode) like crazy. To prevent this, don't configure the logging module globally anymore but only a scoped logger for the evo package.

When using evo_traj, the attitude Angle cannot be aligned.

Bug / Error report

Description:Hello, I use https://github.com/jingpang/LearnVIORB test euroc data set, the output results are no problem, but I found that when using attitude Angle contrast with evo_traj not fitting.

What's wrong with my operation? Thank you!

Can you give an interface to set min and max value of evo_ape's colorbar

Hi, dear author,

Firstly, thank you very much to share this code. It saved me a lot of time in my research work.

I have a little suggestion. When we use evo_ape to evaluating method，the min and max value of the colorbar are auto selected so that different method has a different colorbar. But, I think if we use the same colorbar we can compare the difference between methods and ground truth more intuitively.

So, can you give an interface to set min and max value of evo_ape's colorbar? Thanks in advance.

"ValueError: could not convert string to float" (trailing whitespace)

thank you for your contribution！
I have two file , (1)data.tum is transformed from groundtruth data.csv in EuRoC (2) msckf_traj.txt is the output of my system contain 8 elements(timestamp x y z q_x q_y q_z q_w)
I run "evo_traj tum msckf_traj.txt" , but error like follow:

how can i transform msckf_traj.txt to tum format and compare with data.tum? looking forward to your replay!

Improve documentation

Create a Wiki with more detailed examples of the most important concepts and use cases.

To do:

• installation
• shared CLI concepts (format switch (e.g. kitti), plot_mode, ...)
• trajectory format specs and export (with table from #3)
• metrics
• result comparison
• package settings
• custom configuration files and examples
  ...

Install error: "Can't roll back matplotlib; was not uninstalled"

Bug / Error report

Description:

I have followed the two kinds of installation steps, but fail to install the evo.
Command:

#Just the lines you have shared.

Console output:

#  Can't roll back matplotlib; was not uninstalled
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/matplotlib/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-KqvzVP-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/matplotlib
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    load_entry_point('pip==1.5.4', 'console_scripts', 'pip')()
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 235, in main
    return command.main(cmd_args)
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 161, in main
    text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 29: ordinal not in range(128)

Please give also the following information:

• evo version number shown by evo pkg --version:
• Python version shown by evo pkg --pyversion:
• operating system and version (e.g. Ubuntu 16.04 or Windows 10):
• did you change the source code? (yes / no):
• output of evo_config show --brief --no_color:

#  Ubuntu 14.04

RPY comparison with rotated frame of reference for EuRoC dataset

Bug / Error report

Description:

The rotation angle like yaw of ground truth cannot correspond to the estimation.

Command:

evo_traj tum --ref='/home/vio/evo-master/test/data/MH05_groundtruth.txt' '/home/vio/evo-master/test/data/msckf_MH05.txt' -a -p

Console output:

name:	msckf_MH05
infos:	2221 poses, 101.631m path length, 111000000000.000s duration
--------------------------------------------------------------------------------
name:	MH05_groundtruth
infos:	22212 poses, 97.593m path length, 111055000064.000s duration

Additional files:

evo_traj_rpy.zip

Please give also the following information:

• evo version number shown by evo pkg --version: v0.9.8.4
• Python version shown by evo pkg --pyversion: 2.7.12
• operating system and version (e.g. Ubuntu 16.04 or Windows 10): Ubuntu 16.04
• did you change the source code? (yes / no): no
• output of evo_config show --brief --no_color:

{
    "logfile_enabled": false, 
    "logging_format": "%(message)s", 
    "plot_backend": "Qt4Agg", 
    "plot_export_format": "pdf", 
    "plot_figsize": [
        6, 
        6
    ], 
    "plot_fontfamily": "sans-serif", 
    "plot_fontsize": 12, 
    "plot_hideref": false, 
    "plot_info_text": false, 
    "plot_invert_xaxis": false, 
    "plot_invert_yaxis": false, 
    "plot_linewidth": 1.5, 
    "plot_multi_cmap": "none", 
    "plot_seaborn_style": "darkgrid", 
    "plot_split": false, 
    "plot_texsystem": "pdflatex", 
    "plot_trajectory_cmap": "jet", 
    "plot_usetex": false, 
    "plot_xyz_realistic": true, 
    "save_traj_in_zip": false, 
    "table_export_data": "stats", 
    "table_export_format": "csv", 
    "table_export_transpose": true
}

Using ground truth data in NED coordinates

I am working on gathering ground-truth from GPS and IMU, I have got problem of alignment mismatch as in the image below resulting from evo_ape tum interpolate_ned_output_linear_cut.txt groundtruth2.txt --plot -va command

I follow TUM format ( https://vision.in.tum.de/data/datasets/rgbd-dataset/file_formats) except the coordinate, which I use NED coordinate and I think that Umeyama's method can handle this.
My current format follow TUM is timestamp tx ty tz qx qy qz qw where

• timestamp (float) gives the number of seconds since the Unix epoch.
• tx ty tz (3 floats) give the position of sensors in NED coordinate in meter.
• qx qy qz qw (4 floats) give the orientation of the sensors in quaternion components.

Do I have to convert to coordinate the camera with respect to the world origin as defined by the motion capture system ( Is this mean camera coordinate where origin at center of the camera and x-right, y-up, z-forward).

Could you please suggest me on how to fix this

Full-check result are:

evo_traj tum groundtruth2.txt --full_check --plot
--------------------------------------------------------------------------------
name:	groundtruth2
infos:
	duration (s)	430.330169916
	nr. of poses	13562
	path length (m)	1035.2306277
	pos_end (m)	[ 31.49409783 -89.38959442  -1.95580373]
	pos_start (m)	[ 0.  0. -0.]
	t_end (s)	1526796495.99
	t_start (s)	1526796065.66
checks:
	SE(3) conform	yes
	array shapes	ok
	nr. of stamps	ok
	quaternions	ok
	timestamps	ok
stats:
	v_avg (km/h)	8.674677
	v_avg (m/s)	2.409632
	v_max (km/h)	27.486197
	v_max (m/s)	7.635055
	v_min (km/h)	0.283611
	v_min (m/s)	0.078781

and

evo_traj tum interpolate_ned_output_linear_cut.txt --full_check --plot
--------------------------------------------------------------------------------
name:	interpolate_ned_output_linear_cut
infos:
	duration (s)	480.318549871
	nr. of poses	15142
	path length (m)	1164.95321947
	pos_end (m)	[ 0.65590576  0.39332212 -2.68988697]
	pos_start (m)	[ 0.          0.         -0.69342154]
	t_end (s)	1526796496.05
	t_start (s)	1526796015.73
checks:
	SE(3) conform	yes
	array shapes	ok
	nr. of stamps	ok
	quaternions	ok
	timestamps	ok
stats:
	v_avg (km/h)	8.740063
	v_avg (m/s)	2.427795
	v_max (km/h)	27.999939
	v_max (m/s)	7.777761
	v_min (km/h)	0.000119
	v_min (m/s)	0.000033

Input files:
groundtruth2.txt - position from GPS converted to NED
interpolate_ned_output_linear_cut.txt - position from IMU (NED)
both are gathered from the Pixhawk.

Additional information:

• evo pkg --version: v0.9.9
• Python version : 2.7.12
• Ubuntu 16.04
• output of evo_config show --brief --no_color:

{
    "logfile_enabled": false, 
    "logging_format": "%(message)s", 
    "plot_backend": "TkAgg", 
    "plot_export_format": "pdf", 
    "plot_figsize": [
        6, 
        6
    ], 
    "plot_fontfamily": "sans-serif", 
    "plot_fontscale": 1.0, 
    "plot_hideref": false, 
    "plot_info_text": false, 
    "plot_invert_xaxis": false, 
    "plot_invert_yaxis": false, 
    "plot_linewidth": 1.5, 
    "plot_multi_cmap": "none", 
    "plot_seaborn_palette": "deep", 
    "plot_seaborn_style": "darkgrid", 
    "plot_split": false, 
    "plot_texsystem": "pdflatex", 
    "plot_trajectory_cmap": "jet", 
    "plot_usetex": false, 
    "plot_xyz_realistic": true, 
    "save_traj_in_zip": false, 
    "table_export_data": "stats", 
    "table_export_format": "csv", 
    "table_export_transpose": true
}

How to compare different trajectory formats?

I am so sorry to ask this question, maybe it is samplie. I want to konw how to compare diferent kind trajectories? such as euroc trajectory and kitti trajectory. or how to translate one kind to another kind? for example, translate euroc trajectory and kitti trajectory.
thank you.

time offset should go before alignment

Bug / Error report

Description:

thank you for your great job, it helps a lot to my work

a possible bug found here:
main_traj.py
line 218 do the alignment
line 242 do the time offset
the time offset block has no effect on my time shifted trajectory,
so I change the two blocks, then it works

Non-zero --t_offset doesn't work in metrics

Description:
--t_offset crashes evo_rpe and evo_ape for no reason.

Command:

cd evo/test/data
evo_ape tum fr2_desk_groundtruth.txt fr2_desk_ORB.txt --t_offset 1.0

Console output:

[ERROR] Unhandled error in evo.main_ape
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/evo/entry_points.py", line 87, in launch
    main_module.run(args)
  File "/usr/local/lib/python2.7/dist-packages/evo/main_ape.py", line 215, in run
    est_name=est_name,
  File "/usr/local/lib/python2.7/dist-packages/evo/main_ape.py", line 164, in ape
    ape_metric.process_data(data)
  File "/usr/local/lib/python2.7/dist-packages/evo/core/metrics.py", line 343, in process_data
    raise MetricsException("trajectories must have same number of poses")
MetricsException: trajectories must have same number of poses

[ERROR] evo module evo.main_ape crashed - no logfile written (disabled)

Additional files:
Please attach all the files needed to reproduce the error.
--> see test folder

Please give also the following information:

• evo version number shown by evo pkg --version: v1.0.3
• Python version shown by evo pkg --pyversion: 2.7.12
• operating system and version (e.g. Ubuntu 16.04 or Windows 10): 16.04
• did you change the source code? (yes / no): no
• output of evo_config show --brief --no_color:

{
    "logfile_enabled": false, 
    "logging_format": "%(message)s", 
    "plot_backend": "qt4agg", 
    "plot_export_format": "pdf", 
    "plot_figsize": [
        6, 
        6
    ], 
    "plot_fontfamily": "sans-serif", 
    "plot_fontscale": 1.0, 
    "plot_hideref": false, 
    "plot_info_text": false, 
    "plot_invert_xaxis": false, 
    "plot_invert_yaxis": false, 
    "plot_linewidth": 1.5, 
    "plot_multi_cmap": "none", 
    "plot_seaborn_palette": "deep", 
    "plot_seaborn_style": "darkgrid", 
    "plot_split": false, 
    "plot_texsystem": "pdflatex", 
    "plot_trajectory_cmap": "jet", 
    "plot_usetex": false, 
    "plot_xyz_realistic": false, 
    "save_traj_in_zip": false, 
    "table_export_data": "stats", 
    "table_export_format": "csv", 
    "table_export_transpose": true
}

Windows support?

Hi,
I am trying to install evo on Windows 10 - Anaconda Python 3.6.
During installation I got an error about "pwd" module.

Any help?

Thank you!

File formats for evo_ape euroc

Hello everyone, I want to get ape performed by LSD_Slam on EuRoC diferent sequences of datasets. I got the frame Trajectory using this procedure can be compared to the ground truth of EuRoC dataset(download here) using your evo package.

haidara@haidara-virtual-machine:~/catkin_ws/src/evo/evo/test/data$ evo_ape euroc V1_01_data.csv lsd_V101.txt -va --plot --save_results results/V.zip
--------------------------------------------------------------------------------
forcing trajectory alignment implicitly (EuRoC ground truth is in IMU frame)
--------------------------------------------------------------------------------
loaded 28712 stamps and poses from: V1_01_data.csv
[ERROR]
unhandled error in evo.main_ape
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/evo/entry_points.py", line 100, in launch
    main_module.run(args)
  File "/usr/local/lib/python2.7/dist-packages/evo/main_ape.py", line 254, in run
    args.t_offset,
  File "/usr/local/lib/python2.7/dist-packages/evo/tools/file_interface.py", line 228, in load_assoc_euroc_trajectories
    traj_est = read_tum_trajectory_file(est_file)
  File "/usr/local/lib/python2.7/dist-packages/evo/tools/file_interface.py", line 70, in read_tum_trajectory_file
    mat = np.array(csv_read_matrix(file_path, delim=" ", comment_str="#")).astype(float)
ValueError: invalid literal for float(): 0.0,0.0,0.0,0.0,0.0,0.0,0.0,1.0

[ERROR]
evo module evo.main_ape crashed - no logfile written (disabled)

Questions about rosbag of geometry_msgs/PoseStamped

Feature request

When using rosbag recordto record the geometry_msgs/PoseStamped topic, it used system time and not the timestamps of dataset, so that bag cannot align with the groundtruth. It seems the rosbag record dosen't work to save timestamps, positon and orientation, and I want to know what should I do to use the feature of bag format which can be used to evo repo. Thank you Michael.

Effect of RPE's delta and delta unit on RMSE

Hi, really thanks for your all nice and generous work!
But, there's a small problem which troubles me, looking for your help.

When I run evo_rpe with delta_unit -- 'f', I got a small RMSE error, such as:

But when I run evo_rpe with delta_uint -- 'm', it returns s really big RMSE value, eg:

So, In general, such as in most of papers, How do I choose the delta_unit for a suitable comparison ?
Thanks!

Trajectory origins not colocated in plot

Hi Mr...!
Thank you for your work! Here is a question when I use the following command to plot the rpe colortrajectory:
evo_rpe kitti KITTI_01_gt.txt KITTI_01_ndt.txt -va --plot --plot_mode xz --save_results result_01_RPE/ndt.zip
The origin point doesn't appear at the same pose with the ground truth. But i have tested other sequence on kitti and the result is correct. So what problem you think i have come across.
This is the result figure:

Thank you very much! Looking forward to your reply!

Change timestamp / index handling in evo_res

• change / remove warning message "data lengths/indices are not consistent, plotting could make no sense"
• only warn if:
  • there are no timestamps (KITTI / evo_rpe-for-each) and indices are different
  • or if only non-matching relative timestamps are available
• for results with timestamps:
  • make absolute timestamps the default and relative (seconds from start) optional (remove --use_abs_time and introduce sth. like --use_relative_time)
  • plot with seconds from start if there are no absolute timestamps in the result file (backwards compatibility)

evo_res: duplicates (as in #9) are not removed based only on index

Bug / Error report

Description:
I have encountered the error when I compare the results from rpe w.r.t. translation part.

Command:

evo_res vins_MH04.zip vins_MH05.zip  vins_V102.zip  vins_V202.zip  --save_table rpe_table_ms_vi.csv --plot

Console output:

[WARNING] duplicate indices in error array of vins_MH04 - keeping only first occurrence of duplicates
[WARNING] duplicate indices in error array of vins_MH05 - keeping only first occurrence of duplicates
[ERROR] unhandled error in evo.main_res
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/evo/entry_points.py", line 93, in launch
    main_module.run(args)
  File "/usr/local/lib/python2.7/dist-packages/evo/main_res.py", line 129, in run
    error_df = pd.concat([error_df, new_error_df], axis=1)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/reshape/concat.py", line 213, in concat
    return op.get_result()
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/reshape/concat.py", line 408, in get_result
    copy=self.copy)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/internals.py", line 5207, in concatenate_block_managers
    return BlockManager(blocks, axes)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/internals.py", line 3033, in __init__
    self._verify_integrity()
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/internals.py", line 3244, in _verify_integrity
    construction_error(tot_items, block.shape[1:], self.axes)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/internals.py", line 4608, in construction_error
    passed, implied))
ValueError: Shape of passed values is (2, 2076), indices imply (2, 2068)

[ERROR] evo module evo.main_res crashed - no logfile written (disabled)

Additional files:
Please attach files if you can - for example trajectory files, images...

Please give also the following information:

• evo version number shown by evo pkg --version: v0.9.8
• Python version shown by evo pkg --pyversion: 2.7.12
• operating system and version (e.g. Ubuntu 16.04 or Windows 10): Ubuntu 16.04
• did you change the source code? (yes / no): no
• output of evo_config show --brief --no_color:

{
    "logfile_enabled": false, 
    "logging_format": "%(message)s", 
    "plot_backend": "Qt4Agg", 
    "plot_export_format": "pdf", 
    "plot_figsize": [
        6, 
        6
    ], 
    "plot_fontfamily": "sans-serif", 
    "plot_fontsize": 12, 
    "plot_hideref": false, 
    "plot_info_text": false, 
    "plot_invert_xaxis": false, 
    "plot_invert_yaxis": false, 
    "plot_linewidth": 1.5, 
    "plot_multi_cmap": "none", 
    "plot_seaborn_style": "white", 
    "plot_split": false, 
    "plot_texsystem": "pdflatex", 
    "plot_trajectory_cmap": "jet", 
    "plot_usetex": false, 
    "plot_xyz_realistic": true, 
    "save_traj_in_zip": false, 
    "table_export_format": "csv", 
    "table_export_transpose": true
}

results_vins_rpe.tar.gz

signal_shutdown [atexit]

Bug / Error report

Description:

When I run evo_traj bag ROS_example.bag groundtruth ORB-SLAM S-PTAM in /eva/test/data/.This log always show , and I get no result then.

Command:

# replace this line with the command you used

Console output:

# remove this line and paste your console output HERE

Additional files:
Please attach files if you can - for example trajectory files, images...

Please give also the following information:

• evo version number shown by evo pkg --version:
• Python version shown by evo pkg --pyversion:
• operating system and version (e.g. Ubuntu 16.04 or Windows 10):
• did you change the source code? (yes / no):
• output of evo_config show --brief --no_color:

# remove this line and paste your config HERE

Deprecation warning with new matplotlib version > 2.2

Find out why this warning is shown:

/usr/local/lib/python2.7/dist-packages/matplotlib/axes/_axes.py:6448: UserWarning: 
The 'normed' kwarg is deprecated, and has been replaced by the 'density' kwarg.

Also Python 3.

Histogram function of mpl is the source: https://github.com/matplotlib/matplotlib/blob/4391fe9184b50ba178e80bf356d9d478494cf1ef/lib/matplotlib/axes/_axes.py#L6439