harvard-edge / mavbench

Simulator + benchmark suite for Micro Aerial Vehicle design.

Welcome to MAVBench

This README explains how to setup and use MAVBench, A benchmark and a simulator for Micro Aerial Vehicles.

What is MAVBench? MAVBench is a framework targeting design and development of Micro Aerial Vehicles for hardware/software designers and roboticists. It consists of a closed-loop simulator and an end-to-end application benchmark suite. A closed-loop simulation platform is needed to probe and understand the intra-system (application data flow) and inter-system (system and environment) interactions in MAV applications to pinpoint bottlenecks and identify opportunities for hardware and software co-design and optimization. In addition to the simulator, MAVBench provides a benchmark suite, the first of its kind, consisting of a variety of MAV applications designed to enable computer architects to perform characterization and develop future aerial computing systems. This work is built on top of a host of open source software. A big shout out to Microsoft and ETH Zurich University.

Why MAVBench? We developed MAVBench to accurately model the drone's system and its environment. We identify two main ingredients toward this end.

1. Simulator: Autonomous drones similar to other autonomous machines require a new breed of architectural simulators. Unlike traditional machines (desktops, servers, cellphones and others), information flows in a loop for an autonomous machine . Such flow starts from the machine's environment via sensors, gets processed by the computing subsystem, and flows back out into the environment via actuators. This means, unlike traditional simulators, autonomous machines require a tightly coupled closed-loop feedback simulator for architectural investigation. Our simulator has three core components as shown in the figure bellow. The drone's environments, sensors, and actuators are simulated using a game engine called Unreal augmented with AirSim libraries (top). By using a physics engine, they provide the ability to simulate the drone's behavior, its environment and the interaction between the two such as accurate collision detection. Flight controller (flight stack and the autopilot hardware) is responsible for the drone's stabilization (bottom right). We use a software-simulated flight controller provided by AirSim. However, AirSim also supports other flight controllers, such as the Pixhawk. Much of the drone's perception and trajectory planning is done using an onboard computer, which is generally responsible for running any compute-intensive workloads (bottom left). We used an NVIDIA Jetson TX2, although our setup allows for swapping this embedded board with other platforms like a RISC-V based platform.

2. Benchmark Suite: To quantify the power and performance demands of typical MAV applications, we created a set of workloads that we compiled into a benchmark suite. Our benchmarks run on top of our closed-loop simulation environment. The suite aims to cover a wide range of representative applications. Each workload is an end-to-end application that allows us to study the kernels' impact on the whole application as well as to investigate the interactions and dependencies between kernel. By providing holistic end-to-end applications instead of only focusing on individual kernels, MAVBench allows for the examination of kernels' impacts and their optimization at the application level. This is a lesson learned from Amdahl's law, which recognizes that the true impact of a component's improvement needs to be evaluated globally rather than locally.

Youtube Channel

Please visit our youtube channel.

Building

Instructions to build MAVBench.

Running

Instruction to run MAVBench.

Profiling

Instruction to profile and interpret the results.

Directory Structure

.
├── build_scripts # Scripts for building our repo and subrepos
├── docs          # Documents
│   ├── images    
│   └── read_me   
├── src           # All the src code
│   ├── AirSim
│   ├── darknet
│   ├── mav-bench-apps
│   ├── opencv
│   └── pcl
└── test_benches  # Test benches allowing the user to 1.use pre-defined missions
                  #                                  2. profile
    ├── configs   # Pre-defined missions (you can change this according to your
   		    need)
    └── scripts   # Scripts to load test benches and profile

Paper

More technical details are available in our paper published in Micro 2018.(https://d.pr/f/fqspYT);

Contribute

MAVBench aims at brining the robotics, software and hardware community together. We welcome any contributions such as new sensor/actuator models, new kernels/applications and new hardware setups.

Contributors

Behzad Boroujerdian (UT Austin, Harvard, SiFive)
Hassan Genc (UC Berkeley)
Srivatsan Krishnan (Harvard)
Wenzhi Cui (Google)
Marcelino Almeida (UT Austin)
kayvan Mansoorshahi (UT Austin)
Aleksandra Faust (Google Brain)
Vijay Janapa Reddi (UT Austin, Harvard, Google)

Current Maintainers

[email protected]
[email protected]

FAQ

mavbench's Issues

Object in environment in not moving. - aeiral photography benchmark .(follow_the_leader)

During simulation, there is no mobility in object or person which drone follows.
I tried the following, but the issue remains unresolved.

1. I did not find any other config file related to "follow_the_leader" in test_benches/configs folder. After going through the scripts code, I created new config file. I changed the "map_name" key to "follow_the_leader_simple" and added "time_to_trigger_object" key with value 5 in that new config file.
   Do I need to add any other key-value pair in config file?
2. I run "trigger.exe" file and gave 5 as input.

error while executing ./build_scripts/companion_usr_setup.bash

When I execute the instruction on the TX2, it shows me that the FindEigen.cmake Module in the cmake_modules package is deprecated. I follow the advice to change instances of find_package(Eigen) to find_package(Eigen3).But I failed. How can I solve this?
Thanks in advance.

No generation of trajectory while returing back in package_delivery.

After the delivery, the drone has to turn back and generate the new trajectory to come back. But it is not generating new path and following the same path which was generated

error when processing catkin package: 'octomap_server'

I 'm trying to build the companion computer .But when the terminal process catkin package: 'octomap_server', it shows that "No package 'openni-dev' found" and "No package 'openni2-dev' found" .How can I solve this?
Thanks in advance!

Non colliding paths are not generated in 3D Mapping Benchmark.

In 3d Mapping benchmark, the nbvplanner node has to generate the collision free paths. But in my case all the generated paths are colliding paths as shown below. So there is no progress in the benchmark mission.
I tried chaging the following parameters. But It did not resolved the issue.

1. sensor_max_range - increase
2. planning_resolution - decrease
3. mapping_resolution - decrease
4. system/bbx/x, system/bbx/y, system/bbx/z, system/bbx/overshoot, bbx/minX, bbx/minY, bbx/minZ, bbx/maxX, bbx/maxY, bbx/maxZ, - decrease
   

Eigen Assertion Error in future collision node.

The future_collision node is halting with following assertion failure.

eigen_assert((internal::UIntPtr(eigen_unaligned_array_assert_workaround_gcc47(array)) & (15)) == 0
&& "this assertion is explained here: "
"http://eigen.tuxfamily.org/dox-devel/group__TopicUnalignedArrayAssert.html"
" **** READ THIS WEB PAGE !!! ****");

Why Jetson TX2

I wanted to ask why specifically TX2 is used and not any CUDA capable GPU?

Profiling MAVBench tasks on TX1 CPU and GPU separately

We are trying to use MAVBench for our research, and using a Jetson TX1 for our work. We are studying the implication of running a given task on the onboard CPU vs. GPU (in our case, the Arm A57 or the Maxwell) of the TX1. So, we want to run each of the “nodes” in the applications in Fig. 7 of your paper (e.g. mission planning:scanning) on either the TX1 CPU and GPU, and profile their timing. Within the Github code in your repository, do you know if there is a way to select which device (CPU/GPU) runs a given node?

Build problem of ubuntu1804 on Jetson Xavier NX

I got a lot of building problem using ubuntu1804 on Jetson Xavier Nx. Mainly because of different ros version. Is there any plan of providing building script for ubuntu1804?

Don't have Jetson TX2

Hi Behzad,
I don't have a Jetson TX2. Searched for emulators for Jetson, sadly there isn't.

What can I use instead for this setup and benchmarks?

Drone colliding with the obstacale in package delivery. Unable to detect the obstacle

In the attached images, at the beginning of execution, the images from camera sensor are being received and point cloud is generated.

But using this point cloud, the octomap has to be generated which is not happening. Because of this, the drone is colliding with the wall. We can see in the attached images.
While travelling on the generated path , the obstacle is determined using the octomap. As the octomap is not generated correctly, the drone is colliding with the obstacle.

Error in running example package delivery

Basically, all the steps under "Building" are done without any errors. Issues rise when I tried to run it on Companion Computer (Nvidia Jetson TX2).

After sourcing the relevant files, I run the command as told by the documentation here (https://github.com/harvard-edge/MAVBench/blob/master/docs/read_me/running.md)

roslaunch package_delivery scanning.launch

where it gives the error message as following:

Thanks in advance!