algarianna / event-driven

neuromorphic sensor integration with YARP and iCub

| Read the Documentation | Download the Code |

event-driven

YARP integration for event-cameras and other neuromorphic sensors

rvtfmc.mp4

Libraries that handle neuromorphic sensors, such as the dynamic vision sensor, installed on the iCub can be found here, along with algorithms to process the event-based data. Examples include, optical flow, corner detection and ball detection. Demo applications for the iCub robot, and tutorials for running them, include saccading and attention, gaze following a ball, and vergence control.

@article{Glover2017b,
author = {Glover, Arren and Vasco, Valentina and Iacono, Massimiliano and Bartolozzi, Chiara},
doi = {10.3389/frobt.2017.00073},
journal = {Frontiers in Robotics and AI},
pages = {73},
title = {{The event-driven Software Library for YARP — With Algorithms and iCub Applications}},
volume = {4},
year = {2018}
}

Libraries

Event-driven libraries provide basic functionality for handling events in a YARP environment. The library has definitions for:

• core
  • codecs to encode/decode events to be compatable with address event representation (AER) formats.
  • Sending packets of events in ev::packet that is compatible with yarpdatadumper and yarpdataplayer.
  • asynchronous reading and writing ports that ensure data is never lost and giving access to latency information.
  • helper functions to handle event timestamp wrapping and to convert between timestamps and seconds.
• vision
  • filters for removing salt and pepper noise.
  • sparse event warping using camera intrinsic parameters and extrinsic parameters for a stereo-pair
  • methods to draw events onto the screen in a variety of methods
• algorithms
  • event surfaces such as the Surface of Active Events (SAE), Polarity Integrated Images (PIM), and Exponentially Reduced Ordinal Surface (EROS)
  • corner detection
  • optical flow

TOOLS

• vFramer - visualisation of events streamed over a YARP port. Various methods for visualisation are available.
• calibration - estimating the camera intrinsic parameters
• vPreProcess - splitting different event-types into separate event-streams, performing filtering, and simple augmentations (flipping etc.)
• atis-bridge - bridge between the Prophesee ATIS cameras and YARP
• zynqGrabber - bridge between zynq-based FPGA sensor interface and YARP

Applications

Applications that implement the algorithms available in event-driven are found in our companion repository

event-driven-demos

How to Install:

Comprehensive instructions available for installation.

References

Glover, A., and Bartolozzi C. (2016) Event-driven ball detection and gaze fixation in clutter. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 2016, Daejeon, Korea. Finalist for RoboCup Best Paper Award

Vasco V., Glover A., and Bartolozzi C. (2016) Fast event-based harris corner detection exploiting the advantages of event-driven cameras. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 2016, Daejeon, Korea.

V. Vasco, A. Glover, Y. Tirupachuri, F. Solari, M. Chessa, and Bartolozzi C. Vergence control with a neuromorphic iCub. In IEEE-RAS International Conference on Humanoid Robots (Humanoids), November 2016, Mexico.