yongqli / dbn

Databento Binary Encoding (DBN) - Fast message encoding and storage format for market data

dbn

Databento Binary Encoding (DBN) is an extremely fast message encoding and storage format for normalized market data. The DBN specification includes a simple, self-describing metadata header and a fixed set of struct definitions, which enforce a standardized way to normalize market data.

All official Databento client libraries use DBN under the hood, both as a data interchange format and for in-memory representation of data. DBN is also the default encoding for all Databento APIs, including live data streaming, historical data streaming, and batch flat files.

This repository contains both libraries and a CLI tool for working with DBN files and streams. Python bindings for dbn are provided in the databento_dbn package.

For more details, read our introduction to DBN.

Features

• Performant binary encoding and decoding
• Highly compressible with Zstandard
• Extendable fixed-width schemas

Usage

See the respective READMEs for usage details:

• dbn: Rust library crate
• dbn-cli: CLI crate providing a dbn binary
• databento-dbn: Python package

License

Distributed under the Apache 2.0 License.