segment11 / photondb

A storage engine for modern hardware, built from scratch in Rust.

PhotonDB

A storage engine for modern hardware, built from scratch in Rust.

Design

Progress

The in-memory part of the storage engine is almost done at the moment. We have published the photondb-engine crate v0.0.1.

Example:

[dependencies]
photondb-engine = "0.0.1"

use photondb_engine::tree::{Error, Map, Options};

fn main() -> Result<(), Error> {
    let map = Map::open(Options::default())?;
    map.put(b"hello", b"world")?;
    map.get(b"hello", |value| {
        assert_eq!(value.unwrap(), b"world");
    })?;
    map.delete(b"hello")?;
    map.get(b"hello", |value| assert_eq!(value, None))?;
    Ok(())
}

Benchmark

Some rough benchmarks on the in-memory storage engine with 100M keys:

#threads	get	put	get:put = 4:1
1	638468	521065	444558
2	1245346	1010584	938936
4	2371893	1897404	1768171
6	3317791	2622060	2489307
8	4099652	3240738	3102952
10	4738312	3681498	3418918
12	5281951	4148568	3801134
14	5826484	4636580	4309976
16	6364404	5114875	4812409

References

• The Bw-Tree: A B-tree for New Hardware Platforms
• Building a Bw-Tree Takes More Than Just Buzz Words
• LLAMA: A Cache/Storage Subsystem for Modern Hardware
• Efficiently Reclaiming Space in a Log Structured Store
• The Design and Implementation of a Log-Structured File System
• TinyLFU: A Highly Efficient Cache Admission Policy