theillusionistmirage / sukuna

Sukuna is an experimental log based (SSTable), disk backed, key-value storage engine.

Sukuna

Sukuna is an experimental log based (SSTable), disk backed, key-value storage engine.

Dependencies

• JDK 16 or higher
• Maven 3.8.1 or higher

Build

$ git clone https://github.com/TheIllusionistMirage/sukuna.git
$ cd sukuna
$ ./build.sh

This will build three JAR files corresponding to the modules sukuna-engine, sukuna-service and sukuna-cli-client.

The JAR for sukuna-engine can be embedded into any Java application that wants to control how the engine is hosted.

Trying It Out

Starting the Service

sukuna-service is the default TCP based DB service program.

$ cd sukuna-service
$ java -jar ./target/sukuna-service-<version>-shaded.jar  # Where version is the application version in sukuna-service/pom.xml

Using the Client

sukuna-cli-client is the default TCP based DB client program.

$ cd sukuna-cli-client
$ java -jar ./target/sukuna-cli-client-<version>-shaded.jar  # Where version is the application version in sukuna-cli-client/pom.xml

Currently only two operations are supported:

• get <key>
• set <key> <value>

Documentation

You can find the documentation related to this project in the design directory.

License

Sukuna is licensed under GNU AGPLv3, see LICENSE for more details.

sukuna's Issues

Chore: Add architecture diagram and system design documentation for Sukuna

Currently there is no supporting documentation that describes the design and architecture of Sukuna. It would be beneficial to have them in place.

Tech debt: Add extensive unit test coverage of existing components

There is a lack of proper unit test coverage since the initial focus was to get a prototype up and running quickly. Extensive unit test coverage is needed to ensure health of the code base.

Feature: Improve locking mechanism for active memtables

Currently a re-entrant read-write lock is used for returning read only copies of the active memtables when using them in places like memtable to SSTable conversion. The current mechanism does not actually prevent multi-threading issues and hence has to be properly understood and re-implemented.

Pick a suitable license for Sukuna

Need to pick a suitable license for Sukuna.

Feature: Implement a configuration provider module

Currently, the configuration for the database engine is hard-coded in constant static variables in a Configuration class. There exists a skeleton for a configuration provider (IConfigurationProvider) that needs to be implemented to make the configuration changes more flexible.

Feature: Add a write ahead log to handle DB failures

To handle external failures like OS crashes etc, an append only write ahead log needs to be added. For each incoming write, the write will first be performed on this write ahead log, and then on the currently active memtable. This will ensure that in case of crashes we can consistently rebuild the active memtable. This write ahead log is discarded and a new one created whenever the active memtable is flushed to the disk.

Tech debt: Make logging more consistent

Since this project is experimental and the first iteration was a barely working prototype, consistency in logging has taken a back seat. The logging needs to be more consistent and appropriate use of correct logging levels based on the context needs to be added.

Tech debt: Clean-up the modules for handling compaction

Currently there is support for SSTable compaction but the compactor module has some badly written, unreadable code. This needs to be addressed by properly re-writing some or all parts to make it more readable and maintenance friendly.