ntdls / ntdls.katzebase.server

ACID compliant JSON document-based database engine with SQL language, APIs and GUI.

Katzebase : Server

Katzebase is an ACID compliant document-based database written in C# using .NET 8 that runs on Windows or Linux. By default it runs as a service but the libraries can also be embedded. It supports what you'd expect from a typical relational-database-management-system except the "rows" are stored as sets of key-value pairs (called documents) and the schema is not fixed. The engine is wrapped by MVC controllers and allows access via APIs, a t-SQL like syntax, or by using the bundled management UI (which just calls the APIs).

Documentation and Links

• Check out the full documentation at https://katzebase.com/](https://katzebase.com/
• Server / Service code and releases: https://github.com/NTDLS/NTDLS.Katzebase.Server
• Client Connectivity Libraries code and releases: https://github.com/NTDLS/NTDLS.Katzebase.Client
• SQL Server Migration Tool code and releases: https://github.com/NTDLS/NTDLS.Katzebase.SQLServerMigration
• Management UI code and releases: https://github.com/NTDLS/NTDLS.Katzebase.Management

Features:

• Abortable transactions.
• Caching and write deferment.
• Locking, isolation and atomicity.
• Document indexing.
• Partitioning.
• Logging and health monitoring.
• SQL Query language with support for field list, joins, top(count), where clause, grouping, aggregations, etc.

Client Connectivity?

Grab the nuget package for your project over at nuget.org. Or, maybe you are just looking for the client source code?

Sample Data

To run the included examples, download the sample Katzebase database, which is a compressed archive containing a word list and various relationsips between the words and languages.

SQL Server Migration Tool

We even included a tool to import your schema and data from SQL Server into Katzebase.

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change. If you want to join the project, just email me (its on my profile).

License

MIT

ntdls.katzebase.server's Issues

Implement running only selected UI query text.

Implement REMOVE.

this removes a field from all matched documents.

Implement built in scaler functions.

Implement built in scaler functions: GetDateTime(format), GetDateTimeUtc(format), Random(min, max), Guid(), LeftString(n), RightString(n), SubString(start, length), LengthString(string), PadString(character, length), Sha256String(string).

Implement LEFT OUTER join.

Implement group by.

Implement CREATE INDEX.

Thread pool enhancements.

Fire up the thread pool and add a hash set that contains the queue/completed count for each GUID based workload. This was we can keep the thread pool alive and even share it across queries. The pool can also be safely grown / shrank depending on load.

Implement single line comments.

Add global min threads, max threads and thread multiplier configurations.

Add global min threads, max threads and thread multiplier configurations.

Determine if we enable IO deferment on system created transactions.

Determine if we enable IO deferment on system created transactions. I think we can.

Add transaction rollback.

Implement UPDATE.

Allow it to add fields that do it currently exist.

Separate methods for API and Query processing into separate classes.

The “management” classes for schemas, indexing, documents, etc are very messy. Separate the methods by responsibility.

When running script from UI, split on batch delimiters.

When running script from UI, trim all lines, ensure trailing CrLf and split on "CrLf".

Implement INNER JOIN

Finish implementing TOP limiter.

Implement CREATE SCHEMA.

Add querying for running processes.

Allow to query for running processes and view blocks.

Implement .net functions (like procedures).

Add .net functions (like procedures) that take parameters and are stored as a compiled assembly.

Implement schema aliasing for referencing fields in joined schemas.

I/O safety enhancements.

Add options for flushing IO writes to disk. This can be enabled for LOG or DOCUMENTS exclusive of each other. Use this: . Enable file system write-through caching: The write-through caching policy ensures that every write operation is immediately committed to the disk. You can enable this policy using the FileOptions.WriteThrough flag when creating the FileStream object.

Also look at FileStream.Flush(true).

Evaluate and test how threads handle being a deadlock victims.

Implement ability to rename index.

Implement INSERT.

Remove index cache from MSQ because it’s really covered by the transaction lock cache now.

Implement CLEAR WAITSTATS.

Implement CACHE CLEAR.

Order the joins for performance

Idea: Order the lookups by whether they have joins, then by the descending number of expected records.

UI schema list. Includes nested schema browsing, index creation, and basic script generation (like select * from this schema)

Implement order by.

Implement subqueries.

Add query scope min threads, max threads and thread multiplier configurations.

Implement REBUILD INDEX.

Implement ability to get document ID.

This can assist in tracking down json files.

Implement DROP INDEX.

Implement DROP SCHEMA.

Implement stored procedures.

Implement stored procedures that take parameters.

Implement variable declarations.

In UI, the run state and results tab should be part of the tab page.

In UI, the run state and results tab should be part of the tab page. This is so multiple scripts can be run at a time.

Add system health functions to query system health such as waits.

Implement scalar functions sum, min, max, count.

Rewrite the entire query explanation function. It’s stupid.

Just build an entire expression tree and pass it to ncalc?

Constants in joins and constant to constant comparisons.

When collapsing a condition formula, check to see if either side is a constant or a field and treat it accordingly. This way the constant value can be on the left or the right. It also means we could then use constants in a join condition.

SQL import/export wizard

Create winforms application to export SQL server to Katzebase.

Test exceptions which occur in lookup threads.

Add transaction commit.

Add transaction create.

Implement ability to rename namespace.

Implement DELETE.