cachapa / sql_crdt

Dart implementation of Conflict-free Replicated Data Types (CRDTs) using SQL databases

Dart implementation of Conflict-free Replicated Data Types (CRDTs) using SQL databases.

This project is heavily influenced by James Long's talk CRTDs for Mortals and includes a Dart-native implementation of Hybrid Local Clocks (HLC) based on the paper Logical Physical Clocks and Consistent Snapshots in Globally Distributed Databases.

sql_crdt is based on crdt and the learnings from Libra, StoryArk and tudo.

This package is is abstract and implements the base functionality for CRDTs backed by a relational database. Check sqlite_crdt and postgres_crdt for usable implementations.

See also tudo for a real-world FOSS implementation.

This package is compatible with crdt_sync, a turnkey approach for real-time network synchronization of CRDT nodes.

Notes

sql_crdt is not an ORM. The API is essentially that of a plain old SQL database with a few behavioural changes:

• Every table gets 3 columns automatically added: is_deleted, hlc, and modified.
• Deleted records aren't actually removed but rather flagged in the is_deleted column.
• A reactive watch method to subscribe to database queries.
• Convenience methods getChangeset and merge inherited from the crdt package to sync with remote nodes.

⚠ Because deleted records are only flagged as deleted, they may need to be sanitized in order to be compliant with GDPR and similar legislation.

API

The API is intentionally kept simple with a few methods:

• execute to run non-select SQL queries, e.g. inserts, updates, etc.
• query to perform a one-time query.
• watch to receive query results whenever the database changes.
• getChangeset to generate a serializable changeset of the local database.
• merge to apply a remote changeset to the local database.
• transaction a blocking mechanism that avoids running simultaneous transactions in async code.

Check the examples in sqlite_crdt and postgres_crdt for more details.

Features and bugs

Please file feature requests and bugs in the issue tracker.

sql_crdt's Issues

Conflict resolution for multiple changed attributes on a single entity

Thanks for this rather straightforward looking package!

I have one question about following use case:

entity: id, name, color, category
node1: ch1: changes color and name
node2: ch2: changes category and name

changes on node1 and node2 happen on the same state, means none of those changes is more "correct" than the other.

ch2 arrives at server first, then ch1. Would it work in Last-Write-Wins fashion? Would it retain color and category changes without conflict?
How does the server decide, which change to apply first?

Thanks!

README request

Hello, I am guessing that hlc is hybrid-logical clocks. Can you add a footnote to your readme that unpacks this a bit?

"HLCs use a combination of system timestamp and logical timestamp to have versions as date-time, which can be ordered." I would also be curious to know what data primitive the underlying postgres/sqlite uses to store these.