ipfs-shipyard / tevere

🏞 Decentralized DB over IPFS

Tevere

Decentralized eventually-consistent key-value store over IPFS for the browser. Exposes a Leveldown-compatible API.

Install

$ npm install tevere --save

Use

Here we're using Memdown as a log database, but you can use any database that conforms to the Leveldown interface (Level-js on the browser or Leveldown on Node.js):

const Tevere = require('tevere')
const Memdown = require('memdown')

const db = Tevere('partition name', {
  log: Memdown('partition name')
})

db.put('key', { val: 'ue' }, (err) => {
  if (err) {
    throw err
  }
  console.log('PUT succeeded')
})

Custom merge function

By default, when a conflict exists (two nodes have concucrently performed a change on the same key), Tevere will determinstically choose one of the values for you. Instead, you can provide a synchronous merge function like this:

const db = Tevere('partition name', {
  log: Memdown('partition name'),
  merge: merge
})

// custom merge function that merges two records
function merge (v1, v2) {
  return {
    a: v1.a,
    b: v2.b,
    c: v1.c.concat(v2.c),
    d: Math.max(v1.d, v2.d)
  }
}

Custom merge and determinism

If you define a custom merge function, the result must be deterministic. For every node involved in the same conflict, Tevere guarantees that the order of the values passed into the merge function is the same. In return, you must guarantee that, given the same two values, you always return the same merged value.

This means that you cannot generate data that is not deterministic, like random values or even time stamps.

Invalid merge function:

function merge (v1, v2) {
  return {
    timestamp: Date.now()
  }
}

This is valid, though:

function merge (v1, v2) {
  return {
    timestamp: Math.max(v1.timestamp, v2.timestamp)
  }
}

Tevere's compromise: Given a specific conflict, the order of the two values passed into the merge function is always the same. This means that, if two nodes have conflicting changes, both nodes custom merge functions will be called with the exact same arguments in the exact same order.

Your compromise: Determinism, purely funcional merge function: given a sets of two conflicting values, you always return the same merged value, no matter at which node and at which time the merging occurs.

Tevere API

Tevere (partition, options)

Creates a Tevere instance.

• partition (string, mandatory): identifies the partition this node will participate in.
• options (object, mandatory): some options:
  • ipfsOptions (object, optional). IPFS options object.
  • log (LevelDown-compatible database): this is where the node keeps the log entries (which only have a vector clock and a hash — all the actual data is kept in IPFS).
  • ipfs (IPFS object, optional): an IPFS object instance. If you already can provide an IPFS object, pass it in here.
  • merge (function, optional): a synchronous function that will receive two values and return a new value.

A Tevere instance respects the Leveldown API. Here are the main methods:

Example:

const Leveljs = require('level-js')

const db = Tevere('partition name', {
  log: Leveljs('partition name')
})

db.put (key, value, callback)

Save a value to key.

db.get (key, callback)

Get the value stored in key.

db.iterator (options)

Returns an iterator over the database. Supports the same options described in the Leveldown API.

Events

A Tevere instance emits these event types:

"change" (change)

Every time there is a change (either local or remote), a Tevere instance emits a change event, which is an object that has these properties:

• type (string, either "del" or "put")
• key (string)
• value (any, relevant for put type operations)

Examples

Check the tests dir for some examples.

Internals

Internal workings are documented here.

License

MIT

Why the name "Tevere"?

Tevere is the italian name for the river that goes through Rome. At the time the repo for this project was created, I was hanging out in Rome with some of the IPFS crew, so I guess it made sense at the time...

Contribute

Feel free to join in. All welcome. Open an issue!

This repository falls under the IPFS Code of Conduct.

License

MIT

tevere's Issues

Needs photo of Tevere on the Readme :)

Circle won't start

same problem here.. https://circleci.com/setup-project/gh/ipfs-shipyard/tevere

Persist data across page reloads / multiple browsers

Hello Pedro, thanks again for building this. I was wondering if it's possible to use this as an actual database to persist data across sessions. I might be confused about the inner workings of this or I am mistaking it for something different entirely. Currently I'm using Memdown as the log storage and levelup to access the leveldown API like this:

const Tevere = require('tevere');
const Memdown = require('memdown');
const levelup = require('levelup');

const db = Tevere('coolpartition', {
  log: Memdown('coolpartition'),
});

const tevere = levelup(db);

tevere.put('boo', 'foo', (err) => {
  console.log(err);
});

// This is being done after a page reload
tevere.get('boo', (err, val) => {
  console.log(err);
  console.log(val);
});

but I'm getting a Not Found error when getting the key after a page refresh.

Is it possible to persist the data across sessions and browsers? How's the log db actually used?

Furthermore I'm getting this error when running the above script:

Uncaught Error: no protocol with name: p2p-webrtc-star