Fully replicate two PouchDB or CouchDB databases, while preserving absolutely all document history and conflicts.
Useful for:
- Implementing infinite undo/redo in your app
- Building a Git-like VCS on top of CouchDB/PouchDB
- Building a collaborative wiki, where all edits ever are preserved
Normally, the CouchDB replication protocol only replicates the leafs between two databases. This preserves conflicts (which is awesome!), but it discards non-leafs for performance reasons.
You can visualize it like this. Here's what CouchDB/PouchDB normally does:
The non-leaf revisions 1-x
, 2-x
, and 3-x
are lost during replication, but the conflicting leafs 3-y
and 4-x
are kept. If you try to fetch the non-leafs from the target database (e.g. using db.get('id', {rev: '1-x'})
), then you'll see {"error": "not_found", "reason": "missing"}
.
Incidentally, this is the same thing that happens during compaction.
Now, here's what PouchDB Full Sync does:
Notice that all the revisions are kept, even the non-leafs. So you can call db.get('id', {rev: '1-x'})
and the data will be there!
To use this plugin in the browser, include it after pouchdb.js
in your HTML page:
<script src="pouchdb.js"></script>
<script src="pouchdb.full-sync.js"></script>
It's also available from Bower:
bower install pouchdb-full-sync
Or to use it in Node.js, just npm install it:
npm install pouchdb-full-sync
And then attach it to the PouchDB
object:
var PouchDB = require('pouchdb');
PouchDB.plugin(require('pouchdb-full-sync'));
There are three new methods, which have exactly the same APIs as the old methods:
db.fullyReplicateTo()
emulatesdb.replicate.to()
db.fullyReplicateFrom()
emulatesdb.replicate.from()
db.fullySync()
emulatesdb.sync()
So you will want to refer to the original documentation for replicate()
and sync()
to understand how to use this plugin.
Examples:
var db = new PouchDB('mydb');
var remoteDB = new PouchDB('http://localhost:5984/mydb');
// fully replicate to another database
db.fullyReplicateTo(remoteDB);
// fully replicate from another database
db.fullyReplicateFrom(remoteDB);
// fully sync two databases
db.fullySync(remoteDB);
The same promise/callback/event APIs are supported. So for instance, you can do a one-shot replication with promises:
db.fullySync(remoteDB).then(function () {
// all done!
}).catch(function (err) {
// got an error!
});
Or you can use the familiar event listeners:
db.fullySync(remoteDB, {
live: true,
retry: true
}).on('change', function (info) {
// handle change
}).on('paused', function () {
// replication paused (e.g. user went offline)
}).on('active', function () {
// replicate resumed (e.g. user went back online)
}).on('denied', function (info) {
// a document failed to replicate, e.g. due to permissions
}).on('complete', function (info) {
// handle complete
}).on('error', function (err) {
// handle error
});
If you want to permanently preserve history, you should never call compact()
or enable auto_compaction
. Once you compact, the non-leaf revisions in that database are lost.
Also, your database can grow wildly out of control, because CouchDB/PouchDB is storing your full document history. Be cognizant of browser data limits.
Also, be forewarned that CouchDB 2.0 will deprecate the _compact
API, meaning that it will compact whenever it deems necessary. So at that point, you may need to stick with CouchDB 1.x or PouchDB Server.
This plugin also supports replicating between two CouchDBs (since PouchDB itself can do that), but note that the changes will flow through PouchDB, rather than using the CouchDB replicator at the /_replicate
endpoint.
PouchDB currently has a hard limit of 1,000 revisions (unless you help fix it ๐), at which point it will cut off the revision history. CouchDB has a _revs_limit
API which you can use to configure the limit.
This algorithm is also less efficient than PouchDB's original replication algorithm, because it involves more GET
requests to fetch the parent revisions. When the bulk revs API is finished, it can be improved.
npm install
npm run build
This will run the tests in Node using LevelDB:
npm test
You can also check for 100% code coverage using:
npm run coverage
If you don't like the coverage results, change the values from 100 to something else in package.json
, or add /*istanbul ignore */
comments.
Run npm run dev
and then point your favorite browser to http://127.0.0.1:8001/test/index.html.
The query param ?grep=mysearch
will search for tests matching mysearch
.
You can run e.g.
CLIENT=selenium:firefox npm test
CLIENT=selenium:phantomjs npm test
This will run the tests automatically and the process will exit with a 0 or a 1 when it's done. Firefox uses IndexedDB, and PhantomJS uses WebSQL.