vot / dbclone

Utility to move data between NoSQL databases

Hi, I'm Vot

I write code. Lots of it.

I love Open Source, Open Access and Open Research.

I create tools to fix actual problems, write sustainable code and maintain various services.

I value simplicity and elegance in code - the simpler your code is the easier it will be to maintain.

---

FFbinaries has moved to its own organisation.

Contains ffbinaries-api, ffbinaries-node and ffbinaries-prebuilt repos.

dbclone's Issues

Automated tests and CI?

Lemme know if I can help out. I prefer mocha but lots of sensible options out there.

TODO: Write better documentation

We need JSDoc standard documentation for methods written here.
The coverage of that is not great at the moment and what's there is potentially outdated.

Read the code and fill in the missing JSDoc above functions and review existing JSDoc comments :)

Create unit tests

Now that test framework is scaffolded out in #17, should add some tests 😃

dbclone/test/lib.test.js

Line 6 in 8e0522f

// TODO: Add actual test

I tried to decipher code structure a little bit but now sure how to go about it. Would be great if I got some help to start?

TODO: Add document count

TODO: Add support for keeping MongoID the same between export/import operations

At the moment the _id field gets stored as a string and therefore is not restored correctly.
This will require an update to how the data is stored as well as restored.

The data is dumped into a plain json file, therefore there is a MongoID to string conversion happening on the fly. It needs to be more explicit and the field probably should be tagged as containing MongoID (I'm thinking probably just "_id": "MongoID('5a318871e9211a7ff43a4e3f')").

Such fields will then need to be casted to the correct type while restoring.

TODO: Add support for diffing databases

TODO: Add parsing of CLI arguments

Currently only interactive (prompt-based) mode is available.

Passing arguments into command line to execute commands needs to be added (i.e. dbclone export --host localhost --db example-data-prod --dataDir ~/my-data-dump-dir)

Suppress host output

Currently, calling dbclone.export displays the host output in STDOUT:

Exporting to /home/foo/bar
Host:        mongodb://<username>:<password>@XXXXX.mlab.com:25125/dbclone-test
Selected DB: dbclone-test
------------------
Starting export...
------------------
...

Offer an option to suppress this so we can run in CircleCI which currently does not mask secrets in logs.

Alternately, lets move to TravisCI which does suppress secrets in logs.

TODO: Add support for dropping DBs