merkel
Handles your database migration crisis
merkel is a framework-agnostic database migration tool designed to autonomously run in Continuous Deployment,
with rollbacks in mind.
Installation
npm install --global merkel or npm install --save-dev merkel
Run merkel init to initialize a .merkelrc.json and install a git hook
Is the
.merkelrc.jsonrequired?
No, but it holds the migration directory, and if you use it, you could change it later because the migration directory at any time is known through git.
Is the git hook required?
No, but it helps you type less. Read on to learn more.
Workflow
Make changes to your model files
Let's say you made some changes to your model files that require a database migration.
Generate a migration file
Before you commit, create a new migration file by running merkel generate.
This will generate a new migration file inside your migration directory (default ./migrations).
If a tsconfig.json was detected, the migration file will be in TypeScript.
You can change the migration directory with --migration-dir and provide a custom template with --template.
Like all options, they can also be set in .merkelrc.json or passed through environment variables.
The name of the migration file can be set with --name. By default, a UUID is used.
Why UUIDs?
In opposite to sequential IDs or timestamps, UUIDs allow separate developers to write migration files without any conflicts. There can be migration files introduced in separate git branches or commits with migration files even cherry-picked across repositories, they will not create merge conflicts. This works well with the distributed nature of git. Providing a custom, unique name is equally good.
Write your migration file
You migration file exports two functions: up and down.
The up function is expected to make all necessary database changes compared to the previous commit.
The down function should try to reverse as good as possible. Both functions should return a Promise.
Your migration file can use any dependency you want to execute this task, import a database connection,
use a low-level driver or high-level ORM, use the AWS SDK, spawn child processes...
Example:
const db = require('../db')
exports.up = function up() {
return new Promise((resolve, reject) => db.connect(err => (err ? reject(err) : resolve()))).then(() =>
db.query('ALTER TABLE order_details RENAME COLUMN notes TO order_notes')
)
}
exports.down = function down() {
return new Promise((resolve, reject) => db.connect(err => (err ? reject(err) : resolve()))).then(() =>
db.query('ALTER TABLE order_details RENAME COLUMN order_notes TO notes')
)
}Where db.js could look like this:
const pg = require('pg')
module.exports = new pg.Client(process.env.DB)Commit
Add your model changes and the migration file, and run git commit.
If you installed the git hook, you will see that merkel detected that you added a migration file and included a
command like this in your commit message:
[merkel up d12f99e4-710d-4d4a-94f8-13d9d121bac5]
This command will later be parsed by merkel migrate.
Migrate
After checking out the new commit in Continuous Deployment or on a coworker's machine, run merkel migrate.
merkel will query the database's merkel_meta table for the last migration run, and what the HEAD was when that
migration was run.
To be able to do this, you must provide merkel with a database connection URI.
You can do this either through the --db option or through the MERKEL_DB environment variable.
It is not recommended to save this in the .merkelrc.json file, as connection data differs across environments.
To query the database, merkel needs a database driver.
The driver is detected through the protocol part of the connection URI.
In order to allow many dialects, it is not a dependency of merkel, but instead required from the current working
directory, which means you need one installed in your project (you probably already have). See supported dialects.
merkel then asks git which commits were made since then the last migration.
It then scans the new commits for the merkel commands like you saw in the example above.
Pending migrations:
The confirmation prompt will only show up if run in a TTY context and can be disabled with
--confirm=false. To get only status output, runmerkel status.
merkel will then execute the migrations in that exact order, and log these in the database as they happen.
But...
What if a migration fails?
If a migration fails (throws an exception / returns a rejected promise), the schema your source files expect doesn't match your database schema anymore. You now have two options:
- Quickly fix the migration file in a separate commit.
That commit message should not include any
merkelcommand. The nextmerkel migrateexecution will then start where it migration chain broke and will still run the migration files in the order they were specified in the commit messages, but with the newest version of the migration file. - Completely revert the deployment to the previous state, see reverting migrations
What if I need to revert a deployment?
Reverting a deployment with git revert
When you do a git revert, normally git will create a commit that is the exact inverse of the commits you want to
invert. This means, if one of the commits added a migration file, it will now be deleted. This is not desirable.
Instead of deleting the migration files, you want to keep them, but let merkel migrate them down.
To accomplish this, make sure to run git revert with the --no-commit/-n option.
This will not make the commit immediately, but only stage the proposed changes, allowing you to edit them.
Run git status to see if any migration files got deleted. If yes, you can unstage the whole migration dir with
git reset HEAD migrations
And then bring them back with
git checkout -- migrations
Now we only need to tell merkel that we want to run down migrations.
Run git commit, and in the commit message, add a merkel down command with all the migrations that need to be undone.
The command can look like this:
[merkel down 6e28dcef-16f8-4a81-8783-aedc93043fa4]
[merkel down bab251c6-4aee-4137-8a83-6e6fcab29cdf]
[merkel down 43b15c65-2f6d-447d-bbcf-0efe3a34fd10]
Or multi-line like this:
[
merkel down
6e28dcef-16f8-4a81-8783-aedc93043fa4
bab251c6-4aee-4137-8a83-6e6fcab29cdf
43b15c65-2f6d-447d-bbcf-0efe3a34fd10
]
The order is important here!
Reverting a deployment with git reset
Let's imagine you are using a production branch which always points to a specific commit on master, and regularly
gets updated with git reset to point to a new or older commit.
After you git push --force, the current HEAD will suddenly be behind the last migration run in the database.
merkel will detect this automatically, and run the migrations between HEAD and the last migrations down and in
reverse order.
What if I checkout an older commit on my dev machine?
The same applies as for Reverting a deployment with git reset.
What if I want to change the migration directory?
If you are using a .merkelrc.json that has the migration dir specified, merkel will use git show to get the
migration directory at the time the commit was made.
If not, then this is not possible without using the ability to run old migrations.
Programmatic usage
You can use merkel programmatically, for example in your favourite task runner. API documentation is available here.
TypeScript definitions are included.
Supported dialects
Currently only PostgreSQL. pg version ^6 must be installed in your project.
Wir schaffen das. – Angela Merkel
![greenkeeper[bot] avatar](https://avatars.githubusercontent.com/in/505?v=4 "greenkeeper[bot]")
![renovate[bot] avatar](https://avatars.githubusercontent.com/in/2740?v=4 "renovate[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent