jaemk / migrant

Migration management for PostgreSQL/SQLite/MySQL

Migrant

Basic migration manager powered by migrant_lib

Supported databases/features:

Feature	Backend
postgres	Enable postgres connectivity
sqlite	Enable sqlite connectivity
mysql	Enable mysql connectivity
update	Enable self-update functionality

migrant will manage all migrations that live under <project-dir>/migrations/, where project-dir is the closest parent path that contains a Migrant.toml configuration file (..../<project-dir>/Migrant.toml). The default migration file location can be modified in your Migrant.toml file ("migration_location"). If the migration_location directory doesn't exist, it will be created the first time you create a new migration. migrant stores all applied migrations in a database table named __migrant_migrations.

Note: Configuration values prefixed with env: in your Migrant.toml will be sourced from environment variables. For example, database_user = "env:DB_USER" will use the value of the environment variable DB_USER. If a .env. file exists, it will be "sourced" automatically before your Migrant.toml is loaded.

Note: SQL statements are batch executed as is. If you want your migration to happen atomically in a transaction you must manually wrap your statements in a transaction (begin transaction; ... commit;).

Installation

Binary releases:

See releases for binaries. If you've already installed a binary release, you can update to the latest release via migrant self update.

Building from source:

By default migrant will build without any features, falling back to using each database's cli commands (psql / sqlite3 / mysqlsh). The postgres, rusqlite, and mysql database driver libraries can be activated with their respective feature flags. Note, Some drivers require their dev libraries (postgresql: libpq-dev, sqlite: libsqlite3-dev). Self update functionality (updating to the latest GitHub release) is available behind the update feature. The binary releases are built with all features.

Building from source (crates.io):

# install without features
# use cli commands for all db interaction
cargo install migrant

# install with `postgres`
cargo install migrant --features postgres

# install with `rusqlite`
cargo install migrant --features sqlite

# all
cargo install migrant --features 'postgres sqlite mysql update'

Simple Usage

migrant init [--type <database-type>, --location <project-dir>, --no-confirm] - Initialize project by creating a Migrant.toml file with db info/credentials. When run interactively (without --no-confirm), setup will be run automatically.

migrant setup - Verify database info/credentials and setup a __migrant_migrations table if missing.

migrant new <tag> - Generate new up & down files with the given <tag> under the specified migration_location.

migrant edit <tag> [--down] - Edit the up [or down] migration file with the given <tag>.

migrant list - Display all available .sql files and mark those applied.

migrant apply [--down, --all, --force, --fake] - Apply the next available migration[s].

migrant shell - Open a repl

migrant which-config - Display the full path of the Migrant.toml file being used

migrant connect-string - Display either the connection-string generated from config-params or the database-path for sqlite

migrant self update - Update to the latest version released on GitHub.

migrant self bash-completions install [--path <path>] - Generate a bash completion script and save it to the default or specified path.

Usage as a library

See migrant_lib and examples. migrant itself is just a thin wrapper around migrant_lib, so the full functionality of migration management can be embedded in your actual project.

Development

See CONTRIBUTING

Docker

An image with the binary installed is available at jaemk/migrant:latest

migrant's Issues

Failed to compile self update feature on MacOS

error logs:

Errors:

error[E0277]: the `?` operator can only be applied to values that implement `Try`
   --> /Users/david/.cargo/registry/src/github.com-1ecc6299db9ec823/migrant-0.14.0/src/main.rs:236:23
    |
236 |     let mut builder = self_update::backends::github::Update::configure()?;
    |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the `?` operator cannot be applied to type `self_update::backends::github::UpdateBuilder`
    |
    = help: the trait `Try` is not implemented for `self_update::backends::github::UpdateBuilder`
note: required by `branch`

error[E0277]: the `?` operator can only be applied to values that implement `Try`
   --> /Users/david/.cargo/registry/src/github.com-1ecc6299db9ec823/migrant-0.14.0/src/main.rs:241:18
    |
241 |         .target(&self_update::get_target()?)
    |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^ the `?` operator cannot be applied to type `&str`
    |
    = help: the trait `Try` is not implemented for `&str`
note: required by `branch`

For more information about this error, try `rustc --explain E0277`.
error: failed to compile `migrant v0.14.0`, intermediate artifacts can be found at `/var/folders/4x/rsx335ks4gs8xfw9mr4f1ff80000gn/T/cargo-installyQ4dVb`

Caused by:
  could not compile `migrant` due to 2 previous errors

typo in generated migrant.toml

defeault

Missing tag for v0.14

v0.14 was released quite a while ago, but the source code for it was never tagged. Could you still add that tag? It makes packaging the software slightly nicer.

Cannot connect to postgres database.

Unable to connect to my Postgres instance using migrant.

Error: ConfigError: Cannot connect to postgres database with connection string: "postgres://postgres:postgres@localhost:5432/postgres?sslmode=require". Do the database & user exist?

This is probably due to the fact that the instance is setup with a self-signed cert. Depending on the TLS backend being used by this tool, cert verification will need to be disabled when sslmode=require.

Happy to provide any additional information if needed.

programmable migrations

Add support for writing migrations with rust code. Ideally this could just be a trait implemented for a struct. A migrant_lib migration-config could be provided a Vec<T: Migratable> and provide a wrapper for migration files. This would only be possible when using migrant_lib and so the migration commands would need to be embedded in the app that has the migration code.

Add command to show table schemas

Syntax error in multi statement migration puts you in a bad state

SQLIte project. Made a creation migration that included multiple statements. One of which had a flaw

~/code/babble-note/babble-backend-src/native master*
❯ migrant apply --all
Applying[Up]: "/Users/bachmann/code/babble-note/babble-backend-src/native/migrations/20200615025736_init/up.sql"
Error: MigrationError: Migration was unsucessful...
MigrationError: near "text": syntax error

I fixed the migration but it looks like the original statements were not rolled back (not running in a transaction maybe?)

❯ migrant apply --all
Applying[Up]: "/Users/bachmann/code/babble-note/babble-backend-src/native/migrations/20200615025736_init/up.sql"
Error: MigrationError: Migration was unsucessful...
MigrationError: table tag already exists

But I cant go down because the original migration is not marked as ... up

~/code/babble-note/babble-backend-src/native master*
❯ migrant apply --down
Error: MigrationComplete: No un-applied `Down` migrations found

[migrant_lib] migration complete

Add a convenience method to migrant_lib::errors::Error to check if it's a MigrationComplete variant.

postgres exit status

When running sql/files with psql errors don't exit with a non-zero status code

Add MySQL support

Add a match arm for mysql in runner, shell, and init (maybe make a DBType enum to use instead of &strs). Add mysql and a run_mysql behind a feature flag.

Passing credentials via env vars

Is there a way to pass credentials (database name, username, password) using environmental variables? This way, they can be stored outside of source control and managed as secrets.

Fix travis -_-

Travis fails when attempting to test the sub-crate migrant_lib using cross test:

failed to read `/Cargo.toml`

[migrant_lib] Don't panic when missing command line executables

Return an error instead of panicking if executables are missing instead of printing the unhelpful panicked at 'called `Result::unwrap()` on an `Err` value: Error { repr: Os { code: 2, message: "No such file or directory" } }

connection errors, hostname

db hostname defaulting to empty string instead of localhost

embedding migration files

support embedding migration file contents in executable so migration files aren't needed at run-time