Build flags to select db backend? (e.g., -fmysql) about dbmigrations

[basti1302]

Just to add some (hopefully at least partially relevant info) about other possible consequences of this issue:

I have a bit of code in my app that automatically applies all missing migrations to my (PostgreSQL) database at startup. This runs fine locally, of course I had to install MySQL things to get it working (apt-get install libmysqlclient-dev in my case).

I am also using the Docker integration provided by Stack to deploy the app to production. On the Docker container (based on the fpco/stack-run, which in turn is based on Ubuntu 16.04), I'm getting

error while loading shared libraries: libmysqlclient.so.18: cannot open shared object file: No such file or directory

and the app terminates. In this case, the workaround of "just install some MySQL stuff" does not work because the MySQL version that can be installed via apt-get only provides a more recent version of that library, libmysqlclient.so.20. I haven't been able to solve this.

Any hints about possible workarounds would be appreciated. For now, I think I'll try forking dbmigrations and remove MySQL support and dependencies temporarily.

[jtdaugherty]

I might not be remembering correctly, but will cabal ignore dependencies based on flags?

This has been bothering me, too, and I want to consider another avenue that might be even more beneficial as a whole, which is to factor out backend-specific support into other packages (e.g. dbmigrations-mysql) and leave library code in dbmigrations. This would let you just select what you want on a package basis rather than remembering to configure dbmigrations correctly. It would also guarantee that the dbmigrations library interface would get vetted fully. It's more work but I would also get to factor out MySQL support and get rid of the scary warnings about it in the README. :)

I haven't thought very much about what this would entail for configuration, executable names, etc.

[creswick]

The Cabal cabal file picks dependency versions based on flags, and uses
conditionals based on OS to select the unix dep, so I believe that works.

Factoring out packages is probably a better solution, though. I don't
really trust cabal flags...

On Sat, Mar 19, 2016 at 11:29 AM Jonathan Daugherty <
[email protected]> wrote:

I might not be remembering correctly, but will cabal ignore dependencies
based on flags?

This has been bothering me, too, and I want to consider another avenue
that might be even more beneficial as a whole, which is to factor out
backend-specific support into other packages (e.g. dbmigrations-mysql)
and leave library code in dbmigrations. This would let you just select
what you want on a package basis rather than remembering to configure
dbmigrations correctly. It would also guarantee that the dbmigrations
library interface would get vetted fully. It's more work but I would also
get to factor out MySQL support and get rid of the scary warnings about it
in the README. :)

I haven't thought very much about what this would entail for
configuration, executable names, etc.

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#28 (comment)

[jtdaugherty]

@basti1302 - thanks for this report. Although I agree that this issue wouldn't be coming up if dbmigrations had a Postgres-only driver, I don't think I understand how you're getting into the situation you described. It sounds like your development environment doesn't mirror production, or you're using pre-built binaries from somewhere that should be built from source instead to ensure that your development and production environments are the same. (Bear in mind that I don't use stack or Stackage and I don't know what the stack/docker integration is doing that could be causing this.)

[basti1302]

To be quite honest, I also do not understand exactly why the app needs a mysql shared library during runtime. I'm relatively new to Haskell and do not understand the mechanics of stack completely. (It's probably something I should ask on a stack mailing list.) All I know is that this is due to the mysql dependency of dbmigrations.

Anyways, I would be very interested in one of the two solutions described here, either with cabal build flags or the plug-in mechanism you described earlier, that is, factoring out dbmigrations-mysql into its own package. Can I do anything to help to make progress into this direction?

[jtdaugherty]

@basti1302 I imagine that this is not a yak you necessarily want to shave, but I would be happy to work with someone who wants to help factor backend-specific moo programs out of the dbmigrations package. If someone wants to help with that, I'd be happy to create a ticket milestone and populate it with various tasks to guide the process.

[basti1302]

Actually this might just be the yak I want to shave. The alternatives are maintaining a fork of dbmigrations for quite a while or going without this very useful, fine library :)

My focus isn't the moo executable, though, but running migrations programatically (I do not use moo to apply migrations). So what I would like to do is to make the stuff in Database/Schema independent of any specific DB binding/backend. My guess would be that this goal aligns well enough with your goal of making moo backend-agnostic; maybe a backend agnostic moo can be build on top of a backend-agnostic Database/Schema library. Does that make any sense?

It is probably better to discuss this in more detail in a new issue, as you suggested.

[jtdaugherty]

Right now the library is already backend-agnostic but the only thing we need to do to really complete that is to remove the backend-specific implementations. Afterwards I imagine that we'll have the following packages:

• dbmigrations - backend-agnostic logic, data structures, and command interface;
• dbmigrations-mysql - a moo executable that invokes the Main routines from dbmigrations with a MySQL backend appropriately connected and configured;
• dbmigrations-postgresql - likewise for Postgres.

The logic of the moo program is already part of the dbmigrations core library as Moo.Main so the backend-specific packages would just need to ship executable sources that accept or interpret command-line arguments to configure their backends appropriately, then invoke routines in Moo.Main with the constructed backend value. This would ensure that no matter which backend you happen to be building for, you get a consistent experience with the moo binary.

The big task aside from taking backend-specific bits out of the core library will be to refactor the core library so that it doesn't do all the configuration and environment inspection to choose a backend. That will no longer be desirable once the choice of concrete backend has been done by virtue of selecting a specific package.

Does that make sense?

[jtdaugherty]

With that said, your desire to just use this as a library will be possible after all this refactoring; it's possible now but of course you're stuck building against all the possible backends.

[jtdaugherty]

This has now been resolved as of the recent work by @basti1302 and the creation of the backend-specific packages. I plan on getting all of this onto Hackage soon.