Comments (13)
I'll try to find some time to continue expanding the Reference section in the docs I've been slowly chipping away at. "Custom store" is on that list :)
from goose.
The newly added goose.NewProvider supports adding your own store implementation.
WithStore
This allows your custom dialect to implement the database.Store interface and use goose against that.
I've been meaning to document this better.
There's a bit more context here in case you're interested.
from goose.
I'll fix this up, I want to slightly expand the interface to include a GetLatestVersion
method as a future improvement.
Otherwise, this is ready to go.
For fun, I implemented a custom Store
using an in-memory implementation against a sqlite database.
https://github.com/mfridman/goose-demo/blob/main/customstore/memory/memory.go
Can see the usage here:
https://github.com/mfridman/goose-demo/blob/main/cmd/custom-store/main.go
I also want to add Redis and then move on to some more interesting ones like duckdb.
Obviously these are contrived examples, but they give an idea that goose should work with any correctly implemented Store.
from goose.
Yes, I typically batch a few changes and release every 1-3 months. I'll probably do the next release this weekend.
from goose.
Sounds like there's a way to do this. Will close. Thank you for such a quick response!
from goose.
Looking at the WithStore
option for bringing our own Provider (https://github.com/pressly/goose/blob/42eab2b6e805616fd54ebb494ec7d1b90863b1d3/database/store.go#L45C2-L45C9):
// Each database dialect requires a specific implementation of this interface. A dialect represents
// a set of SQL statements specific to a particular database system.
type Store interface {
// Tablename is the version table used to record applied migrations. Must not be empty.
Tablename() string
// CreateVersionTable creates the version table. This table is used to record applied
// migrations. When creating the table, the implementation must also insert a row for the
// initial version (0).
CreateVersionTable(ctx context.Context, db DBTxConn) error
// Insert inserts a version id into the version table.
Insert(ctx context.Context, db DBTxConn, req InsertRequest) error
// Delete deletes a version id from the version table.
Delete(ctx context.Context, db DBTxConn, version int64) error
// GetMigration retrieves a single migration by version id. If the query succeeds, but the
// version is not found, this method must return [ErrVersionNotFound].
GetMigration(ctx context.Context, db DBTxConn, version int64) (*GetMigrationResult, error)
// ListMigrations retrieves all migrations sorted in descending order by id or timestamp. If
// there are no migrations, return empty slice with no error. Typically this method will return
// at least one migration, because the initial version (0) is always inserted into the version
// table when it is created.
ListMigrations(ctx context.Context, db DBTxConn) ([]*ListMigrationsResult, error)
// TODO(mf): remove this method once the Provider is public and a custom Store can be used.
private()
}
Looks like there's a private member to that interface. Are there any other options?
from goose.
Do you plan on releasing a new version with this latest change soon?
from goose.
@mfridman Hi, thank you for the option. I tried this out recently and I could not find a way to actually update the global variable store by just using goose.NewProvider("", db, files, goose.WithStore(NewBQStore()))
. And after that I tried to use goose.Up()
to migrate the db, but it does not use the store I implemented but still the default (postgres in my case) so I am getting sql format error.
I am wondering is there a way to actually set the store to be custom store, something similar as "goose.SetDialect()"?
from goose.
I appreciate how confusing this might be. The short answer is goose.NewProvider
has no global state (except for globally registered Go migrations1 for backwards compatibility), so you should run:
provider, err := goose.NewProvider(...)
res, err := provider.Up()
When adding the new goose provider we went back and forth on whether to cut a new major version (/v4
) or implement it in a backwards-compatible way within the current /v3
. We opted for the latter to reduce fragmentation, but this does mean overhead for new users or those not keeping up with the project.
Please let us know if you need an example or clarification on the README or docs.
Maybe in the future, we might add a global goose.SetStore
so that all global (non-provider) functions behave as you expect. Sorry for the confusion.
Lastly, I'm curious why you need a custom Store implementation for postgres when goose natively supports it?
Footnotes
-
Even this can be scoped to the provider using a functional option WithGoMigrations ↩
from goose.
@mfridman I am adding a bigquery provider. And in our code, we are already using the already-implemented postgres dialect. We are trying to use goose to manage all migrations in the code for different types of databases.
Used provider.UP()
and it worked! Thank you very much.
BTW, I do have a fork with bigquery dialect implemented, are you interested in adding those to official goose? If so, I can continue to work on it to match your code standard and tests.
from goose.
Yep, contributions are always welcome. I'm sure other folks would benefit from an implementation.
The only tricky part is adding a supported community driver to the CLI. It's a bit of a pain nowadays because some drivers are supporting only the latest Go version, whereas with goose we try to lag 1-2 releases behind to avoid breaking folks.
I'm trying to see if this is the correct path forward #664
from goose.
I cut a v3.20.0 release this morning. I'd like to keep this issue open as a reminder to also update the documentation website.
from goose.
I started to expand the docs here:
https://pressly.github.io/goose/documentation/custom-store/
I'll close this issue, but if you have any issues feel free to continue commenting or re-open.
from goose.
Related Issues (20)
- Need advice and guidance on our case. HOT 3
- Semicolon Detection Fails for "--" in Strings in SQL Statements HOT 3
- Support for quiet operations HOT 4
- Creating two migrations at the same time HOT 1
- New feature `up-preview` `down-preview`? HOT 3
- Request to release 3.18.1 HOT 4
- Question: Is it possible to conditionally run different statements based on type of database? HOT 5
- Build breaks: undefined symbol: duckdb_xx HOT 4
- Feature request: goose seed HOT 4
- failed to insert new goose version: Error 1054 (42S22): Unknown column '$1' in 'field list' HOT 2
- "NO TRANSACTION" migrations should mark database as dirty if not successful HOT 3
- clickhouse: getting a sql.ErrSkip at createVersionTable() HOT 4
- Error when executing SQL migration with specific comments HOT 2
- Add support for ssl ca-cert for Postgres HOT 1
- Update README.md
- fail when there is a pending migration and out of order migrations are not allowed HOT 2
- NO TRANSACTION annotation is case sensitive HOT 7
- Feature request: explore WASM-based migrations
- Question about providing our own Logger implementation HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from goose.