Coder Social home page Coder Social logo

go-gormigrate / gormigrate Goto Github PK

View Code? Open in Web Editor NEW
1.0K 11.0 96.0 284 KB

Minimalistic database migration helper for Gorm ORM

Home Page: https://pkg.go.dev/github.com/go-gormigrate/gormigrate/v2

License: MIT License

Go 100.00%
go gorm schema migrations schema-migrations database db

gormigrate's Introduction

Gormigrate

Latest Release Go Reference Go Report Card CI | Lint CI | Test

Gormigrate is a minimalistic migration helper for Gorm. Gorm already has useful migrate functions, just misses proper schema versioning and migration rollback support.

IMPORTANT: If you need support to Gorm v1 (which uses github.com/jinzhu/gorm as its import path), please import Gormigrate by using the gopkg.in/gormigrate.v1 import path.

The current Gorm version (v2) is supported by using the github.com/go-gormigrate/gormigrate/v2 import path as described in the documentation below.

Supported databases

It supports any of the databases Gorm supports:

  • MySQL
  • MariaDB
  • PostgreSQL
  • SQLite
  • Microsoft SQL Server
  • TiDB
  • Clickhouse

Usage

package main

import (
	"log"

	"github.com/go-gormigrate/gormigrate/v2"
	"github.com/google/uuid"
	"gorm.io/driver/sqlite"
	"gorm.io/gorm"
	"gorm.io/gorm/logger"
)

func main() {
	db, err := gorm.Open(sqlite.Open("./data.db"), &gorm.Config{
		Logger: logger.Default.LogMode(logger.Info),
	})
	if err != nil {
		log.Fatal(err)
	}

	m := gormigrate.New(db, gormigrate.DefaultOptions, []*gormigrate.Migration{{
		// create `users` table
		ID: "201608301400",
		Migrate: func(tx *gorm.DB) error {
			// it's a good pratice to copy the struct inside the function,
			// so side effects are prevented if the original struct changes during the time
			type user struct {
				ID   uuid.UUID `gorm:"type:uuid;primaryKey;uniqueIndex"`
				Name string
			}
			return tx.Migrator().CreateTable(&user{})
		},
		Rollback: func(tx *gorm.DB) error {
			return tx.Migrator().DropTable("users")
		},
	}, {
		// add `age` column to `users` table
		ID: "201608301415",
		Migrate: func(tx *gorm.DB) error {
			// when table already exists, define only columns that are about to change
			type user struct {
				Age int
			}
			return tx.Migrator().AddColumn(&user{}, "Age")
		},
		Rollback: func(tx *gorm.DB) error {
			type user struct {
				Age int
			}
			return db.Migrator().DropColumn(&user{}, "Age")
		},
	}, {
		// create `organizations` table where users belong to
		ID: "201608301430",
		Migrate: func(tx *gorm.DB) error {
			type organization struct {
				ID      uuid.UUID `gorm:"type:uuid;primaryKey;uniqueIndex"`
				Name    string
				Address string
			}
			if err := tx.Migrator().CreateTable(&organization{}); err != nil {
				return err
			}
			type user struct {
				OrganizationID uuid.UUID `gorm:"type:uuid"`
			}
			return tx.Migrator().AddColumn(&user{}, "OrganizationID")
		},
		Rollback: func(tx *gorm.DB) error {
			type user struct {
				OrganizationID uuid.UUID `gorm:"type:uuid"`
			}
			if err := db.Migrator().DropColumn(&user{}, "OrganizationID"); err != nil {
				return err
			}
			return tx.Migrator().DropTable("organizations")
		},
	}})

	if err = m.Migrate(); err != nil {
		log.Fatalf("Migration failed: %v", err)
	}
	log.Println("Migration did run successfully")
}

Having a separate function for initializing the schema

If you have a lot of migrations, it can be a pain to run all them, as example, when you are deploying a new instance of the app, in a clean database. To prevent this, you can set a function that will run if no migration was run before (in a new clean database). Remember to create everything here, all tables, foreign keys and what more you need in your app.

type Organization struct {
	gorm.Model
	Name    string
	Address string
}

type User struct {
	gorm.Model
	Name string
	Age int
	OrganizationID uint
}

m := gormigrate.New(db, gormigrate.DefaultOptions, []*gormigrate.Migration{
    // your migrations here
})

m.InitSchema(func(tx *gorm.DB) error {
	err := tx.AutoMigrate(
		&Organization{},
		&User{},
		// all other tables of you app
	)
	if err != nil {
		return err
	}

	if err := tx.Exec("ALTER TABLE users ADD CONSTRAINT fk_users_organizations FOREIGN KEY (organization_id) REFERENCES organizations (id)").Error; err != nil {
		return err
	}
	// all other constraints, indexes, etc...
	return nil
})

Options

This is the options struct, in case you don't want the defaults:

type Options struct {
	// TableName is the migration table.
	TableName string
	// IDColumnName is the name of column where the migration id will be stored.
	IDColumnName string
	// IDColumnSize is the length of the migration id column
	IDColumnSize int
	// UseTransaction makes Gormigrate execute migrations inside a single transaction.
	// Keep in mind that not all databases support DDL commands inside transactions.
	UseTransaction bool
	// ValidateUnknownMigrations will cause migrate to fail if there's unknown migration
	// IDs in the database
	ValidateUnknownMigrations bool
}

Who is Gormigrate for?

Gormigrate was born to be a simple and minimalistic migration tool for small projects that uses Gorm. You may want to take a look at more advanced solutions like golang-migrate/migrate if you plan to scale.

Be aware that Gormigrate has no builtin lock mechanism, so if you're running it automatically and have a distributed setup (i.e. more than one executable running at the same time), you might want to use a distributed lock/mutex mechanism to prevent race conditions while running migrations.

Contributing

To run integration tests, some preparations are needed. Please ensure you have task and docker installed. Then:

  1. Ensure target or all databases are available and ready to accept connections. You can start databases locally with task docker:compose:up
  2. Copy integration-test/.example.env as integration-test/.env and adjust the database connection ports and credentials when needed.
  3. Run integration test for single database or for all
# run test for MySQL
task test:mysql

# run test for MariaDB
task test:mariadb

# run test for PostgreSQL
task test:postgres

# run test for SQLite
task test:sqlite

# run test for Microsoft SQL Server
task test:sqlserver

# run test for all databases
task test:all

Alternatively, you can run everything in one step: task docker:test

gormigrate's People

Contributors

andreynering avatar arshsingh avatar avakarev avatar cgetzen avatar dacohen avatar dependabot[bot] avatar dinos80152 avatar dseevr avatar gitter-badger avatar hashlash avatar heww avatar j16r avatar lonng avatar mdales avatar mennanov avatar mkozjak avatar tboerger avatar tomfeigin avatar yutita avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gormigrate's Issues

Allow fake migrations

Having recently discovered this project, and trying to migration an existing project - I realized a blocker for migrating existing project for a lot of people would do fake migrations.

I'd model this against Django, which describes it as:

Marks the migrations up to the target one (following the rules above) as applied, but without actually running the SQL to change your database schema.

This is intended for advanced users to manipulate the current migration state directly if theyโ€™re manually applying changes; be warned that using --fake runs the risk of putting the migration state table into a state where manual recovery will be needed to make migrations run correctly.

I shouldn't have to manually muck around with SQL queries when integrating this library.

I can take a stab at it if this looks like a reasonable feature.

Migrations ordering issue

If you have InitSchema migration and another migrations on the first start, schema initialization migration is applied after other migrations. There are no changes in the database structure, but migrations are recorded in the migrations table. Because of migrations are applied to non-existent database schema.

func Migrate() error {
	connection, err := db.Db.GetConnection(true)
	if err != nil {
		return fmt.Errorf("get database connection: %v", err)
	}

	m := gormigrate.New(connection, migrationsOptions, []*gormigrate.Migration{
		m201901291100,
		m201902041157,
		m201902151152,
		m201903121503,
		m201903130943,
		m201903151255,
	})

	m.InitSchema(func(tx *gorm.DB) error {
		fmt.Println("Init database schema")
		err := tx.AutoMigrate(
			&entities.Notification{},
		).Error
		if err != nil {
			return fmt.Errorf("init schema: %v", err)
		}

		return nil
	})

	if err := m.Migrate(); err != nil {
		return fmt.Errorf("migrate: %v", err)
	}

	return nil
}

No way to tell which order migrations were applied in or when they were applied

Hi,

Thanks for making this library.

Because there is no auto-incrementing primary key column and there is no column tracking when migrations were applied, there is no way to tell which order the migrations were applied in.

How do you feel about adding an applied column similar to Django?

mysql> show full columns from django_migrations;
+---------+--------------+--------------------+------+-----+---------+----------------+---------------------------------+---------+
| Field   | Type         | Collation          | Null | Key | Default | Extra          | Privileges                      | Comment |
+---------+--------------+--------------------+------+-----+---------+----------------+---------------------------------+---------+
| id      | int(11)      | NULL               | NO   | PRI | NULL    | auto_increment | select,insert,update,references |         |
| app     | varchar(255) | utf8mb4_unicode_ci | NO   |     | NULL    |                | select,insert,update,references |         |
| name    | varchar(255) | utf8mb4_unicode_ci | NO   |     | NULL    |                | select,insert,update,references |         |
| applied | datetime(6)  | NULL               | NO   |     | NULL    |                | select,insert,update,references |         |
+---------+--------------+--------------------+------+-----+---------+----------------+---------------------------------+---------+
4 rows in set (0.01 sec)

If you have no objections, I can add this + tests.

List migrations

There should be a way to list which migrations have been applied, and which are pending. This would be immensely helpful in trying to apply migrations to a real life environment, stage or prod, where you don't want to run queries without knowing what queries would be run. Django provides a good model for this.

Also kinda related is the concept of dry run in the migrate method. In a similar vein as above, dry run would not do any writes to the database, but just list which queries it would've run if you didn't use the dry-run option. Django doesn't support this option, but again this would be really useful is increasing confidence when using this library on live environments such as stage or prod.

Implementation: I'd add a separate function (perhaps ShowMigrations) that lists all migrations with a status of whether they have been applied or not. For dry run, perhaps a new method MigrateWithOptions(options) would be best, so you can extend it in the future without breaking function signature. Migrate method could just be changed with this signature in v2 if that works.

I can take a stab at it if the this sounds reasonable.

Feature request: Rename the table

Is it not possible to specify TableName with this library?

Putting this code won't work

	Migrate: func(tx *gorm.DB) error {
		// it's a good pratice to copy the struct inside the function,
		// so side effects are prevented if the original struct changes during the time
		type Person struct {
			gorm.Model
			Name string
		}
                func (Person) TableName() string {
	                return "justpeople"
                }
		return tx.AutoMigrate(&Person{})
	},

Rollback failed

goVersion: go1.13.5
platform: linux/amd64

Migrations

		Rollback: func(tx *gorm.DB) error {
			if err := tx.Migrator().DropColumn("statistic_items", "parent_id"); err != nil {
				return err
			}
			return tx.Migrator().DropColumn("statistic_items", "embed_link")
		},

Error

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x8b75d6]

goroutine 1 [running]:
gorm.io/gorm/migrator.Migrator.DropColumn.func1(0xc00024e000, 0xc00024e000, 0xc0001114f8)
	/root/go/pkg/mod/gorm.io/[email protected]/migrator/migrator.go:287 +0x46
gorm.io/gorm/migrator.Migrator.RunWithValue(0x7c00000000000000, 0xc0001eaf30, 0xb08d80, 0xc0001b8780, 0x95ff60, 0xaecd40, 0xc000111508, 0xc000111550, 0xc000111540)
	/root/go/pkg/mod/gorm.io/[email protected]/migrator/migrator.go:44 +0xb3
gorm.io/gorm/migrator.Migrator.DropColumn(0x8f8800, 0xc0001eaf30, 0xb08d80, 0xc0001b8780, 0x95ff60, 0xaecd40, 0xa2b122, 0x9, 0xc0001b8780, 0xc0001b8780)
	/root/go/pkg/mod/gorm.io/[email protected]/migrator/migrator.go:286 +0xba
main.m202011060001.func2(0xc0001eaf30, 0xc0001fb020, 0xc0001fb010)

Source Code
image

Validating the migration ID exist in the list of migrations

With MigrateTo(migrationID) if migrationID does not exist in the list of migrations, gormigrate will migrate to the latest migration like MigrateTo(""). Similarly, RollbackTo(migrationID) will also rollback all migrations if the migrationID does not exist. So a simple typo will have catastrophic consequences.

It seems to make more sense for the desired effect to return an error when the migrationID does not exist. MigrateTo already checks for checkDuplicatedID. We can add something like checkIDExist to MigrateTo and RollbackTo as such

// ErrMigrationIDDoesNotExist is returned when migrating or rolling back to a migration ID that
// does not exist in the list of migrations
var ErrMigrationIDDoesNotExist = errors.New("gormigrate: Tried to migrate to an ID that doesn't exist")

func (g *Gormigrate) checkIDExist(migrationID string) error {
	if migrationID == "" {
		return nil
	}
	for _, migrate := range g.migrations {
		if migrate.ID == migrationID {
			return nil
		}
	}
	return ErrMigrationIDDoesNotExist
}

Dependencies

This is not an issue, just a question.

The demo code has a comment saying that we should copy the struct corresponding to a table to the migration function itself. This makes sense to me.

However, what if I'm trying to make a model/table that has a foreign key to another table, which has a key to another table, which has a foreign key to a fourth table. It doesn't seem quite right to have a bunch of copies of every struct, with every migration.

What's the recommended pattern here?

Any plan on releasing a new minor / patch version?

The current version 2.0.0 is still using gorm.io/gorm v1.20.0 which is the version I'm using.
I want to upgrade my gorm to its latest (1.23.4) but it isn't 100% compatible with gormigrate v2.0.0.

Is it possible to release a new version please?

Failed to migrate if no migration is defined

Hi,

I have the following code:

m := gormigrate.New(d.db, gormigrate.DefaultOptions, []*gormigrate.Migration{})

m.InitSchema(func(tx *gorm.DB) error {
	return tx.AutoMigrate(
		&MyTable{},
	).Error
})

if err := m.Migrate(); err != nil {
	log.Fatalf("Could not migrate: %v", err)
}

When I run this I get the following error:
Could not migrate: gormigrate: No migration defined

This happens because the following line of code (which was recently introduced I believe):
https://github.com/go-gormigrate/gormigrate/blob/master/gormigrate.go#L121-L123

Could you fix this please?

Unexpected query error using `gorm.io/driver/clickhouse` when `UseTransaction = true`

Running the example of the README gives a 101 Unexpected query when running multiple migrations with UseTransaction = true.

package main

import (
	"github.com/go-gormigrate/gormigrate/v2"
	"github.com/google/uuid"
	"gorm.io/driver/clickhouse"
	"gorm.io/gorm"
	"gorm.io/gorm/logger"
	"log"
)

func main() {
	db, err := gorm.Open(clickhouse.Open("clickhouse://clickhouse:9000"), &gorm.Config{
		Logger: logger.Default.LogMode(logger.Info),
	})
	if err != nil {
		log.Fatal(err)
	}

	m := gormigrate.New(db, &gormigrate.Options{
		TableName:                 "migrations",
		IDColumnName:              "id",
		IDColumnSize:              255,
		UseTransaction:            true,
		ValidateUnknownMigrations: false,
	}, []*gormigrate.Migration{{
		// create `users` table
		ID: "201608301400",
		Migrate: func(tx *gorm.DB) error {
			// it's a good pratice to copy the struct inside the function,
			// so side effects are prevented if the original struct changes during the time
			type user struct {
				ID   uuid.UUID `gorm:"type:UUID;primaryKey;uniqueIndex"`
				Name string
			}
			return tx.Migrator().CreateTable(&user{})
		},
		Rollback: func(tx *gorm.DB) error {
			return tx.Migrator().DropTable("users")
		},
	}, {
		// add `age` column to `users` table
		ID: "201608301415",
		Migrate: func(tx *gorm.DB) error {
			// when table already exists, define only columns that are about to change
			type user struct {
				Age int
			}
			return tx.Migrator().AddColumn(&user{}, "Age")
		},
		Rollback: func(tx *gorm.DB) error {
			type user struct {
				Age int
			}
			return db.Migrator().DropColumn(&user{}, "Age")
		},
	}, {
		// create `organizations` table where users belong to
		ID: "201608301430",
		Migrate: func(tx *gorm.DB) error {
			type organization struct {
				ID      uuid.UUID `gorm:"type:UUID;primaryKey;uniqueIndex"`
				Name    string
				Address string
			}
			if err := tx.Migrator().CreateTable(&organization{}); err != nil {
				return err
			}
			type user struct {
				OrganizationID uuid.UUID `gorm:"type:UUID"`
			}
			return tx.Migrator().AddColumn(&user{}, "OrganizationID")
		},
		Rollback: func(tx *gorm.DB) error {
			type user struct {
				OrganizationID uuid.UUID `gorm:"type:UUID"`
			}
			if err := db.Migrator().DropColumn(&user{}, "OrganizationID"); err != nil {
				return err
			}
			return tx.Migrator().DropTable("organizations")
		},
	}})

	if err = m.Migrate(); err != nil {
		log.Fatalf("Migration failed: %v", err)
	}
	log.Println("Migration did run successfully")
}

The error received is an code: 101, message: Unexpected packet Query received from client after the first migration.
The last function called before the commit is the insertMigration function. Slightly tweaking this has different results.

For example, the following works without error:

func (g *Gormigrate) insertMigration(id string) error {
	record := map[string]interface{}{g.options.IDColumnName: id}
	return g.tx.Create(record).Error
}

But even something like return g.tx.Table("migrations").Create(record).Error fails.

Funnily enough, the following does work, most likely because the transaction is committed earlier.

func (g *Gormigrate) insertMigration(id string) error {
	record := map[string]interface{}{g.options.IDColumnName: id}
	return g.db.Table(g.options.TableName).Create(record).Error
}

System

  • go 1.21.0
  • gorm.io/driver/clickhouse v0.5.1
  • gorm.io/gorm v1.25.5
  • clickhouse-server:23.9-alpine

Add ability to rollback all migrations

This is a great package and it had helped me a lot. But, the missing ability to "rollback all migrations" have been troubling me for a while.
If I run RollbackTo with the first migration's id, it will keep the first migration unrolled. I can only call RollbackLast multiple times or call it after RollbackTo the first migration.
This feature is needed when we want to clean the database and rebuild a fresh new one. While developing an application, this is a common need by developers. A simple method RollbackAll would be nice.

Noisy log message `transaction has already been committed or rolled back` during regular usage

Since we use this pattern:

tx.begin()
defer tx.rollback()

This means rollback is called even when a transaction is committed. This log message is mostly harmless, but it makes it difficult to properly diagnose improper use of transactions in our code base.

Screen Shot 2019-04-30 at 13 14 19

Other than doing:

tx.begin()
if err != nil {
  tx.rollback()
  return err
}
return tx.commit()

Maybe this could be handled by suppressing logging from gormigrate's use of rollback or by checking if a commit has already been run and turning rollback into a no-op. Thoughts?

Init schema applies few times

In my app I have for right now only initSchema function and no migrations.

func (g *Gormigrate) isFirstRun() bool {
	var count int
	g.db.
		Table(g.options.TableName).
		Count(&count)
	return count == 0
}

As I understand this function calculates amount of migrations executed before. But in my case it always 0 and that is causing error on 2nd launch of application.

go get -u is broken when using modules

Using go1.11.2 darwin/amd64

Any 'go get -u' or 'go get -u ./...' fails and stops with

go: github.com/go-gormigrate/[email protected]: parsing go.mod: unexpected module path "gopkg.in/gormigrate.v1"

With verbose logging also adds

go: error loading module requirements

Regardless if I use "github.com/go-gormigrate/gormigrate" or "gopkg.in/gormigrate.v1" as import the result is the same.

Migration breaks in gorm.io/gorm 1.25.6 when using pgx driver for PostgreSQL

Specifically the change go-gorm/gorm@v1.25.5...v1.25.6#diff-7539aa7c170a85138fa67c7846b65fda95a51169bcf8637d5961ea570307d755 will end up with an error LastInsertId is not supported by this driver.

It seems to be the hasReturning(*gorm.DB,bool)(bool, gorm.ScanMode) in
gorm.io/gorm/callbacks/helper.go:96 that breaks existing functionality.
This file doesn't appear to be changed in the gorm.io version upgrade.

Might it look like that Gorm has started to require a RETURNING on the kind of INSERT/Create() that gormigrate perform?

Versions:
Go 1.21.5 linux/amd64
Gorm.io 1.25.6
Gormigrate 2.1.1 (v2 packages)

Works with versions:
Go 1.21.5 linux/amd64
Gorm.io 1.25.5
Gormigrate 2.1.1 (v2 packages)

Rollback function not called automatically

Hi everybody,

Rollback isn't executed when an error occured in the migration script. The issue have been raised in #42 but the solution provided isn't working as expected because RollbackLast only rollback the last succeed migration, not the last that have failed :( .

Dependencies

gorm.io (so v2) version: v1.21.12
go migrate version: v2.0.0
postgresql 12
go version: 1.16

Migrations

package main

var Migrations = []*gormigrate.Migration{
   {
		ID: "2021082409555553",
		Migrate: func(tx *gorm.DB) error {
			return tx.Exec(`ALTER TABLE "todos" ALTER COLUMN "text" TYPE varchar(200)`).Error
		},
		Rollback: func(tx *gorm.DB) error {
			return tx.Exec(`ALTER TABLE "MMMMMMM" ALTER COLUMN "text" TYPE varchar(200)`).Error
		},
	},
}

Execute Migrate

package main

func Migrate(db *gorm.DB) error {
	m := gormigrate.New(db, gormigrate.DefaultOptions, Migrations)

	return m.Migrate()
}

Actual

The migration function is called, fails because the todos table doesn't exist and the rollback function isn't called.

Note: Logs have been wrapped with logrus

DEBU[0000] SELECT count(*) FROM "migrations" WHERE id = '2021082409555553'  error="<nil>" rows=1
DEBU[0000] ALTER TABLE "todos" ALTER COLUMN "text" TYPE varchar(200)  error="ERROR: relation \"todos\" does not exist (SQLSTATE 42P01)" rows=0
FATA[0000] ERROR: relation "todos" does not exist (SQLSTATE 42P01)  error="ERROR: relation \"todos\" does not exist (SQLSTATE 42P01)"

Expectations

The first ALTER TABLE will fail because the table doesn't exists and then Rollback function must be called.

Anybody have a clue ?

Thanks !

Migration rollback does not working

Hi, contributors
Rollback does not seem to be executed when an error occurs in the migration.
am I using it wrong?

  • dependencies:
    • gorm version: v1.9.14
    • go migrate version: v1.6.0
    • mariadb 10.4.3
    • go version: 1.12

Migrations

package main

var Migrations = []*gormigrate.Migration{
	{
		ID: "202006181800",
		Migrate: func(tx *gorm.DB) error {
			group := new(entities.GroupExtend)
			visitor := new(entities.VisitorExtend)
                         
                        if err := tx.AutoMigrate(group, visitor).Error; err != nil {
				return err
                        }

                        // this code will always fail.
                        if err := tx.Model(visitor).AddForeignKey("group_id", "group_s(id)", "CASCADE","CASCADE").Error; err != nil {
				return err
			}
                         return nil
                },
                Rollback: func(tx  *gorm.DB) error {
                         return tx.DropTableIfExists("group_extends", "visitor_extends").Error
                }
         },
}

Execute Migrate

package main

func Migrate(db *gorm.DB, migrations []*gormigrate.Migration) error {
	db.LogMode(true)
	m := gormigrate.New(db, gormigrate.DefaultOptions, Migrations)

	return m.Migrate()
}

Expectations

I get an error when registering a foreign key, I must execute the Rollback method.

DROP TABLE ....(SQL..)

Actual

[2020-06-25 16:51:32]  Error 1005: Can't create table `visitor_extends` (errno: 150 "Foreign key constraint is incorrectly formed") 

[2020-06-25 16:51:32]  [16.23ms]  ALTER TABLE `visitor_extends` ADD CONSTRAINT visitor_extends_group_id_group_ex_id_foreign FOREIGN KEY (group_id) REFERENCES group_s(id) ON DELETE CASCADE ON UPDATE CASCADE;  

Problem

As you can see from the results, i can't see the record of the Rollback function being executed.

Maintenance status of this project (2023)?

This library looks great, thank you! As someone who is considering start using it I am a little worried about the maintenance status. Could someone comment on this? At the time of writing (Feb 2023) it seems that there has been no release with features/bugfixes in 3 years (only two releases updating dependencies), there are several large PRs open without any comments or explanations ... ๐Ÿค”

This is not meant as a criticism towards the maintainers but as guidance for people considering using this project!

callback functions after successful migration

Hey, great work with this package so far. What I miss is a way to hook into a chain of migrations/rollbacks and execute some code after each successful migration. This could be used for example to report to current progress of the migrations.
What I have in mind is something like extending the Options struct as follows. The methods would be called after each migration step.

type Options struct {
	// ...
	AfterMigration func(m *Migration)
	AfterRollback func(m *Migration)
}

If you are happy with that change, I can provide a PR for it.

Improve readme for more gomodules clarity

I'm using gormigrate in a project, but I'm encountering some difficulties with importing the package. The readme could be of more help in this instance.

The error I'm getting is:

go: github.com/go-gormigrate/[email protected]: parsing go.mod: unexpected module path "gopkg.in/gormigrate.v1"
go: error loading module requirements

Which is possibly a conflict in import urls, but changing over ALL import paths doesn't really sound that normal to me like in issue #22.

Is a cli on the horizon?

Is a cli on the horizon?

Would be great to use it like:

gormigrate -m for all migrations

or

gormigrate -m 201608301400

or

gormigrate -r 201608301400 for rollback

and so on...

Provide a method to rollback all migrations?

A method to rollback all migrations would be very helpful in testing. While working on library/shared code, it is impossible to know the first ID without doing an extra query.

UseTransaction should be true by default

I have a mult-steps migration like:

return &gormigrate.Migration{
	Migrate: func(tx *gorm.DB) error {
		// This creates the table successfully
		if err := tx.Migrator().CreateTable("table1"); err != nil {
			return err
		}

		// Failure happens here
		return migrator.CreateTable("table2")
	},
}

Running this result in a partial failure; part of the migration is executed with no problem but it fails after.
The result is that I end up with a corrupt migration state. Since this can happen and can lead to potential bugs and or manual intervention to correct the db state, I believe that by default migrations should run inside a tx unless it is stated otherwise.
This will be simply fixed by making DefaultConfiguration return UseTransaction: true

Provide option to lock migration table during migration to prevent concurrent migrations

I started trying to solve this in gormigrate, but before I get too stuck into this, I thought I'd seen what, if any, appetite there is for solving this here.

The problem I have is that in a production environment where I have many nodes talking to a single DB backend the migrations may race. To solve this I'd like the migrate function to have the option to lock the migrations table for the duration of migrations.

However given how different the syntax is between Postgres, MySQL and SQLServer, this will require a number of DB specific implementations (I wouldn't plan to support this feature on SQLite).

Is this worth doing here? Logically it's the right place to put the logic I think, but looking at the existing code there's no real DB specific code in there yet, so I don't want to write all the code and have it rejected as no something the gormigrate community want.

Gormigrate doesn't allow specifying a schema on the TableName option

Hi,
First of all thank you for gomigrate. It`s very usefull.
I have a problems with postgres database with several schemas.
In case if we got sql with additional shemas in migration, gorm HasTable method returns false, even if table really exists, so createMigrationTableIfNotExists trying to create table and fails(even if I add schema to migration table name in options). I know this is gorm issue, but after checking their repo issues I found that this known problem and will not to be fixed soon.

What about custom method to check table existing?

Thank you

How does people table name refer to Person struct in the example/tests?

๐Ÿ‘‹
I'm trying to understand how the people table name linked to Person{} as specified in README and tests.

Initially I was trying to set a custom name, and in gorm we usually has the func similar to following to specify the table name:

func (Person) TableName() string {
	return "people"
}

But since the suggestions is to freeze the struct with the migration by specifying inside the function, it's not clear how the following code really works or has a typo:

		// create persons table
		{
			ID: "201608301400",
			Migrate: func(tx *gorm.DB) error {
				// it's a good pratice to copy the struct inside the function,
				// so side effects are prevented if the original struct changes during the time
				type Person struct {
					gorm.Model
					Name string
				}
				return tx.AutoMigrate(&Person{})
			},
			Rollback: func(tx *gorm.DB) error {
				return tx.Migrator().DropTable("people")
			},
		},

Once the InitSchema function is called, the contents of []*gormigrate.Migration are not executed as expected.

My database is clean, without any tables.

I want to InitSchema and do some other migrations, just like this:

	m := gormigrate.New(db, gormigrate.DefaultOptions, []*gormigrate.Migration{
		&gormigrate.Migration{
			ID: "20230716",
			Migrate: func(tx *gorm.DB) error {
				return tx.Exec("INSERT INTO user (username) VALUES ('admin')").Error
			},
			Rollback: func(tx *gorm.DB) error {
				return tx.Exec("DELETE FROM user WHERE username='admin'").Error
			},
		},
	})
	m.InitSchema(func(tx *gorm.DB) error {
		err := tx.AutoMigrate(
			&User{},
			// all other tables of you app
		)
		return err
	})
	if err := m.Migrate(); err != nil {
		log.Fatalf("Migration failed: %v", err)
	}
	log.Println("Migration did run successfully")

However, the sql INSERT INTO user (username) VALUES ('admin') was not executed as expected.

I then checked the source code of this library. There is only insertMigration in this place:

gormigrate/gormigrate.go

Lines 347 to 351 in 293e5ee

for _, migration := range g.migrations {
if err := g.insertMigration(migration.ID); err != nil {
return err
}
}

Is migration.Migrate(g.tx) missing. Or should it be call g.runMigration(migration.ID) instead of g.insertMigration(migration.ID)?

Relation <migration table name> already exists

I'm having an issue where gormigrate fails when calling the Migrate function, with the error:
pq: relation <table name specified in gormigrate.Options> already exists"
Indeed, the table does exist in the database, as it should, so I don't understand why this should result in an error. Should InitSchema be called once and only once, or in other words, should my code only run InitSchema if the migration table does not exist?

The only way I can see this error occurring is if the line here:

if g.tx.Migrator().HasTable(g.options.TableName) {

does not correctly evaluate to true for the existing table, and therefore allows the attempt to create the table to go through.

Pure-Go SQLite implementation

Is _ "github.com/jinzhu/gorm/dialects/sqlite" required for gormigrate to work?
I'm trying to replace my usage of github.com/mattn/go-sqlite3 (which requires CGO to cross-compile) with pkg.go.dev/modernc.org/sqlite.

I've been testing with github.com/glebarez/sqlite (which is an alternative to sqlite driver for GORM go-gorm/gorm#4101) and it seems to be working fine.

Just wanted your confirmation.

import (
	"log"

	"github.com/go-gormigrate/gormigrate/v2"
	"gorm.io/gorm"
	_ "github.com/jinzhu/gorm/dialects/sqlite"
)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.