okjiang / dumpling

Dumpling is a fast, easy-to-use tool written by Go for dumping data from the database(MySQL, TiDB...) to local/cloud(S3, GCP...) in multifarious formats(SQL, CSV...).

🥟 Dumpling

Dumpling is a tool and a Go library for creating SQL dump from a MySQL-compatible database. It is intended to replace mysqldump and mydumper when targeting TiDB.

You may read the design document, English user guide and 中文使用手册 for details.

Features

Dumpling is currently in early development stage, and most features are incomplete. Contributions are welcomed!

• SQL dump is split into multiple files (like mydumper) for easy management.
• Export multiple tables in parallel to speed up execution.
• Multiple output formats: SQL, CSV, ...
• Write to cloud storage (S3, GCS) natively
• Advanced table filtering

Any questions? Let's discuss in #sig-migrate in Slack!

Building

1. Install Go 1.16 or above
2. Run make build to compile. The output is in bin/dumpling.
3. Run make test to run the unit tests.
4. Run make integration_test to run integration tests. For integration test:

• The following executables must be copied or generated or linked into these locations:
  • bin/sync_diff_inspector (download from tidb-enterprise-tools-latest-linux-amd64)
  • bin/tidb-server (download from tidb-master-linux-amd64)
  • bin/tidb-lightning (download from tidb-toolkit-latest-linux-amd64)
  • bin/minio (download from https://min.io/download)
• The following programs must be installed:
  • mysql (the CLI client)
• There must be a local mysql server listening on 127.0.0.1:3306, and an active user with no password that can be connected through this TCP address.

License

Dumpling is under the Apache 2.0 license. See the LICENSE file for details.