woolfg / mysql-backup-sidecar Goto Github PK

Thank you for your work on this handy container.

A fairly straightforward request...please consider adding curl to the container so that through the use of your post-script hook, we can communicate with external APIs. For example, I send an API post to a Slack channel that looks much like the following:

curl -i -X POST -H "Content-Type: application/json" -d "${PAYLOAD}" "${CONN}"

Giving me a sense of status for the cron job.

Can you update MariaDB version to 10.5 ?

when the db content doesn't change in between two backups, the lsn numbers will be the same. the rotation from current to archive doesn't work in this case.

This looks awesome and I think is what I need for my meriamdb inside a docker.
However, I'm not great with docker and I'm having dificulty understanding the settings.
i.e. how do you pass in the password for the user?
Could you provide some sample docker run ... commands?

Thanks!

An option to run the backup at container startup would be nice. Nothing fancy, just a simple boolean true/false option.

It is useful when doing debugging or when setting up a new backup, that it creates a backup right away, instead of waiting for the schedule to run.

See resticker image for inspiration:
https://github.com/djmaze/resticker/

Hi,

First of all thank you for your work.

I'm using your service with mysql 8.0.21 ; the service is installing xtrabackup 8.0.9 which is not compatible with mysql > 8.0.19 (cf https://www.percona.com/blog/2020/04/28/percona-xtrabackup-8-x-and-mysql-8-0-20/).

As a workaround, I'm able to connect to the machine with "docker exec -it bash" in order to update manually the repo, then the xtrabackup install to the version 8.0.14. Of course, each time I'm rebuilding the image, it will pull the 8.0.9 version, forcing me to use my workaround again.

Would it be possible to correct this ? I'm guessing it's juste the "apt update" that is missing from the dockerfile.

In advance, thanks for your attention.

(PS : I'm absolutely not used to Github, if my issue has something missing, feel free to point it to me)

The sidecar can produce a lot of backups which grow fast. Especially, when uploading backups somewhere (e.g. by using a script or woolfg/mysql-backup-sidecar-gs) , it should be possible to define a pattern when old backups get deleted. Maybe just by specifiying a max age in days which would then delete all full backups older than the defined value.

Hi,

Thank you again for your work, it is working great.

It is not realy an issue but more asking for same advice.

As the backup is running at night, I would like to get a status of it.
To be more generic, I was thinking of running a script after the bakup.sh or rotate.sh are executed. We could use two variable SCRIPT_AFTER_BACKUP = "myscript.sh" and SCRIPT_AFTER_ROTATE="myscript2.sh" when launching the container.

I am not realy sure how to obtain a valid status of the backup, how does mariabackup behave in case of errors?

Thank you for your help.
Guillaume

When multiple incremental backups are to be taken, backup script fails because a folder with YYYYMMDD already exists.

Folder name should also contain hour and minute information.

2023-07-10 12:35:01 2023-07-10T09:35:01.609231-00:00 0 [Note] [MY-011825] [Xtrabackup] Connecting to MySQL server host: db, user: root, password: set, port: 3306, socket: not set
2023-07-10 12:35:01 2023-07-10T09:35:01.625462-00:00 0 [Note] [MY-011825] [Xtrabackup] Redo Log Archiving is not set up.
2023-07-10 12:35:01 xtrabackup: Can't create/write to file '/backup/current/20230710/xtrabackup_logfile' (OS errno 17 - File exists)
2023-07-10 12:35:01 2023-07-10T09:35:01.730288-00:00 0 [ERROR] [MY-011825] [Xtrabackup] failed to open the target stream for 'xtrabackup_logfile'

If the backup fails, backup.sh returns an error code of 0. Instead, it should return an error code.

In case of failed backups, the following script rotate.sh should probably not remove any old backups, since it cannot detect corrupted ones. It should rather wait until the backup works again.

Great job indeed.

Yet I feel like I need some more information while considering applying mysql-backup-sidecar on my production mariadb pod like the followings:

• it reads 'non-blocking' but as far as I concern, it's executing mariabackup --backup then almost the very first query to be executed would be FLUSH TABLE WITH READ LOCK so it will definitely block all DML queries while the backup is being done.
• would it be safe to use on InnoDB(storage engine), MyISAM tables while those are being consistently changed.

If I misunderstood, would be very appreciated if you could point it out. Thanks in advance.

As per the above, although cron is triggering correctly from the cron.d folder, and appears to be correctly executing ... any output is not delivered to the /backup folder within the container.

The current build pipeline running on docker hub is no longer active as it has become a premium feature which is no longer for free. Therefore, we should build images using github actions and push it to docker hub.

Restoring an incremental backup requires multiple commands, so it would really be helpful to have a script to automate (and test) restoring.

To that end, the dependancy qpress should also be included.

Then it should also be straight-forward to run MySQL/MariaDB server to actually test the restored data. Because a backup is only worth it if it actually works...

mysql-backup-sidecar-backup-1 | 2023-08-08T22:37:01.442335-00:00 0 [Warning] [MY-011825] [Xtrabackup] --compress using quicklz is deprecated and the ability to take backups using this compression algorithm will be removed in a future release. Please use ZSTD or LZ4 instead.