p-bizouard / cloudbackup Goto Github PK

• Add a field in BackupConfiguration to set a regex to ignore errors dumped on stderr
• If the field is valued and the error output is empty after ignoring lines, the command should be considered as success

It will allow to backup object storages with files name > 255 chars on a filesystem that does not support this, and consider the backup as a success even if few files were not backuped

Command "rebuild-index" is deprecated, Use "repair index" instead

All velero restic volume backup from a same namespace are saved in one restic repository. So if we have multiple pvc, restic snapshots --latest may return more than one result (at leat one by pvc).

We want to filter the result based on the volume name, ex: restic snapshots --latest 1 --tag volume=php-uploads-pvc --json

Add custom restic argument in the BackupConfiguration ?
Add jq or php filter ?

Sometime we create a backup configuration with empty database, repository, ... so we set to 0 octet, but when we go live we do not remember to set a real value. And the data will growth overtime, so if a backup is not full for some reason, we may not be alerted if the minimum expected size is not updated.

• Checkbox to ignore the error on the BackupConfiguration
• Visual error on dashboard and backup
• How much lighter should throw an error ? Should it be configurable on the BackupConfiguration ?

The forget command rclone delete --rmdirs --min-age "${KEEP_DAILY}d" "${REMOTE_STORAGE_BACKUP}" --config "${RCLONE_CONFIG}" throws errors when ${REMOTE_STORAGE_BACKUP} is non-existent.

Error executing backup - rclone delete - 2023/06/01 14:00:08 ERROR : : error listing: directory not found
2023/06/01 14:00:08 ERROR : Attempt 1/3 failed with 2 errors and: directory not found
2023/06/01 14:00:08 ERROR : : error listing: directory not found
2023/06/01 14:00:08 ERROR : Attempt 2/3 failed with 2 errors and: directory not found
2023/06/01 14:00:08 ERROR : : error listing: directory not found
2023/06/01 14:00:08 ERROR : Attempt 3/3 failed with 2 errors and: directory not found
2023/06/01 14:00:08 Failed to delete with 2 errors: last error was: directory not found

Check before executing the command with rclone lsd and expect to find a backup directory in stdout ?

https://rclone.org/commands/rclone_size/

We should check the remote integrity with https://rclone.org/commands/rclone_check/ or
https://rclone.org/commands/rclone_cryptcheck/
This should be a select choice between never/daily/weekly in the BackupConfiguration.

When a backup of the same volume is done, it saves the host path. So if the pod start on 2 different nodes, it will create two different snapshots in the history, so --latest 1 may return more than 1 result.

We should sort the resulted array based on the timestamp, with jq or in php

restic snapshots --latest 1 --json | jq 'sort_by(.time) | reverse'

2023/08/31 22:27:42 Failed to delete: remove /data/xxx/rclone/xxx/backup/2023-09-01/documents: directory not empty

We should remove files with rclone delete without --rmdirs arguments, then use rclone rmdirs --leave-root to clean empty directories left.

Just like restic snapshots, restic stats should filter on tags

restic snapshots --json --last -q - Flag --last has been deprecated, use --latest 1