nafets227 / backup Goto Github PK

Include also size, directory

using root is a security issue by definition.
when using rsync it may be needed to copy access rights and files of other users
when accessing the backup target, it may be unwanted, so keep the target files under a "backup" user.

Needs to be thoroughly investigated what's the best tradeoff between security and usability

When scheduling the second backup of the day, sometimes it fails with the message "found a newer backup yyyy/mm/dd.in-progress

This happens when the first backup started but did not complete, thus left the .in-progress directory.

Expected behavior is to reuse the .in-progress directory, update it and upon completion rename to yyyy/mm/dd without .in-progress suffix.

... using the new feature introduced in clone 1.59.
See https://rclone.org/onedrive/#onedrive-access-scopes

according to https://forum.rclone.org/t/rclone-hard-links-and-updated-files/2618 rclone does not behave well with hard links; we use hard links when doing history backups.

Our tests also showed that it overwrote previous versions, compromising the whole backup!

Possible workaround is to use --backup-dir parameter of rclone, forcing it to first move files to the backup-dir and then create a new file to write the downloaded content into. backup-directory is not really needed, so it can be delete after the backup.

to be analyzed if putting in the same repository or create a new one just containing the helm chart.

Send an Email on success (or not) of backup.

Especially when doing cloud backups, like copying OneDrive locally, all files already downloaded are valuable even if the complete backup did not succeed.

Idea is to have an option (e.g. --keep-in-progress) that avoids to delete the .in-progress directory before starting history backup.
It has to be considered if reusing also aborted backups (.in-progress directories) of previous days. The scenario here is a massive upload of data to OneDrive, where download may be too big to succeed in one day. Restarting next day from scratch will lead to an infinite loop with no backup.

maybe --stat-* options of clone can resolve this requirement