rsnapshot / rsnapshot

a tool for backing up your data using rsync (if you want to get help, use https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss)

RSNAPSHOT

rsnapshot comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions. See the GNU General Public Licence for details.

rsnapshot is a filesystem snapshot utility based on rsync. rsnapshot makes it easy to make periodic snapshots of local machines, and remote machines over ssh. The code makes extensive use of hard links whenever possible, to greatly reduce the disk space required.

It is written entirely in perl with no module dependencies, and has been tested with versions 5.004 through 5.24.3. It should work on any reasonably modern UNIX compatible OS. It has been tested successfully on the following operating systems:

• Debian: 3.0 (woody), 9.9 (stretch)
• Redhat: 7.x, 8.0
• RedHat Enterprise Linux: 3.0 ES, 5, 6, 7
• Fedora Core: 1, 3
• Fedora: 17, 18
• CentOS: 3, 4, 5, 6, 7
• WhiteBox Enterprise Linux 3.0
• Slackware 9.0
• SuSE: 9.0
• Gentoo Linux
• FreeBSD 4.9-STABLE
• OpenBSD 3.x
• Solaris 8 (SPARC and x86)
• Mac OS X
• IRIX 6.5

If this is your first experience with rsnapshot, you may want to read the man page which will give you a detailed walk-through on how to get rsnapshot up and running and also serve as a reference of all available commands.

If you are upgrading from version 1.1.6 or earlier, make sure you read the file Upgrading from 1.1.

For installation or upgrade instructions please read the INSTALL doc.

If you want to work on improving rsnapshot please read the CONTRIBUTING doc.

If you want to ask a question or have a general discussion use the Mailing List.

COMPATIBILITY NOTICES (Please read)

1. Note that systems which use GNU cp version 5.9 or later will have problems with rsnapshot versions up to and including 1.2.3, if cmd_cp is enabled (and points at the later gnu cp). This is no longer a problem since rsnapshot 1.2.9, as it strips off trailing slashes when running cp.

2. If you have rsync version 2.5.7 or later, you may want to enable the link_dest parameter in the rsnapshot.conf file.

   If you are running Linux but do not have the problem above, you should enable the cmd_cp parameter in rsnapshot.conf (especially if you do not have link_dest enabled).

   Be advised that currently link_dest doesn't do well with unavailable hosts. Specifically, if a remote host is unavailable using link_dest, there will be no latest backup of that machine, and a full re-sync will be required when it becomes available. Using the other methods, the last good snapshot will be preserved, preventing the need for a re-sync. We hope to streamline this in the future.

CONFIGURATION

Once you have installed rsnapshot, you will need to configure it. The default configuration file is /etc/rsnapshot.conf, although the exact path may be different depending on how the program was installed. If this file does not exist, copy /etc/rsnapshot.conf.default over to /etc/rsnapshot.conf and edit it to suit your tastes. See the man page for the full list of configuration options.

When /etc/rsnapshot.conf contains your chosen settings, do a quick sanity check to make sure everything is ready to go:

$ rsnapshot configtest

If this works, you can see essentially what will happen when you run it for real by executing the following command (where interval is alpha, beta, etc):

$ rsnapshot -t [interval]

Once you are happy with everything, the final step is to setup a cron job to automate your backups. Here is a quick example which makes backups every four hours, and beta backups for a week:

0 */4 * * *     /usr/local/bin/rsnapshot alpha
50 23 * * *     /usr/local/bin/rsnapshot beta

In the previous example, there will be six alpha snapshots taken each day (at 0,4,8,12,16, and 20 hours). There will also be beta snapshots taken every night at 11:50PM. The number of snapshots that are saved depends on the "interval" settings in /etc/rsnapshot.conf.

For example:

retain	alpha		6

This means that every time rsnapshot alpha is run, it will make a new snapshot, rotate the old ones, and retain the most recent six (alpha.0 - alpha.5).

If you prefer instead to have three levels of backups (which we'll call beta, gamma and delta), you might set up cron like this:

00 00 * * *     /usr/local/bin/rsnapshot beta
00 23 * * 6     /usr/local/bin/rsnapshot gamma
00 22 1 * *     /usr/local/bin/rsnapshot delta

This specifies a beta rsnapshot at midnight, a gamma snapshot on Saturdays at 11:00pm and a delta rsnapshot at 10pm on the first day of each month.

Note that the backups are done from the highest interval first (in this case delta) and go down to the lowest interval. If you are not having cron invoke the alpha snapshot interval, then you must also ensure that alpha is not listed as one of your intervals in rsnapshot.conf (for example, comment out alpha, so that beta becomes the lowest interval).

Remember that it is only the lowest interval which actually does the rsync to back up the relevant source directories, the higher intervals just rotate snapshots around. Unless you have enabled sync_first in your configuration-file, in which case only the sync pseudo-interval does the actual rsync, and all real intervals just rotate snapshots.

Also remember that a higher interval will rotate the oldest snapshot of the interval one step lower as its newest snapshot. For instance if you set daily and weekly snapshots and set daily to keep 30 snapshots, the newest snapshot for weekly will be the oldest (30 day old) daily snapshot after a rotation.

For the full documentation, type man rsnapshot once it is installed. The HOWTO also has a detailed overview of how to install and configure rsnapshot, and things like how to set it up so users can restore their own files.

If you plan on using the backup_script parameter in your backup scheme, take a look at the utils/-directory in the source distribution for several example scripts. The utils/rsnapreport.pl script is well worth a look.

AUTHORS

Please see the AUTHORS file for the complete list of contributors.

rsnapshot's Issues

ERROR: /usr/bin/rsync returned 0.09375

cron was mailing me these crazy error messages, turned out the code does "return ($retval / 256);
" for some reason. Is there any reason for obfuscating rsync return values?

rsnapshot fails on BTRFS

For modern file systems, I would be useful to have a custom mkdir command for non-temporary directories, eg.

cmd_mkdir=/sbin/btrfs subvolume create

how to use newest snapshot instead of oldest in rotation?

Hello,

When using tool to make backup, we generally want something like:
-1 backup per day (let's say 7 for 1 week)
-1 backup per month (first or last day of month, choose date)

As far as I understand, rsnapshot will rotate the oldest per day backup to make the first per month backup. So I won't have a monthly backup that match what was present on 1st day of the month (or whatever day I started rsnapshot with monthly argument).

I could use 2 specific config files so they ignore each other but I won't have benefit of hardlink between the two (and so basically host twice full data).

Anyway to achieve that?

I know it's question and not directly issue but couldn't find a support mailing list from here.

Thanks in advance,

Mathieu Chateau

Postexec should be run after lazy deletes

If you use preexec & postexec scripts to make the snapshot directory available (mount, toggle between read-only and read/write, etc.) and have lazy deletes turned on, deletes will always fail because the postexec runs before the deletes. When using lazy deletes, the postexec should not run until after the deletes are finished.

snapshot_root needs to be defined before backup points

rsnapshot 1.3.1 on CentOS 6.6

In my /etc/rsnapshot.conf:

snapshot_root /mnt/nfs/nas/encodia-server/

/mnt/nfs/nas/encodia-server/ is a NFS share on a NAS; it's mounted and writeable (I can rsync to it).

If i run sudo rsnapshot configtest I get:

----------------------------------------------------------------------------
rsnapshot encountered an error! The program was invoked with these options:
/usr/bin/rsnapshot configtest 
----------------------------------------------------------------------------
ERROR: /etc/rsnapshot.conf on line 29:
ERROR: snapshot_root /mnt/nfs/nas/encodia-server/ 
ERROR: /etc/rsnapshot.conf on line 198:
ERROR: backup /etc/ localhost/ - snapshot_root needs to be defined before \
         backup points 
ERROR: /etc/rsnapshot.conf on line 200:
ERROR: backup /usr/local/ localhost/ - snapshot_root needs to be defined \
         before backup points 
ERROR: ---------------------------------------------------------------------
ERROR: Errors were found in /etc/rsnapshot.conf,
ERROR: rsnapshot can not continue. If you think an entry looks right, make
ERROR: sure you don't have spaces where only tabs should be.

Thank you.

Copy & Paste?

In the region around https://github.com/rsnapshot/rsnapshot/blob/master/rsnapshot-program.pl#L762 I’m like

duuuude, Y U NO refactor?

rsnapshot bailing out on rsync error == 0, should continue, instead...

When for some reason one backup fails (typically because that server is down...), the whole rsnapshot process dies, and all following snapshots are not executed...

I would suggest to change, in sub handle_rsync_error:

if (0 == $retval) { bail('retval == 0 in handle_rsync_error() ('.$$bp_ref{'src'}.')'); }

with

if (0 == $retval) {
    print_err ("$config_vars{'cmd_rsync'} error 0 on $$bp_ref{'src'} (check host is up and accepting ssh connections...)", 2);
    syslog_err("$config_vars{'cmd_rsync'} error 0 on $$bp_ref{'src'} (check host is up and accepting ssh connections...)");
  }

preexec fails at least for rsnapshot daily.

I wanted to use
cmd_preexec /bin/mount -o remount,rw /mnt/Backup/
cmd_postexec /bin/mount -o remount,ro /mnt/Backup/
to keep my backup disk in a safe read only state most of the time and only switch it to rw during backup.
After few months I recognized that I had only fresh hourly backups, but no recent daily, weekly and monthly backups. So I found out, that rsnapshot daily fails trying to write the backup disk ("rm ...") as the preexec command had not been issued, while with rsnapshot hourly everything went well.

More ideas for tests

• file local vs absolute/link/directory traversal checks for:
  • -c?
  • include_conf (with args?)
  • cmd_rsync (no args)
  • cmd_ssh (no args)
  • cmd_cp (no args)
  • cmd_rm (no args)
  • cmd_logger (with args?)
  • cmd_du (no args)
  • cmd_rsnapshot_diff (no args)
  • cmd_preexec (with args?)
  • cmd_postexec (with args?)
  • linux_lvm_cmd_lvcreate (no args, backup level too)
  • linux_lvm_cmd_lvremove (no args, backup level too)
  • linux_lvm_cmd_mount (no args, backup level too)
  • linux_lvm_cmd_umount (no args, backup level too)
  • logfile
  • include_file (backup level too?)
  • exclude_file (backup level too?)
  • backup
  • backup_script
• Mixing retain and interval
• Mixing log methods
• Check command line options:
  • -t: verify that no changes are made for a series of commands
  • -c: probably covered by all of the other tests
  • -x: specify a custom "rsync" and check for -x with it
  • -v, -q, -V, -D: unsure
  • Check multiple retains
• Test lockfiles hanging around (on every test?)

Rsnapshot clean up (rm -rf) killing drives

Hey,
I am already using a c3 nice level for rsync which made the backup task unnoticeable. But unfortunately as soon as the max backups are archived and the olds are being removed it is killing the hard drives.
Is there any way to set an io nice for the remove task or other ways to optimize?

File system is already ext4

(the rm causes the system to slow down extremely for 5-10 minutes)

lockfile is not being removed when calling `rsnapshot sync` with `use_lazy_deletes = 1`

how to reproduce:

• enable both use_lazy_deletes and sync_first
• call rsnapshot sync

what happens:

• the lockfile (e.g. /var/run/rsnapshot.pid) doesn't get removed.
• if rsnapshot sync is called again subsequently, that command has the warning WARNING: Removing stale lockfile. The exit status is then 2 (warning), even if the actual command succeeded.

what should happen:

• the lockfile should have been removed.
• subsequent commands should not be influenced

This bug has been introduced by commit 8147e70.

More in detail:

There are two possibilities where the remove_lockfile() subroutine is called:

• if use_lazy_deletes is off: on line 291
• if use_lazy_deletes is active: in the handle_interval() subroutine, line 2953.

However, in the latter case remove_lockfile() is called only in some cases, that is, only when the _delete.$$ directory exists. That directory does not exist rsnapshot sync is called.

I'm going to create a pull request.

rsnapreport does not work with backup_script

Using backup_script does not output anything useful when piped through rsnapreport.

My understanding is that rsnapreport relies on the statistics generated by rsync, however backup_script uses a custom "sync_if_different" method instead of using rsync. Therefore, there are no statistics and no report.

with sync_first, sync should not roll back if failed

see the old entry, I think this is still needed:
https://sourceforge.net/p/rsnapshot/patches/9/

do rotation only, if maximum previous level exists [patch]

I created a feature request a whila ago on
http://sourceforge.net/p/rsnapshot/feature-requests/30/

Since sourceforge seems to be dead or at least taken over by criminals [1], I repeat it here and add the patch file.
Hi,
here I have a simple patch that prevents rotation in a level, if there is nothing to roll into it from the lower level (e.g. no daily.{max} -> do not rotate weekly to prevent disappearing weekly.0).
Until now rotation is always done, and leads to holes in the backup scheme (e.g. weekly.[013] exist,weekly.[2] missing due to forced rotation).
This can then later lead to holes in the next levels (e.g. monthly.0 missing, if weekly.3 missing during monthly rotation), too.

I regard this as bug and the attached patch is quite simple.
Somebody should check and decide,
a) if there should be an option for the old behaviour (the 'dumb'/forced rotate).
b) if there are any problems with just bailing out then (I dont think so, but there are some options checked in the following parts).

Thanks for looking into it,
Martin

[1] http://www.gluster.org/2013/08/how-far-the-once-mighty-sourceforge-has-fallen/

use POSIX::lchown

It's been in core since 5.8.6 (? certainly 5.10). On older perls, fall back to Lchown if installed.

rsnapshot-diff doesn't cope with symlink loops?

Reported by LuKreme to mailing list in [email protected] on 6 Mar 2013

rsnapshot diff hourly.0 hourly.3

Comparing hourly.3 to hourly.0
Use of uninitialized value in numeric eq (==) at /opt/local/bin/rsnapshot-diff line 221.
Can't compare across devices.
(looking at
hourly.3/www/usr/local/www/munged/wordpress/wp-content/cache/supercache/munged/category/no-link/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=/%3Ciframe%20width=)

exclude_file does not work

I am on CentOS 6.5 and am using rsnapshot 1.3.1-12 from the EPEL repo. I have made an exclude file that lists directories and files to be included and excluded (+ -) for rsync to use in the config I have. Whenever I perform a backup however, it still backups everything and isnt using the exclude file I made. Rather annoying really...

Command I use to perform the backup
/usr/bin/rsnapshot -c /etc/rsnapshot/necc-data.backup.conf weekly

rsnapshot config file
http://pastebin.com/qbi0i2m1

rsnapshot exclude file
http://pastebin.com/rn2JQYR0
(Pastebin makes it seem like part of the directory path goes on a new line, but checking with gedit and nano on my machine has it on one line so its fine)

Whats the problem here? I guess a temporary fix is to use the command backup command and list each and every directory manually...although I still need to exclude one or two things

Destructive rotations, redux.

This may be an extension to issue # 33 [https://github.com//issues/33], but I want to simplify the description.

With sync_first enabled, running rsnapshot for any retain interval will rotate and delete the backups even if there's no newer backup to take their place. This eventually leads to every backup being removed.

For example, if you have this in rsnapshot.conf:
retain weekly 3
And then run "rsnapshot weekly" three times all your weekly backups will be gone.

rsnapshot should not rename any backup to X+1 unless there's an X-1 (or .sync) being promoted. When X+1 hits the retain value the backup is deleted whether or not any more recent backups exist.

Scheduling the rsnapshot cron jobs as described right on the man page leaves the daily, weekly, and monthly rotations vulnerable to this because they're not dependent on a successful sync job.

release 1.4

release 1.4 sourcecode todo-list

• update buildsystem/makefiles and update gitignore
  • the PR is at #86
• test travis and travis with github releases again (has to be done in seperate repo)
  • done. but can't give 100% guarantee, that deployment will work with the latest token
• make a good basis for unittests
  • the PR is at #88
• tidy the code with perltidy, to make use of one proper code-formatting-style
  • the PR for tidiing is there: #85
• write changelog
• look at the distro-tracker to merge in all other common patches
• create release-candidates?
  • nope
• update authors field and current maintainer status

release 1.4 todo-list outside of sourcecode-repository

• create a new website
  • @DrHyde should correct my english 😉
  • the URL should be made correct.
• port the mailinglist to somewhere else and give sourceforge the last shot in the head
  • delayed. Not a goal for 1.4
• create announcement

remove of LV snapshot not happening if mount error

I was looking in existing bug reports / issues if this problem was already mentioned but couldn't find anything. Please close this issue and point me to existing the bug report if there is one.

It appears that rsnapshot does not remove a logical volume snapshot if the snapshot could not be mounted. See the following log:

[10/Sep/2014:02:07:43] /sbin/lvcreate --snapshot --size 200M --name rsnapshot /dev/vgdata/mylv
[10/Sep/2014:02:07:44] /bin/mount /dev/vgdata/rsnapshot /mnt/lvm-snapshot
[10/Sep/2014:02:07:44] /usr/bin/rsnapshot -c /etc/rsnapshot.backup.conf daily: ERROR: Mount LVM snapshot failed: 8192
[10/Sep/2014:02:07:44] rm -f /var/run/rsnapshot.pid

Reason for the mount error was, that the mountpoint (/mnt/lvm-snapshot) didn't exist.
In the log you see, that the snapshot was not removed.

On the next run of rsnapshot, the creation of the LV snapshot failed, because it already existed (from the previous run):

[10/Sep/2014:09:30:08] /sbin/lvcreate --snapshot --size 200M --name rsnapshot /dev/vgdata/mvlv
[10/Sep/2014:09:30:08] /usr/bin/rsnapshot -c /etc/rsnapshot.backup.conf daily: ERROR: Create LVM snapshot failed: 1280
[10/Sep/2014:09:30:08] rm -f /var/run/rsnapshot.pid

The created LVM snapshot should have been removed in the first run when the mount error appeared.

Or verify that the defined mountpoint for LVM snapshots does exist when rsnapshot starts up. If the mount point is not found, throw error and exit rsnapshot without doing anything.

Development Model?

Hey all,

I'm a little bit confused about how things are moving forward in terms development. The CONTRIBUTING.md seems out of sync with the reality.

Last I checked there was talk of taking master back to 1.3.1 and creating a 1.3.2 release. I guess that is not happening now, it would have been a lot of work. So what is the plan? Could we branch a dev from master now, and cut a 1.3.2 release pretty much now, or in a coupe of commits? Maybe create a milestone for 1.3.2.

Please accept whitespaces as separators in rsnapshot.conf

See also https://bugs.debian.org/261677

If config-file is a folder, config_version is not found!

bebe@butter > ls
bin  files  rsnapshot  rsnapshot.conf  rsnapshot.log  testcase
bebe@butter > rsnapshot -c rsnapshot daily
Config file "rsnapshot" does not exist or is not readable.
bebe@butter > ./rsnapshot/bin/rsnapshot -c rsnapshot daily
----------------------------------------------------------------------------
rsnapshot encountered an error! The program was invoked with these options:
./rsnapshot/bin/rsnapshot -c rsnapshot daily
----------------------------------------------------------------------------
ERROR: config_version was not defined. rsnapshot can not continue.
bebe@butter >

rsnapshot is (1.3.1-5 Archlinux, roughly 1.3.1), ./rsnapshot/bin/rsnapshot is built from the current master.

Probably the commit 051d5b5 introduced it. I haven't tested this yet.

official home of project?

is this repo the official home of the project now? I see quite some recent activity here but the website at http://www.rsnapshot.org seems broken (according to archive.org since early Feb. 2015) and the sourceforge page does not mention GitHub while hosting releases and still active mailing lists. Would be great if someone could confirm the state of the project. Maybe I just came here in a moment of transition?

HOWTO is missing?

The HOWTO document that the readme refers to is not present on the main web site. I can see its source in the documentation directory though, but it's not readable easily.

exclude not working as expected

my config looks like this

include_conf    config/GLOBAL.rsnap

snapshot_root   /home/BACKUPS/rsnap/demo/

exclude "/e/"

# nailworks
#backup nailor@localhost:/home/nailor/code/datenplan/rsnapshot/demo/    demo/
backup  /home/nailor/code/datenplan/rsnapshot/demo/ demo/

the demo folder looks like this

nailor@sting:*datenplan/rsnapshot$ tree demo                
demo
├── a
├── b
├── c
│   ├── c1
│   └── c2
├── d
│   └── d1
└── e
    ├── e1
    └── ee
        └── ee1

4 directories, 7 files

I call rsnapshot sync (not the first run, so not all files are transferred) and this is the output

echo 31274 > /home/nailor/code/datenplan/rsnapshot/rsnap.lock 
/usr/bin/rsync -avPx --numeric-ids --partial --delete --delete-excluded \
    --stats --exclude="/e/" /home/nailor/code/datenplan/rsnapshot/demo/ \
    /home/BACKUPS/rsnap/demo/.sync/demo/ 
sending incremental file list
e/
e/e1
              0 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=2/12)
e/ee/
e/ee/ee1
              0 100%    0.00kB/s    0:00:00 (xfr#2, to-chk=0/12)

Number of files: 12 (reg: 7, dir: 5)
Number of created files: 4 (reg: 2, dir: 2)
Number of deleted files: 0
Number of regular files transferred: 2
Total file size: 0 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 364
Total bytes received: 71

sent 364 bytes  received 71 bytes  870.00 bytes/sec
total size is 0  speedup is 0.00
touch /home/BACKUPS/rsnap/demo/.sync/ 
rm -f /home/nailor/code/datenplan/rsnapshot/rsnap.lock

WHY IS e TRANSFERRED!?!

If I run the exact rsync command listed in the log, e is not transferred / it is deleted when rsnapshot created it before:

nailor@sting:*datenplan/rsnapshot$ /usr/bin/rsync -avPx --numeric-ids --partial --delete --delete-excluded \
>     --stats --exclude="/e/" /home/nailor/code/datenplan/rsnapshot/demo/ \
>     /home/BACKUPS/rsnap/demo/.sync/demo/
sending incremental file list
deleting e/ee/ee1
deleting e/ee/
deleting e/e1
deleting e/

Number of files: 8 (reg: 5, dir: 3)
Number of created files: 0
Number of deleted files: 4 (reg: 2, dir: 2)
Number of regular files transferred: 0
Total file size: 0 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 187
Total bytes received: 56

sent 187 bytes  received 56 bytes  486.00 bytes/sec
total size is 0  speedup is 0.00

I am running Ubuntu 14.04 and the packaged rsnaphot is rsnapshot 1.3.1.

Any ideas?

Thanks,
Michael

pod2man from perl-5.18 doesn't like rsnapshot-1.3.1

The error is
pod2man -c '' -n 'rsnapshot' -r '' rsnapshot > rsnapshot.1
rsnapshot around line 6723: You forgot a '=back' before '=head1'
POD document had syntax errors at /usr/pkg/bin/pod2man line 71.

Subtle bug in rsnapshot invocation in tests

This:

ok(! `/usr/bin/perl ./rsnapshot -c t/support/etc/gnu_cp.conf hourly`);

explicitly uses /usr/bin/perl when rsnapshot might be configured to use /usr/local/bin/perl or some other random executable. By the time 'make test' runs the tests, ./rsnapshot should have had its #! line frobbed and be executable so the /usr/bin/perl (or rather, the @Perl@ in the source) can be cut out.

Running multiple instances at once

I have the following problem: I am using systemd timers to automate the backups as described here. Since my machine is not running 24/7 I use the Persistent option: If some timers were not run due to the machine being switched off, then the corresponding units are immediately run after startup.

The problem is that this leads to the {hourly,daily,monthly} instances being run at the same time. This leads to some units failing because the lockfile already exists. I am wondering whether there is a way around this problem. Maybe rsnapshot could wait for the process related to the lockfile to terminate?

Customize temp location for backup_script

In exec_backup_script(), rsnapshot creates a temporary directory for the backup script to dump its data into. This is hard-coded to be rsnapshot_root/tmp:
https://github.com/rsnapshot/rsnapshot/blob/e2f14aa/rsnapshot-program.pl#L4261

It would be great if this location was configurable in case rsnapshot_root lives on a slow disc or remote computer. I guess I could send a PR if you're interested. What would be a suitable name? Something like backup_script_root (defaulting to rsnapshot_root if omitted)?

Root is created before preexec

This is not necessarily a bug but very unexpected or unfortunate for some NAS related use cases. A papercut if you want.

The situation:

The snapshot_root should be on a NAS. But the nas is powered off all the time, started via wake-on-lane and mounted with a preexec script.

• rsnapshot version 1.3.1 from Debian Wheezy.

Here the relevant parts of the config in the right order. But it does not matter if snapshot_root is before cmd_preexec or not.

config_version  1.2

snapshot_root   /mnt/nas-03/rsnapshot-backup

cmd_preexec /root/wakup-and-mount-nas03.sh
cmd_postexec    /root/umount-nas03.sh

When i test the config with option -t i get the following sequence of commands:

root@atlas:/# rsnapshot -c /etc/rsnapshot-auf-nas03.conf -t daily
echo 27062 > /var/run/rsnapshot-nas03.pid 
mkdir -m 0700 -p /mnt/nas-03/rsnapshot_backup/ 
/root/make-nas03-ready.sh
mkdir -m 0755 -p /mnt/nas-03/rsnapshot_backup/daily.0/ 
/usr/bin/rsync -a --delete --numeric-ids --relative --delete-excluded /home \
    /mnt/nas-03/rsnapshot_backup/daily.0/atlas/ 
    […] all the folders
touch /mnt/nas-03/rsnapshot_backup/daily.0/ 
/root/umount-nas03.sh

What happens is that snapshot_root is created in /mnt/mountpoint/snapshot_root then preexec runs and mounts the storage above the snapshot_root. Now the folder is not accessible and gets created on the NAS by mkdir … -p mnt/mountpoint/snapshot_root/daily.0/ anyway. This leaves an orphaned folder in /mnt/mointpoint/ called snapshot_root which is never used and only visible if the storage is unmounted.

IMHO preexec should be executed before rsnapshot writes any file related to storing the backup.

Request: do not perform an initial backup, hardlink instead.

The normal operation this tool is to create a backup (copy) of a directory and then use hardlinks as the backup is rotated. I would like to request the ability to use hardlinks initially (so that no additional disk space is used). This is useful in the case that the files we want to "backup" are already backups.

I had the situation where I was already backing up files from a remote server to a backup server. I then wanted to have snapshots created on the backup server without further duplicating the data.

convert README/INSTALL/etc. files to markdown

convert files to markdown. The goal is to have later the fancy github-pages feature.

warnings that lead to a nonzero exitcode aren't logged even with loglevel==3 (Verbose)

Well, yeah. My loglevel is set to 3, and I have this in the logfile (this was an initial/full backup, hence the long running time):

[21/Aug/2014:05:25:02] /usr/bin/rsnapshot -c /etc/rsnapshot.tack.conf sync: started
[21/Aug/2014:05:25:02] echo 5295 > /var/run/rsnapshot.tack.pid
[21/Aug/2014:05:25:02] mkdir -m 0755 -p /mnt/backup1/tack/.sync/
[21/Aug/2014:05:25:02] /usr/bin/rsync -a --delete --numeric-ids --relative --delete-excluded --exclude=/dev/** --exclude=/mnt/** --exclude=/media/** --exclude=/tickhome/** --exclude=/shares/** --exclude=/var/chroots/ub1204/sys/** --exclude=/var/chroots/ub1204/proc/** --exclude=/dos/** --exclude=/proc/** --exclude=/run/** --exclude=/swapfile* --exclude=/sys/** --exclude=/tmp/** --exclude=mnt/backup1/tack /. /mnt/backup1/tack/.sync/tack/
[21/Aug/2014:17:52:43] touch /mnt/backup1/tack/.sync/
[21/Aug/2014:17:52:43] rm -f /var/run/rsnapshot.tack.pid
[21/Aug/2014:17:52:43] WARNING: /usr/bin/rsnapshot -c /etc/rsnapshot.tack.conf sync: completed, but with some warnings

So, there was a warning that was serious enough to cause a nonzero exit code (which in my case led to the subsequent rotation not being performed), and that fact was logged, but the warning itself was not, with loglevel==3. That can't be right, right? The documentation states "Verbose level 2: Print errors and warnings only" -- mine was 3, and it didn't print that warning. So this seems to be a documentation bug at least.

Add environment-variables containing cmd_* configuration-options to shell-executions.

Reported by admwiggin at sourceforge on 2011-04-02 07:21 UTC
It would be extremely useful if configuration values (especially "cmd_ssh", "cmd_rsync", and "ssh_args") could be passed forward to a command run using backup_script using environment variables such as $RSNAPSHOT_cmd_ssh or something. Would make writing scripts that do remote connections much simpler, since I wouldn't have to duplicate parts of my configuration, and could more easily use the same scripts in multiple rsnapshot configuration files without worrying about them trying to use the wrong SSH identify file, for example.

👍 This probably requires writing a new subroutine launching shell-commands properly in a standardized way.

Re: [sourceforge-issues] [Feature] Please add the possibility to run rsnapshot in push mode

Reported by eddyp on 2013-03-28 21:08 UTC
If the machine holding the backups must be given access to all backed-up machines, then any compromise of the backup machine will compromise all the clients.

A more secure approach is to push backups from the clients to the backup machine. On the backup machine SSH could be configured to keep the users in isolation with ChrootDirectory and ForceCommand internal-sftp.

This way, the client can decide to encrypt the data before the push, rendering useless any compromise of the backup machine.

Also, if any of the clients are compromised, the ssh key of that machine can simply be removed from the backup machine and any possible access will cease.

umount and lvremove on SIGTERM etc.

It's nice that rsnapshot catches signals like SIGTERM, but it doesn't call umount and lvremove. If rsnapshot would do that, the next backup (after an aborted backup) could run without any manual cleanup. This would allow someone to shutdown the machine without taking care about eventually running backups.

I'm using sync_first, and IMHO with that option enabled, nothing would be broken if the LVM logical volume is removed, as the backup will be synced again at the next run.

False directory conflict

Getting false error message about conflicting directories when configuring similar dir names for backup and backup_script simultaneously.
In config:

backup          /opt/ets/FileRepository/                                lotcenter/etp_files/
backup_script   /root/rsnapshot_scripts/postgres_backup.sh ets_etp      lotcenter/etp/

Running rsnapshot i get this:

# rsnapshot daily
----------------------------------------------------------------------------
rsnapshot encountered an error! The program was invoked with these options:
/usr/bin/rsnapshot daily
----------------------------------------------------------------------------
ERROR: destination conflict between "lotcenter/etp_files/" and "lotcenter/etp/" in backup / backup_script entries

Changing destination dir name from etp_files to files_etp in config solves the problem. There are no problems if configuring two backup or two backup_script entries.

Feature request: multithread cleaning

Hello,

when starting job, it may prune old snapshot like through rm-rf :
[25/Jul/2015:19:15:15] /usr/bin/rm -rf /mypath/snapshots/hourly.1/

It's taking 3 hours on my system (glusterFS behind with tons of small files).
Would it be possible to multithread it, like 1 thread per root folder below hourly.1 ?

Destructive rotations

Hello,
rsnapshot backup rotations does not take into account, that backup may fail. There should be check how old is the oldest daily backup and rotate the weekly only if the oldest daily is at least week old. When a lot of backups fail, for example when trying to backup a missing laptop, rotations will eventualy delete all old backups, which is definitely wrong (at when cron is used to rotate as recommended in man page).

I use following script to invoke rsnapshot:

echo "$host: /"

if ping -c 20 -i 0.2 -W 10 -n "$host" >/dev/null
then
        # do backup
        rsnapshot -c "$conf" sync \
        && rsnapshot -c "$conf" daily

        # rotate only older than 6.5 day
        last_daily="`date -r daily.6 +%s || echo 0`"
        first_weekly="`date -r weekly.0 +%s || echo 0`"
        #echo "last_daily - first_weekly = $last_daily - $first_weekly = "$(($last_daily - $first_weekly))
        [ $(( $last_daily - $first_weekly )) -gt 561600 ] \
        && rsnapshot -c "$conf" weekly

        # update symlinks
        rm [0-9][0-9][0-9][0-9]-* 2>/dev/null
        find -mindepth 1 -maxdepth 1 -type d \
        | while read d
        do
                ln -s "$d" "`date -r "$d" +'%Y-%m-%d %H:%M:%S'`"
        done

else
        echo "     ... offline. (No ping response.)"
fi

I use only daily and weekly backups. When backup fails, daily rotation is not performed. When oldest daily backup is older than 6.5 day (to avoid rounding errors when one day is backup faster than other), weekly rotation is performed.

Goal is to have weekly backups at least 6.5 days from each other.

When there is more daily backups than expected, extra daily backups are rotated away. And next weekly backup is created when the rotated-away backup is old enough.

When daily backups are missing, weekly backup is created earlier than after 7 daily rotations. So if laptop is connected in LAN only once a week, daily backups will be in week interval and each daily backup will become a weekly backup. This will remove too old backups.

Final part of the script creates symlinks with date of backup. Very simple and very practical. But a little bit off topic here.

What do you think about real-time based rotations?

"ssh_args" does not work due to lack of quoting

If you set ssh_args to anything it gets added to the rsync command directly-

--rsh=/usr/bin/ssh -p 222

As a result those extra arguments look like arguments for rsync itself. Enclosing them in quotes would make them actually work-

--rsh="/usr/bin/ssh -p 222"

The relevant code is on line 3735.

rsnapshot is rolling back the wrong destination, if a source host cannot be reached

Hi,

first of all, rsnapshot is doing a great job for my backup needs, thanks.

I fiddled a bit with the current version 1.4.1 in order to expand its impact, but stumbled upon something strange: one of my systems ("pal") was off in this session. Consequently, connection failed, but rsnapshot was rolling back a different destination ("oskar"), which was backed up before in the same session. That operation resulted in retcode 23, that shouldn't do any rollback at all, if I read the source correctly.

Here's the log:

/usr/bin/rsync -vax --delete --numeric-ids --relative --delete-excluded \
    --acls --xattrs --exclude=/home/*/.cache/ --exclude=/home/*/.thumbnails/ \
    --exclude=/home/*/tmp/ --exclude=/tmp/ --exclude=/var/tmp/ \
    --rsh=/usr/bin/ssh --link-dest=/backup/rsync.daytime/daily.1/oskar/ \
    root@oskar:/ /backup/rsync.daytime/daily.0/oskar/ 
/usr/bin/rsync -vax --delete --numeric-ids --relative --delete-excluded \
    --acls --xattrs --exclude=/home/*/.cache/ --exclude=/home/*/.thumbnails/ \
    --exclude=/home/*/tmp/ --exclude=/tmp/ --exclude=/var/tmp/ \
    --rsh=/usr/bin/ssh --link-dest=/backup/rsync.daytime/daily.1/oskar/ \
    root@oskar:/home/ /backup/rsync.daytime/daily.0/oskar/ 
----------------------------------------------------------------------------
rsnapshot encountered an error! The program was invoked with these options:
/usr/bin/rsnapshot -c /etc/rsnapshot.conf.daytime daily 
----------------------------------------------------------------------------
ERROR: /usr/bin/rsync returned 0.08984375 while processing root@oskar:/home/
/usr/bin/rsync -vax --delete --numeric-ids --relative --delete-excluded \
    --acls --xattrs --exclude=/home/*/.cache/ --exclude=/home/*/.thumbnails/ \
    --exclude=/home/*/tmp/ --exclude=/tmp/ --exclude=/var/tmp/ \
    --rsh=/usr/bin/ssh --link-dest=/backup/rsync.daytime/daily.1/kuno/ \
    root@kuno:/ /backup/rsync.daytime/daily.0/kuno/ 
/usr/bin/rsync -vax --delete --numeric-ids --relative --delete-excluded \
    --acls --xattrs --exclude=/home/*/.cache/ --exclude=/home/*/.thumbnails/ \
    --exclude=/home/*/tmp/ --exclude=/tmp/ --exclude=/var/tmp/ \
    --rsh=/usr/bin/ssh root@ziggy:/ /backup/rsync.daytime/daily.0/ziggy/ 
/usr/bin/rsync -vax --delete --numeric-ids --relative --delete-excluded \
    --acls --xattrs --exclude=/home/*/.cache/ --exclude=/home/*/.thumbnails/ \
    --exclude=/home/*/tmp/ --exclude=/tmp/ --exclude=/var/tmp/ \
    --rsh=/usr/bin/ssh root@pal:/ /backup/rsync.daytime/daily.0/pal/ 
ERROR: /usr/bin/rsync returned 0.99609375 while processing root@pal:/
WARNING: Rolling back "oskar/"
/usr/bin/rm -rf /backup/rsync.daytime/daily.0/oskar/ 
/usr/bin/cp -al /backup/rsync.daytime/daily.1/oskar \
    /backup/rsync.daytime/daily.0/oskar 
touch /backup/rsync.daytime/daily.0/ 
rm -f /var/run/rsnapshot-daytime.pid

Further reading the source points to some of Beneedikt's latest changes, that I'm going to comment in the related commit (e2f14aa).

Consider Hostname for rotation

If I backup from two hosts to the same snapshot_root on an NFS share, the two instances interfere each other. As example: I installed a new host and want to run rsnapshot for the first time. My expected behavior was, that rsnapshot would create another subfolder named like the hostname under, for example, the folder daily.0 and not to work directly on the daily.0 folder. So at the moment the new host would delete the daily.6 folder and start the rotation, this is not the behavior I would expect. It would be great if the rotation task would be aware of the different host names and only rotate the correct folders.

Suppress Uid message

Running rsnapshot gives me lots or rsync errors like:
uid 4294967295 (-1) is impossible to set on "/backup/rsnapshot/salsa/daily.0/c_partition/0213807470eac823c6779e"

when backing up windows hosts. Is there a proper way of suppressing these messages?

-v flag is setting verbosity lower than it is defined in conf-file

If I define the verbosity in the rsnapshot.conf with verbose 5 and invoke rsnapshot with the -v flag, the verbosity is set down to 3. IMO in this case the -v flag is confusing, as I want to have more chatty output, but in reality it limits it.

The behaviour to achieve should be, to set verbose only if the level is higher than it is right now.

rsync's exit value is lost

Seems that rsnapshot looses rsync's exit value, e.g. if the remote host is unreachable. Sample debug output (verbose 5):

:
/usr/bin/rsync -SHavx --delete --numeric-ids --relative --delete-excluded \
--exclude=Cache --exclude=tmp --rsh=/usr/bin/ssh \
--link-dest=/export/rsnapshot/alpha.1/daffy/ root@daffy:/ \
/export/rsnapshot/alpha.0/daffy/
ssh: connect to host daffy port 22: No route to host
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
rsync succeeded
:

The old rsnapshot 1.3.1 used to do

:
/usr/bin/rsync -SHavx --delete --numeric-ids --relative --delete-excluded \
--exclude=Cache --exclude=tmp --rsh=/usr/bin/ssh \
--link-dest=/export/rsnapshot/hourly.1/daffy/ root@daffy:/ \
/export/rsnapshot/hourly.0/daffy/
ssh: connect to host daffy port 22: No route to host
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
----------------------------------------------------------------------------
rsnapshot encountered an error! The program was invoked with these options:
/usr/bin/rsnapshot hourly 
----------------------------------------------------------------------------
ERROR: /usr/bin/rsync returned 255 while processing root@daffy:/
:

This breaks rollback.

Cannot use non-anonymous rsync protocol with username

When use this backup config, rsnapshot assumes I want to use ssh:

backup rsync://oernii@android:/rootfs/ android/

workaround: return 0 in is_ssh_path when the path starts with "rsync://"
sub is_ssh_path {
...
if ($path =~ m/^rsync:///) { return (0); }

ignores all changes on ntfs to ext4

When using rsnapshot to back up a directory on a ext4 partition and a directory on a ntfs partition to an external hard drive (also ext4), all changes in the ntfs directory are just ignored.

Additional per-backup excludes aren't properly merged

How to reproduce:

1. Specify additional per-backup exclude file (4th column in backup statement with '+' sign):

   backup  xxx@YYY:/etc/        ./t3    +exclude_file=/ZZZ/etc.exclude
   

   where

   # cat etc.exclude
   2.tmp
   

   And there is also global exclude file:

   # cat test.excl
   1.tmp
   

   specified earlier as

   exclude_file    /ZZZ/test.excl
   

2. Run configtest:

   # rsnapshot configtest
   Use of uninitialized value $parsed_opts{"exclude_file"} in concatenation (.) or string at /usr/local/bin/rsnapshot line 1809, <GEN0> line 228.
   require Lchown
   Lchown module loaded successfully
   Syntax OK
   

3. Run backup:

   # rsnapshot daily
   Use of uninitialized value $parsed_opts{"exclude_file"} in concatenation (.) or string at /usr/local/bin/rsnapshot line 1809, <GEN0> line 228.
   require Lchown
   Lchown module loaded successfully
   Setting locale to POSIX "C"
   echo 34130 > /var/run/rsnapshot.pid
   mkdir -m 0755 -p /ZZZ/daily.0/./
   /usr/local/bin/rsync -av --delete --numeric-ids --relative \
       --delete-excluded --exclude-from= --rsh=/usr/bin/ssh \
       xxx@YYY:/etc /ZZZ/daily.0/./t3
   receiving incremental file list
   created directory /ZZZ/daily.0/./t3
   etc/
   etc/1.tmp
   etc/2.tmp
   [..]
   

As you can see, '--exclude-from' option of rsync has no argument and both
files (excluded globally and per-backup) are included. When global exclude is
not used all will be similar ('--exclude-from' will not have argument).

Upd. rsnapshot 1.3.1

Ignore git working directory

Did you consider adding an option for not saving the content of a directory when it's a git repo, and only save .git/ ? .git already contains the history of the project. It may be difficult to implement using rsync, though.