digint / btrbk Goto Github PK
View Code? Open in Web Editor NEWTool for creating snapshots and remote backups of btrfs subvolumes
Home Page: https://digint.ch/btrbk/
License: GNU General Public License v3.0
Tool for creating snapshots and remote backups of btrfs subvolumes
Home Page: https://digint.ch/btrbk/
License: GNU General Public License v3.0
Add support for btrfs-progs v3.14.1 (stable on Ubuntu 14.10 / Debian wheezy backport).
btrfs-progs 3.14 do not support "btrfs subvolume list -R" flag, which makes it impossible to check send/receive relationship between source and backup subvolume.
Have an option flag that pipes the transfer through the pv utility. pv monitors data going through a pipe, alowing things like counting the bytes transfered.
Should be something like
btrfs send $SRC | pv | btrfs receive $DEST
Note that pv has no way of knowing the total amount of data that needs to be tranfered since send/receive doesn't know, it just counts the bytes. (If I am mistakened and there is a way to determine the side of a snapshot then there is a flag to add that.)
(originally by @diraimondo)
I'm wondering if it would be possible to add an option (command-line and/or config) to let btrbk attempt to transfer all the already-created snapshots on source that are missing on target. This skipping any attempt to create further snaps.
This would fit in the following scenario:
Thank you for your time.
I would keep in a unique configuration file all the remote/local subvolume managed by the backup server. Such volumes could be managed using different strategies (once per day, once per week, only sync because snapshots are made remotely using recent introduced features, ...). It would nice if I could restrict on command-line the selected action on a specified subvolume (not all). As long as I understand, it is not possible at the moment: all the subvolumes are involved.
Hello, I've add some changes to btrbk, to be able to choose ssh cipher for better performance while send/recieve subvolume snapshots via ssh. I have added two ciphers to choose of aes128-ctr and arcfour256. Here are changes:
btrbk.conf:
ssh_cipher aes128-ctr
btrbk:
79a80
> ssh_cipher => { default => undef, accept => [ "arcfour256", "aes128-ctr" ] },
238a240
> my $ssh_cipher = config_key($config, "ssh_cipher");
241a244
> push(@ssh_options, '-c', $ssh_cipher) if ($ssh_cipher);
Speed improvements (without compression):
Cheers!
I'm running BTRBK as a timer on a systemd-enabled system. I had my back up drive disconnected from my system for a couple of days, and I recently connected it today. Upon running btrbk, I took a look at the executed commands and found that it will attempt to send the first missed snapshot under the current day's name like so:
btrfs send -p /mnt/subvolroot/server/srv/gameservers/tekkit/backups/world.20150323 /mnt/subvolroot/server/srv/gameservers/tekkit/backups/world.20150326
Currently running BTRBK 0.14
I think it would be a great feature to be able to set custom backup names instead of it always being date created. It could be done with a command line option like -n or in the config file or both.
Hi,
first of all: Thanks a lot for this great tool! I use it regularly to transfer snapshots to my usb-drive.
While doing so, I tried the following scenario:
To do so, I have these settings in my config file:
snapshot_create_always yes
resume_missing yes
Mount point of my usb-drive is /mnt/btrfs_bkp. But if I want to run the script without having the disk attached, it will fail, throwing the following error (since the path /mnt/btrfs_bkp exists, but does not contain a btrfs-filesystem):
WARNING: No real path provided by "btrfs subvolume show" for subvolume "/mnt/btrfs_bkp", using: /mnt/btrfs_bkp
ERROR: process died unexpectedly (btrbk v0.17.0)
Please contact the author: *********
Stack Trace:
----------------------------------------
Use of uninitialized value in hash element at /usr/sbin/btrbk line 879, <FILE> line 95.
at /usr/sbin/btrbk line 103, <FILE> line 95.
main::__ANON__("Use of uninitialized value in hash element at /usr/sbin/btrbk"...) called at /usr/sbin/btrbk line 879
main::btr_tree(HASH(0x2b9c100)) called at /usr/sbin/btrbk line 909
main::vinfo_subvol_list(HASH(0x2b9c100)) called at /usr/sbin/btrbk line 262
main::vinfo_root(HASH(0x2b9c100)) called at /usr/sbin/btrbk line 1574
Is there any way to avoid this? --> Creating the snapshots, but only transfering if the destination path constains a btrfs-filesystem?
I am trying to make snapshots of entire volume. Can the config file be configured for use without subvolumes? Commenting out the subvolume line leads to errors.
Thanks in advance
Hi,
It seems that even though I specify not to keep any snapshots on the source, btrbk
is keeping snapshots and not deleting them.
I wonder if this is an configuration issue on my side or an bug.
Directory Listing
drwxr-xr-x 1 root root 12 Mar 30 2013 home/
drwxr-xr-x 1 root root 12 Mar 30 2013 home.20150820/
drwxr-xr-x 1 root root 12 Mar 30 2013 home.20150829/
drwxr-xr-x 1 root root 12 Mar 30 2013 home.20150831/
drwxr-xr-x 1 root root 12 Mar 30 2013 home.20150902/
drwxr-xr-x 1 root root 12 Mar 30 2013 home.20150903/
drwxr-xr-x 1 root root 12 Mar 30 2013 home.20150904/
drwxr-xr-x 1 root root 186 Jul 25 16:44 root/
drwxr-xr-x 1 root root 186 Jul 25 16:44 root.20150820/
drwxr-xr-x 1 root root 186 Jul 25 16:44 root.20150829/
drwxr-xr-x 1 root root 186 Jul 25 16:44 root.20150831/
drwxr-xr-x 1 root root 186 Jul 25 16:44 root.20150902/
drwxr-xr-x 1 root root 186 Jul 25 16:44 root.20150903/
drwxr-xr-x 1 root root 186 Jul 25 16:44 root.20150904/
drwxrwxr-x 1 root vmware 138 Jul 19 15:14 vm/
drwxrwxr-x 1 root vmware 138 Jul 19 15:14 vm.20150820/
drwxrwxr-x 1 root vmware 138 Jul 19 15:14 vm.20150829/
drwxrwxr-x 1 root vmware 138 Jul 19 15:14 vm.20150831/
drwxrwxr-x 1 root vmware 138 Jul 19 15:14 vm.20150902/
drwxrwxr-x 1 root vmware 138 Jul 19 15:14 vm.20150903/
drwxrwxr-x 1 root vmware 138 Jul 19 15:14 vm.20150904/
Configuration
% cat /etc/btrbk/btrbk.conf
resume_missing no
snapshot_preserve_daily 1
snapshot_preserve_weekly 0
snapshot_preserve_monthly 0
target_preserve_daily 1
target_preserve_weekly 1
target_preserve_monthly 0
volume /var/lib/btrfs/r0-ssd
subvolume home
target send-receive /var/lib/btrfs/r1-sata/backups
subvolume root
target send-receive /var/lib/btrfs/r1-sata/backups
subvolume vm
target send-receive /var/lib/btrfs/r1-sata/backups
target_preserve_weekly 0
target_preserve_monthly 0
Dry-Run
Using: btrbk -v -l debug -c /etc/btrbk/btrbk.conf dryrun
btrbk command line client, version 0.20.0-dev (Sat Sep 5 18:03:54 2015)
Using configuration: /etc/btrbk/btrbk.conf
### 'btrfs' 'subvolume' 'show' '/var/lib/btrfs/r0-ssd' 2>&1
Command execution successful
found btrfs root: /var/lib/btrfs/r0-ssd
### 'btrfs' 'subvolume' 'list' '-a' '-c' '-u' '-q' '-R' '/var/lib/btrfs/r0-ssd'
Command execution successful
Parsed 21 total subvolumes for filesystem at: /var/lib/btrfs/r0-ssd
Found 21 subvolume children of: /var/lib/btrfs/r0-ssd
### 'btrfs' 'subvolume' 'show' '/var/lib/btrfs/r1-sata/backups' 2>&1
Command execution successful
Parsed 12 subvolume detail items: /var/lib/btrfs/r1-sata/backups
### 'btrfs' 'subvolume' 'list' '-a' '-c' '-u' '-q' '-R' '/var/lib/btrfs/r1-sata/backups'
Command execution successful
Parsed 12 total subvolumes for filesystem at: /var/lib/btrfs/r1-sata/backups
Found 9 subvolume children of: /var/lib/btrfs/r1-sata/backups
### 'btrfs' 'subvolume' 'show' '/var/lib/btrfs/r1-sata/backups' 2>&1
Command execution successful
Parsed 12 subvolume detail items: /var/lib/btrfs/r1-sata/backups
Found 9 subvolume children of: /var/lib/btrfs/r1-sata/backups
### 'btrfs' 'subvolume' 'show' '/var/lib/btrfs/r1-sata/backups' 2>&1
Command execution successful
Parsed 12 subvolume detail items: /var/lib/btrfs/r1-sata/backups
Found 9 subvolume children of: /var/lib/btrfs/r1-sata/backups
Snapshot creation enabled (snapshot_create=always)
Creating subvolume snapshot for: /var/lib/btrfs/r0-ssd/home
[btrfs] snapshot (ro):
[btrfs] source: /var/lib/btrfs/r0-ssd/home
[btrfs] target: /var/lib/btrfs/r0-ssd//home.20150905
>>> /var/lib/btrfs/r0-ssd//home.20150905
### (dryrun) 'btrfs' 'subvolume' 'snapshot' '-r' '/var/lib/btrfs/r0-ssd/home' '/var/lib/btrfs/r0-ssd//home.20150905'
<SNIP>
Creating subvolume backup (send-receive) for: /var/lib/btrfs/r0-ssd/home
Found 6 snapshot children of: /var/lib/btrfs/r0-ssd/home
Found 1 receive targets in "/var/lib/btrfs/r1-sata/backups/" for: /var/lib/btrfs/r0-ssd/home.20150904
Latest common snapshots for: /var/lib/btrfs/r0-ssd/home: src=/var/lib/btrfs/r0-ssd/home.20150904 target=/var/lib/btrfs/r1-sata/backups/home.20150904
Receiving from snapshot: /var/lib/btrfs/r0-ssd//home.20150905
Incremental from parent snapshot: /var/lib/btrfs/r0-ssd/home.20150904
>>> /var/lib/btrfs/r1-sata/backups/home.20150905
[btrfs] send/receive (incremental):
[btrfs] source: /var/lib/btrfs/r0-ssd//home.20150905
[btrfs] parent: /var/lib/btrfs/r0-ssd/home.20150904
[btrfs] target: /var/lib/btrfs/r1-sata/backups
### (dryrun) 'btrfs' 'send' '-p' '/var/lib/btrfs/r0-ssd/home.20150904' '-v' '/var/lib/btrfs/r0-ssd//home.20150905' | 'btrfs' 'receive' '-v' '/var/lib/btrfs/r1-sata/backups/'
<SNIP>
Cleaning backups of subvolume "/var/lib/btrfs/r0-ssd/home": /var/lib/btrfs/r1-sata/backups/home.*
Filter scheme: preserving all within 1 days
Filter scheme: preserving first in week (starting on sunday), for 1 weeks
Filter scheme: preserving last weekly of month, for 0 months
=== /var/lib/btrfs/r1-sata/backups/home.20150829: preserved weekly: 1 weeks ago, +6 days after sunday
=== /var/lib/btrfs/r1-sata/backups/home.20150831: preserved weekly: 0 weeks ago, +1 days after sunday
<<< /var/lib/btrfs/r1-sata/backups/home.20150903
=== /var/lib/btrfs/r1-sata/backups/home.20150904: preserved daily: 1 days ago
[btrfs] delete:
[btrfs] subvolume: /var/lib/btrfs/r1-sata/backups/home.20150903
### (dryrun) 'btrfs' 'subvolume' 'delete' '/var/lib/btrfs/r1-sata/backups/home.20150903'
Deleted 1 subvolumes in: /var/lib/btrfs/r1-sata/backups/home.*
Cleaning snapshots: /var/lib/btrfs/r0-ssd//home.*
Filter scheme: preserving all within 1 days
Filter scheme: preserving first in week (starting on sunday), for 0 weeks
Filter scheme: preserving last weekly of month, for 0 months
Deleted 0 subvolumes in: /var/lib/btrfs/r0-ssd//home.*
<SNIP>
Completed within: 0s (Sat Sep 5 18:03:54 2015)
--------------------------------------------------------------------------------
Backup Summary (btrbk command line client, version 0.20.0-dev)
Date: Sat Sep 5 18:03:54 2015
Config: /etc/btrbk/btrbk.conf
Legend:
=== up-to-date subvolume (source snapshot)
+++ created subvolume (source snapshot)
--- deleted subvolume
*** received subvolume (non-incremental)
>>> received subvolume (incremental)
--------------------------------------------------------------------------------
/var/lib/btrfs/r0-ssd/home
+++ /var/lib/btrfs/r0-ssd//home.20150905
>>> /var/lib/btrfs/r1-sata/backups/home.20150905
--- /var/lib/btrfs/r1-sata/backups/home.20150903
/var/lib/btrfs/r0-ssd/root
+++ /var/lib/btrfs/r0-ssd//root.20150905
>>> /var/lib/btrfs/r1-sata/backups/root.20150905
--- /var/lib/btrfs/r1-sata/backups/root.20150903
/var/lib/btrfs/r0-ssd/vm
+++ /var/lib/btrfs/r0-ssd//vm.20150905
*** /var/lib/btrfs/r1-sata/backups/vm.20150905
--- /var/lib/btrfs/r1-sata/backups/vm.20150810
Maybe you can shed some light on this?
Thanks!
from: http://www.digint.ch/btrbk/doc/readme.html
A link exists to: http://www.digint.ch/btrbk/doc/doc/FAQ.md, which is broken
I hope you don't mind a question rather than a bug here (you might want to consider adding it to the FAQ).
I have been using btrbk for a few months now -- thank you for your efforts. I have just upgraded my debian stretch system and my cron job fails because the "tree" command has been removed.
Can you suggest any way to get similar information with the new commands? I can't find a "list" option which shows a similar approach. "list snapshots" is the closest, but I found the tree formatting to be really useful to see which of my backups was generating the most snapshot activity.
Actually, I don't really need the full "tree" output. I would really like to see, for each source_subvol, how many snapshot_subvol's there are for it, and the name of the most recent one (whether it is up to date or not would be useful as well).
Any suggestion on getting that info would be great.
Linux package maintainers seem to have no consent on where the btrfs
binary should reside in. Here is what I found for some distros (thanks Henrik for pointing me to this):
/sbin/btrfs
/bin/btrfs
/sbin/btrfs
/usr/bin/btrfs
This is cumbersome, as for security reasons, btrbk sets $PATH=""
and always calls /sbin/btrbk
, which now fails on some distros with recent btrfs-progs.
Usually it is the task of the package maintainer to fix things like this (i.e. by changing hardcoded paths before installing), but in the debian case, this is not possible since they use different locations for different versions of btrfs-tools.
I see no other option than relaxing the security measures a bit by setting PATH=/sbin:/bin:/usr/sbin:/usr/bin
, and call btrfs <subcommand> ...
instead of /sbin/btrfs <subcommand> ...
from the btrbk executable (as well as in ssh_filter_btrbk.sh
). From a security perspective, this should still be ok since an attacker cannot modify the files in either /sbin
, /bin
, /usr/sbin
and /usr/bin
(unless he has root privileges).
I'm running arch with btrfs-progs 4.1-1. btrbk -l debug -v dryrun
produces the following output: http://pastebin.com/nHfAqwvg
I downgraded to btrfs-progs v 3.17.3 and it worked like a charm, as it used to.
It looks as though the format changed from:
/backup/backup-root
Name: backup-root
uuid: 4c791925-6637-7f49-9d94-8c1ee9ff97d4
Parent uuid: -
Creation time: 2015-06-19 19:21:44
Object ID: 585
Generation (Gen): 1292
Gen at creation: 1087
Parent: 5
Top Level: 5
Flags: -
Snapshot(s):
to
/backup/backup-root
Name: backup-root
UUID: 4c791925-6637-7f49-9d94-8c1ee9ff97d4
Parent UUID: -
Received UUID: -
Creation time: 2015-06-19 19:21:44 -0500
Subvolume ID: 585
Generation: 1292
Gen at creation: 1087
Parent ID: 5
Top level ID: 5
Flags: -
Snapshot(s):
I left the backup volume disconnected from my laptop for about a week or two, and went to reconnect it today to transfer over the missing snapshots. The most recent backup on the drive is from March 25th, and the laptop has snapshots from April 5th to today. I was expecting btrbk to continue incremental backups using the March backup as the parent source, but instead it started a full backup from April.
Creating subvolume snapshot for: /mnt/subvolroot/rtfs
>>> /mnt/subvolroot/.snapshots/rtfs.20150415
Checking for missing backups of subvolume "/mnt/subvolroot/rtfs" in: /mnt/backups/Laptop/btrbk/
Resuming subvolume backup (send-receive) for: /mnt/subvolroot/.snapshots/rtfs.20150405
Receiving from snapshot: /mnt/subvolroot/.snapshots/rtfs.20150405
No common parent subvolume present, creating full backup
>>> /mnt/backups/Laptop/btrbk/rtfs.20150405
At subvol /mnt/subvolroot/.snapshots/rtfs.20150405
At subvol rtfs.20150405
The laptop snapshot directory
root [/root]$ ls /mnt/subvolroot/.snapshots
boot.20150405 boot.20150410 boot.20150415 etc.20150409 etc.20150414 home.20150410 home.20150415 rtfs.20150410 rtfs.20150415
boot.20150406 boot.20150411 etc.20150405 etc.20150410 etc.20150415 home.20150411 rtfs.20150405 rtfs.20150411
boot.20150407 boot.20150412 etc.20150406 etc.20150411 home.20150405 home.20150412 rtfs.20150406 rtfs.20150412
boot.20150408 boot.20150413 etc.20150407 etc.20150412 home.20150406 home.20150413 rtfs.20150407 rtfs.20150413
boot.20150409 boot.20150414 etc.20150408 etc.20150413 home.20150407 home.20150414 rtfs.20150409 rtfs.20150414
the backup volume directory
root [/root]$ ls /mnt/backups/Laptop/btrbk
boot.20150323 boot.20150325 etc.20150324 home.20150323 home.20150325 rtfs.20150324 rtfs.20150405
boot.20150324 etc.20150323 etc.20150325 home.20150324 rtfs.20150323 rtfs.20150325
I understand that the backup volume snapshots are relatively old, but shouldn't an incremental send still be possible with the March snapshots as a parent? There is little difference between the March backups and the recent ones (the differences only being a couple of documents that I worked on and some downloads, no updates or significant changes were made).
Running version 0.19.3, attempting to declare subvolumes like "VirtualBox VMs", "BitTorrent Sync", etc. in the config returns "Ambiguous file for option "subvolume""
I tried using double and single quotes, as well as "" to escape the space, and \040, of which none appeared to work.
I'd like to seed the remote backup target with snapshot(s) delivered on a USB drive. What I'm thinking is to run 'btrbk' with a temporary target of the USB mount point. I can then move the USB drive to the target server and 'btrfs send | btrfs receive' into the backup volume. It's not clear to me if this will cause the parent relationships to be preserved and incremental backups to happen from then on. Does this work? Is there another way to accomplish the same thing if not?
When I tried a dry run of this by send/receiving the 'root' snapshot to the USB drive and then running btrbk resume targeting it, it seemed to be copying the entire thing again.
File root subvolume on my file system is labelled @
following the ubuntu naming convention (https://help.ubuntu.com/community/btrfs).
When I perform a dryrun using this label I get the error:
btrbk command line client, version 0.12 (Thu Mar 19 09:16:40 2015)
ERROR: Ambiguous file for option "subvolume" in "btrbk-#####.conf" line 83: @
ERROR: Failed to parse configuration file
This also happens for my subvolume labelled @home
.
btrbk command line client, version 0.12 (Thu Mar 19 09:17:40 2015)
ERROR: Ambiguous file for option "subvolume" in "btrbk-#####.conf" line 88: @home
ERROR: Failed to parse configuration file
Hey,
I like the ability to quickly run btrbk whenever I feel like it, e.g. before an update, experimental work and so forth. In particular, I may run it multiple times a day. At the same time, I don't preserve snapshots (i.e. all snapshot_preserve_* options are set to "0", all target_preserve_* to "all").
The problem is that in this case, the first run will generate the snapshot
SomeSub.YYYYMMDD
The second run will create a new snapshot and delete the old one, so we have in the source directory (as per the documentation)
SomeSub.YYYYMMDD_1
Now if I run btrbk a third time, it fails because the next snapshot it creates has once again the name SomeSub.YYYYMMDD and a subvolume with this name already exists at the target location.
There are several ways this could be fixed, e.g. looking for the highest _N there is in the source folder (taking into account missing folders) and use _(N+1), or using random suffixes. However, my suggestion would be (and it would be much more informative also!) to use not only the date but also the time as the timestamp suffix. Maybe I'm overlooking something, but this would even make it more elegant since you don't have to look up the largest existing _N. Okay, two gotchas could be multiple snapshots within a second (or whatever precision is chosen for the timestamp), and possibly snapshots taken during daylight saving time changes (in particular, duplicate names may arise, and timestamps may not be ordered chronologically, in case this matters). But anyway, just as a thought, maybe this could be made an option.
I've been using snapper to maintain snapshots on my laptop's to and recover accidentally deleted files. I found btrbk while looking for something that would provide the same onto external hardware, but it seems to me that it fits that requirement AND can replace snapper, since it maintains in-volume snapshots anyway. Is that so or am I missing something?
I think this may be related to issue #15. In an ubuntu system, I have '@' subvolume for / and @home for /home (from fstab):
UUID=d34c6201-ad16-4e25-81d7-e4d4b6eadcb2 / btrfs defaults,compress=lzo,subvol=@ 0 1
UUID=d34c6201-ad16-4e25-81d7-e4d4b6eadcb2 /home btrfs defaults,compress=lzo,subvol=@home 0 2
btrbk config is:
snapshot_preserve_daily 14
snapshot_preserve_weekly 0
snapshot_preserve_monthly 0
target_preserve_daily 20
target_preserve_weekly 10
target_preserve_monthly 12
snapshot_dir btrbk_snapshots
volume /
subvolume @
target send-receive /mnt/backups/andor
subvolume @home
target send-receive /mnt/backups/andor
However, when trying to dryrun, it complains about not finding the subvolumes:
WARNING: Skipping subvolume "//@": Failed to fetch subvolume detail: finding real path for '//@', No such file or directory
WARNING: Skipping subvolume "//@home": Failed to fetch subvolume detail: finding real path for '//@home', No such file or directory
--------------------------------------------------------------------------------
Backup Summary (btrbk command line client, version 0.19.2)
Date: Mon Jun 8 17:54:12 2015
Config: /etc/btrbk/btrbk.conf
Legend:
=== up-to-date subvolume (source snapshot)
+++ created subvolume (source snapshot)
--- deleted subvolume
*** received subvolume (non-incremental)
>>> received subvolume (incremental)
--------------------------------------------------------------------------------
//@
!!! Aborted: Failed to fetch subvolume detail: finding real path for '//@', No such file or directory
//@home
!!! Aborted: Failed to fetch subvolume detail: finding real path for '//@home', No such file or directory
NOTE: Some errors occurred, which may result in missing backups!
Please check warning and error messages above.
NOTE: Dryrun was active, none of the operations above were actually executed!
Any ideas?
Hi !
I've set up a backup initiated from my computer to my local NAS.
I would like now to make remote backups to have a real backup plan in case of stealing, fire, asteroid...
Since WAN connection is way too much slow compared to gigabit local network, I'd like the NAS to take care of doing the remote backups, since it's online 24/24 and it can do this every nights.
Myhost --> LAN NAS --> Remote NAS
Here is the configuration :
- MyHost
volume /btrfs/pool_hdd
snapshot_create always
resume_missing yes
target_preserve_daily 7
target_preserve_weekly 4
target_preserve_monthly 12
subvolume home/alex
snapshot_dir home_snapshots/
snapshot_name alex.home
target send-receive ssh://nas/btrfs/pool_1/backups/alex/alex-pc
- LAN NAS
All backups of alex-pc are located in /btrfs/pool_1/backups/alex/alex-pc
e.g. alex.home.YYYYMMDD, root.YYYMMDD ...
The backups from LAN NAS to remote NAS are last resort backups, so I want to keep only one snapshot on the remote NAS, no need to have history.
And from this point, I failed to see how I can again backup the backups... Since it has timestamps...
How could I get around this ?
--raw-output
and a little scripting.--- Solution 1 : snapshoting the latest snap
Directories would be like that :
/btrfs/pool_1/backups/alex/alex-pc
- alex.home.YYYYMMDD
- alex.home.latest
/btrfs/pool_1/backups/alex/alex-pc-sending-remotely
- alex.home.YYYYMMDD newly created snap from alex.home.latest
volume /btrfs/pool_1/backups/alex
snapshot_create always
resume_missing no
snapshot_preserve_daily 1
snapshot_preserve_weekly 0
snapshot_preserve_monthly 0
target_preserve_daily 1
target_preserve_weekly 0
target_preserve_monthly 0
subvolume alex-pc/alex.home.latest
snapshot_dir alex-pc-sending-remotely/
snapshot_name alex.home
target send-receive ssh://remote.nas/backups/alex/alex-pc
--- Solution 2 : directly send the latest snapshot
/btrfs/pool_1/backups/alex/alex-pc
- alex.home.YYYYMMDD
- alex.home.latest
volume /btrfs/pool_1/backups/alex
snapshot_create no
resume_missing no
target_preserve_daily 1
target_preserve_weekly 0
target_preserve_monthly 0
subvolume alex-pc/alex.home.latest
target send-receive ssh://remote.nas/backups/alex/alex-pc
I tend to prefer solution 1 because if I don't do another snapshot and the LAN NAS go down during a certain amount of time,
it could be possible that the common subvolume between LAN NAS and remote NAS get's deleted during the backup of MyHost (following target_preserve policy) ?
I'd really like your thoughts about all this, and if you see a better way to implement this.
It's complicated, don't hesitate to ask me again if I wasn't clear with my explanations and questions...
Thanks a lot !
It would be nice to have the ability to configure BTRBK to send all snapshots that were unable to be sent to a backup volume once the volume appears again. Currently, it'll only send the current day's snapshots, without attempting to re-send previously failed transfers.
I wanted to use btrbk to create managed local snapshots and not backup to a different btrfs pool. I thought that I should be able to do this by not defining a target.
So I just left the target out of the subvolume I wanted to snapshot. I received this error:
/mnt/btrfs/vm_images
!!! Subvolume "vm_images" aborted: No targets defined for subvolume: /mnt/btrfs/vm_images
Is there any way to do this? Or am I missing something?
It's also fine if this is not how it's intended to be used, but I would think it would be an easy enhancement to add.
I'm attempting to substitute my personal bash-script to remotely backup my laptop on a local server using btrfs send/receive feature. In my attempt the backup is started by the server. My problem is related to the cases where the transfer is interrupted for a network problem (or because if go away with the laptop). In this case, if 'resume_missing' is 'yes', btrbk attempts to resume previous snapshot but this fails. This looks a bug (it should delete previous backup).
$ sudo btrbk -l info run
btrbk command line client, version 0.17.0 (Sat May 9 11:13:13 2015)
Using configuration: /etc/btrbk/btrbk.conf
Creating subvolume snapshot for: {gandalf}/mnt/btrfs-pool/PACMAN
>>> {gandalf}/mnt/btrfs-pool/snapshots/PACMAN.20150509
Creating subvolume backup (send-receive) for: {gandalf}/mnt/btrfs-pool/PACMAN
Receiving from snapshot: {gandalf}/mnt/btrfs-pool/snapshots/PACMAN.20150509
No common parent subvolume present, creating full backup
>>> /mnt/btrfs-backup-pool/BTRBK/PACMAN.20150509
At subvol /mnt/btrfs-pool/snapshots/PACMAN.20150509
At subvol PACMAN.20150509
^C
$ sudo btrbk -l info run
btrbk command line client, version 0.17.0 (Sat May 9 11:14:03 2015)
Using configuration: /etc/btrbk/btrbk.conf
Creating subvolume snapshot for: {gandalf}/mnt/btrfs-pool/PACMAN
>>> {gandalf}/mnt/btrfs-pool/snapshots/PACMAN.20150509_1
Checking for missing backups of subvolume "{gandalf}/mnt/btrfs-pool/PACMAN" in: /mnt/btrfs-backup-pool/BTRBK/
Resuming subvolume backup (send-receive) for: {gandalf}/mnt/btrfs-pool/snapshots/PACMAN.20150509
Receiving from snapshot: {gandalf}/mnt/btrfs-pool/snapshots/PACMAN.20150509
No common parent subvolume present, creating full backup
>>> /mnt/btrfs-backup-pool/BTRBK/PACMAN.20150509
At subvol /mnt/btrfs-pool/snapshots/PACMAN.20150509
At subvol PACMAN.20150509
ERROR: creating subvolume PACMAN.20150509 failed. File exists
ERROR: Failed to send/receive btrfs subvolume: {gandalf}/mnt/btrfs-pool/snapshots/PACMAN.20150509 -> /mnt/btrfs-backup-pool/BTRBK
ERROR: Error while resuming backups, aborting
Resumed 1 backups
WARNING: Skipping cleanup of snapshots for subvolume "{gandalf}/mnt/btrfs-pool/PACMAN", as at least one target aborted earlier
Completed within: 2s (Sat May 9 11:14:05 2015)
--------------------------------------------------------------------------------
Backup Summary (btrbk command line client, version 0.17.0)
Date: Sat May 9 11:14:03 2015
Config: /etc/btrbk/btrbk.conf
Legend:
+++ created subvolume (source snapshot)
--- deleted subvolume
*** received subvolume (non-incremental)
>>> received subvolume (incremental)
--------------------------------------------------------------------------------
{gandalf}/mnt/btrfs-pool/PACMAN
+++ {gandalf}/mnt/btrfs-pool/snapshots/PACMAN.20150509_1
!!! /mnt/btrfs-backup-pool/BTRBK/PACMAN.20150509
!!! Target "/mnt/btrfs-backup-pool/BTRBK" aborted: Failed to send/receive subvolume
NOTE: Some errors occurred, which may result in missing backups!
Please check warning and error messages above.
I'm not really interested in resuming the transfer of all the previous backups: if I use 'resume_missing' as 'no', then btrbk doesn't attempt to resume the previous transfer. It shots another fresh snapshot on the laptop and it tries to transfer it on the sever. This fits well in my scenario but I would like that btrbk removes all the server-located incomplete snapshots: they are left on the server dirtying my server file-system. I can't even distinguish between completed and uncompleted backups.
Note: my bash-scripts manages this problem adding keywords like 'transfering' / 'synced' to the name of the snapshots.
Hi!
First of all, thank you for this wonderful program ! Very useful and handy !
I open this issue because I've some concerns about security when btrbk operates over ssh with the ssh_filter_btrbk script.
Let me explain you this:
For example:
btrfs subvolume delete
accepted for all subvolumes)btrfs sub snap
and btrfs send
)Maybe I'm paranoid and this kind of attack is highly unlikely but, I think It would be good to restrict the path in which the btrfs command can proceed and if out of this path: reject_and_die !
If you think this is relevant and needs more attention, I've made a modification of the ssh_filter_script to take into consideration the working path.
(as an example, my authorized_keys looks like this :
from="192.168.0.1",command="/usr/local/bin/ssh_filter_btrbk.sh --log --sudo --restrict-path /btrfs/pool_1/backups/pc1"
)
I'm no programmer, and even less a bash expert... so my modifications are probably bad but
tell me what you think of all this !
/bin/bash
, I don't know of to do this regular expression (with =~ operator) valid for all shellsbtrfs sub delete
../
in the path, we should be good ?use_sudo
because I'm not connecting to remote root user and btrfs command needs these permissions#!/bin/bash
set -e
set -u
export PATH=/sbin:/bin:/usr/sbin:/usr/bin
enable_log=
restrict_path=
use_sudo=
while [ "$#" -ge 1 ]
do
key="$1"
case $key in
-l|--log)
enable_log=1
;;
-p|--restrict-path)
restrict_path="$2"
shift # past argument
;;
-s|--sudo)
use_sudo="sudo"
;;
*)
;;
esac
shift
done
log_cmd()
{
if [ -n "$enable_log" ]; then
logger -p $1 -t ssh_filter_btrbk.sh "$2 (Name: ${LOGNAME:-<unknown>}; Remote: ${SSH_CLIENT:-<unknown>}; Restrict-path: $restrict_path; $use_sudo): $SSH_ORIGINAL_COMMAND"
fi
}
reject_and_die()
{
log_cmd "auth.err" "btrbk REJECT"
/bin/echo "ERROR: ssh command rejected" 1>&2
exit 1
}
run_cmd()
{
log_cmd "auth.info" "btrbk ACCEPT"
$use_sudo $SSH_ORIGINAL_COMMAND
}
check_path()
{
if [ -n "$restrict_path" ]; then
regex="^btrfs[-a-zA-Z0-9 ]* ([ ]*$restrict_path[^ ]*)*[ ]*$"
if [[ $SSH_ORIGINAL_COMMAND =~ $regex ]]; then
run_cmd
else
reject_and_die
fi
else
run_cmd
fi
}
case "$SSH_ORIGINAL_COMMAND" in
*\$*) reject_and_die ;;
*\&*) reject_and_die ;;
*\(*) reject_and_die ;;
*\{*) reject_and_die ;;
*\;*) reject_and_die ;;
*\<*) reject_and_die ;;
*\>*) reject_and_die ;;
*\`*) reject_and_die ;;
*\|*) reject_and_die ;;
*"../"*) reject_and_die;; #prevent directory traversal
btrfs\ subvolume\ show\ *) check_path ;; #run_cmd ;; # mandatory
btrfs\ subvolume\ list\ *) check_path ;; #run_cmd ;; # mandatory
btrfs\ subvolume\ snapshot\ *) check_path ;; #run_cmd ;; # mandatory if this host is backup source
btrfs\ send\ *) check_path ;; #run_cmd ;; # mandatory if this host is backup source
btrfs\ receive\ *) check_path ;; #run_cmd ;; # mandatory if this host is backup target
btrfs\ subvolume\ delete\ *) check_path ;; #run_cmd ;; # mandatory if scheduling is active
btrfs\ subvolume\ find-new\ *) check_path ;; #run_cmd ;; # needed for "btrbk diff"
btrfs\ filesystem\ usage\ *) check_path ;; #run_cmd ;; # needed for "btrbk info"
*) reject_and_die ;;
esac
The following use-case was not considered (thanks Bernd):
volume /
subvolume @home
target send-receive /mnt/btrfs_backup
This will be fixed in refactored btrbk, to be released soon.
Hello,
I have enabled quota support and btrfs creates a qgroup with id 0/{subvolid} whenever a subvolume is created. But this qgroup is not deleted when the subvolume is deleted. Could you add an option to delete the correct qgroup when the subvolume is deleted?
While btrbk will already create hourly snapshots if it is run every hour via cron, there is no way to manage the expiration of them separately from the daily ones. For example, a configuration which keeps hourly snapshots for 48 hours, but keep dailies for 14 days is not possible.
In order for this to work, several changes are needed:
{snapshot,target}_preserve_hourly
configuration option, and will break the current preserve logic: currently, setting {snapshot,target}_preserve_daily 14
means "keep ALLsnapname.YYYYMMDD-HH[_N]
. This implies a new configuration option snapshot_timestamp_format
, as most btrbk users will nothourly_logic yes
could automatically enable this alternative timestamp, and wouldsnapshot_create onschedule
configuration option.Summing up, this feature introduces more complexity and confusion, while a main design goal of btrbk is "keep things simple and easy to configure".
Any thoughts?
I'm looking at encrypting my main btrfs pool using dm-crypt/luks and wonder about backups using btrbk(really in general i suppose) if i setup a ecryptfs folder on my backup harddrive ex: /mnt/usbdrive/.backup mounted at /mnt/usbdrive/backup and create a subvol called @homebackup inside this ecryptfs mount would this cause any potential problems to btrbk? In the end i suppose this is more a general "how well does ecryptfs work with btrfs subvols" but i figured this was as good a place as any to ask plus it might help others.
There is a new branch adding a feature to create raw backups on non-btrfs targets ("target_raw" branch). This is very useful if you want to make backups to some cloud storage or your backup file server does not support btrfs. This will probably be merged (as an experimental feature) into the next release.
The idea is to do a btrfs send
directly to a file, which can then be restored by cat <file> | btrfs receive
.
Features so far:
Bugs:
btrbk.conf:
raw_target_compress xz
raw_target_encrypt gpg
gpg_keyring /etc/btrbk/gpg/pubring.gpg
gpg_recipient [email protected]
volume /mnt/btr_pool
subvolume home
target raw ssh://myserver.mydomain.com/non-btrfs-directory
This will create encrypted/compressed raw files on myserver.mydomain.com:
/non-btrfs-directory/home.20150101.<received_uuid>.<parent_uuid>.btrfs.xz.gpg
Hello, i'm trying to setup btrbk with send/recieve, i've simple setup with such config:
volume /mnt/vol1
subvolume BACKUP
snapshot_preserve_daily 0
snapshot_preserve_weekly 0
snapshot_preserve_monthly 0
target send-receive ssh://xx.xx.xx.xx/mnt/vol1/BACKUP/
target_preserve_daily 0
target_preserve_weekly 4
target_preserve_monthly 12
On dst server its ok:
/mnt/vol1/BACKUP
+++ /mnt/vol1//BACKUP.20150911_8
>>> {xx.xx.xx.xx}/mnt/vol1/BACKUP/BACKUP.20150911_8
--- {xx.xx.xx.xx}/mnt/vol1/BACKUP/BACKUP.20150911_7
On src server btrbk ignores snapshot_preserve_* settings and number of snapshots keeps growing:
root@ol71a:/etc/btrbk# bls
ID 261 gen 94 top level 5 path BACKUP
ID 306 gen 83 top level 5 path BACKUP.20150911
ID 307 gen 84 top level 5 path BACKUP.20150911_1
ID 308 gen 85 top level 5 path BACKUP.20150911_2
ID 309 gen 86 top level 5 path BACKUP.20150911_3
ID 310 gen 87 top level 5 path BACKUP.20150911_4
ID 311 gen 89 top level 5 path BACKUP.20150911_5
ID 312 gen 90 top level 5 path BACKUP.20150911_6
ID 313 gen 92 top level 5 path BACKUP.20150911_7
ID 314 gen 94 top level 5 path BACKUP.20150911_8
root@ol71b:/mnt/vol1/BACKUP# bls
ID 257 gen 225 top level 5 path BACKUP
ID 301 gen 193 top level 257 path BACKUP/BACKUP.20150911
ID 309 gen 222 top level 257 path BACKUP/BACKUP.20150911_8
Any advice, thanks.
Not sure if this is something i am doing of if it's really broken. I know i did a diff before just can remeber on what version. With .16 this is the output of trying to diff 2 sub volumes
btrbk -c /etc/btrbk/btrbk.conf diff "/media/Storage3/Backup/@HomeBackup/@Home.20150417" "/media/Storage3/Backup/@HomeBackup/@Home.20150420"
ERROR: process died unexpectedly (btrbk v0.16)
Please contact the author: Axel Burri [email protected]
Died at /usr/bin/btrbk line 204.
at /usr/bin/btrbk line 101.
main::ANON("Died at /usr/bin/btrbk line 204.\x{a}") called at /usr/bin/btrbk line 204
main::config_key(undef, "btrfs_progs_compat") called at /usr/bin/btrbk line 523
main::btr_subvolume_list("/media/Storage3/Backup/@HomeBackup/@Home.20150417", undef, "subvol_only", 0) called at /usr/bin/btrbk line 649
main::btr_tree("/media/Storage3/Backup/@HomeBackup/@Home.20150417") called at /usr/bin/btrbk line 1220
My btrbk.conf
btrfs fails on a config like this (if symlinked_subvol
is a symlink to a real subvolume):
volume /mnt/btr_pool
subvolume symlinked_subvol
This needs further investigation. For now, the only known workaround is not to use symlinks.
If the source snapshot directory ("snapshot_dir" in btrbk.conf) is a subvolume instead of a regular directory, btrbk fails to find the snapshots within. As a consequence, it always wants to make a full backup instead of an incremental one.
Hello, I have set up for BTRBK snapshot and backup a Ubuntu server. I'm not sure if I did it right. I believe that I have a flaw. In particular diie logic to create the Subvolumes was not easy to recognize. I have the home- and Rootpartionionen from Standard "/" after "/mnt/c" reverse hanging. Would I dri can restore? Does the snapshot without error even when the current root partition?
/etc/fstab:
UUID=..... /mnt/c/rootfs btrfs defaults,subvol=@ 0 1
UUID=.... /mnt/c/home btrfs defaults,subvol=@home 0 2
UUID=.... none swap sw 0 0
/dev/sda1 /mnt/c/server btrfs defaults 0 0
/etc/btrbk/backup.conf:
snapshot_preserve_daily 14
snapshot_preserve_weekly 0
snapshot_preserve_monthly 0
target_preserve_daily 20
target_preserve_weekly 10
target_preserve_monthly all
snapshot_dir btrbk_snapshots
volume /mnt/c
subvolume rootfs
target send-receive /mnt/c/server
subvolume home
target send-receive /mnt/c/server
I'm sure I'm missing something obvious, but I've tried everything and haven't been able to figure it out.
Running btrbk with the backup volume connected works as expected.
Disconnecting the backup volume and running again, I get this output:
WARNING: Skipping target "/mnt/backups/Laptop/btrbk": Failed to fetch subvolume detail
WARNING: Skipping target "/mnt/backups/Laptop/btrbk": Failed to fetch subvolume detail
WARNING: Skipping target "/mnt/backups/Laptop/btrbk": Failed to fetch subvolume detail
WARNING: Skipping target "/mnt/backups/Laptop/btrbk": Failed to fetch subvolume detail
ERROR: process died unexpectedly (btrbk v0.17.0)
Please contact the author: Axel Burri <[email protected]>
Stack Trace:
----------------------------------------
Died at /usr/bin/btrbk line 1725, <FILE> line 97.
at /usr/bin/btrbk line 103, <FILE> line 97.
main::__ANON__("Died at /usr/bin/btrbk line 1725, <FILE> line 97.\x{a}") called at /usr/bin/btrbk line 1725
My config file is as follows:
snapshot_dir .snapshots
incremental yes
snapshot_create_always yes
resume_missing yes
preserve_day_of_week sunday
snapshot_preserve_daily 14
snapshot_preserve_weekly 4
snapshot_preserve_monthly 0
target_preserve_daily 40
target_preserve_weekly 32
target_preserve_monthly all
btrfs_commit_delete after
btrfs_progs_compat no
volume /mnt/subvolroot
subvolume rtfs
target send-receive /mnt/backups/Laptop/btrbk
subvolume boot
target send-receive /mnt/backups/Laptop/btrbk
subvolume home
target send-receive /mnt/backups/Laptop/btrbk
subvolume etc
target send-receive /mnt/backups/Laptop/btrbk
I first noticed the issue with version 0.17.0.
Hello,
I just tried out your great backup script and did and Pull based backup from one remote host.
The first time the backup runs fine. But if I did it the second time the error below appears.
please help.
btrbk command line client, version 0.11 (Tue Mar 10 20:45:11 2015)
ERROR: process died unexpectedly (btrbk v0.11)
Please contact the author: Axel Burri <axel@####.##>
Use of uninitialized value $& in concatenation (.) or string at /usr/sbin/btrbk line 673, line 108.
at /usr/sbin/btrbk line 97, line 108.
main::ANON('Use of uninitialized value $& in concatenation (.) or string ...') called at /usr/sbin/btrbk line 673
main::btr_subtree('/mnt/btrbk/kerio', 'HASH(0x13ca968)') called at /usr/sbin/btrbk line 1222
Yesterday, I messed around with a mainline kernel as there was a bug in BTRFS prohibiting me from converting my setup from RAID0 to raid10. The conversion worked, but I lost power three quarters of the way through and corrupted the file system.
So, I went ahead and re-created it, using the snapshots taken from my backup disk via btrbk.
After a couple of hours of btrfs send/receive, I have a fully functional system again and thought I couldn't be happier, until I attempted to run btrbk to resume my backups.
Upon running, I get this error:
ERROR: process died unexpectedly (btrbk v0.19.2)
Please contact the author: Axel Burri <[email protected]>
Stack Trace:
----------------------------------------
Died at /usr/bin/btrbk line 904, <FILE> line 134.
at /usr/bin/btrbk line 119, <FILE> line 134.
main::__ANON__("Died at /usr/bin/btrbk line 904, <FILE> line 134.\x{a}") called at /usr/bin/btrbk line 904
main::btr_tree(HASH(0x10c6790)) called at /usr/bin/btrbk line 956
main::vinfo_subvol_list(HASH(0x10c6790)) called at /usr/bin/btrbk line 296
main::vinfo_root(HASH(0x10c6790)) called at /usr/bin/btrbk line 1679
Running with btrbk -l trace run, the error is printed right after:
### btrfs subvolume list -a -c -u -q -R '/mnt/subvolroot'
... Command output:
ID 258 gen 2054 cgen 7 top level 5 parent_uuid - received_uuid abd4df05-160d-964b-9cd1-d7e97900ab43 uuid 33e0bc86-10f2-1d44-ae52-5bd716662491 path server
ID 295 gen 1929 cgen 52 top level 258 parent_uuid - received_uuid 40b49bc9-9a40-de4a-854e-9c8704f2e55f uuid f7b2eb3a-3b2f-fc4e-8912-a84fc4deed6c path <FS_TREE>/server/etc
ID 296 gen 2054 cgen 55 top level 5 parent_uuid - received_uuid 2286e4be-44fe-5344-9caf-cae657284e15 uuid b379e5bd-5fd4-8146-a5ac-82b2d48c2f3a path desktop
ID 324 gen 1044 cgen 119 top level 335 parent_uuid - received_uuid e32e9aa2-0eeb-bc47-8c85-e256d4c04265 uuid 92709d52-1f6e-0147-b367-0a9a37414ca6 path <FS_TREE>/server/var/lib/machines/guest_os
ID 332 gen 2054 cgen 143 top level 258 parent_uuid - received_uuid b94d353e-6292-7c44-bfa6-ff9944ecf535 uuid 72eae412-c5e4-a241-bbc8-f8b2e39830d5 path <FS_TREE>/server/var/lib/boinc
ID 335 gen 1044 cgen 152 top level 258 parent_uuid - received_uuid bab52f3b-cca5-6d41-b407-13e45fb34424 uuid c16575f9-d75e-874f-b87d-e754858274c3 path <FS_TREE>/server/var/lib/machines
ID 336 gen 1044 cgen 157 top level 296 parent_uuid - received_uuid 10ead242-eb04-234e-bbaa-41ebc1d7bde6 uuid a9776a67-1dd0-9c4f-a203-4b34b2f8841e path <FS_TREE>/desktop/home/[username]/temp
ID 350 gen 1044 cgen 198 top level 258 parent_uuid - received_uuid - uuid 50fb67cb-e42f-ad46-b86e-5811e7c1e7f6 path <FS_TREE>/server/srv/gameservers
ID 351 gen 1044 cgen 202 top level 350 parent_uuid - received_uuid - uuid 112d9f84-d858-724f-bbcf-d3c0a4ab68bf path <FS_TREE>/server/srv/gameservers/tekkit
Command execution successful
Parsed 9 total subvolumes for filesystem at: /mnt/subvolroot
... btr_tree: processing subvolume list of: /mnt/subvolroot
My btrbk config is as follows:
### GLOBALS ###
snapshot_create onchange
snapshot_preserve_daily 7
snapshot_preserve_weekly 4
snapshot_preserve_monthly 2
target_preserve_daily 28
target_preserve_weekly 16
snapshot_preserve_monthly all
btrfs_commit_delete after
preserve_day_of_week sunday
resume_missing yes
incremental yes
### END GLOBALS ###
### BEGIN LOCAL CONFIG ###
volume /mnt/subvolroot
snapshot_dir backups
subvolume server
target send-receive /mnt/backups/Desktop/btrbk/server
subvolume server/srv/gameservers/tekkit
snapshot_dir server/srv/gameservers/backups
target_preserve_daily 7
target send-receive /mnt/backups/Desktop/btrbk/tekkit
subvolume server/etc
target send-receive /mnt/backups/Desktop/btrbk/server
subvolume desktop
target_preserve_weekly 40
target send-receive /mnt/backups/Desktop/btrbk/desktop
subvolume server/var/lib/machines/guest_os
snapshot_dir server/var/lib/machines/.snapshots
target send-receive /mnt/backups/Desktop/btrbk/machines
### END LOCAL CONFIG ###
All snapshot directories and subvolumes appear to be accounted for.
I'm not sure how I should proceed.
So this is probably somewhere on the net but i can't seem to find anything with good examples. It would be nice to have an example of how one would go about restoring from an incremental snapshot/backup. Now that i think about it if the original full backup is gone due to retention policy then are any files not modified between it and the last incremental before it was deleted no longer a part of the "backup". The reason i ask is because i'm thinking of switching distros and would like to just use the snapshots/backups i already have to restore my home partition instead of creating a new one.
Right now I'm setting identity files via the config and port numbers via ssh_config. It would be nice to have the ability to specify ssh's port number via btrbk's config for servers that use a non-standard port.
hi there :)
first off, thank you for this nifty little tool. makes my backup-workflow easier.
back to the matter:
it would be nice to be able to run a command or a script before and/or after the backup is executed.
i for example do not have mounted my subvolid=0's by default so for now i put btrbk in a shell script where i first mount the subvolid=0 of the disks needed, run the backups/snapshots and then unmount them again. it works but it's yet another wrapper-script flying around...
bonus would be an action on error.
Cheers!
Running btrbk 0.19.3.
Running btrbk via cron sends me an email even if there aren't any errors.
I tried running the command outside of cron gives me:
sudo btrbk -q -c /etc/btrbk/btrbk-######.conf run
At subvol /mnt/btr_system/btrbk_snapshots/@.20150709_3
At subvol /mnt/btr_system/btrbk_snapshots/@home.20150709_3
At subvol /mnt/btr_system/btrbk_snapshots/@vms.20150709_3
Which is the same output I'm being emailed daily.
I'm trying to run btrfs-tools only from the main repo (3.17) on Kubuntu 15.04 on my desktop. This doesn't happen on my main ubuntu server where I run the latest btrfs-progs.
I guess it might not be a btrfs-tools or btrbk bug and just a problem with my system.
If this isn't an easy fix I can just download and build the latest btrfs-progs, but I thought I should let you know.
Hi!
A dryrun with this config file
snapshot_dir btrbk_snapshots
incremental yes
snapshot_create always
resume_missing yes
preserve_day_of_week monday
snapshot_preserve_daily 14
snapshot_preserve_weekly 8
snapshot_preserve_monthly all
target_preserve_daily 14
target_preserve_weekly 8
target_preserve_monthly all
btrfs_commit_delete after
volume /home
subvolume user1
target send-receive /mnt/btrfs_bkp
fails like this
btrbk command line client, version 0.19.1 (Sat Jun 6 11:09:09 2015)
Using configuration: /foo/bar/btrbk.conf
ERROR: process died unexpectedly (btrbk v0.19.1)
Please contact the author: Axel Burri <[email protected]>
Stack Trace:
----------------------------------------
Use of uninitialized value $key in concatenation (.) or string at /usr/sbin/btrbk line 367, <FILE> line 95.
at /usr/sbin/btrbk line 119, <FILE> line 95.
main::__ANON__("Use of uninitialized value \$key in concatenation (.) or strin"...) called at /usr/sbin/btrbk line 367
main::check_file("ERROR", HASH(0x1c49170)) called at /usr/sbin/btrbk line 632
main::btrfs_subvolume_detail(HASH(0x1b8bf08)) called at /usr/sbin/btrbk line 291
main::vinfo_root(HASH(0x1b8bf08)) called at /usr/sbin/btrbk line 1728
Do you have any hint?
So i was in the process of encrypting my entire system and recreating all my btrfs volumes/subvolumes in the process i made a mistake and created my snapshot dir as @backup instead of @BackuPs like was set up originally and defined in btrbk.conf. When i ran dryrun it did not show me any errors:
btrbk -c /etc/btrbk/btrbk.conf dryrun
--------------------------------------------------------------------------------
Backup Summary (btrbk command line client, version 0.19.2)
Date: Thu Jun 11 08:43:35 2015
Config: /etc/btrbk/btrbk.conf
Legend:
=== up-to-date subvolume (source snapshot)
+++ created subvolume (source snapshot)
--- deleted subvolume
*** received subvolume (non-incremental)
>>> received subvolume (incremental)
--------------------------------------------------------------------------------
/mnt/btrfsroot/@Home
+++ /mnt/btrfsroot/@Backups/@Home.20150611
*** /media/Storage3/Backup/@HomeBackup/@Home.20150611
/mnt/btrfsroot/@Storage
+++ /mnt/btrfsroot/@Backups/@Storage.20150611
*** /media/Storage3/Backup/@StorageBackup/@Storage.20150611
NOTE: Dryrun was active, none of the operations above were actually executed!
but then when i ran run it failed(as it should since the @BackuPs subvol is not present):
btrbk -v -c /etc/btrbk/btrbk.conf run
btrbk command line client, version 0.19.2 (Thu Jun 11 08:44:04 2015)
Using configuration: /etc/btrbk/btrbk.conf
Creating subvolume snapshot for: /mnt/btrfsroot/@Home
>>> /mnt/btrfsroot/@Backups/@Home.20150611
ERROR: can't access '/mnt/btrfsroot/@Backups'
ERROR: Failed to create btrfs subvolume snapshot: /mnt/btrfsroot/@Home -> /mnt/btrfsroot/@Backups/@Home.20150611
WARNING: Skipping subvolume section: Failed to create snapshot: /mnt/btrfsroot/@Home -> /mnt/btrfsroot/@Backups/@Home.20150611
Creating subvolume snapshot for: /mnt/btrfsroot/@Storage
>>> /mnt/btrfsroot/@Backups/@Storage.20150611
ERROR: can't access '/mnt/btrfsroot/@Backups'
ERROR: Failed to create btrfs subvolume snapshot: /mnt/btrfsroot/@Storage -> /mnt/btrfsroot/@Backups/@Storage.20150611
WARNING: Skipping subvolume section: Failed to create snapshot: /mnt/btrfsroot/@Storage -> /mnt/btrfsroot/@Backups/@Storage.20150611
Completed within: 0s (Thu Jun 11 08:44:04 2015)
--------------------------------------------------------------------------------
Backup Summary (btrbk command line client, version 0.19.2)
Date: Thu Jun 11 08:44:04 2015
Config: /etc/btrbk/btrbk.conf
Legend:
=== up-to-date subvolume (source snapshot)
+++ created subvolume (source snapshot)
--- deleted subvolume
*** received subvolume (non-incremental)
>>> received subvolume (incremental)
--------------------------------------------------------------------------------
/mnt/btrfsroot/@Home
!!! Aborted: Failed to create snapshot: /mnt/btrfsroot/@Home -> /mnt/btrfsroot/@Backups/@Home.20150611
/mnt/btrfsroot/@Storage
!!! Aborted: Failed to create snapshot: /mnt/btrfsroot/@Storage -> /mnt/btrfsroot/@Backups/@Storage.20150611
NOTE: Some errors occurred, which may result in missing backups!
So it seems that some part of dryrun is not testing for the presence of the snapshot dir as it should.
btrfs subvol list /mnt/btrfsroot
ID 259 gen 9 top level 5 path @Backup
ID 646 gen 1303 top level 5 path @Home
ID 901 gen 1307 top level 5 path @Storage
Upon transferring several snapshots to my backup volume, I received this error message.
At subvol /mnt/subvolroot/.snapshots/rtfs.20150531
ERROR: process died unexpectedly (btrbk v0.19.0)
Please contact the author: Axel Burri <[email protected]>
Stack Trace:
----------------------------------------
Use of uninitialized value in string ne at /usr/bin/btrbk line 2240, <FILE> line 26.
at /usr/bin/btrbk line 119, <FILE> line 26.
main::__ANON__("Use of uninitialized value in string ne at /usr/bin/btrbk lin"...) called at /usr/bin/btrbk line 2237
Believing there was something wrong with that particular rtfs snapshot, I deleted the snapshot for 20150531 and re-ran it. The same error came up with the previous snapshot as well, despite being transferred successfully.
Hi,
I'm using btrbk 0.16 on Arch Linux. I have source subvolumes named 0, 1, 2, ... which I would like to backup. However, the config file parsing fails when encountering the "subvolume 0" line (however, e.g. "subvolume 1" works):
...
config: adding volume "<path omitted>" to root context
ERROR: Failed to parse configuration file
When I tried to change "0" to "0/" in the config file, the error gets even worse:
...
ERROR: process died unexpectedly (btrbk v0.16)
Please contact the author: Axel Burri <e-mail address omitted>
Stack Trace:
----------------------------------------
Died at /usr/bin/btrbk line 1371, <FILE> line 116.
at /usr/bin/btrbk line 101, <FILE> line 116.
main::__ANON__("Died at /usr/bin/btrbk line 1371, <FILE> line 116.\x{a}") called at /usr/bin/btrbk line 1371
I'm pretty sure i have everything setup right so here is the problem:
config file:
cat /etc/btrbk/btrbk.conf.example
snapshot_dir @BackuPs
snapshot_create_always yes
incremental no
preserve_day_of_week sunday
snapshot_preserve_daily all
snapshot_preserve_weekly 4
snapshot_preserve_monthly 1
target_preserve_daily all
target_preserve_weekly 4
target_preserve_monthly 1
btrfs_commit_delete after
receive_log no
volume /mnt/btrfsroot
subvolume @Home
target send-receive /media/Storage3/@HomeSnaps
btrfs subvolume list /media/Storage3/@HomeSnaps
ID 5683 gen 51069 top level 5 path @HomeSnaps
btrfs subvolume show /media/Storage3/@HomeSnaps
/media/Storage3/@HomeSnaps
Name: @HomeSnaps
uuid: 1bb1ed43-5c5d-7542-9a6e-87b36eae5f48
Parent uuid: -
Creation time: 2015-03-19 17:43:48
Object ID: 5683
Generation (Gen): 51069
Gen at creation: 51069
Parent: 5
Top Level: 5
Flags: -
Snapshot(s):
btrbk -c /etc/btrbk/btrbk.conf.example dryrun
WARNING: Skipping target: Failed to read btrfs subvolume list for "/media/Storage3/@HomeSnaps"
Backup Summary (btrbk command line client, version 0.13)
Date: Fri Mar 20 12:24:14 2015
Config: /etc/btrbk/btrbk.conf.example
/mnt/btrfsroot/@Home
+++ /mnt/btrfsroot/@Backups/@Home.20150320
!!! Target "/media/Storage3/@HomeSnaps" aborted: Failed to read btrfs subvolume list for "/media/Storage3/@HomeSnaps"
NOTE: Some errors occurred, which may result in missing backups!
Please check warning and error messages above.
NOTE: Dryrun was active, none of the operations above were actually executed!
i've tried creating the subvol with and without @. At first i was creating it at /media/Storage3/Backup/HomeSnaps but even moving it to the root of the FS i have the same problem. I am using arch linux with kernel 3.19.1-1-ck and btrfs-progs v3.19. Is it something i am doing wrong or a bug?
I have a USB drive as one target for my snapshots which isn't connected most of the time.
What I would like to do, is connect it once in a while and sync the snapshot (copy missing and delete old). At the moment this means that I have to connect the drive, make sure it is mounted and wait for the cron-job to happen. Or start the backup manually which will result in two snapshots for the same day.
What I'm missing is a new command or a option for the "run" command to prevent btrbk from making new snapshots, just making sure all targets are up to date.
This would also be useful in case a remote server was down during the backup.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.