lzkelley / bkup_rpimage

Script to backup a Raspberry Pi disk image

bkup_rpimage

Script to backup a Raspberry Pi disk image

Author / Origin:

Script developed on the RaspberryPi.org forums: The Raspberry Pi Backup Thread. The script, as copied over to github.com, was primarily written by user: jinx. I found this via a raspberrypi.stackexchange.com answer by user: ppumpkin.

The script was taken from here, a post from Mon Nov 10, 2014 7:05 am. I copied the attached file on Wed Feb 01, 2017 to this git repository.

Usage from Jinx and corrected the mistake of using rsync instead of start in the examples

bkup_rpimage.sh v1.0 by jinx

Usage:

• bkup_rpimage.sh start [-cslzdf] [-L logfile] sdimage
• bkup_rpimage.sh mount [-c] sdimage [mountdir]
• bkup_rpimage.sh umount sdimage [mountdir]
• bkup_rpimage.sh gzip [-df] sdimage

Commands:

• start - starts complete backup of RPi's SD Card to 'sdimage'
• mount - mounts the 'sdimage' to 'mountdir' (default: /mnt/'sdimage'/)
• umount - unmounts the 'sdimage' from 'mountdir'
• gzip - compresses the 'sdimage' to 'sdimage'.gz

Options:

• -c creates the SD Image if it does not exist
• -l writes rsync log to 'sdimage'-YYYYmmddHHMMSS.log
• -z compresses the SD Image (after backup) to 'sdimage'.gz
• -d deletes the SD Image after successful compression
• -f forces overwrite of 'sdimage'.gz if it exists
• -L logfile writes rsync log to 'logfile'
• -s define the size of the image file

Examples:

Start backup to rpi_backup.img, creating it if it does not exist:

bkup_rpimage.sh start -c /path/to/rpi_backup.img

Start backup to rpi_backup.img, creating it if it does not exist limiting the size to 8000Mb

bkup_rpimage.sh start -s 8000 -c /path/to/rpi_backup.img

Use the RPi's hostname as the SD Image filename:

bkup_rpimage.sh start /path/to/$(uname -n).img

Use the RPi's hostname and today's date as the SD Image filename, creating it if it does not exist, and compressing it after backup:

bkup_rpimage.sh start -cz /path/to/$(uname -n)-$(date +%Y-%m-%d).img

Mount the RPi's SD Image in /mnt/rpi_image:

bkup_rpimage.sh mount /path/to/$(uname -n).img /mnt/rpi_image

Unmount the SD Image from default mountdir (/mnt/raspi-2014-11-10.img/):

bkup_rpimage.sh umount /path/to/raspi-2014-11-10.img

bkup_rpimage's Issues

Backup a USB HD partition

Hi,

I've just set up mi RPi to boot from SD (mmcblk0) and innmediatelly mount the rootfs from a USB HD as described in HOWTO: Move the filesystem to a USB stick/Drive. I also added a couple of additional partitions to mount /var/log, /data and a /backup mountpoints

I want to schedule daily backups of /boot (on SD) and / (on USB HD) and store the images on /backup in order to be able to burn a new SD and boot from it in case of HD failure.

Your script assumes the source partitions are on /dev/mmcblk0 and use that device to get partition sizes and types to create the target image. However, it then rsync's from / and /boot, that in my system are in the USB HD /dev/sdc.

How do I tell the script to take those params from the proper places? (/dev/mmcblk0 and /dev/sdc2)

I'm also planning as the next step to boot directly from the USB /boot partition (/dev/sdc1 in my case) and remove the SD card from the RPi. I guess that in that case, the bkup_image script will have trouble to get the image geometry from an inexistent SD card.

Any hint?
Thanks!

routine for mounting compressed Images is missing/not working

it would be nice it mounting compressed images would be possible.

pi@rpi2:~ $ sudo ./bkup_rpimage.sh mount /usb-stick/backup/SD_Card_rpi2_2018-11-20.img.gz /mnt
Attaching /usb-stick/backup/SD_Card_rpi2_2018-11-20.img.gz to /dev/loop0
losetup: /usb-stick/backup/SD_Card_rpi2_2018-11-20.img.gz: Warning: file does not fit into a 512-byte sector; the end of the file will be ignored.
partx: /dev/loop0: failed to read partition table
Mounting /dev/loop0p1 and /dev/loop0p2 to /mnt
mount: special device /dev/loop0p2 does not exist
mount: special device /dev/loop0p1 does not exist
SD Image has been mounted and can be accessed at:
    /mnt

pi@rpi2:~ $ ll /mnt
drwxr-xr-x 24 root root 4096 Nov 11 15:25 ..
drwxr-xr-x  2 root root 4096 Nov 20 17:14 boot
drwxr-xr-x  3 root root 4096 Nov 20 17:14 .
pi@rpi2:~ $ ll /mnt/boot/
drwxr-xr-x 3 root root 4096 Nov 20 17:14 ..
drwxr-xr-x 2 root root 4096 Nov 20 17:14 .

It is backing up itself !

I have this problem. My external drive (for backup and other stuff) is mounted at boot in fstab and shared with smb. The location for the mount in mnt/usbsd.
When I start the backup I see that it back up also mnt/usbsd, where there is already the .img file.
So it makes a back up of itself??
Is there something I can do to avoid it?
or maybe add a function to skip "itself" back up?

/dev/root 100% usage after backup

Hi all.
I wanted to backup SD 16G card on a 32gb pendrive.
sudo sh bkup_rpimage.sh start -c /media/mypibackup/rpi_backup.img
After about 20 minutes, I received information:

rsync: write failed on "/mnt/rpi_backup.img/media/pibackup/rpi_backup.img": No s             pace left on device (28)
rsync error: error in file IO (code 11) at receiver.c(374) [receiver=3.1.3]
Filesystem     1M-blocks  Used Available Use% Mounted on
/dev/loop0p1         256    40       217  16% /mnt/rpi_backup.img/boot
/dev/loop0p2       14631  3244     10625  24% /mnt/rpi_backup.img
-e Flushing to disk
-e Unmounting /dev/loop01 and /dev/loop02 from /mnt/rpi_backup.img/
-e Detaching /media/pibackup/rpi_backup.img from /dev/loop0
-e SD Image backup process completed.

The system started to work very slowly. After issuing the command: df -h :

Filesystem      Size  Used Avail Use% Mounted on
/dev/root        15G   15G     0 100% /
devtmpfs        459M     0  459M   0% /dev
tmpfs           464M     0  464M   0% /dev/shm
tmpfs           464M  6.3M  457M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           464M     0  464M   0% /sys/fs/cgroup
/dev/mmcblk0p1  253M   40M  213M  16% /boot
tmpfs            93M     0   93M   0% /run/user/1000

HELP!

Remove image and remove bkup_rpimage

Hello.
I'm running Raspbian Buster on a Raspberry PI 4. I'm not able to make bkup_rpimage to work, and want to remove it from my system. I have two image-files that shows up when I goes to the Documents folder and select the mnt folder, I have tried to use the sudo sh bkup_rpimage.sh umount /mnt/rpi_backup.img, but I'm only receiving a message that the mount doesn't exist. Can anyone help me?

image does not boot on recent Raspbian versions (using PARTUUIDs)

Recent Rasbian versions use PARTUUIDs.
When creating an image with this tool, the image and the partitions within it get a new PARTUUID. These are different from those defined in /etc/fstab and /boot/cmdline.txt. If you write the image as such on a new sdcard, this card won't boot.

Solution: change the PARTUUIDs manually in the above files in the image.

notes:

• rpi-clone (https://github.com/billw2/rpi-clone) alters those files automatically. The main branch, however, does not allow to create images. This fork/branch (https://github.com/SeeSpotRun/rpi-clone/tree/image) does, and although it output some errors, the image seems to work.

• this forum post (https://www.raspberrypi.org/forums/viewtopic.php?t=191775#p1203116) contains a script to change PARTUUIDs on an image. I did not test this

Destination: does it have to be as big as the RPI SD card?

Hi,

I have a 64gb SD card in my RPI with just 5gb in use. I also have an USB memory stick attached of 16gb (pretty much all free).

# df -h
/dev/root        57G  5.0G   49G  10% /
devtmpfs        458M     0  458M   0% /dev
tmpfs           462M     0  462M   0% /dev/shm
tmpfs           462M   13M  450M   3% /run
tmpfs           5.0M  8.0K  5.0M   1% /run/lock
tmpfs           462M     0  462M   0% /sys/fs/cgroup
/dev/mmcblk0p6   66M   24M   43M  36% /boot
tmpfs            93M  8.0K   93M   1% /run/user/1000
/dev/mmcblk0p5   30M  801K   27M   3% /media/pi/SETTINGS
/dev/sdb1        15G   44M   14G   1% /media/sd

When I try to run /apps/bkup_rpimage/bkup_rpimage.sh start -c /media/sd/rpi.img, this is what happens:

Starting SD Image backup process
Creating sparse /media/sd/rpi.img, the apparent size of /dev/mmcblk0
0+0 records in
0+0 records out
0 bytes copied, 0.00106547 s, 0.0 kB/s
Attaching /media/sd/rpi.img to /dev/loop0
Copying partition table from /dev/mmcblk0 to /dev/loop0
Checking that no-one is using this disk right now ... OK

Disk /dev/loop0: 59.6 GiB, 64021856256 bytes, 125042688 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x095a474a

Old situation:

>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Created a new DOS disklabel with disk identifier 0x0001a4b4.
/dev/loop0p1: Created a new partition 1 of type 'W95 FAT32 (LBA)' and of size 1.6 GiB.
/dev/loop0p2: Created a new partition 2 of type 'Extended' and of size 58.1 GiB.
/dev/loop0p3: Created a new partition 5 of type 'Linux' and of size 32 MiB.
/dev/loop0p6: Created a new partition 6 of type 'W95 FAT16 (LBA)' and of size 66 MiB.
/dev/loop0p7: Created a new partition 7 of type 'Linux' and of size 58 GiB.
/dev/loop0p8: Done.

New situation:

Device       Boot   Start       End   Sectors  Size Id Type
/dev/loop0p1         8192   3300781   3292590  1.6G  c W95 FAT32 (LBA)
/dev/loop0p2      3300782 125042687 121741906 58.1G  5 Extended
/dev/loop0p5      3301376   3366909     65534   32M 83 Linux
/dev/loop0p6      3366912   3502079    135168   66M  e W95 FAT16 (LBA)
/dev/loop0p7      3506176 125042687 121536512   58G 83 Linux

The partition table has been altered.
Calling ioctl() to re-read partition table.
Re-reading the partition table failed.: Invalid argument
The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8).
Syncing disks.
Formatting partitions
mkfs.fat 4.1 (2017-01-24)
mke2fs 1.43.4 (31-Jan-2017)
Found a dos partition table in /dev/loop0p2
Proceed anyway? (y,N) y
mkfs.ext4: inode_size (128) * inodes_count (0) too big for a
	filesystem with 0 blocks, specify higher inode_ratio (-i)
	or lower inode count (-N).

Mounting /dev/loop0p1 and /dev/loop0p2 to /mnt/rpi.img/
mount: /dev/loop0p2: can't read superblock
Skipping rsync since /mnt/rpi.img/ is not a mount point
Flushing to disk
Unmounting /dev/loop0p1 and /dev/loop0p2 from /mnt/rpi.img/
umount: /mnt/rpi.img/: not mounted
rmdir: failed to remove '/mnt/rpi.img/': Directory not empty
Detaching /media/sd/rpi.img from /dev/loop0
SD Image backup process completed.

Any clue about how to fix this issue?

Compact (not compress) resulting image file

Using this great tool to clone a system which boots from USB (no SD on the raspi), the backup runs OK but since it's a 16GB pendrive where 40MB boot partition and 4ishGB root partition are, I would expect 4GBish images, rather it creates 15gbish images.
I understand as it created an image "the apparent size of $SDCARD", is there a way to allocate only the real part1+part2 size instead? Or should I necessarily specify the image size with -s? This second option should work, but it statically allocates the destination image on the secondary pendrive making the process reeeaaaaal slow

Getting bunch of errors

I have a freshly installed Raspbian Stretch Lite apart from installing git and enabling SSH. It's installed on a USB instead of a SD Card so I've changed the SDCARD variable on the top of the script to point to the rootfs partition of the USB, which is /dev/sda2.

I ran the script with the following command:
./bkup_rpimage.sh start -s 4000 -c /mnt/JetFlash16/rpi_stage_1.img

Script output:

Starting SD Image backup process
Creating sparse /mnt/JetFlash16/rpi_stage_1.img, the apparent size of /dev/sda2
0+0 records in
0+0 records out
0 bytes copied, 0.000842968 s, 0.0 kB/s
Attaching /mnt/JetFlash16/rpi_stage_1.img to /dev/loop0
Copying partition table from /dev/sda2 to /dev/loop0
Checking that no-one is using this disk right now ... OK

Disk /dev/loop0: 3.9 GiB, 4194304000 bytes, 8192000 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xa6e4b0cf

Old situation:

sfdisk: /dev/sda2: does not contain a recognized partition table

Done.

New situation:

The partition table has been altered.
Calling ioctl() to re-read partition table.
Re-reading the partition table failed.: Invalid argument
The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8).
Syncing disks.
Formatting partitions
mkfs.fat 4.1 (2017-01-24)
mkfs.vfat: unable to open /dev/loop0p1: No such file or directory
mke2fs 1.43.4 (31-Jan-2017)
The file /dev/loop0p2 does not exist and no size was specified.
e2fsck 1.43.4 (31-Jan-2017)
e2fsck: No such file or directory while trying to open /dev/loop0p2
Possibly non-existent device?
tune2fs 1.43.4 (31-Jan-2017)
tune2fs: option requires an argument -- 'U'
Usage: tune2fs [-c max_mounts_count] [-e errors_behavior] [-f] [-g group]
[-i interval[d|m|w]] [-j] [-J journal_options] [-l]
[-m reserved_blocks_percent] [-o [^]mount_options[,...]]
[-p mmp_update_interval] [-r reserved_blocks_count] [-u user]
[-C mount_count] [-L volume_label] [-M last_mounted_dir]
[-O [^]feature[,...]] [-Q quota_options]
[-E extended-option[,...]] [-T last_check_time] [-U UUID]
[-I new_inode_size] [-z undo_file] device

Welcome to fdisk (util-linux 2.29.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): Disk /dev/loop0: 3.9 GiB, 4194304000 bytes, 8192000 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xa6e4b0cf

Command (m for help):
Expert command (m for help):
Enter the new disk identifier: Incorrect value.

Expert command (m for help):
Command (m for help):
Disk /dev/loop0: 3.9 GiB, 4194304000 bytes, 8192000 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xa6e4b0cf

Command (m for help): The partition table has been altered.
Calling ioctl() to re-read partition table.
Re-reading the partition table failed.: Invalid argument

The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8).

Mounting /dev/loop01 and /dev/loop02 to /mnt/rpi_stage_1.img/
mount: special device /dev/loop0p2 does not exist
mount: special device /dev/loop0p1 does not exist
Skipping rsync since /mnt/rpi_stage_1.img/ is not a mount point
df: /dev/loop0p1: No such file or directory
df: /dev/loop0p2: No such file or directory
Flushing to disk
Unmounting /dev/loop01 and /dev/loop02 from /mnt/rpi_stage_1.img/
umount: /mnt/rpi_stage_1.img//boot: not mounted
umount: /mnt/rpi_stage_1.img/: not mounted
rmdir: failed to remove '/mnt/rpi_stage_1.img/': Directory not empty
Detaching /mnt/JetFlash16/rpi_stage_1.img from /dev/loop0
partx: specified range <1:0> does not make sense
SD Image backup process completed.

I've tried to execute suggested methods of the "kernel still uses the old table" message, but with no luck. Anything else is beyond my knowledge at this point, so I would appreciate the help.

Feature request: specify backup source from /dev/disk/by-id/ folder

Just like the title.
I use USB boot, and this tool is awesome for backups, especially in the latest version where you can set image size and the PARTUUID's are automagically exported as well.
Awesome for real.
Now, since my raspi's work with several things hooked on USB, it MIGHT happen that the boot drive is sda today, and sdb tomorrow after I reboot, that's why whenever I need to refer to a drive, I prefer using the disk/partition identifiers in /dev/disk/by-id/ folder.
With this script though, it seems disk identifiers under /dev/disk/ don't work, and I am forced to use /dev/sd*.

Is it a known limitation or can it be made so that they can be used to point to the source drive?

What about "hard link" ?

Hi,
rsync without the -H option should convert hardlinks into files. In addition to wasting space, this compromises many uses which assume the hardlinks are in place. (My current image has 2081 links, many in Raspbian itself.).
Any drawback to the "-H" addition to the rsync command line ?

What about ACL and xattrs?

You may want to add -A and -X to the rsync command to preserve ACLs and xattrs.
This is particularly important for stuff like "ping" which has capabilities set in the xattrs:

Original:

$ getcap /bin/ping
/bin/ping = cap_net_raw+ep
$

Backup:

$ getcap /bin/ping
$

Hardlinks (-H) would also be good but there is already an open issue for that.

exclude directory

Is there a way to exclude certain directories on the backup?

Thanks
Mike

Exclude NFS server mounted

Hello, Is there a way to exclude a nfs share mounted on /mnt by /etc/fstab on boot?