restic / restic

Fast, secure, efficient backup program

Introduction

restic is a backup program that is fast, efficient and secure. It supports the three major operating systems (Linux, macOS, Windows) and a few smaller ones (FreeBSD, OpenBSD).

For detailed usage and installation instructions check out the documentation.

You can ask questions in our Discourse forum.

Quick start

Once you've installed restic, start off with creating a repository for your backups:

$ restic init --repo /tmp/backup
enter password for new backend:
enter password again:
created restic backend 085b3c76b9 at /tmp/backup
Please note that knowledge of your password is required to access the repository.
Losing your password means that your data is irrecoverably lost.

and add some data:

$ restic --repo /tmp/backup backup ~/work
enter password for repository:
scan [/home/user/work]
scanned 764 directories, 1816 files in 0:00
[0:29] 100.00%  54.732 MiB/s  1.582 GiB / 1.582 GiB  2580 / 2580 items  0 errors  ETA 0:00
duration: 0:29, 54.47MiB/s
snapshot 40dc1520 saved

Next you can either use restic restore to restore files or use restic mount to mount the repository via fuse and browse the files from previous snapshots.

For more options check out the online documentation.

Backends

Saving a backup on the same machine is nice but not a real backup strategy. Therefore, restic supports the following backends for storing backups natively:

• Local directory
• sftp server (via SSH)
• HTTP REST server (protocol, rest-server)
• Amazon S3 (either from Amazon or using the Minio server)
• OpenStack Swift
• BackBlaze B2
• Microsoft Azure Blob Storage
• Google Cloud Storage
• And many other services via the rclone Backend

Design Principles

Restic is a program that does backups right and was designed with the following principles in mind:

• Easy: Doing backups should be a frictionless process, otherwise you might be tempted to skip it. Restic should be easy to configure and use, so that, in the event of a data loss, you can just restore it. Likewise, restoring data should not be complicated.

• Fast: Backing up your data with restic should only be limited by your network or hard disk bandwidth so that you can backup your files every day. Nobody does backups if it takes too much time. Restoring backups should only transfer data that is needed for the files that are to be restored, so that this process is also fast.

• Verifiable: Much more important than backup is restore, so restic enables you to easily verify that all data can be restored.

• Secure: Restic uses cryptography to guarantee confidentiality and integrity of your data. The location the backup data is stored is assumed not to be a trusted environment (e.g. a shared space where others like system administrators are able to access your backups). Restic is built to secure your data against such attackers.

• Efficient: With the growth of data, additional snapshots should only take the storage of the actual increment. Even more, duplicate data should be de-duplicated before it is actually written to the storage back end to save precious backup space.

Reproducible Builds

The binaries released with each restic version starting at 0.6.1 are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that are contained in the builder repository.

News

You can follow the restic project on Mastodon @resticbackup or subscribe to the project blog.

License

Restic is licensed under BSD 2-Clause License. You can find the complete text in LICENSE.

Sponsorship

Backend integration tests for Google Cloud Storage and Microsoft Azure Blob Storage are sponsored by AppsCode!

restic's Issues

Add high-level documentation for repository

Create high-level documentation that describes the repository layout and files stored within, maybe as an inkscape svg graphic file.

repository structure: rename blob to data

blob is too generic, directory contains data. Rename in code.

Efficient way to show difference between snapshots

Addition to list snapshots: As a user I want to see the increments between snapshots (in files + file size).

Implement concurrency (for Backup)

Backup backend needs to support concurrent read and write

Annotate all error messages, drop external dependency

Add a string that describes which function has thrown the error

Add elapsed and estimated time for backup

Implement Continuous Integration

Storage Backend: Amazon S3

Implement Storage Backend Amazon S3

Find file system boundaries

Add option to stay on the same file system.

Add high-level documentation for crypto

Create high-level overview documentation for the cryptographic primitives used and a graphic representation (e.g. inkscape SVG file).

Store number of changed files + filesize

Store number of changed files + filesize in Snapshot metadata

Add Integration Tests to CI

khepri init should require -r

khepri init creates repository in current directory if -r not provided. Repository init should require -r.

 ./khepri init /mnt/usb/backup                
enter password for new backend: 
enter password again: 
created khepri backend at khepri-backup
.-(~GOPATH/bin)----------------------------------------------------------------------------------------------(fd@leno)-
`--> ls
khepri  khepri-backup

Write manpage

Create lightning talk slides

Create slides for the lightning talk at 31c3.

Key Management: List Keys

Create function to list encryption keys

Configuration File

Configuration File to store the default settings. Just run "restic" to create next snapshot based on settings in config file.

Connect #288

Integrity Check

Implement Integrity Checker (check snapshots and existance of related data files)

• Verify specific snapshot (Download Data + decrypt + verify checksum)
• Verify remote snapshot (Do not download data, just verify existance of data files and their size)

Key Management: Add Keys

Create Function to add new encryption keys

Decide license

khepri is currently licensed as 2-clause-BSD, stay with it or change to GPL?

Cryptographic review

Find reviewers, start review process

Save and restore extended attributes

This issue tracks saving and restoring extended attributes like POSIX ACL.

Check Timestamps, efficient backup

If a user initiates a backup, availability of previous snapshots shall be verified. Based on the snapshot, local verification shall take place wether new backup for a particular file or directory is required.

implement restic -V

• Frontend: Output version information
• Backend: define version during compile time

Correctly restore timestamps

Some timestamps for directories are not restored, this should be corrected.

Add Website

Add website hosted at with github pages

Key Management: Remove Keys

Create function to remove encryption keys. Implement safeguard against accidential deletion of keys.

Compare snapshots

Implement feature to compare two or more snapshots, show differences of files + size

• Show added files/directories
• Show deleted files/directories
• Show modified files/directories

Use case: find a specific version of a file, compare directory structure of multiple snapshots, etc.

Implement Compression

This issue is a tracking issue for the purposes of tracking discussions and other issues/PRs related to the request for implementing compression.

The following issues/PRs are related to this topic (and may therefore be closed in favor of this one):

• PR #2441
• PR #3666

Define Command Line Structure

Define Command Line Parameter structure.

Sub commands:

backup
restore
show

Show parameters should work for restore

Delete Specific Snapshot

Function to delete Snapshots + related data (that is not needed by any other snapshot)

Find File in Snapshots

Find specific file or Directory in Snapshots

• limit number of snapshots
• limit date range
• return snapshot ID + Snapshot Information

Show number of changed files + file size

linked to #1 and #13

Implement HTTP Server/Client and backend

Add a custom HTTP server (REST) and add that a as a storage backend

Implement fuse file system for snapshots

Implement function to delete old snapshots according to a retention policy.

Idea: Use snapshot selectors from #1, remove snapshots, then run fsck with prune #50

Have a config like "keep hourly snapshots for the last two days, daily backups for the last four weeks, and weekly backups for everything that's older".

Vendor code

Use godep https://github.com/tools/godep to include all third-party library code into the khepri repository.

Implement fsck

Detect dangling blobs

Store ID of base snapshot

New Snapshot should have a reference to the old snapshot it is based upon (store ID of base snapshot in new snapshot metadata)

List all Snapshots

List following Parameters:

• creation date + timestamp (Default in Local time, Option to show Z-Time)
• Snapshot ID (shortened SHA 256, Option to show full SHA 256 ID)
• Source Hostname
• Source Directory

Create Arch Linux PKGBUILD

Key Management: Change Password

Delete Files from Existing Snapshot

In cases of accidential backup of e.g. too large files, I would like to be able to delete specific files or directories (incl. recursion) from existing snapshots

Select snapshots by various criteria

Implement command to list snapshots by the following criteria:

• Hostname
• Last X Snapshots
• Username
• Date / Time Range

Correct symlink bug with older kernels

Linked to #24
Integration test on wercker showed:
https://app.wercker.com/#build/546878b3c6bc8c046515bef0

testfiles: test/test-backup.sh
error for /tmp/khepri-testsuite-uavS1t/fake-data-restore/fake-data/0/tests/testfile-symlink: too many levels of symbolic links
error for /tmp/khepri-testsuite-uavS1t/fake-data-restore/fake-data/0/tests: too many levels of symbolic links
error for /tmp/khepri-testsuite-uavS1t/fake-data-restore/fake-data/0: too many levels of symbolic links
error for /tmp/khepri-testsuite-uavS1t/fake-data-restore/fake-data: too many levels of symbolic links
error executing command "restore": too many levels of symbolic links
run: test-backup failed!

Select scrypt parameter based on computing power and memory

Users with strong computing power will automatically get higher key derivation standards.

Add documentation to explain terminology

Add documentation that explains the terminology used in khepri, at least:

• backend
• repository
• blob, data, tree, snapshot, key

Support MS Windows

Implement Metadata Cache

Metadata should be cached locally for multiple operations.

• Feature to limit cache size

Make blobs read only

Set files with blobs read-only so that users do not accidentally delete blobs.