azure / azfilebackup

Backup tool for VM file systems to Azure Blob Storage.

Filesystem backup utility for Azure Blob Storage

This Python-based tool uses tar to perform backups of a file system and upload the archive to Azure Blob Storage. In most cases, you should first evaluate Azure Backup which is a fully-managed backup service for virtual machines running both in Azure and on-premises. If for any reason Azure Backup is not applicable in your environment, this tool might be useful to you.

Current status: beta

Features and limitations

This tool requires Python 2.7 and should work on any Linux distribution. It was tested on SUSE Enterprise Linux 12.

Features:

• Perform full backups
• Can be automated via cron
• Configure file system to backup and files to exclude
• Storage account to use is determined by an instance tag
• Storage container to use is determined by configuration file
• Backup schedule is defined using instance tags

Requirements:

• Azure Virtual Machine: this tool is designed to run on an Azure virtual machine. It uses the Azure environment, like tags in the instance metadata, to determine how it should run. This allows you to control most of the behaviour of the tool through environment changes, instead of having to log on to the virtual machine to edit a configuration file.

• Azure Blob Storage: this tool is designed to upload the backup archives to an Azure Blob Storage account. It uses managed identities for authentication, which means that storage credentials don't need to be saved on the machine.

Installation using pip

Dependency to psutil

The tool has a dependency to the psutils Python library. This library only provides a source distribution, which means that it must be compiled on the target machine. If the target machine does not have development tools installed, the installation of the psutil library will fail. Please install the development tools before installing using the pip method.

Using virtualenv

The less intrusive way to install and test the tool is to use virtualenv. Here is how you can create and activate a virtual environment:

pip install --user virtualenv
~/.local/bin/virtualenv --python=python2.7 ~/azfilebak
source ~/azfilebak/bin/activate

Install the tool

You can install directly from a release URL:

pip install https://github.com/Azure/azfilebackup/releases/download/v1.0-alpha1/azfilebak-1.0b1.tar.gz

Or if you downloaded the distribution archive locally:

pip install azfilebak-1.0b1.tar.gz

Configuration

Storage account and managed identity

You will need an Azure Storage account to store the archives, and you will need to configure a system-assigned managed identity to authorize the virtual machine to access the storage account. You can follow this tutorial to set this up: Use a Linux VM system-assigned managed identity to access Azure Storage.

Tags

Tags are used to govern the backup schedule and configure certain parameters. This allows controlling the backup process without having to modify the configuration file on the machine. The files test-set-vm-tags-arm.json and test-set-vm-tags.sh how tags should be defined.

Usage

If the backup configuration file is not in the default location (/usr/sap/backup/backup.conf), use -c to specify an alternate location:

sudo azfilebak -c ./test_backup.conf -x

Run a full backup now, disregarding the schedule constraints:

sudo azfilebak --full --force

List existing backups:

sudo azfilebak --list

Download a backup archive (it will be saved to /tmp by default):

azfilebak --restore fs_test-backup_full_20181122_094011.tar.gz

Stream the contents of the archive to a local tar command, so you can dreictly list or extract files:

azfilebak --restore fs_test-backup_full_20181122_094011.tar.gz --stream | tar tvzf -

Development

The tool requires Python 2.7.

Install requirements:

pip install -r requirements.txt

Run tests:

make test

Tests

Some of the tests need to access a real Azure storage account. The name of the account to use can be changed in the file sample_instance_metadata.json:

"tags": "StorageAccount:sahec99az1backup0001;

The storage account key can be specified via an environment variable:

export STORAGE_KEY='xxx'

The tool requires the GNU version of the tar command. If you are developing and running the tests in a non-GNU environment (e.g. macOS), you can install GNU tar. On macOS for example, you can use Homebrew: brew install gnu-tar.

Packaging

The release directory contains instructions and a Dockerfile that are used to generate an RPM file suitabled for deployment on a system without impact on existing Python installations.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

azfilebackup's Issues

Script should sanity check the backup size before running tar

If the total backup file size (.tar.gz archive) exceeds the maximum uncommitted block count for an Azure Blob (e.g. 100,000) then this will cause an error (e.g. BlockCountExceedsLimit) that will interrupt the backup process, possibly after long hours of upload. Possibly a similar error would be raised at the end of the upload if the backup size exceeds the maximum size for Azure blob (e.g. 50,000 times the block size).

The script should make an attempt at evaluating the backup size before running tar, and warn or fail if that size exceeds the maximum Blob size.

Notification messages do not contain proper UTC epoch values

backup file name

test

tar command hangs on some machines

stderr is redirected to pipe, but the parent process never reads the stream. On some machines the tar command hangs after printing large numbers of errors to stderr.

https://github.com/Azure/azfilebackup/blob/master/azfilebak/executableconnector.py#L62

Proposed fix: do not redirect stderr (delete the line).

The script does not handle the tar return code

In some cases, the tar command will fail for some reason, but the script does not look at the return code from the command execution. In this case the problem is not properly detected (i.e. error log + failure notification).

Exclude /mnt/resource from backup

OSError if the notification command does not exist

After a backup successfully completes, if the default notification command does not exist on the system, the tool fails with the following error:

OSError: [Errno 2] No such file or directory

tar return code 1 should not be considered a failure

tar return code 1 means that "some files were changed while being archived and so the resulting archive does not contain the exact copy of the file set". This should not be considered a failure but a warning condition, since this is a situation that can happen frequently on active production systems.

Backups are limited to 200GB

Because of the MAX_BLOCK_SIZE constant:

https://github.com/Azure/azure-storage-python/blob/master/azure-storage-blob/azure/storage/blob/blockblobservice.py#L84-L94

Add --sparse option to tar command

Script fails with KeyError: 'bkp_storage_account'

if the TAG 'bkp_storage_account' does not exist,
script musts report this fact somehow and compose the storage-account name
with known naming-convention.

But it throws an error message and exit:

hec45v101770:/usr/sap/backup # /usr/share/python/bin/azfilebak -x
Traceback (most recent call last):
  File "/usr/share/python/bin/azfilebak", line 11, in <module>
    sys.exit(main())
  File "/usr/share/python/lib/python2.7/site-packages/azfilebak/__main__.py", line 15, in main
    Runner.main()
  File "/usr/share/python/lib/python2.7/site-packages/azfilebak/runner.py", line 173, in main
    for line in backup_agent.get_configuration_printable(output_dir=output_dir):
  File "/usr/share/python/lib/python2.7/site-packages/azfilebak/backupagent.py", line 414, in get_configuration_printable
    "azure_storage_account_name:         {}".format(self.backup_configuration.get_azure_storage_account_name())
  File "/usr/share/python/lib/python2.7/site-packages/azfilebak/backupconfiguration.py", line 173, in get_azure_storage_account_name
    account = self.instance_metadata.get_tags()['bkp_storage_account']
KeyError: 'bkp_storage_account'

RPM package is not signed

When installing with zypper:

Package is not signed!

azfilebackup-1.0-0.pre.a2.x86_64 (Plain RPM files cache): Signature verification failed [6-File is unsigned]
Abort, retry, ignore? [a/r/i] (a):

You have to select the ignore option to install the package.

"timestamp-bkp-begin": "" - must not be empty

Please check value "timestamp-bkp-begin": "". It is always empty but required value for our message processing. Every backup should have a start time, so why it is empty? Could you please correct that?

Thanks
Robert