riotkit-org / br-backup-controller Goto Github PK

Project should be renamed from Bahub into Backup Controller to less confuse users. We have Backup Maker, so the components needs to be distencted from each other.

Backup Controller should have a possibility to generate JSON Web Tokens (that gives access to Backup Repository server) on-demand, when a backup is planned to be done.

Temporary JWT should be injected into Backup Maker environment. After successful upload or download the token would be deactivated by the server.

Currently it is marked as todo

Just like Docker Adapter, we need an adapter for Kubernetes.

There should be two options:

1. Calling exec on existing pod to backup it's contents
2. Creating a Kubernetes job that will perform a backup by accessing persistent volumes of the application. Bahub should observe the job realtime (synchronous) and collect logs, redirect into the stdout.

Architecture change

Bahub should no longer upload files into Backup Repository. Bahub should take care of scheduling the pod, and injecting a binary that would upload the backup by itself.

Why such architecture change? Reasons:

1. It was always a big pain to supervise the process remotely - passing through the data using pipes. New solution would reduce it to only logs collection in Bahub
2. The cluster can have multiple nodes in multiple regions, job's pod can be scheduled to be on the same node as application, or at least in the same zone, where the application data is stored to drastically reduce upload time

The architecture change should be applied to non-kubernetes adapters as well to allow us to fully move the upload & encryption code into a external-tiny application. Of course the GNUPG encryption would stay as it is using native gpg binary without creating own solutions.

The new "uploader" tiny application should be written in Go language, so it would be injected as a simple binary pre-compiled for supported architectures, without need to have Python installed.

Communication

The job should be scheduled by Bahub with following input parameters:

• Temporary token that allows to upload successfully 1 file within X time (in case the job will restart we need to have this token still valid)
• If upload should be streamed directly, or saved to disk at first
• Command to execute
• GPG Public key used to encrypt the file (or a private key if we are decrypting)
• Future: An information that the backup is paired with other backups and as a group it consists a complete application state

Todo

• Backup
• Restore
• Download and save into file
• GPG support (encryption + decryption) with specyfing a key. Temporary GPG directory should be created and keys should be imported
• Environment variables support to use instead of commandline switches optionally
• Unit tests
• Use backup-maker in Bahub
• Documentation
• Examples how to run in Kubernetes
• #3
• #4
• Rename Bahub into Backup Controller