I write code, I write articles, I wrote a book (I like TensorFlow).
It looks like I write all day ๐ค
An easy-to-use backup tool designed for servers.
License: Apache License 2.0
I write code, I write articles, I wrote a book (I like TensorFlow).
It looks like I write all day ๐ค
This is somehow a must-have feature.
Especially if we are using some free-tier remote that has limited storage
The logging is still poor, need to add an info message when:
I have to start from somewhere:
Google Drive support as a remote would be nice!
The support for Google Cloud storage is nice to have.
The new type should implement the Uploader
trait and behave similarly to AWSBucket.
Currently, the code is not well documented, any kind of documentation is gold.
Let's say we have a containerized service made in this way:
All these services are inside a docker container, and they are configurable through configuration files often mounted inside a volume.
Doing the backup of the configuration files is easy: we can just use the "folders" service.
But what about the data of the different databases? We could use the folders service and copy and archive all of them, but if the database is in use (and it's often the case), there's a non-zero chance of copying some file that's in a nonstable state.
Hence, the possibility of invoking the correct commands on the container should be provided, in order to ask the running DB to do the dump, store it inside the volume (accessible from the host), and then back up the correctly created dump.
An interface for being compatible with docker should be defined.
I want to start this project using Rust, so I have a good excuse to learn the language by having a project to complete.
Backing up a whole VPS is stupid, backing up only the data might be a good idea, but the data we want to backup depends on the services hosted on the VPS; the problem is not trivial.
The obvious way of using rsync + systemd timers or whatever just bores me, and it's old and not trivially configurable (since everything depends on the services we want to backup).
What the user wants. There might be thousands of different scenarios, like:
pg_dumpall
periodically and backup this dump.There are TONS of use cases, and every use case is completely user-dependent (e.g. I like to configure my system as I want, and my systems are almost completely customized. All the standard backup tools just don't work for me).
It depends. Do you like the cloud? I don't. But there's some good cheap storage on "the cloud", so you might be interested in backing up your stuff there.
The self-hosted server can be trivial or can be something nice, a bacup client and a bacup server might be nice to have.
It should be trivial.
Every time I configure a new server, I should just:
The configuration part should be trivial. I should be able to:
foo:
cloud: aws
address: whatever
authentication: the_token
dbname:
type: postgres
what: cluster
when: daily
incremental: true
where: foo
and if PostgreSQL is in some well-known location, then the pg_dump_all -f filename.sql
can be executed every day, to create the backup file and once created upload it to the configured cloud provider.
With the same simplicity I should be able to backup: files (invoking rsync in the background if possible, why not), folders, and invoke the correct backup tool for the specified service (if it has it, like PostgreSQL).
So this is more or less the idea.
Any comment/suggestion is welcome.
The design of the software architecture will be in a different issue - this issue is only to brainstorm about the idea, the simplicity of usage, how it should work from the user point of view (yeah it's like a user-story, although I don't "love" all these agile things).
Since this tool is designed for servers, a folder misc/system
(or other scripts) that contains a service file would be really handy.
It's a very strange way of doing backup, but it's something I've already seen (and done, shame on me).
If a local folder is a git repository, then we can basically do:
git checkout <backup-branch> # optional?
git add . -A
git commit -m 'backup'
git push origin
Using SSH is mandatory, hence we can re-use the private key control part of the SSH remote + adding the which(git)
and be sure that git
is installed in the system, and create a new type that implements the Uploader
trait and it behaves similarly to the SSH service.
The support for backing up compressed files and compressed folders can be added too, although it's horrific loading archives on Git (especially for backing up). But anyway, it's a feature we can offer just in case.
on the latest stable rust version:
error[E0308]: mismatched types
--> /home/debian/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-cron-scheduler-0.7.6/src/job/mod.rs:346:41
|
346 | .checked_add_signed(time::Duration::seconds(
| ______________________------------------_^
| | |
| | arguments to this method are incorrect
347 | | instant.duration_since(Instant::now()).as_secs() as i64,
348 | | ))
| |_____________________^ expected `TimeDelta`, found `Duration`
|
note: method defined here
--> /home/debian/.cargo/registry/src/index.crates.io-6f17d22bba15001f/chrono-0.4.35/src/datetime/mod.rs:442:12
|
442 | pub fn checked_add_signed(self, rhs: TimeDelta) -> Option<DateTime<Tz>> {
| ^^^^^^^^^^^^^^^^^^
error[E0308]: mismatched types
--> /home/debian/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-cron-scheduler-0.7.6/src/job/mod.rs:444:41
|
444 | .checked_add_signed(time::Duration::seconds(duration.as_secs() as i64))
| ------------------ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `TimeDelta`, found `Duration`
| |
| arguments to this method are incorrect
|
note: method defined here
--> /home/debian/.cargo/registry/src/index.crates.io-6f17d22bba15001f/chrono-0.4.35/src/datetime/mod.rs:442:12
|
442 | pub fn checked_add_signed(self, rhs: TimeDelta) -> Option<DateTime<Tz>> {
| ^^^^^^^^^^^^^^^^^^
error[E0308]: mismatched types
--> /home/debian/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-cron-scheduler-0.7.6/src/job/mod.rs:568:62
|
568 | .and_then(|nt| nt.checked_add_signed(time::Duration::seconds(r as i64)))
| ------------------ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `TimeDelta`, found `Duration`
| |
| arguments to this method are incorrect
|
note: method defined here
--> /home/debian/.cargo/registry/src/index.crates.io-6f17d22bba15001f/chrono-0.4.35/src/datetime/mod.rs:442:12
|
442 | pub fn checked_add_signed(self, rhs: TimeDelta) -> Option<DateTime<Tz>> {
| ^^^^^^^^^^^^^^^^^^
error[E0308]: mismatched types
--> /home/debian/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-cron-scheduler-0.7.6/src/scheduler.rs:167:67
|
167 | ... nt.checked_add_signed(time::Duration::seconds(r as i64))
| ------------------ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `TimeDelta`, found `Duration`
| |
| arguments to this method are incorrect
|
note: method defined here
--> /home/debian/.cargo/registry/src/index.crates.io-6f17d22bba15001f/chrono-0.4.35/src/datetime/mod.rs:442:12
|
442 | pub fn checked_add_signed(self, rhs: TimeDelta) -> Option<DateTime<Tz>> {
| ^^^^^^^^^^^^^^^^^^
For more information about this error, try `rustc --explain E0308`.
error: could not compile `tokio-cron-scheduler` (lib) due to 4 previous errors
warning: build failed, waiting for other jobs to finish...
Another very common database used server-side is MariaDB/MySQL.
Similar to the PostgreSQL type, this new type must implement the Service
trait and only depend on the system commands for doing the dump. Do not depend on libraries DB specific.
We have some tests, but we don't run them in the CI.
A TravisCI or something new like GitHub actions must be implemented.
NOTE: some tests require third-party services to be running in the background (e.g. testing the dump of a database, requires the database running and the possibility of calling the DB utils from CLI).
Perhaps I have already configured some network filesystem and I have a working mount point (sshfs, nfs, samba, whatever), everything bacup has to do is just copy the files in the correct local location.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.