fmoledina / snapraid-btrfs-runner Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
nvm, already figured it out.
Hope you don't mind me asking another question..
Before using your script, my sync & cleanup commands were part of a nightly cleanup bash script, run via cron (non-root). It contained this:
# Create a temp file to indicate maintenance is running
touch /tmp/maintenance-is-running
(perform actions)
# Delete temp file, follow up tasks can continue
rm /tmp/maintenance-is-running
This way, my next nightly scheduled cronjob (bash script creating backups, requires root) runs about an hour later via root cron, but waits for that temporary file to be deleted. This way, I made sure snapraid was finished before performing other disk intensive duties:
# Make sure maintenance is finished before performing backups via btrbk
while [[ -f /tmp/maintenance-is-running ]] ; do
sleep 10 ;
done
Now it seems silly to use a bash script to run your script, but now my root backup script can no longer detect if snapraid is running. Is there a way for me to simply figure out if your script is finished, before performing other tasks?
; specify snapper-configs and/or snapper-configs-file as specified in snapraid-btrfs; only one instance of each can be specified in this config
snapper-configs =
snapper-configs-file =
It is unclear to me why the script needs this.
Snapraid-btrfs finds the configs in /etc/snapper/configs. I have 3 configs there representing subvolumes on 3 disks.
Just 2 more questions if you don't mind! Thanks for the help so far:
sync
and cleanup
. Now, since I run your script, I notice in the Snapper-GUI, there is also a diff
type snapshot created. Those diffs are not cleaned up with your runner script.exclude /.snapshots/
but I still get these warnings with snapraid-btrfs and with your script:[OUTERR] WARNING! Ignoring mount point '/mnt/disks/cache/Users/.snapshots/7/snapshot/.snapshots' because it appears to be in a different device
I understand it cannot harm since the actual snapshots within are subvolumes themselves so they are excluded. But why the message? It is driving me crazy. Note my disks are actually mounted at /mnt/disk/diskname but each contains 3 subvolumes, I am only covering one called Users. My snapper config files contain the exact same path as in the snapraid.conf (/mnt/disks/cache/Users for example):
# Defines the data disks to use
# The name and mount point association is relevant for parity, do not change it
# WARNING: Adding here your /home, /var or /tmp disks is NOT a good idea!
# SnapRAID is better suited for files that rarely changes!
# Format: "data DISK_NAME DISK_MOUNT_POINT"
data d0 /mnt/disks/cache/Users
data d1 /mnt/disks/data1/Users
data d2 /mnt/disks/data2/Users
# Excludes hidden files and directories (uncomment to enable).
#nohidden
# Defines files and directories to exclude
# Defines files and directories to exclude
# Remember that all the paths are relative at the mount points
# Format: "exclude FILE"
# Format: "exclude DIR/"
# Format: "exclude /PATH/FILE"
# Format: "exclude /PATH/DIR/"
exclude /.snapshots/ # snapraid-btrfs (snapper) snapshot folder
exclude /.snapraid/ # snapraid content subvolume folder
exclude /.filerun.trash/ # filerun trash folder
exclude /.stversions/ # Syncthing deleted files (by peer, synced to this device)
exclude /incoming/incomplete/ # temp torrent download subvolume folder
exclude *.unrecoverable
exclude /tmp/
exclude /lost+found/
exclude .tmp/
exclude .cache/
exclude /lost+found/
exclude .AppleDouble
exclude ._AppleDouble
exclude .DS_Store
exclude .Thumbs.db
exclude .fseventsd
exclude .Spotlight-V100
exclude .TemporaryItems
exclude .Trashes
exclude .AppleDB
exclude .err
exclude .bak
exclude .dmp
exclude .dump
exclude .old
exclude .tmp
exclude .temp
Can we use shutil.which
instead of os.path.isfile
for detecting executables?
This has the advantage of looking through the PATH so that we don't need absolute paths but works either way.
If snapper
, snapraid-btrfs
, and snapraid
are in PATH, these would no longer need to be changed in the config.
Hi, I am using your script for my project https://github.com/TheLinuxGuy/free-unraid and I am looking to validate that the cleanup function is working as intended.
My understanding is that if cleanup = true
upon successful execution of the script that there should be no btrfs snapshot volumes on disk.
This doesn't seem to be what I am seeing, aren't these left over from previous runs?
root@nas:/mnt# btrfs subvolume list /mnt/disk1/
ID 256 gen 5076 top level 5 path data
ID 258 gen 5078 top level 5 path content
ID 7617 gen 5077 top level 256 path .snapshots
ID 7620 gen 4291 top level 7617 path .snapshots/3/snapshot
ID 7621 gen 4296 top level 7617 path .snapshots/4/snapshot
ID 8842 gen 5072 top level 7617 path .snapshots/1/snapshot
ID 8843 gen 5073 top level 7617 path .snapshots/2/snapshot
ID 8844 gen 5074 top level 7617 path .snapshots/5/snapshot
ID 8845 gen 5076 top level 7617 path .snapshots/6/snapshot
root@nas:/mnt# btrfs subvolume list /mnt/disk2/
ID 257 gen 5485 top level 5 path data
ID 258 gen 5487 top level 5 path content
ID 15158 gen 5486 top level 257 path .snapshots
ID 15161 gen 4127 top level 15158 path .snapshots/3/snapshot
ID 15162 gen 4167 top level 15158 path .snapshots/4/snapshot
ID 19034 gen 5481 top level 15158 path .snapshots/5/snapshot
ID 19035 gen 5482 top level 15158 path .snapshots/6/snapshot
ID 19036 gen 5483 top level 15158 path .snapshots/7/snapshot
ID 19037 gen 5485 top level 15158 path .snapshots/8/snapshot
I think I would expect there to be no .snapshots/XXX/snapshot
in the above output if the script was working as intended, since only a single (or none) .snapshots
should exist?
I would like to suggest checking data volumes for bit rot or otherwise corruption using btrfs scrub before updating the snapraid parity. Cheers.
Not sure how/if this is possible, but it would be a nice to have to only sent email if there is anything to report.
If no email is received, it means it ran well.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.