Coder Social home page Coder Social logo

snapraid-btrfs-runner's People

Contributors

dli7319 avatar fmoledina avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

snapraid-btrfs-runner's Issues

How to check if runner is still running?

Hope you don't mind me asking another question..
Before using your script, my sync & cleanup commands were part of a nightly cleanup bash script, run via cron (non-root). It contained this:

# Create a temp file to indicate maintenance is running
touch /tmp/maintenance-is-running

(perform actions)
# Delete temp file, follow up tasks can continue
rm /tmp/maintenance-is-running

This way, my next nightly scheduled cronjob (bash script creating backups, requires root) runs about an hour later via root cron, but waits for that temporary file to be deleted. This way, I made sure snapraid was finished before performing other disk intensive duties:

# Make sure maintenance is finished before performing backups via btrbk
while [[ -f /tmp/maintenance-is-running ]] ; do
   sleep 10 ;
done

Now it seems silly to use a bash script to run your script, but now my root backup script can no longer detect if snapraid is running. Is there a way for me to simply figure out if your script is finished, before performing other tasks?

Why specify snapper configs? snapraid-btrfs finds them automatically

; specify snapper-configs and/or snapper-configs-file as specified in snapraid-btrfs; only one instance of each can be specified in this config
snapper-configs = 
snapper-configs-file =

It is unclear to me why the script needs this.
Snapraid-btrfs finds the configs in /etc/snapper/configs. I have 3 configs there representing subvolumes on 3 disks.

2 more questions.. diffs are not cleaned up and /.snapshots/ exclude still shows warnings

Just 2 more questions if you don't mind! Thanks for the help so far:

  1. I don't use Snapper for anything else but snapraid-btrfs (I use btrbk for backups). I only used sync and cleanup. Now, since I run your script, I notice in the Snapper-GUI, there is also a diff type snapshot created. Those diffs are not cleaned up with your runner script.
    I ran the script 4 times to test the email server, and now I have 4 diff snapshots for each disk, even though cleanup=true.
    Is this expected behaviour?
  2. In my snapraid.conf I have exclude /.snapshots/ but I still get these warnings with snapraid-btrfs and with your script:
[OUTERR] WARNING! Ignoring mount point '/mnt/disks/cache/Users/.snapshots/7/snapshot/.snapshots' because it appears to be in a different device

I understand it cannot harm since the actual snapshots within are subvolumes themselves so they are excluded. But why the message? It is driving me crazy. Note my disks are actually mounted at /mnt/disk/diskname but each contains 3 subvolumes, I am only covering one called Users. My snapper config files contain the exact same path as in the snapraid.conf (/mnt/disks/cache/Users for example):

# Defines the data disks to use
# The name and mount point association is relevant for parity, do not change it
# WARNING: Adding here your /home, /var or /tmp disks is NOT a good idea!
# SnapRAID is better suited for files that rarely changes!
# Format: "data DISK_NAME DISK_MOUNT_POINT"
data d0 /mnt/disks/cache/Users
data d1 /mnt/disks/data1/Users
data d2 /mnt/disks/data2/Users

# Excludes hidden files and directories (uncomment to enable).
#nohidden

# Defines files and directories to exclude
# Defines files and directories to exclude
# Remember that all the paths are relative at the mount points
# Format: "exclude FILE"
# Format: "exclude DIR/"
# Format: "exclude /PATH/FILE"
# Format: "exclude /PATH/DIR/"  
exclude /.snapshots/             # snapraid-btrfs (snapper) snapshot folder
exclude /.snapraid/              # snapraid content subvolume folder
exclude /.filerun.trash/         # filerun trash folder
exclude /.stversions/            # Syncthing deleted files (by peer, synced to this device)
exclude /incoming/incomplete/    # temp torrent download subvolume folder
exclude *.unrecoverable
exclude /tmp/
exclude /lost+found/
exclude .tmp/
exclude .cache/
exclude /lost+found/
exclude .AppleDouble
exclude ._AppleDouble
exclude .DS_Store
exclude .Thumbs.db
exclude .fseventsd
exclude .Spotlight-V100
exclude .TemporaryItems
exclude .Trashes
exclude .AppleDB
exclude .err
exclude .bak
exclude .dmp
exclude .dump
exclude .old
exclude .tmp
exclude .temp

Detecting executables

Can we use shutil.which instead of os.path.isfile for detecting executables?
This has the advantage of looking through the PATH so that we don't need absolute paths but works either way.
If snapper, snapraid-btrfs, and snapraid are in PATH, these would no longer need to be changed in the config.

How to validate 'cleanup' is working as intended?

Hi, I am using your script for my project https://github.com/TheLinuxGuy/free-unraid and I am looking to validate that the cleanup function is working as intended.

My understanding is that if cleanup = true upon successful execution of the script that there should be no btrfs snapshot volumes on disk.

This doesn't seem to be what I am seeing, aren't these left over from previous runs?

root@nas:/mnt# btrfs subvolume list /mnt/disk1/
ID 256 gen 5076 top level 5 path data
ID 258 gen 5078 top level 5 path content
ID 7617 gen 5077 top level 256 path .snapshots
ID 7620 gen 4291 top level 7617 path .snapshots/3/snapshot
ID 7621 gen 4296 top level 7617 path .snapshots/4/snapshot
ID 8842 gen 5072 top level 7617 path .snapshots/1/snapshot
ID 8843 gen 5073 top level 7617 path .snapshots/2/snapshot
ID 8844 gen 5074 top level 7617 path .snapshots/5/snapshot
ID 8845 gen 5076 top level 7617 path .snapshots/6/snapshot
root@nas:/mnt# btrfs subvolume list /mnt/disk2/
ID 257 gen 5485 top level 5 path data
ID 258 gen 5487 top level 5 path content
ID 15158 gen 5486 top level 257 path .snapshots
ID 15161 gen 4127 top level 15158 path .snapshots/3/snapshot
ID 15162 gen 4167 top level 15158 path .snapshots/4/snapshot
ID 19034 gen 5481 top level 15158 path .snapshots/5/snapshot
ID 19035 gen 5482 top level 15158 path .snapshots/6/snapshot
ID 19036 gen 5483 top level 15158 path .snapshots/7/snapshot
ID 19037 gen 5485 top level 15158 path .snapshots/8/snapshot

I think I would expect there to be no .snapshots/XXX/snapshot in the above output if the script was working as intended, since only a single (or none) .snapshots should exist?

btrfs scrub pre snapraid sync

I would like to suggest checking data volumes for bit rot or otherwise corruption using btrfs scrub before updating the snapraid parity. Cheers.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.