Comments (9)
There's also reflinking of files (for some filesystems, such as btrfs). I think this is one more reason why no hard-/ref-linking should happen by default (because it's not clear what should be preferred), but rather only if a per-alternative config option is set. In addition, hardlinking by default changes the behaviour when the files in the alternative collection are modified (e.g. by a player writing rating tags): Currently, this will not affect files in the main beets library.
The cp
command has a --reflink=[never|auto|always]
flag, maybe the hardlink option could also take these values, with the same meaning.
Try to hardlink whenever a file is copied and fall back to copying on errors. This may be ineffecient when copying many files.
I doubt that this would really the bottleneck when updating alternatives (of course, I haven't measured it). I suppose that the system call to hardlink fails rather quickly if it is not supported by the filesystem.
from beets-alternatives.
Symlinks could be a solution, but might be unstable if the main library is modified. Symlinks would break if a file is moved to a different path or deleted, requiring an alt update
to repair. Hardlinks would still point to the same data, even if the main library is changed. For my use case, hardlinks would work better, but I can see why it might be troublesome.
What seems best to me would be to make it an option, so the user could pick from copy
, link
/symlink
, hardlink
or reflink
. Keeping the default value as copy
seems the most sane, but gives the user the ability to pick whatever option works best for them.
from beets-alternatives.
I'd say, all of symlink
, hardlink
, reflink
could be implemented (or one for now, adding others as requested), everyone could then choose his or her preferred method. For example, we could add the options
alt:
phone:
# ...
link: [never|auto|always]
linktype: [hardlink|symlink|reflink]
If I understand @nichobi correctly, symlinks might be somewhat inconvenient, because they'd require special care when copying to the other devices, e.g. cp --dereference
or rsync --copy-links
since a simple cp
or rsync
would copy the link.
Symlinks would break if a file is moved to a different path or deleted, requiring an alt update to repair.
On the other hand, the old, hardlinked/copied files might contain stale (meta)data. I don't think the validity of an alternative collection after changes to the beets database and before the next alt update
is something we should care too much about.
from beets-alternatives.
I'd say, all of
symlink
,hardlink
,reflink
could be implemented (or one for now, adding others as requested), everyone could then choose his or her preferred method.
This seems to be the right approach given that every option has their own benefits and drawbacks and the user probably knows best what they want.
For configuring this I’d condense your approach @wisp3rwind: We would just provide one link
option per alternative with values false
, hardlink
, symlink
, and reflink
. I don’t see why need an auto
option.
from beets-alternatives.
For configuring this I’d condense your approach @wisp3rwind: We would just provide one
link
option per alternative with valuesfalse
,hardlink
,symlink
, andreflink
. I don’t see why need anauto
option.
In that case, I think alt update
should abort if a hardlink/symlink/reflink fails. Otherwise, with a silent fallback to copying, it's somewhat hard to verify that you've configured the alternative in a way that is supported on your filesystem.
from beets-alternatives.
Seems link a good idea. os.link()
is what we would need to use.
If you feel like, please open a PR, @nichobi. I’m happy to help with any questions.
from beets-alternatives.
I had a look around and found a hardlink() function in beets util that simplifies implementation. What I'm unsure of is how to determine when to hardlink vs copy. Some options I've considered:
- Somehow check whether files are on the same filesystem and hardlinks are supported. Have not found any easy way to check this.
- Add a config option (per alternatives directory). What should be done if hardlink is set to true but fails?
- Try to hardlink whenever a file is copied and fall back to copying on errors. This may be ineffecient when copying many files.
- Attempt a hardlink for the first file or a test file, use the result to determine whether to hardlink or copy remaining files. What if our test files succeeds but future files fail?
Do you have any opinion on which to go with? I'm happy to open a PR, just not sure in what manner to start working.
from beets-alternatives.
I wasn’t aware of beets.util.hardlink()
. Makes sense to use it.
Thanks for the analysis on the different approaches. I think the last one makes the most sense. We could also add a config option that disables this behavior, for example if the user knows that the collection is on a different filesystem. But this is something we can always add later.
from beets-alternatives.
In addition, hardlinking by default changes the behaviour when the files in the alternative collection are modified (e.g. by a player writing rating tags): Currently, this will not affect files in the main beets library.
Excellent point! This is indeed a good reason not to use hardlinking by default. In general it makes me wonder whether hardlinking is a good idea. Reflinking is a lot better but also less widely supported. (It’s only available on some file systems and not in the Python stdlib yet although there is a package for it.)
An alternative solution for your use case, @nichobi, would be to enable symlinks alongside transcoding. Basically we would add a flag that would symlink files instead of copying them if they don’t need to be transcoded. This is different from format: link
where all files are symlinked by default. Would this be acceptable @nichobi?
from beets-alternatives.
Related Issues (20)
- Respect convert's threads config HOT 3
- Check for existing files / Invoke convert's convert_item HOT 2
- Copy album art to destination folder (without embedding) HOT 2
- Create m3u for newly imported tracks HOT 2
- beet alt update doesn't always remove obsolete links HOT 3
- Remove external items when the corresponding item is removed from beets
- Update performance for SymlinkViews
- `beet alt update foo`: FileNotFoundError HOT 1
- Crashes when the destination file exists, but wasn't created by this plugin HOT 1
- Symlink artwork in SymlinkViews?
- Drop Python 2.7 support HOT 3
- Disable album art embedding HOT 3
- Can't upgrade to 0.10.2 with pip HOT 2
- File exists during link of paths using symlink mode HOT 5
- Failure on case insensitive filesystem when case changes HOT 2
- Encoded files in alt are encoded again during `alt update` HOT 7
- custom paths with inline plugin broken? HOT 2
- ebmedart doesn't seem to work HOT 5
- Every album re-copied on each update HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from beets-alternatives.