Comments (6)
For plain TAR files you could read in the metadata and memory map the files as they are in the archive, they are back to back. As for using stdin or compressed archives it might be possible to just allocate anonymous maps as you go.
I understand that these are potentially quite invasive changes to make, I'm just trying to float some ideas.
from dwarfs.
Hi and thanks for your suggestion.
I believe something like this came up already in an earlier discussion, so you're not alone. :)
I agree this would be a nice feature, however it's definitely not trivial to implement. At the moment, mkdwarfs
likes to memory-map files for basically any input operation, which makes it really hard to use anything other than "real" files for input.
I'll keep this open, but please don't expect any progress anytime soon.
from dwarfs.
Yeah, plain TAR would probably be easy enough, but usually you have .tar.{xz,bz2,gz}
and it'd be hard to argue to support one but not the other. Anonymous maps would work up to a point, but probably not if we're talking tens of gigabytes of data or more. Also, unless you hold on to these mappings, you'd need to be able to rewind (hard with stdin), as mkdwarfs
usually needs more than one pass over the input data.
from dwarfs.
I understand that it isn't super simple, at the minimum plain old TAR would be nice because that alone would eliminate the need for root in some edge cases like I described.
from dwarfs.
I assume the easiest way to shoehorn this into mkdwarfs
would be to provide an os_access
implementation on top of a plain TAR file. I'm also leaving this gist here for reference.
from dwarfs.
I'm still not sure whether it should be done, but at least this would seem like a solution that wouldn't increase the complexity of the code base too much.
from dwarfs.
Related Issues (20)
- Add integrity check from dwarfsextract to mount as well HOT 8
- UIDs/GIDs truncated to 16 bits HOT 6
- Command line option (for windows) to set a folder where writable files go [] HOT 10
- how do I use you software? HOT 3
- Consider supporting zstd dictionaries HOT 4
- Trouble compiling: 'utf8cpp/utf8.h' file not found HOT 6
- dwarfs 0.7.3 doesn't use section index properly HOT 3
- error initializing file system: unsupported compression type: ZSTD HOT 8
- Issues when building into different architectures and on musl HOT 3
- Kernel level implementation HOT 1
- Build fails when tests enabled HOT 14
- Invalid UTF-8 (Core Dump) HOT 4
- Ability to view files in the archive HOT 8
- null_decompressor not implemented (core dumped) HOT 3
- Ricepp tries to add nonexistent googletest directory even if -DPREFER_SYSTEM_GTEST=ON is set HOT 2
- failed opening file: The system cannot find the file specified. (windows) HOT 3
- Idea: LZAV compression algorithm HOT 2
- Unable to detect duplicate data between game savefiles of 30Mb each HOT 2
- fails to build with google-glog 0.7.0 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dwarfs.