Comments (4)
Ah, you cut after compression, not before.
In this case, this is more like :
cat *.zst* | zstd -d > dest.tar
Same idea here, this only works if expansion of *.zst*
is properly ordered.
from zstd.
That worked! You rock and thank you! Now, to simplify the command and also untar 😄
For others on the internet, the one liner I made use of to tar and compress with zstd.
I backed up ~120GB SSD onto a FAT32 1.5TB disk drive in less than 15 minutes with default compression algorithms 🎉
Thank you zstd! 🙌
Compress / Decompress Huge Files via zstd with PROGRESS indication via PV
The required packages are:
- tar
- pv
- zstd
- dosfsck
Check the filesize in chunks for the FAT32 drive
sudo dosfsck -v -n /dev/sda1
File size is 4294967295 bytes, cluster chain length is 0 bytes.
Split the tar into files to transport between drives e.g. FAT32 formatted drive and filter/compress with zstd with -I option into another drive, while visualizing a progress counter with pv.
tar -I zstd -cf - Desktop/ | (pv -p --timer --rate --bytes | split --bytes=4294967295 - /run/media/username/Backups/System/Linux/Desktop/Desktop.backup.tar.zst)
Decompress multiple zst files between drives
cat *.zst* | zstd -d > /home/username/dest.tar
Then extract the archive
tar -xvf dest.tar
or extract the archive into specific directory
tar -xvf dest.tar -C /opt/files
from zstd.
The *.zst
suffix requirement is because the CLI must determine a destination filename.
If it doesn't need to generate such a name, the *.zst
requirement is dropped.
So one way to achieve your objective could be :
zstd -d *.zst* -c > dest.tar
This however implies that *zst*
shell expansion is properly ordered, which I'm not sure of.
There are probably more elaborated way to generate this list of files that would guarantee proper alphabetical order.
But you get the idea for the zstd
part.
from zstd.
I tried that and received and error, but the file size seems to have added another file; each zst is 4.3GB and the dest.tar is now 8.2GB..or maybe that is the actual size uncompressed
:
zstd -d *.zst* -c > /home/username/dest.tar
top.backup.tar.zstaa : Read error (39) : premature end
zstd: Desktop.backup.tar.zstab: unsupported format
zstd: Desktop.backup.tar.zstac: unsupported format
zstd: Desktop.backup.tar.zstad: unsupported format
zstd: Desktop.backup.tar.zstae: unsupported format
zstd: Desktop.backup.tar.zstaf: unsupported format
zstd: Desktop.backup.tar.zstag: unsupported format
Also,
*** Zstandard CLI (64-bit) v1.5.5, by Yann Collet ***
from zstd.
Related Issues (20)
- Should zstd delete incomplete archives? HOT 5
- 32-bit x86 build failure with 1.5.6 HOT 3
- v1.5.6 breaks 32-bit Windows clang-cl build HOT 3
- Replication of bug #3517 HOT 29
- Separate dictionary references to enable dictionary usage for any combination of window size and content size HOT 1
- Decompression speed regression in zstd 1.5.6 (win)
- Embed hash of raw dictionary in compressed resource (optionally) HOT 4
- Decompression crash after upgrading from zstd 1.4.5 to 1.5.6 HOT 12
- Missing check on failed allocation leads to NULL-ptr dereference
- libzstd.lib missed in package, also VC sample seems include wrong mem.h or ambigious including!
- Environment variable for --memory HOT 2
- Improve misleading wording in the streaming decompression howto HOT 2
- erro
- Add library and cli flags for file format with embedded dictionary
- Question about ZSTD protocole HOT 1
- Building on MacOS 13 and targeting MacOS 11 and SDK 11.3 (or any other MacOS version) does not work HOT 2
- Integrating the library with an external thread pool HOT 2
- Is it safe to move compression and decompression contexts between threads? HOT 1
- ZDICT_trainFromBuffer_cover is not thread safe
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zstd.