Coder Social home page Coder Social logo

Comments (4)

Cyan4973 avatar Cyan4973 commented on May 16, 2024 2

Ah, you cut after compression, not before.

In this case, this is more like :

cat *.zst* | zstd -d > dest.tar

Same idea here, this only works if expansion of *.zst* is properly ordered.

from zstd.

oDinZu avatar oDinZu commented on May 16, 2024 1

That worked! You rock and thank you! Now, to simplify the command and also untar 😄

For others on the internet, the one liner I made use of to tar and compress with zstd.

I backed up ~120GB SSD onto a FAT32 1.5TB disk drive in less than 15 minutes with default compression algorithms 🎉

Thank you zstd! 🙌

Compress / Decompress Huge Files via zstd with PROGRESS indication via PV

The required packages are:

  • tar
  • pv
  • zstd
  • dosfsck

Check the filesize in chunks for the FAT32 drive

sudo dosfsck -v -n /dev/sda1

File size is 4294967295 bytes, cluster chain length is 0 bytes.

Split the tar into files to transport between drives e.g. FAT32 formatted drive and filter/compress with zstd with -I option into another drive, while visualizing a progress counter with pv.

tar -I zstd -cf - Desktop/ | (pv -p --timer --rate --bytes | split --bytes=4294967295 - /run/media/username/Backups/System/Linux/Desktop/Desktop.backup.tar.zst)

Decompress multiple zst files between drives

cat *.zst* | zstd -d > /home/username/dest.tar

Then extract the archive

tar -xvf dest.tar

or extract the archive into specific directory

tar -xvf dest.tar -C /opt/files

from zstd.

Cyan4973 avatar Cyan4973 commented on May 16, 2024

The *.zst suffix requirement is because the CLI must determine a destination filename.
If it doesn't need to generate such a name, the *.zst requirement is dropped.

So one way to achieve your objective could be :

zstd -d *.zst* -c > dest.tar

This however implies that *zst* shell expansion is properly ordered, which I'm not sure of.
There are probably more elaborated way to generate this list of files that would guarantee proper alphabetical order.
But you get the idea for the zstd part.

from zstd.

oDinZu avatar oDinZu commented on May 16, 2024

I tried that and received and error, but the file size seems to have added another file; each zst is 4.3GB and the dest.tar is now 8.2GB..or maybe that is the actual size uncompressed:

zstd -d *.zst* -c > /home/username/dest.tar

top.backup.tar.zstaa : Read error (39) : premature end 
zstd: Desktop.backup.tar.zstab: unsupported format 
zstd: Desktop.backup.tar.zstac: unsupported format 
zstd: Desktop.backup.tar.zstad: unsupported format 
zstd: Desktop.backup.tar.zstae: unsupported format 
zstd: Desktop.backup.tar.zstaf: unsupported format 
zstd: Desktop.backup.tar.zstag: unsupported format

Also,
*** Zstandard CLI (64-bit) v1.5.5, by Yann Collet ***

from zstd.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.