crabz: multi-threaded gzip (like pigz, but in Rust)
https://github.com/sstadick/crabz| Installer Source| Releases (json) (tab)
crabz: multi-threaded gzip (like pigz, but in Rust)
https://github.com/sstadick/crabz| Installer Source| Releases (json) (tab)
To update or switch versions, run webi crabz@stable (or @0.8, @beta, etc).
crabzbrings the power of multi-core compression to gzip and deflate.
(and a few other formats + other useful features)
gzip, faster.
crabz -I ./example.json
crabz -d -I ./example.json.gz
Compressing (gzip) with 8 threads at compression level 6.
Decompressing (gzip) with 8 threads available.
These are the files/directories that are created and/or modified with this install:
~/.config/envman/PATH.env
~/.local/bin/crabz
| Flag | Value | Comments |
|---|---|---|
-l, --compression-level |
1-9 | higher is slower |
-p, --compression-threads |
8 | set to the number of available cores |
| (but no more than 4 for decompression) | ||
-P, --pin-at |
0 | pin to physical cores, starting at N |
| (so 4 threads starting at 0 is 0, 1, 2, 3) |
crabz -l 9 -p 8 -I ./example.tar
crabz -d -p 4 -I ./example.tar.gz
Tar and then compress:
tar cv ./example/ | crabz -o ./example.tar.gz
Or decompress and then untar:
crabz -d ./example.tar.gz | tar xv
crabz supports most of the LZ77 with Huffman coding compression formats:
| Format | Extension | Notes |
|---|---|---|
gzip |
.gz |
of GNU fame |
bgzf |
.gz |
supports random-access decompression |
mgzip |
.gz |
of python fame |
zlib |
.zz |
of PNG fame, also .z |
snap |
.sz |
of LevelDB and MongoDB fame |
deflate |
.gz |
the O.G. LZ77 |
crabz --format mgzip -I ./example.tar
# DO NOT decompress in-place
crabz --format mgzip -d ./example.tar.gz -o ./example.tar
# verify before removing the original
tar tf ./example.tar
⚠️ Warnings:
gunzip will work correctly on files compressed with mgzip or
bgzf, some combinations (ex: decompressing from mgzip with bgzf) could
result in corruption!tar xvf and gzip -l may report incorrect information, even though gunzip
will workSee also:
(p.s. zip isn't in the list because it's a container format like tar, not a
zip format)