This is known topic and we were using in our day to day activity to compress and decompress files & folders. You might already know zip,tar,7-zip, etc and you would have used all these application for your requirement.
Even today also we are going to discuss about similar kind of topic, the tool name is Zstandard. It is super fast data compression tool and compression ratio is very very low.
Zstd is lossless data compression algorithm developed by Yann Collet at Facebook.
Due to high data compression ratio many of the popular companies and databases are using this tool.
What is Zstandard
Zstandard called zstd as short version, is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios.
It’s backed by a very fast entropy stage, provided by Huff0 and FSE library. Zstd can also offer stronger compression ratios at the cost of compression speed. Speed vs Compression trade-off is configurable by small increments.
Decompression speed is preserved and remains roughly the same at all settings, a property shared by most LZ compression algorithms, such as zlib or lzma. It also offers a special mode for small data, called dictionary compression, and can create dictionaries from any sample set.
These compression gains are achieved while simultaneously providing faster compression and decompression speeds. This utility producing and decoding .zst, .gz, .xz and .lz4 files.
How to install Zstandard
Zstandard is available in most of the distribution official repository that can be installed through package manager.
For Debian/Ubuntu, use APT-GET Command or APT Command to install Zstandard.
$ sudo apt install zstd
For RHEL/CentOS, use YUM Command to install Zstandard.
$ sudo yum install zstd
For Fedora, use DNF Command to install Zstandard.
$ sudo dnf install zstd
For Arch Linux, use Pacman Command to install Zstandard.
$ sudo pacman -S zstd
For openSUSE, use Zypper Command to install Zstandard.
$ sudo zypper in zstd
How to use Zstandard
It uses standard Linux format so, just use zstd command followed by the filename to compress the file.
$ zstd filename
The above command, create a compressed file with appending .zst extension to the original filename.
Add -o option followed by the file name to create a compressed file with new name.
$ zstd filename -o newfile.zst
Run below commands to uncompress any file compressed with zstandard.
$ zstd -d filename.zst $ unzstd filename.zst
Zstandard Comparison Table
As discussed in the beginning of article. Zstandard offer super speed and high compression ratio. Hence, we are going to comparing this with other compression tools called zip & tar.
I have a .sql file with 102MB for this testing and the results are follow.
$ ls -lh daygeekc_magi.sql -rw-rw-r-- 1 daygeek daygeek 102M Jun 21 16:15 daygeekc_magi.sql
For Zstd: It taken only 1 seconds and created 5.8MB of compressed file.
$ date; zstd daygeekc_magi.sql; date Thu Jun 21 16:16:26 IST 2018 daygeekc_magi.sql : 5.62% (106770640 => 6001513 bytes, daygeekc_magi.sql.zst) Thu Jun 21 16:16:27 IST 2018 $ ls -lh daygeekc_magi.sql.zst -rw-rw-r-- 1 daygeek daygeek 5.8M Jun 21 16:15 daygeekc_magi.sql.zst
For zip: It taken only 2 seconds and created 9.7MB of compressed file.
$ date; zip daygeekc_magi.sql.zip daygeekc_magi.sql; date Thu Jun 21 16:17:02 IST 2018 adding: daygeekc_magi.sql (deflated 90%) Thu Jun 21 16:17:04 IST 2018 $ ls -lh daygeekc_magi.sql.zip -rw-r--r-- 1 daygeek daygeek 9.7M Jun 21 16:17 daygeekc_magi.sql.zip
For tgz: It taken only 2 seconds and created 9.7MB of compressed file.
$ date; tar czf daygeekc_magi.sql.tgz daygeekc_magi.sql; date Thu Jun 21 16:18:37 IST 2018 Thu Jun 21 16:18:39 IST 2018 $ ls -lh daygeekc_magi.sql.tgz -rw-r--r-- 1 daygeek daygeek 9.7M Jun 21 16:18 daygeekc_magi.sql.tgz
