From |kry|ov @end|ng |rom d|@root@org  Sun Jan 12 09:14:40 2025
From: |kry|ov @end|ng |rom d|@root@org (Ivan Krylov)
Date: Sun, 12 Jan 2025 11:14:40 +0300
Subject: [Rd] Package compression benchmarks for zstd vs gzip
In-Reply-To: <CAFDcVCS6kfkqzwR7ximTu=EJiHeuSBa0hU_r0Ze7yRi87_cLbw@mail.gmail.com>
References: <CABFfbXvHddYPsHs7j9ZzvTrPo-NX880mbh6=quZWrd1+eCjD=g@mail.gmail.com>
 <CAL6gwnL0eLEFu0kyxqubmy1CeqnCGUa3FqQ+wFUghEazyxEV1g@mail.gmail.com>
 <CAFDcVCS6kfkqzwR7ximTu=EJiHeuSBa0hU_r0Ze7yRi87_cLbw@mail.gmail.com>
Message-ID: <20250112111440.598e4f54@Tarkus>

On Sat, 11 Jan 2025 16:05:46 -0800
Henrik Bengtsson <henrik.bengtsson at gmail.com> wrote:

> It's probably also worth looking at package compression with 'xz'
> compression. In [1], Mike FC has a graph where 'bzip2' and 'xz' seem
> to give the best compression ratios, at least for RDS files.

'bzip2' can be surprisingly good on very repetitive payloads. It
compresses 0x80000000 zero bytes to only 1.5 KiB, much better than 'xz
-9' with 305 KiB (with compression settings not making much
difference), although the compression is not perfect. One terabyte of
zeros can be compressed to 697202 bytes of repetitive compressed stream
that can be bzipped further to 248 bytes.

Binary packages are probably the most obvious target for new
compression methods because there is no need to install them on older
versions of R.

-- 
Best regards,
Ivan



[Rd] Package compression benchmarks for zstd vs gzip
Ivan Krylov