[gravitas]

AUR users -- the default settings in /etc/makepkpg.conf (delivered by the pacman package as of 5.2.1-1) are still at xz, you must manually edit your local config:

  PKGEXT='.pkg.tar.zst'

The largest package I always wait on perfect for this scenario is `google-cloud-sdk` (the re-compression is a killer -- `zoom` is another one in AUR that's a beast) so I used it as a test on my laptop here in "real world conditions" (browsers running, music playing, etc.). It's an old Dell m4600 (i7-2760QM, rotating disk), nothing special. What matters is using default xz, compression takes twice as long and appears to drive the CPU harder. Using xz my fans always kick in for a bit (normal behaviour), testing zst here did not kick the fans on the same way.

After warming up all my caches with a few pre-builds to try and keep it fair by reducing disk I/O, here's a sampling of the results:

  xz defaults  - Size: 33649964
  real  2m23.016s
  user  1m49.340s
  sys   0m35.132s
  ----
  zst defaults - Size: 47521947
  real  1m5.904s
  user  0m30.971s
  sys   0m34.021s
  ----
  zst mpthread - Size: 47521114
  real  1m3.943s
  user  0m30.905s
  sys   0m33.355s

I can re-run them and get a pretty consistent return (so that's good, we're "fair" to a degree); there's disk activity building this package (seds, etc.) so it's not pure compression only. It's a scenario I live every time this AUR package (google-cloud-sdk) is refreshed and we get to upgrade. Trying to stick with real world, not synthetic benchmarks. :)

I did not seem to notice any appreciable difference in adding the `--threads=0` to `COMPRESSZST=` (from the Arch wiki), they both consistently gave me right around what you see above. This was compression only testing which is where my wait time is when upgrading these packages, huge improvement with zst seen here...

[Foxboron]

It should be noted that the makepkg.conf file distributed with pacman does not contain the same compression settings as the one used to build official packages.

pacman:

    COMPRESSZST=(zstd -c -z -q -)

https://git.archlinux.org/svntogit/packages.git/tree/trunk/m...

devtools:

    COMPRESSZST=(zstd -c -T0 --ultra -20 -)

https://github.com/archlinux/devtools/blob/master/makepkg-x8...

[gravitas]

The man page for zstd mentions that using the --ultra flag will cause decompression to take more RAM as well when used to compress. Does this indicate a huge increase in memory to decompress, or just a trivial amount per package, say something large like... `libreoffice-fresh`? Or `go`? They're two of the largest main repo packages I have installed... (followed by linux-firmware)

[telendram]

Without `--ultra`, the decompression memory budget is capped at 8 MB. At `--ultra -20`, it's increased to 32 MB.

That's still less than XZ, which reaches 64 MB.

[JyrkiAlakuijala]

The respective flag for brotli would be `--large_window 25 --quality 11`

Brotli defines memory use as log2 on command line, i.e., 32 MB = 1 << 25

zstd uses a lookup table where the number given by the user is mapped to a decoding-time memory use. The user just needs to look it up if they want to control decoder memory use.

If one benchmarks zstd with `20` and brotli with `20`, zstd may be using 32 MB of decoding memory, where one is specifying 1 MB for brotli. By default zstd tends to use 8 MB for decoding (however it is variable with encoding effort setting) and brotli 4 MB for decoding (not changing with the encoding effort setting).