Hacker News new | ask | show | jobs
by ncmncm 2360 days ago
Zstd has an enormous advantage in compression and, especially, decompression speed. It often doesn't compress quite as much, but we don't care as much as we once did. We rebuild packages more than we once did.

This looks like a very good move. Debian should follow suit.

1 comments

I build packages periodically from the AUR, and compression is the longest part of the process much of the time. For a while, I disabled compression on AUR packages because it was becoming enough of a problem for me to look into solutions. If it's annoying for me, I can imagine it's especially problematic for package maintainers. I can only imagine how much CPU time switching the compression tool will save.
I love the AUR, but every single time I have to wait for it to compress Firefox nightly, and then wait for it to immediately decompress it again because the only reason I was building the package in the first place was to install it I about lose my mind. Hopefully this helps, but I really wish AUR helpers would just disable compression and call it a day so I don't have to go mess with config files that would also change my manual package building routine.

EDIT: nevermind, this doesn't seem to have made this the default for building packages locally, just for ones you download from the official repos. Guess I'll go change that by hand and then still be sad that I can't have it easily disabled entirely for AUR helpers but build my packages with compression.

This isn't a function of an AUR helper but rather makepkg itself.

In makepkg.conf, ommit compression by specifying:

    PKGEXT='.pkg.tar'
More information can be found at https://wiki.archlinux.org/index.php/Makepkg#Tips_and_tricks
Then it happens for packages I build myself (and want compressed) and those that I just want to install.
You can also override PKGEXT as an environment variable when invoking makepkg, so you can have compression by default but easily skip it when it matters:

  PKGEXT=.pkg.tar makepkg
EDIT: By the way, it's often a big win to use multithreaded compression (pigz, zstd -T0, etc.) in makepkg.conf. With this, it's fast enough that I hardly ever override PKGEXT anymore.
If you care about space more than you care about speed you may want to stick with xz, it is hard to beat or impossible by zstd. So set your own priorities rather than adopt the ones of Arch devs.

As long as there will be support within the tools for xz individual builders of packages for their own use can use either or more.

I don't really care about space all that much, but packages I build tend to get uploaded and the people downloading them may not have fast internet. Meanwhile, packages built by my AUR helper I care about speed (seriously, it takes ages to compress then immediately decompress firefox). The problem isn't that I want to optimize for one or the other, it's that AUR helpers generally have a different need than I do when building my packages myself, but for some reason AUR helpers don't override the compression setting for just their install. Probably due to caching like I said which means they can't assume everyone will want compression off all the time, but I'm not sure, that's just a guess.
I doubt anybody today has legitimately enough reason to care about space to prefer xz over zstd. The only plausible reason is that some customer, or customer's customer, is only equipped to unpack archaic formats.
> EDIT: nevermind, this doesn't seem to have made this the default for building packages locally, just for ones you download from the official repos. Guess I'll go change that by hand and then still be sad that I can't have it easily disabled entirely for AUR helpers but build my packages with compression.

I believe you can supply it via an environment variable if your AUR helper has the ability to set those for `makepkg`.

I'll have to experiment with that, thanks. Still, I don't understand why they don't just do it for me. Maybe they're aggressively caching packages and don't want to take the size hit or something.
Why did you re-enable it? Seems to work fine in my experience?