[bennettnate5]

Fascinating--I wonder to what extent the performance improvements are due to the better constraints Rust hands over to LLVM for optimization (specifically thinking `noalias` rules) vs aggressiv code optimization by the developers to be to the point where they beat the C zlib. The latter could be more difficult to achieve given how long the original zlib has been getting incrementally improved, though.

[cratermoon]

Reading the linked blog post:

-Cllvm-args=-enable-dfa-jump-thread option, which recovers most of the performance here. It performs a kind of jump threading for deterministic finite automata, and our decompression logic matches this pattern.

https://en.wikipedia.org/wiki/Jump_threading

[TinkersW]

zlib is really old and out of date, no real reason to use it except legacy. I think most optimization effort goes into newer compression libs like zstd.

[0cf8612b2e1e]

Legacy is a bit much. Deflate is probably the only supported compression format in many existing/live standards. For a greenfield project where you can control all clients, sure, use whatever, but zlib is likely going to continue to be used for decades.

[daveguy]

EDIT: doh! Mixed up Zstd and Zlib. Nothing to see here. (Can't delete with a reply.)

[dralley]

I think you've confused Zstd (which they were not discussing) for Zlib (the primary implementation of DEFLATE, which is most commonly used by Gzip)

Gzip/DEFLATE is mediocre on most metrics relevant to compression but it's widely used of course. Zstd is better all-around, but much newer and thus doesn't have the same level of adoption.

[dralley]

The comparison is against zlib-ng, not the OG zlib. So this is still very impressive.

I suspect the results won't be quite as good on Aarch64 or other architectures though. zlib-ng has a pretty wide range of hand-optimized intrinsics whereas zlib-rs seems to only really have x86_64.

[sanxiyn]

zlib-rs does use ARM CRC32 instructions though.