Hacker News new | ask | show | jobs
by burch45 234 days ago
This post’s conclusions are odd. It has a bunch of extensive benchmarks showing that zstd is by far the worst performing across every metric except a slight increase in compression ratio and then says the conclusion is zstd is the best choice. Unless I’m missing something in the data.
3 comments
Dylan16807 234 days ago
In the first benchmark it gets a ratio of 4 instead of 2.7, fitting 36-40% more data with 75% more CPU. It looks great.

The next two show it fitting 20% more data with 2-3x the CPU, which is a tougher tradeoff but still useful in a lot of situations.

The rest of the post analyzes the CPU cost in more detail, so yeah it's worse in every subcategory of that. But the increase in compression ratio is quite valuable. The conclusion says it "provides the highest compression ratio while still maintaining acceptable speeds" and that's correct. If you care about compression ratio, strongly consider zstd.

link
buildbot 234 days ago
I have had similar experience, with ZFS zstd dropped IOPs and throughput by 2-4x compared to lz4! On a 64 core Milan server chip…
link
colechristensen 234 days ago
ZFS lz4 in my experience is faster in every metric than no compression.
link
Havoc 233 days ago
Only if the data in question is at least somewhat compressible
link
colechristensen 233 days ago
Not really, it goes so fast through the CPU that the disk speed is at worst the same and the CPU overhead is tiny (in other words it's not fast while saturating the CPU, it's fast while consuming a couple percent of the CPU)

technically sure you're correct but the actual overhead of lz4 was more or less at the noise floor of other things going on on the system to the extent that I think lz4 without thought or analysis is the best advice always.

Unless you have a really specialized use case the additional compression from other algorithms isn't at all worth the performance penalty in my opinion.

link
1oooqooq 234 days ago
the context is missing.

but for vps, where the cpu usage is extremely low and ram is expensive, it might make sense to sacrifice a little performance for more db cache maybe. can't say without more context

link