Hacker News new | ask | show | jobs
by klauspost 1430 days ago
Here is an updated chart - and the data (2019): https://community.centminmod.com/threads/round-4-compression...

Yes, though decompression speed (+memory to some extent) should also be considered. Also different input types has different characteristics, so a single type isn't very representative.

While ultimate compression efficiency is interesting, in practice I find that most often you are looking for a "good enough", meaning "give me the best at xxxMB/s". For most purposes (de)compression at 50MB/s is just too slow to bother.

For a server anything considerably below 1GB/s/core is too slow, otherwise you will too easily saturate your CPU before your NIC or disks are saturated.

1 comments

Well according to the chart, "give me the best at 1Gb/sec" would seem to be zstd level 2...

Also, zstd seems to operate over the widest range - a simple algorithm to use that would be: "Each second, check... Is network utilization over 90% and CPU utilization under 90%? If so, increase compression level by one. If the reverse, decrease compression level by one"