Hacker News new | ask | show | jobs
by mananaysiempre 1608 days ago
> There is no tradeoff, [Zstd] is always better.

Well of course there’s the tradeoff in memory usage, utterly irrelevant on a desktop and probably even on a server, but you’re never decompressing a Zstandard file (compressed with standard options) on an STM32 (36 to 144 MHz ARM Cortex-M) micro or a similar RAM-impoverished but not entirely wimpy system. That is no secret, though, IIRC part of the original motivation for Zstandard was that modern hardware makes old window size choices obsolete.

A less obvious point is that a modern implementation of Deflate is, if not necessarily always faster, not as catastrophically slow compared to Zstandard as Zlib[1]. (In other benchmarks I’ve seen libdeflate be something like half again as slow as Zstandard, but at least it’s not multiple times slower.)

[1] https://lemire.me/blog/2021/06/30/compressing-json-gzip-vs-z...

1 comments

If you are working on a microcontroller, odds are you will be keeping your entire file in RAM as that is all you have, in which case you should not need any memory for a window at all, as you can just use the file itself.

I don't know if the zstd implementation supports working like this, though.

I believe you can compress into the source buffer and keep the read offset ahead of the write offset, but I don't think you can decompress this way.
This is about the window buffer, not the source and destination buffers. They are still distinct, but you can combine the uncompressed buffer and the window buffer, if you have, or are going to have, the entire uncompressed data in memory.