[scottlamb]

Sort of. If I'm understanding correctly:

The filetypes it's designed for are ones that specify the "deflate" compression algorithm: they can be thought of as headers+deflate(raw). It applies the better (higher-compression-ratio) lzma algorithm to the raw data. That's better than applying it to the deflate form as a .zip file compressor would try to do. Ie, zip might try lzma(deflate(raw)) where this tries lzma(raw).

So the novel things are:

* It understands these specific file formats well enough to pick out deflate part that it should uncompress (when it applies its own compression) and recompress (when it applies its decompression).

* I think it promises to exactly reproduce the original deflate(raw) form. You can't necessarily do that from lzma(raw) alone. I'm not an expert, but from a wikipedia article skim, apparently deflate works with a combination of dictionary encoding and Huffman coding. This program probably stores the original tables for those (with the value sides as ranges of the original file rather than repeating them), and maybe some other details to handle this. Like, if the originally compressing program called flush more frequently, it produces a different stream, and this needs to be able to reconstruct that. Maybe it stores like a list of deflate operations rather than either the deflate-compressed result or the raw data. This strikes me as kind of a tricky problem, not in any theoretical computer science sense, but in a practical getting-all-the-details-right software engineering sense.

[toast0]

If that's the case, it sounds like Dropbox's Lepton[1], which recompresses jpegs, but can be losslesly decompressed back to the original jpeg file.

[1] https://dropbox.tech/infrastructure/lepton-image-compression...

[wolf550e]

To match the original zlib/gzip stream, it needs to match the exact hash table and parsing algorithm used by the exact zlib implementation used originally, to choose the same matches. It would not do to use a stronger deflate compressor that produces optimal parsing to generate smaller deflate streams but then not be able to match the original file. The hash of the compressed data and even the length of the compressed data might be important to the application using the file.

[scottlamb]

Looks like it uses the preflate library [1] which back-references precomp in its README:

> "precomp" is a tool which used to be able to do the bit-correct reconstruction very efficiently, but only for deflate streams that were created by the ZLIB library. (It only needed to store the three relevant ZLIB parameters to allow reconstruction.) It bailed out for anything created by 7zip, kzip, or similar tools. Of course, "precomp" also handles JPG, PNG, ZIP, GIF, PDF, MP3, etc, which makes a very nice tool, and it is open source. UPDATE The latest development version of "precomp" incorporates the "preflate" library.

so precomp used to work as you describe: it depended on having exactly the same zlib algorithm as the one that produced the original data. Now it uses preflate, which has some hybrid approach:

> for deflate streams created by ZLIB at any compression level, we want to be able to reconstruct it with only a few bytes of reconstruction information (like "precomp")

> for all other deflate streams, we want to be able to reconstruct them with reconstruction information that is not much larger than "reflate"

> "reflate" can reconstruct any deflate stream (also 7zip, kzip, etc), but it is only close to perfect for streams that were created by ZLIB, compression level 9. All lower compression levels of ZLIB require increasing reconstruction info, the further from level 9, the bigger the required reconstruction data. "reflate" only handles deflate streams, and is not open source. As far as I know, it is also part of PowerArchiver.

I'd be curious to learn a little more about the "reconstruction information" but I don't see a write-up. So far I'm not quite curious enough to dig through preflate's source code.

[1] https://github.com/deus-libri/preflate