Hacker News new | ask | show | jobs
by pixelglow 3647 days ago
You can definitely compose a zip file "on the fly" and stream it out. The central directory at the end of the zip file can be determined from all the content already streamed.

The only wrinkle is that each entry has a header which typically states the compressed size and checksum. Either you have to compress each entry content in some temp buffer or file to figure out the compressed size and checksum, then write out the header and compressed content. Or you write out the header with these fields zeroed, then compress the content on the fly and write it out, then write out a data descriptor with the compressed size and checksum.

1 comments

What the parent is saying is that the client has to buffer the entire zip file before they receive the index and can begin decompressing it. "Streaming compression" usually implies streaming decompression as well—importantly because a non-negligible use case for streaming compression is sending streams of compressed data that won't fit on the destination together with the decompressed copy; or sending continuous streams of compressed data that could never reach the end to be decompressed. What the parent is saying is that zip cannot work for these use-cases.