Hacker News new | ask | show | jobs
by jonstewart 387 days ago
That’s true, but streamable formats often don’t need an index.

A team member just created a new tool that uses the tar format (streamable), but then puts the index as the penultimate entry, with the last entry just being a fixed size entry with the offset of the beginning of the index.

In this way normal tar tools just work but it’s possible to retrieve a listing and access a file randomly. It’s also still possible to append to it in the future, modulo futzing with the index a bit.

(The intended purpose is archiving files that were stored as S3 objects back into S3.?