Hacker News new | ask | show | jobs
by klauspost 1406 days ago
For MinIO (an S3 compatible server), we add an index for each part, which contains uncompressed -> compressed offset pairs.

Since we already used a Snappy-derived method, each 1MB block is stored without backreferences. With this we only have to decode at most 1MB-1 extra bytes to respond with a specific range offset.