[klauspost]

You only need a single part to calculate a specific offset, assuming you have part sizes stored in metadata already (a good idea).

Each part can be max 5GiB as per S3 spec. 5120 * 4 = 20KiB.

Even if you unpack to 8*2 bytes in memory when decoding, you are still not talking a huge amount of memory.

The on-disk space is ~0.0004% as blibble calculated, and should easily be offset by the compression achieved. In MinIO we don't store indexes for files < 8MiB, so for small files there is no overhead.

If the added metadata is a problem for whatever system you are looking at, then that is a characteristic of that system and not a general problem.