[herf]

Mostly you shouldn't have to hide backend latency by reordering, and if you don't reorder, you can just use JPEG image strips/sprites, which don't need double compression. JPEGs also stream, but in an order determined by the client, not the server.

In my experience, Dropbox is quite a bit slower (by 10-20x) than services optimized for serving images - using 1500ms to deliver a 25k file is very slow on the web, but it is very common when requesting files via the Dropbox web API. (I can only speculate about the reasons, but Amazon's s3 has big latencies too.)

[ziga]

We thought of spriting the thumbnails on the server, but this introduces a few problems:

- All thumbs would have to be available on the server before the sprite can be encoded, which defeats progressive rendering. (Progressively encoding JPEG might be a possibility, but this isn't something we explored.)

- The additional cost of encoding/decoding a large image on the server/client.

-- Ziga (author)

[herf]

Yes, you can stream JPEG pretty easily as long as you know the output size.

Also to combine JPEGs into a sprite, you can avoid all DCT/iDCT/color operations and use only the Huffman portion of the codec (decode, concat, encode). Since Huffman throughput is a considerable multiple of gzip (~10x?), I think you'd be quite a ways ahead, for memory and time.

[WhitneyLand]

1) Not necessarily, you may not even need progressive rendering if sprite batches bring it down to a manageable number of concurrent requests, right?

2) How is it any more work to encode? It's the same total number of pixels and the memory footprint isn't that large for server or client.

I don't mean to dis your efforts, looks like you've done some nice work that performs well.

I just think cloud/media management is an interesting problem with a lot of different ways to look at it.