[staycoolboy]

throughput vs latency.

If I want to download a 1GB file, I do a TLS handshake once, and then send huge TCP packets. I can get almost 50MB/s from my AWS S3 bucket on my 1GB fiber, so it takes ~20secons.

However, If I split that 1GB up into 1,000,000 1KB files, I incur 1,000,000 the handshake penalty, plus all of the OTHER overhead with nginx/apache and file system or whatever is serving the request, so my bandwidth is significantly lower. I just did an SCP experiment and got 8MB/s average download speed and cancelled the download.

The problem here is throughput is great with few big files, but hasn't improved with lots of little files.