[ccurtsinger]

Little's law certainly applies here. The article's author does have a good point though: if there is no queue and requests are processed one by one, your only option is good old serial program optimization.

Still, I think most large systems where latency is a major concern have some level of concurrency and/or queueing that can be manipulated to reduce latency.