[8note]

A queue distributes the latency increase to all requests whereas a stack only increases the latency for some requests when you're overloaded.

This means that if you catch up to the incoming new requests, a queue keeps every request running slower due to the time spent in the queue.

The steady state stack on the other hand, gives the same couple items worse and worse latency, while all the new items go back to normal.

Chances are the long latency requests will be retried, and there's a roughly fixed number of items to be retried, so you don't have to worry too much about the ones stuck in the stack. For a queue, the retries lengthen the queue, and that waiting time is added to every item, making them more likely to retry too, lengthening the queue further etc

Having the stack leak items at the bottom gets the same benefit - you don't have to fulfil them, but if you can get the items back out of the stack quickly, it's still worth working on and completing them before the client needs to send a retry. The more of them you get in, the more 9s you get to look more like your p50 than your p100