[avidiax]

> You might also want to consider how requesting behavior will change when requests are stacked instead of queued. As soon as people have learned that you keep requests in a stack, the correct way to make a request is to make it, wait for a very small amount of time, and then, if your request hasn't already succeeded, repeat it.

It would be very hard to learn this so long as the queue is a very small fraction of the total throughput. If the queue depth is 100, and you receive 10,000qps, but process 9,900 qps, the queue will get full, and roughly 97 calls will go unanswered. Ideally you should have another mechanism to time these out, which most systems do. Whatever queue type you pick, you are going to reject 1% of the inbound, but with a FIFO queue, you will also delay 100% of the responses. Do that at several layers, and you can even end up with the client timing out even though their request wasn't even rejected at any stage.