We serve millions of requests per day and have some slow responses (75+ ms) but on any given day our servers handle 175 requests per second without breaking a sweat. =/
Keep in mind that we're a platform, not a website, and we have very little control over things like schemas, slow queries, inefficient looping requests that our developers send us. We can't optimize 500 thousand apps' queries for them.
So for example if an app does something bad like performing a full table scan on every request against a 300 million collection, so every request to that backend is timing out at 30 seconds, and there are thousands of them per second, well -- pretty soon your fixed pool will be full of requests timing out to that backend.
So for example if an app does something bad like performing a full table scan on every request against a 300 million collection, so every request to that backend is timing out at 30 seconds, and there are thousands of them per second, well -- pretty soon your fixed pool will be full of requests timing out to that backend.