[fuckaj]

Is the pragmatic solution to return 503 and have clients back off.

Or, if possible make latency a feature (embrace the queue!). For service to service internal stuff e.g. something like a request to hard delete something, this can always be a queue.

And obviously you can scale up as the queue backs up.

I do love the maths tho!