[evanj]

Fair queuing is an interesting and complementary approach that I have not considered, thanks for the suggestion.

I usually work with "internal" systems that don't always have an obvious "user" identifier, but it would be fairly easy to add one of some sort (e.g. an application id or similar). However, for that to be useful, there still has to be some sort of limit. In your case, the limit seems to be 1 concurrent request per user, up to a maximum of 100. In the case of some internal system, like say a metrics aggregator, what is the correct limit? I'm not sure, and I think it would require tuning.

That said, given a reasonable limit, this would definitely ensure that the "misbehaving" application (e.g. the one flooding the metrics aggregator with packets) is the one that gets punished, rather than everyone else. I'll have to think about this more. Thanks!