[Twirrim]

Couldn't Apache httpd just do all of that for you? mod_buffer provides request buffering, and mod_proxy_balancer provides load balancing capabilities.

[luhn]

Can Apache do request queuing?

[Twirrim]

Yes, under MPM, you have a listen backlog. If there's a spare worker available to process a request, they'll be picked off the backlog.

https://httpd.apache.org/docs/2.4/mod/mpm_common.html#listen...

[luhn]

I don't think that's what I'm looking for, that's queuing at the front of the pipe but I need it queuing at the end of the pipe. Apache should be buffering and queuing lots of connections (with a timeout) and sending them single-file in gunicorn.

This lays out what I'm trying to achieve: https://aws.amazon.com/builders-library/using-load-shedding-...