[luhn]

Unfortunately HAProxy doesn't buffer requests*, which is necessary for a production deployment of gunicorn. And for anybody using AWS, ALB doesn't buffer requests either. Because of this I'm actually running both HAProxy and nginx in front of my gunicorn instances—nginx in front for request buffering and HAProxy behind that for queuing.

If anybody is interested, I've packaged both as Docker containers:

HAProxy queuing/load shedding: https://hub.docker.com/r/luhn/spillway

nginx request buffering: https://hub.docker.com/r/luhn/gunicorn-proxy

* It does have an http_buffer_request option, but this only buffers the first 8kB (?) of the request.

[Twirrim]

Couldn't Apache httpd just do all of that for you? mod_buffer provides request buffering, and mod_proxy_balancer provides load balancing capabilities.

[luhn]

Can Apache do request queuing?

[Twirrim]

Yes, under MPM, you have a listen backlog. If there's a spare worker available to process a request, they'll be picked off the backlog.

https://httpd.apache.org/docs/2.4/mod/mpm_common.html#listen...

[luhn]

I don't think that's what I'm looking for, that's queuing at the front of the pipe but I need it queuing at the end of the pipe. Apache should be buffering and queuing lots of connections (with a timeout) and sending them single-file in gunicorn.

This lays out what I'm trying to achieve: https://aws.amazon.com/builders-library/using-load-shedding-...