[stormbeard]

From what I've seen, there's usually some hard requirement that a single instance of the application can handle N in-flight requests. This is fine when you have a single load balancer that uses some flavor of "least request" or "least connection" balancing algorithm. The problem arises when you have a fleet of load balancers- they don't share state, so each LB can't know the state of any individual backend without sending a request. This results in a failed request if the backend is already at full capacity.

Your tone is coming off as condescending and I'm not sure if it's intentional. I guess I should mention that I'm _very_ familiar with how load balancing works. This is a real problem and you can't "tune" your way into a consistent global view of the backend states.

[sargstuff]

But do have control over how the data parcels causing load are partitioned/preped before being parceled out. (which requires knowledge of resource metrics/topology-- distributed resources within computing box vs. distributed external computing resources ).

Pre-modern computing, load balancing was a telecommunications field thing. Cloud computing is the modern 'load balancing' take.

[stormbeard]

I'm familiar with the "modern load balancing" take. Modern load balancers, like you'd find in any cloud vendor or OSS project like envoy/nginx/haproxy, learn about the endpoints it is balancing across via some service discovery mechanism. DNS is usually what you find, but there are other ways like envoy's xds mechanism.

When you generalize this by saying "cloud computing", it glosses over the fact that there's still a fleet of load balancers somewhere. When you use an NLB or ALB in AWS, there are many machines behind the scene and a very complex control plane providing those machines with the information they need to balance load.

The problem I'm talking about still exists here. I _know_ they have this problem because I'm familiar with how those systems are built and I know what shortcomings they have.