[kevincox]

> the ALB itself did not scale fast enough

On the other hand sharing an ALB across 100 apps means that a single app's fluctuations are less significant. If your apps have completely independent traffic patterns then a 10x surge for one service would only be a 10% surge for the ALB if all applications have equal amounts of traffic. This likely can be handled with the current utilization buffer so the ALB scaling isn't even required (of course you hope that it will still scale up quickly to refill the buffer).

Of course in real life everything isn't this perfect, your traffic patterns are correlated between services and one service is the vast majority of your traffic but it can still be a nice buffer.