[wahnfrieden]

When your workloads are unpredictable and spike suddenly such that you can't scale quickly enough to avoid having a bunch of spare capacity waiting around and have HA requirements. In this scenario more is spent on avoiding variable spend to achieve a "flat" rate

[withinboredom]

In 20 years of writing software, I have never seen an amount of legitimate influx of traffic that can swamp a whole pool of servers faster than it can scale. I’m not saying it can’t happen, I’ve just not worked on any code or infrastructure that couldn’t keep up with the demands of scale. Is there an industry this regularly happens in where this is a recurring issue?

I write software that a billion users see every day, so maybe I’m jaded by the sheer scale and challenges of writing code at scale that I just can’t imagine these types of problems.

[wahnfrieden]

You are looking at your own experiences I guess. In edtech it is common for large classrooms to suddenly come online and do things in tight coordination and no predictive scaling isn’t predictable enough for this problem. You can also look at ecommerce, Black Friday type events to see how capacity planning can easily require runway on spare capacity before scaling can react several minutes in.

Do you think EC2 capacity on AWS is on average kept in high utilization? Everyone runs (non truly elastic resources) with headroom to varying degrees

[withinboredom]

Ah, yeah, I’m only familiar with the industries I’ve worked in and never worked in edtech. That’s a pretty good example of any industry that gets sudden, unpredictable load.