[arter45]

If an application gets 4 thousand req/s for an hour, and an additional 10% requests in the rest of the day, it is handling nearly 15 million reqs/day, which is completely different and of course requires scaling in most cases.

That said, even then, there are a lot of business cases where you are not constrained by the time required to sort or traverse a custom data structure, because you spend more time waiting for an answer from a database (in which case you may want to tune the db or add a cache),or the time needed to talk to a server or another user, or a third party library, or a payment processing endpoint.

There are also use cases (think offline mobile apps) where the number of concurrent requests is basically 1, because each offline app serves a single user, so as long as you can process stuff before a user notices the app is sluggish (hundreds of milliseconds at least) you're good.

What do you do with those 4 thousand req/s? That's what makes the difference between "processing everything independently is fast enough for our purposes", "we need to optimize database or network latency", or "we need to optimize our data structures".

[johnmwilkinson]

A peak of 4k/s does not mean they get that for the entire hour. The point I was trying to make is that simply computing the mean over a 24hr period will almost certainly ensure you size things incorrectly.

If a stretch of road was used by an average of 10 cars per minute over a 24 hour period, is it congested?

In both cases, you need more specific traffic data to size things properly.