[account42]

Clearly not the case with most websites. And "milliseconds" are already a huge amount of time. Video games simulate huge worlds and render complex 3D graphics within 16ms or even much less with the >60 framerates that are expected these days.

[jgalt212]

You've gotten off topic here. The giant cache you speak of approaches the size of the content database when designing for the long tail of bots. A giant cache is non economic and thus not a solution unless you're an AWS salesman.

[londons_explore]

Yes, the cache ends up being bigger than the content database, but for text content that's typically not a problem. The human effort to type some text always hugely exceeds the cost of a few kilobytes of flash to store what they typed in a ready-to-serve form.

The generation process of taking the raw text and assembling the page around it is typically rather expensive for most CMS systems. Sure - it isn't theoretically expensive, but unless you want to engineer a CMS from scratch most people just pick one off the shelf and then end up having to pay the CPU time overhead of wordpress etc.

[account42]

Not at all. You can either design your website so that pages can be retrieved in sub-millisecond time (and that doesn't have to mean throwing money at cloud providers) or you can cry about bots.