[tempfile]

> require login

this is not a solution if you want a public internet (and sites that don't care about the public internet already don't have a problem)

[londons_explore]

for read-only content, I just stick it behind a cache and let the bots go wild.

[jgalt212]

You can't cache this stuff for bot consumption. Humans only want to see the popular stuff. Bots download everything. The size of your cache then equals the size of your content database.

[account42]

But you can still make sure that you save the data in a form where generating the served webpage takes the least amount of time. For most websites this means saving the HTML - in a giant cache or with a more deliberate pre-generation setup.

[jgalt212]

The data structure to html conversion takes milliseconds. That's distinction without a difference.

[account42]

Clearly not the case with most websites. And "milliseconds" are already a huge amount of time. Video games simulate huge worlds and render complex 3D graphics within 16ms or even much less with the >60 framerates that are expected these days.

[jgalt212]

You've gotten off topic here. The giant cache you speak of approaches the size of the content database when designing for the long tail of bots. A giant cache is non economic and thus not a solution unless you're an AWS salesman.

[dartos]

That’s just passing the buck.

Someone still needs to pay for that traffic. If it gets too much for cloud flare or whoever, you’re gonna get the bill.

[account42]

Traffic is not all that expensive though if you are not using a cloud provider where that's how they squeeze captured customers.

[tempfile]

I presume OSM has already considered this and ruled it out (probably because the map should be dynamic)