[shanemhansen]

I wonder how well this will actually work.

The core problem is that alot of crawlers aren't spending their money. They are part of a botnet so they are just spending the victim's money.

But hopefully most of the crawlers aren't botnets or funded by free VC money so they have an economic incentive to avoid crawling systems requiring proof-of-work.

[theshrike79]

It's not about stopping, more about slowing down.

(numbers for presentation purposes only)

If every request goes from 10ms to 100ms - a human won't care as they'll click like 5-6 times total, while reading the output between page load, but a bot will crawl the site 10 times slower.

[Havoc]

Pretty sure the AI crawlers aren't botnets

[thayne]

I'd bet some of them, run by less scrupulous entities, are.