[reconnecting]

> When most of your server capacity is going to answering the scrapers it matters

I've been dealing with the web since the previous century and still haven't managed to build a website that could be hurt by scrapers visiting it.

If you went through the logs, you'd probably see that these bots are on a single IP or subnet, which can be easily detected and blocked instead of closing off search to non-registered users.

[trumpdong]

That's incorrect, they use residential proxy networks.

[LorenPechtel]

Botnet.

Our offending searches were coming from many addresses.