[spiderfarmer]

They work because they offer ways to opt out, they honor crawl delay, setting ideal scraping times, IndexNow, etc.

And they give you real, valuable traffic in return.

[avazhi]

Most offer ways to opt out, some don’t. Scraping somebody’s website might be annoying or problematic traffic-wise but that’s a far (very far) step removed from saying scrapers should be criminalised. The latter statement is outright laughable.