[matheusmoreira]

> at scale, to the degree that you're taking down my site

Fair. Scrapers should be polite and do their utmost to consume the smallest possible amount of resources.

> without my consent by masking your user-agent

Your consent is not required. It's my user agent. I set it to whatever I want.

> for the purpose of stealing data I didn't authorize you to have

Data can't be "stolen", only copied.

You set up an HTTP server that literally sends people the data when they request it. Don't do that if you don't want people to have the data. Secrecy is the only possible defense.

[oasisbob]

> Data can't be "stolen", only copied

Though, scrapers can certainly steal capacity through conversion for their own use. When they do so, they permanently deprive the site owner and other users the beneficial use of that capacity at that time.

I'm speaking morally, not legally. Though, in the US at least, there are parallels with free newspapers. You're allowed to take one for free. It's not legal to clear the whole rack.

[matheusmoreira]

That's a much better argument, and one I agree with. Scrapers should be polite and consume the least amount of resources possible, ideally less than what a person would consume by browsing the website normally.

[lixtra]

> You set up an HTTP server that literally sends people the data when they request it. Don't do that if you don't want people to have the data.

By the same argument I could say: If I send you an exploit and you execute it, don’t complain that your setup fell for it. Just don’t download and run random data from the internet.

In reality there’s a consent and expectation beyond the pure technicals.

[matheusmoreira]

> Just don’t download and run random data from the internet.

I don't. I go out of my way to filter everything. Scraping is but one of the tools I use to do it. I want just the data that I actually care about, not people's javascripted hot mess websites full of malware-vectoring ads, fingerprinting and tracking.

I don't let my computers talk to strangers either. My servers don't respond to just anyone, they only reply to me, and only after I've cryptographically authenticated. When others try to talk to them it's like they're not even there.

But people want their computers to talk to strangers, don't they? They want to serve pages and pages of ads to massive audiences. Unlike your exploitation example, nobody's actively invading their computers and exfiltrating data. Breaking into someone else's computers and dumping their private databases is one thing. We're just requesting the exact same data that they're more than happy to send out to literally anyone who shows up with a browser, through the exact same channels even. So I really have no sympathy.

[pocksuppet]

The expectation with scraping is that you won't cause excessive server load. The "AI" scrapers are not respecting that. But there are plenty of other scrapers that do.

Here's some stuff on Adversarial Interoperability, which is an incredibly good thing that can reverse enshittification if it becomes more widespread: https://www.eff.org/deeplinks/2019/10/adversarial-interopera...