[raincole]

If Google doesn't provide a way to say "please don't scrape my site", then it 100% unethical.

We have robots.txt. If Google doesn't respect that, it's unethical. Don't you think so?

[RobotToaster]

Does twitter's robots.txt forbid scraping? Judging by the fact it shows up in Google I'd assume not.

[pcthrowaway]

Maybe it's time for an llm.txt

Not that the people you want to respect that would

[raincole]

The tricky part is it's much more harder to prove that they didn't respect that.