[tlonny]

As I understand it, anything publicly viewable is free to be scraped without issue.

If you scrape stuff behind a sign-up, you potentially open yourself up to being sued for breach of ToS.

Now I'm sure the Chinese labs don't give a rats arse about that threat, but I imagine it stops the likes of Google, OAI, Anthropic from scraping and instead forces them to purchase the data.

[benjiro29]

Not how it works ... Anything public viewable is still subject to a TOS. And the whole copyright or whatever still applies.

If your argument was valid, we can scrape news websites and show their content freely. No ... You get sued and lose the case.

See Google News that got their behinds in court and lost. Short summaries are allowed / transformative, but you can not just take content (even without a login). Not without getting into civil court if somebody wants to press the matter.

Now, if you scrap and never make that data public or transform it (LLMs). Then it becomes a harder matter to deal with.