Hacker News new | ask | show | jobs
by PunchyHamster 77 days ago
Given that the scraping doesn't do any rate limiting and pisses on robots.txt, yes it is abuse
1 comments
singpolyma3 77 days ago
Is there any evidence OpenAI has been ignoring robots.txt for scraping purposes? AFAIK the main sources of that traffic are still unknown.
link