[itsbjoern]

Always find it amusing when people write about „blocking“ requests using robots.txt as if they are deploying a firewall

[gitgud]

Agreed, all it takes is another site to copy the content, then an LLM could just scrape that…

An open web that block scraping… is likely “not an open web”