Hacker News new | ask | show | jobs
by janmo 73 days ago
> I closed port 443

> Now closing https service is obviously just a temporary fix

Probably the best starting point would be to edit the robots.txt file and disallow LLM bots there. Currently the file allows all bots: http://acme.com/robots.txt

2 comments

The LLM scraper bots ignore robots.txt
If the bots respected robots.txt, I would not have a viable business plan.