Hacker News new | ask | show | jobs
by pfych 481 days ago
I use a robots.txt[^1] to try to prevent them from calling my site. But they're probably doing it regardless ¯\_(ツ)_/¯. Don't want to completely kill my site from indexing, but if I did I'd 100% set up a tarpit.

[^1]: https://github.com/ai-robots-txt/ai.robots.txt