[Sammi]

No robots.txt can't fix this.

Have you tried Anubis? It was all over the internet a few months ago. I wonder if it actually works well. https://github.com/TecharoHQ/anubis

[EPendragon]

This is a really cool tool. I haven't seen it before. Thank you for sharing it!

On their README.md they state:

> This program is designed to help protect the small internet from the endless storm of requests that flood in from AI companies. Anubis is as lightweight as possible to ensure that everyone can afford to protect the communities closest to them.

I love the idea!