Hacker News new | ask | show | jobs
by bhaney 62 days ago
I get a few points for having a robots.txt with rules specific to AI-crawlers, even though those rules are complete bans. Shame, I was hoping to get a 0.