Y
Hacker News
new
|
ask
|
show
|
jobs
by
bhaney
62 days ago
I get a few points for having a robots.txt with rules specific to AI-crawlers, even though those rules are complete bans. Shame, I was hoping to get a 0.