Hacker News new | ask | show | jobs
by Thorrez 463 days ago
The problem is many crawlers pretend to be humans. So to ban the rest of the crawlers by default, you'll have to ban humans.