Hacker News new | ask | show | jobs
by wimplash 4966 days ago
honeypot entry to ensure that crawlers are complying with robots.txt?