Hacker News new | ask | show | jobs
by yuhong 4224 days ago
What is funny is that using robots.txt would have been enough.