Hacker News new | ask | show | jobs
by setr 2333 days ago
Robots.txt doesn't interfere with anything. It's a suggestion.