Hacker News new | ask | show | jobs
by quesera 4933 days ago
Respecting robots.txt is probably the best plan.