Hacker News new | ask | show | jobs
by rvbissell 631 days ago
Someone who doesn't care about polluting our corpus is not going to care about your robots.txt