Hacker News new | ask | show | jobs
by sjg007 2434 days ago
I don't believe that robots.txt has ever had the backing of law.