Hacker News new | ask | show | jobs
Why Relying on Robots.txt Disallow Directive Isn't Enough (lugassy.net)
2 points by mluggy 1081 days ago