Hacker News new | ask | show | jobs
by daveoc64 149 days ago
All the comments there seem to suggest that there has been no change and that robots.txt isn't required.