Hacker News new | ask | show | jobs
by kjell 4045 days ago
They're there, at least the two I looked at.

https://web.archive.org/web/20130413152316/http://www.state....

Each line is missing `/documents` in the snippet of the `robots.txt`