Hacker News new | ask | show | jobs
by xnx 1034 days ago
The Allow: directive was an extension to robots.txt added later.