Y
Hacker News
new
|
ask
|
show
|
jobs
by
viddi
3229 days ago
To clear things up: robots.txt can retroactively
hide
content from the archive. If it's changed back to allowing the archive's crawler, content from before the ban can be accessed again.