Hacker News new | ask | show | jobs
by pseudalopex 1047 days ago
> Agree with their position 100%.

Including the part where they blamed anyone but themselves for hiding old snapshots when robots.txt changed?

1 comments

no I didn't see that in there, I just agree that robots.txt makes sense for web crawlers but not archival purposes