Hacker News new | ask | show | jobs
by lucb1e 919 days ago
Presumably due to https://neatnik.net/robots.txt
2 comments

Wow, thanks for pointing that out, that made me never want to join omg.lol
That's not present on another omglol site that was linked elsewhere in the thread, though.

I would agree with you if they automatically set this for everyone. I'm not sure how come that other sites aren't showing up in the archive. (I'm not a customer of theirs or anything, I'm already hosting my own stuff)

> User-agent: ia_archiver

> Disallow: /

Denied!