Hacker News new | ask | show | jobs
by nydel 2531 days ago
i move my not-to-be-indexed stuff around a lot: renaming, archiving, etc. & i've a bit of shell scripting and a commonlisp program that automatically add things to robots.txt so that barely any of the things listed won't 404, and the ones that won't are protected via htaccess.

not sure why i did this aside from that it was fun!