|
|
|
|
|
|
by nydel
2531 days ago
|
|
|
i move my not-to-be-indexed stuff around a lot: renaming, archiving, etc. & i've a bit of shell scripting and a commonlisp program that automatically add things to robots.txt so that barely any of the things listed won't 404, and the ones that won't are protected via htaccess. not sure why i did this aside from that it was fun! |
|
|