[spacecadet]

I had this same thought, more along the lines of expanding the usage of robots.txt