[JohnFen]

This is why I stopped relying on robots.txt a long time ago. I still use it, but I also have my server check the user agent for crawlers and return a 403 to them.