[paulcole]

In a way it’s even worse. Listing the stuff you don’t want crawled might be more like a text file with a list of vulnerabilities that hackers shouldn’t use against you.