[mdekkers]

To properly allow the Googlebot to crawl your site, you usually combine checking googlebot with an IP whois lookup. This is also what Google recommends.