[birken]

The different search engines generally give you ways to validate their crawlers, either with publicly disclosed IP ranges or DNS lookups.