Hacker News new | ask | show | jobs
by tylerl 2502 days ago
The bot doesn't need to crawl your site for it to be indexed; it crawls other sites that link to yours.

You can specify your index preferences in Webmaster Tools. Don't know if there's a domain-wide off switch in there, but there probably is.

1 comments

Using Webmaster Tools is not a good option since it requires you register with the exact company you are probably trying to not interact with.