[reitzensteinm]

There's a scary side to this - what if Google themselves gave you better rankings or lots of free adwords if you agreed to only be indexed by them (selectively block the other crawlers)?

Of course they have no reason to now, and maybe not ever, and even if they did maybe there would be anti trust issues. But it's still a scary amount of lock in - probably even more powerful than Wintel.