[joshuamorton]

Having disk space is only a small part of being able to make use of something like a search index. Arguably the least difficult part.

One of your suggestions was

> That's why the stipulation that other entities be allowed to mirror the index - they can optimize the index for their own purposes and rankings on their own hardware.

And the point is that there's nobody who can do this outside of Google, Microsoft (who also does), Facebook, and Amazon.

Not to mention the problems of actually getting the data. You're at the scale of data where trucks of disks are faster data transfer than cables unless you have direct fiber backbone connections.

[jeffbee]

99% agreement, except for the amerocentric viewpoint. I think it is likely that Baidu has the scale.

[joshuamorton]

I knew I was missing someone. Yes, Baidu (who also already runs a large search index) could probably do the same thing.