[throwawayinfo]

I don't think the index is the challenge. Anyone with enough machines and gigabits can index a big enough portion of "the web". The real difficult part is ranking and understanding web pages themselves.

[ogre_codes]

> The real difficult part is ranking and understanding web pages themselves.

This is what I was referring to.