[sarah180]

Google and Microsoft were not the first web spiders.

A famous example, Yahoo!, didn't walk away from search and partner with Microsoft because of the difficulty of building an index. They did it because it was going to cost billions per year to try to keep up with Google in producing results.

I'm not arguing there's no work to do in building an index, but the problems of crawling and indexing can be solved by cash. They're a moat against small challengers, but not against well capitalized ones. Ranking and filtering require lots of research and tuning. This is the moat against even the well capitalized.

Put another way: do you really disagree that Google would still easily dominate search based on result quality even if small startups got access to their index data?