[mahranch]

> we want to maintain an artisanal search index of 1B documents. Then our cost comes down to $12M/yr.

Yeah, no. It doesn't cost that much to maintain an index of that size. Not even remotely close. My roommate coded a search engine back in 2001-02 as part of a class project/hobby and it easily had an index that large, probably larger till he shut it down. The key lies in not crawling it all in a single day. And your index doesn't populate overnight, it takes months of slow crawling. You can maintain an index of practically unlimited size for chump change (well, compared to the millions OP was throwing around). Do people think Excite, Hotbot, Lycos and Yahoo! in the early years had $12 million per year to spend on their indexing? Hell no. Every single one of them would have went bankrupt in a week. Those companies weren't even worth that much back in the mid to late 90s. Your biggest cost is going to be user loads on your servers (CPU/Ram/Bandwidth), not maintaining an index.