well, I toyed around with writing my own search engine (have a pretty good proof of concept) but in the end the bandwidth costs would have been prohibitive... so there :)
The funny thing is while coding that stuff the bigger problems were financial and the enormous amount of cruft that is the web. The actual search engine wasn't that hard at all.
Yep - that's exactly it. Setting up the infrastructure to handle large, web-scale content analysis is the real challenge. (Shameless plug alert) That's why we setup 80legs: to help everyone not called Google/Yahoo/Microsoft to have comparable capabilities when it comes to this.
(This intrigues me. I had imagined the long tail queries were really hard. I mean, the places where Google succeeds and Bing fails, or vice-versa seem to me the "gaps" where for whatever reason its difficult to get things right, be they for spam reasons or scoring difficulties.