How about xmarks (used to be foxmarks)? They highlight search results depending on which of their users have bookmarked them (they are a bookmark sync service so they have access to that data). After all, bookmarking is a way of upmodding. http://www.xmarks.com/about/features/smarter_search
Xmarks search is indeed a cool concept (much like delicious search), but I don't bookmark everything useful I find on the web - so bookmark-based search will never be quite as comprehensive.
In a way, search engines already account for up-modding by factoring in which links you click on when you are given a page of search results. But you are more likely to click on results already at the top, so I wouldn't give too much weight to that information.
The SearchWiki concept has a lot of potential if Google decides to go down that path though.
I agree that a bookmark based search will never be comprehensive. Neither will any other kind of social solution - I doubt anything will do sort of a search engine that looks at semantics.
The real problem with SearchWiki (or other upvoting/downvoting) is that you keep going back to the search page while your search is failing. As soon as you click through and find the page that you want, you stop - you do what you wanted and get on with your life. Whereas if that is the page you need and bookmark it (granted your point that we don't bookmark everything), that is a true and accurate indicator of success.
Since HN was explicitly mentioned upthread, it would be interesting to know how many people upvote/downvote stories (which you have to get back to the main page to do) versus comments (which are in your face in their totality when you are asked to vote). I can only speak for myself, but I vote on comments massively more often than I vote on stories, exactly for the reason I described. So in order for SearchWiki to really work for me, I want to express my happiness when I am viewing the target page, not the search page. (I am sure this can be accomplished in theory, I just don't think the support is currently there).
How about a history-based search engine? A plugin that simply submits every link you visit (when your privacy features are off) to the aggregator, along with how long you kept each page open before you closed it/surfed away. I know it would be incredibly invasive, but it would basically be a Nielsen Box for the Internet.
I don't think you can do that just via the history. A lot of pages are just sitting there in my tabs because I am not paying attention to them, not because I am engrossed in them. You need to be able to see what the user is really doing.
I remember a while back in the UK there was a study to actually measure real attentiveness while the TV was on. Cameras were installed inside people's TVs to record the viewers so their focus could be noted. This was all very clearly explained to them, but as time went by, and people being people, they eventually started to forget the camera was there; so in some cases the poor researchers got a bigger share of, err, domestic activities than they bargained for.