[lacker]

It was clear that Yahoo.com was the definitive result for the query "yahoo" so it would score a 10. Other Yahoo pages would be ok (perhaps a 5 or 6). Irrelevant pages stuffed with Yahoo-related keywords would be spam.

As someone who worked on search quality at Google for some time, this bit jumped out at me as a terrible mistake. The correct way to judge results for the query [yahoo] is:

(a) Where is yahoo.com? At the top?

(b) There is no (b).

It seems like a slight difference, but it leads to the wrong priorities. For the query [yahoo], it does not matter if spam or non-spam is in spot #5. The only thing that matters is where you put yahoo.com.

[bitops]

Can you elaborate a bit more on this? I think I understand your point but a few more details would be great. Thanks!

[ricardobeat]

Most people don't look farther than the first 3 results, so what comes after that isn't very high priority. Might be the reason some more specific/obscure queries on google return terrible results :)

[Zakharov]

More specifically, if people find what they're looking for in the first result, they ignore the rest.