[Animats]

The value of looking at queries is that it allows learning what questions users ask. The front end of the search process is to infer from the query what the user really should be given. That's a machine learning problem. The head of Google search remarked recently that "as the search engine gets smarter, the queries get dumber".

This is reflected in Google's search results. A Google query which can possibly be interpreted as related to a popular culture item usually will be. Google has become more aggressive about this over the years. Their "Did you mean" result tag once offered an alternative for a second search. Now, they return results for the more popular interpretation first.

The back side of search, page quality and ranking, is weaker than many think. Links are less useful than they used to be. Most links to business sites are now from "social" sites or forums, which are easily spammed. Using social signals was a disaster back in 2012, when, for a few months, Google went all-in on social signals. Google tried to recognize sites that "look like spam", but everybody knows that now and spam sites look better than ever. (The same thing happened with spam emails a decade ago.) Google doesn't recognize provenance, so they can be fooled by scraper sites. Google doesn't recognize the business behind the web page, so they can be fooled by marginal businesses. There are even SEO companies using machine learning to reverse engineer Google's algorithms, to find out how far they can go with keyword stuffing before a penalty kicks in.

Google does far more manual adjustment than they did two years ago. There's an army of people doing manual ranking, and a smaller unit handing appeals from manual penalties. There was a time when Google boasted they did no manual adjustments to ranking. The automation is starting to fail.

[sanxiyn]

1noon (Korean web search startup) tried to recognize provenance and was somewhat successful. But that wasn't enough to win in the market. Naver acquired 1noon.