[Reuzel]

> Does anyone have any actual statistics or quantitative data on the quality of Google search results?

Google has. They use this data expertly to improve search. Common sense and technological advancement tells us that, quantitatively, Google search has become better year over year, for all their relevant metrics/cost functions.

And likely, exactly because it has become better for all its users in aggregate, it has to become a bit worse for a certain group of power users. There, we can only rely on anecdotes and personal experience, but these tell us it actually has gotten worse.

Similarly, the web can become both worse and better. The really useful articles today are better researched, multi-modal, solid web of links, internet-first. Spam has also evolved. And "top 10 ways to do X"-McContent outranks better articles, because that is what the majority of Google users wants to see and clicks on. They truly have a better experience, while others' experiences suffer. It depends on what you measure.

[willhinsa]

> Common sense and technological advancement tells us that, quantitatively, Google search has become better year over year, for all their relevant metrics/cost functions.

lmfao. so you're telling me "quantitatively" that google search results have gotten better, without citing any data at all, but with an appeal to common sense and "technological advancement"?

what if i told you that search is an adversarial problem, and that it's possible for google's tech to be getting better slower than the aggregate tech power used to game google search is getting better? is this not a patently obvious possibility? it's not some kind of gotcha impossibility for google's tech to get much worse over time, even if they weren't hamstringing themselves by lots and lots of user-hostile changes which benefit google's interests rather than their users.

[Reuzel]

> lmfao. so you're telling me "quantitatively" that google search results have gotten better, without citing any data at all, but with an appeal to common sense and "technological advancement"?

Yes. If that sounds so unacceptable or strange to you, I suggest you try it. Works really well when reasoning about unavailable data, or researching a field with slow peer-review process.

> what if i told you that search is an adversarial problem

Then I get an adversarial reaction and a downvote from you.

> it's possible for google's tech to be getting better slower than the aggregate tech power used to game google search is getting better? is this not a patently obvious possibility?

Yes, that's plausible. Should be measurable quantitively too. Can you cite some data on this? :)

We could compare to the available data on the quality of (HTML) e-mail spam filtering over the years, which all have kept up. Like pg said: Spam is solved, when skilled spammers start creating content which does not look, feel, or talk like spam. So webcontent-spammers still are on the first 2 pages of Google, but with content not classifiable as spam/content farm.

> which benefit google's interests rather than their users.

One of Google's interest is their user. But perhaps not the type of user you are. Studies have shown the value that the tech of Google is delivering its users per year. This value was in the thousandths, and this value has risen. Meanwhile, Google makes about tens of dollars per user per year, less for technical users which don't click ads or block these.

> it's not some kind of gotcha impossibility for google's tech to get much worse over time

It really is, no way to mince it. Google search is funded by Google ad tech. Google ad tech has improved ML by a ton. To say Google tech is getting worse, is to totally overlook deep learning revolution, word2vec, transformers, BERT, etc. etc. etc. To state that, is to reveal the truth that you are ignorant of major technological advances in the past decade, only looking at the issue from the viewpoint of a single atypical Google-search user. What would you even do with quantitive SEQ data?

Do you really think it is possible that Google runs an implementation test of a new ranking model, and deploys it, while all measurements, human labeling, and user tests show it is doing worse? Of course not! Only if you think you are smarter, could you think that Google search changed and has gotten worse.

If Google does ML like the rest of the industry, all model changes move up their designed levers, or these changes are not committed. So if Google was unable to improve search, then Google would have looked exactly like 2008 Google. The fact that it does not, shows either you or Google is wrong. If I had to make a bet...

[breakfastduck]

> Yes, that's plausible. Should be measurable quantitively too. Can you cite some data on this? :)

It's a bit bloody rich for you to come out with that attitude when your entire point revolves around your opinion of 'common sense'

> Do you really think it is possible that Google runs an implementation test of a new ranking model, and deploys it, while all measurements, human labeling, and user tests show it is doing worse? Of course not! Only if you think you are smarter, could you think that Google search changed and has gotten worse.

You seem to be confusing the concepts of 'Google have made their tech more profitable' and 'Google have made their search capability better'.

[Reuzel]

Of all things, I am sure that this discussion thread did not add value to this community conversation, and as such, serves as spam. Apologies, and let's hope Google bot finds ways to ignore low-information content in a threaded forum. Maybe they could even locate the exact post which caused the derail, and apply some authority penalty on its author.

[breakfastduck]

Smartness system

[Reuzel]

No, I used common sense, but I actually have the quantitative data that poster was asking about. Right now, I am doing exact keyword matches and trying to find myself. Will get back to you when my analysis is done.

> You seem to be confusing the concepts of 'Google have made their tech more profitable' and 'Google have made their search capability better'.

No, you are confused. Try to Google the article I was talking about. It talks of perceived value to the user. What value would you lose without access to Google maps, search, youtube, gmail, etc.?

Google made their tech more valuable. That sounds like an improvement to me.

> Google have made their search capability better

If you want an explanation for this obvious statement (and not be demanded to ask in return how capitalism works), I suggest you first try to code a simple search engine. I think a 100-line Python script with some imports would do. Only this would make talking about capabilities possible.

[ichydkrsrnae]

I'm not buying the tribal argument, that I'm the wrong user for Google. The results are shite across the board regardless of tribe.

I think they've accepted that SEO has killed past ranking algorithms and are rebuilding rank from the ground up using ML with the entire internet as guinea pig. All of the crap results we're weeding through now is grist for the ML mill.

I am literally typing exact phrases for content I know is there and not getting the results I should.

I think they've cut the cord with past algorithms, not an incremental update, a major one.

My thinking so doesn't make it so, however, so just my two cents.

[astrange]

> We could compare to the available data on the quality of (HTML) e-mail spam filtering over the years, which all have kept up. Like pg said: Spam is solved, when skilled spammers start creating content which does not look, feel, or talk like spam. So webcontent-spammers still are on the first 2 pages of Google, but with content not classifiable as spam/content farm.

I've actually noticed Gmail spam detection doesn't work as well in the last year or two. I get an obvious spam message maybe every other day. Ads with all images and all.

[Doctor_Fegg]

> So webcontent-spammers still are on the first 2 pages of Google, but with content not classifiable as spam/content farm.

The exact clones of StackOverflow and Wikipedia are all over the first two pages of Google.

[skinkestek]

Because Stack Overflow is its own worst enemy and routinely deleted extremely useful questions because they "didn't fit in"?

Is it the any surprise if it ranks highly on alternative sites, spam or not?

[hnaccount141]

This presumes that the metrics they optimize for are intended to represent usefulness to actual users and not, say, ad revenue. Even if they do intend to optimize for usefulness, this doesn't mean that they have metrics that accurately represent that.

I also think you're underestimating average users. Anecdotally I've heard my parents complain repeatedly about the incoherent, auto-generated, affiliate link spam that plagues product searches.

[Reuzel]

> This presumes that the metrics they optimize for are intended to represent usefulness to actual users and not, say, ad revenue. Even if they do intend to optimize for usefulness, this doesn't mean that they have metrics that accurately represent that.

They have multiple levers, of which user search quality is a big set. There is always trade-offs and a balance that must be found, which aligns with company vision and strategy. Having these levers allows business-decision makers to direct focus top-down (on a certain set of users, on producing great ad numbers, etc.).

It is clearly hard and important to design these levels and find the right balance, given a rapidly changing company and user-base. So a lot of expertise and power is invested to measure the right things, and to find the right balance (an incorrect/risky balance should also be adjustable with other levers).

So for me: either Google is trying really hard, but essentially failing. Or they have the best of the world, with all the right context, designing these levers. While hard and sometimes wrong, I do not expect to contribute anything which may improve their lever settings. If someone does know, Google would like to hire them.

So while true, that accurately measuring things with proxies, is really hard, and sometimes done wrong at companies. I do not think Google gets this wrong, or at least, gets this to be the best of breed. If their metrics still cause long-term search engine quality loss, would show them to not know what they are doing. I think they do know very well, better than me at least.

I would agree too that the balance of levers right now is in line with Google's strong market position. Search engine quality could take a small hit, if justified with extra adsense income. But when search engine quality noticeably start going down, then all other metrics will suffer. You should have teams with sole focus on improving quality. Other teams will have to realize that favoring their lever over the search-engine-quality lever must lead to worse outcomes for Google in general.

About product searches, I myself was not able to do this satisfactory 15 years back. It improved. But need to stop viewing things as a single lever, a single metric. To say search has become "worse" in general, is to exactly fall into the trap of not accurately measuring and losing too much nuance/details for competing objectives.

[skinkestek]

> So for me: either Google is trying really hard, but essentially failing.

If Google really really wanted they could find out why they more often than not include results that doesn't contain my keywords even after I have put doublequotes around them and hunted down and applied their verbatim setting!

After that they could think really hard about how relevant the text:

> and something someone said xyz.

>

>abc is next up and something something

is for someone searching for "xyz.abc"

Or maybe see if they can dig out an old cheat sheet with all the operators they used to support and invite some old Googlers to secretly come in and teach about it but that can wait until they got the basics working again.

[bsanr2]

I don't think this is necessarily true. Your standard desktop computer has gotten much easier for a casual user to navigate over time, but not any less robust for power users. This is because powerful tools for customization and building are still exposed to power users. Google has slowly stripped away many of these tools. One has to wonder why, and my guess would be that it's because they would expose either the unethical ways Google deals with results, or the failure of its search model.