[ravenstine]

That's a nice page, and I'll definitely bookmark that one. But I don't see most of the public using something like that over WebMD.

WebMD is complete trash (I even block it in my Kagi search preferences), but it succeeds over sites like the NHS one because of blog SEO. WebMD is really a glorified blog with posts about every condition you might get paranoid over. I don't know how precisely they achieve this, but they're doing something right in an SEO sense if The Google continues to put them near the top of results after all these years.

Meanwhile, there are also sites like Merck Manuals, which is both a terrific resource and privately run, but I don't recall ever seeing it coming up for a search query like "what is that bump on the side of my neck."

https://www.merckmanuals.com/

The resources are out there. Whether The Google thinks the average person should read them is a different story. I don't believe The Google is going to ever filter out WebMD, so there must be a middle ground where sites like the ones you and I mentioned find a way to make themselves more WebMD-like without sacrificing their more academically-minded content they already have.

[tmoravec]

So there's a private option that's terrible (WebMD) and a private option that's terrific (Merck Manuals). And it's the terrible one that ranks well at Google.

Sounds like a Google problem and apparently only a Google problem.

[ajmurmann]

That's where I continue to want to see what results in a search engine would look like that heavily punishes presence of advertisements in a result. All the SEO spam pages are ad-driven, so cutting out anything following that incentive should result in removal of all pages that follow that terrible SEO spam pattern that ruins search results.

[hombre_fatal]

Punishment/vengeance is a popular idea around here, but you have to also remember that a search engine is supposed to bring you the most relevant results.

Filtering out, say, Stack Overflow or Reddit because it has ads doesn't help you when it answers your question and is perhaps the only thing on the internet that truly does.

People seem to think there's this ad-less replica of the internet, sitting right behind our ad-riddled internet, where everything they want exists for free, it's just hidden. In reality, the websites making money are the ones providing the vast majority of things people are looking for.

Use https://search.marginalia.nu if you want to severely punish ads.

[nvrspyx]

Maybe instead of heavily punishing websites with ads, a search engine could instead punish heavily ad-driven websites. A lot of the SEO-exploiting blog mills are filled to the brim with ads where the goal is to get you to visit to view as many ads as possible, not provide good content that's funded by ads.

[function_seven]

Some sort of ratio algorithm would be nice.

Does this site (in general, not just this page) have more than 5 advertisements per page? Between 2 and 4?

Does this site attempt to load 12 trackers? "Only" 4 trackers? Just 1?

Does an AI text analysis of the first few paragraphs match on this nonsense?:

> Fixing your gadget is important. Many people find that their gadget sometimes breaks. Gadget helps us do action easier, and improves our lives. We all hate it when our Gadget doesn't work the way we expect it to. It can be frustrating. Read below for tips on how to fix your gadget. (Followed by 3 more paragraphs of filler before getting to regurgitated gems like "reboot it".

I'm sure we have the AI tech now to semantically see this bullshit and downrank it. Right? (Ok, maybe I overestimate how easy this would be. Forgive me, I'm just ranting here)

[gregmac]

Do you mean the advertising company that runs a search engine should punish pages in the results that... show their ads? Or just when it's a "lot" of their ads? Or should they only do that if the pages are showing ads from their competitors?

[derefr]

I’m honestly surprised that Google thinks that a page with N ads deserves N times the CPM. The more ads, the less attention each ad can grab, no? I wonder whether just treating ads as zero sum (regardless of ad provider) — such that a page with 5 ads, 2 of which are Google ads, gets a payout of 2/5 the CPM of a page with one Google ad and no other ads — would basically drive all these SEO mills out of business. While also not really impacting honest ad-sponsored sites (like Reddit), that only tend to run one ad per page.

[nvrspyx]

I mean when it's a lot of ads. It could perhaps be an ad to content ratio? It seems SEO spam contains more ads than content most of the time.

[kelnos]

Or maybe the search engine shouldn't care about ads at all, and just figure out what is good content and what is bad content, and what actually answers queries well.

[ajmurmann]

When I said "punish", I meant that the ranking algorithm should do that. It's not about vengeance, it's about filtering out SEO spam. The problem with filtering out SEO spam by detecting it as such is that it's by definition an arms race. That's why I propose to instead of looking for the symptom (SEO spam) pull it out at the wrong incentive structure that's causing it (ads).

[zeruch]

"that heavily punishes presence of advertisements in a result." while that is pleasing to read at face value, it has two fundamental problems:

1. it's orthogonal to relevance of content (semi-solvable algorithmically I suspect) 2. it's antithetical to Google's core business model (a lot tougher nut to crack)

[ajmurmann]

> 1. it's orthogonal to relevance of content

The entire point of my comment was that it's not orthogonal. The ads are what fuels the click bait and SEO-driven articles. Nobody for example would ever pay a subscription to a website that is just waffle filler. While stackoerflow has ads, it's much better in that regard to the SEO spam pages.

[kelnos]

Aren't you contradicting yourself there? Stack Overflow is ad-supported, but is good. But you want search engines to penalize sites that have ads?

I hate ads, but I don't think we should be focusing on them here. Some sites that have ads have garbage content, and some sites that have ads have useful content. Just... find the useful content, and return it in search results. I know "just" is doing an awful lot of heavy lifting there, but I don't think "has/does not have ads" is as important a signal to a search engine's algorithm as you think it is.

[fredophile]

> it's orthogonal to relevance of content

I disagree. The way content is presented matters. Splitting an article into 4-6 pages and filling those pages with ads makes me not want to read that content. I'd much rather go somewhere that has the same text in a single page and only a few ads.

[beambot]

What about playing 2-3 unskippable video ads before watching the actual content? Thus, Google should degrade YouTube search results as well!

[ajmurmann]

The ideal search engine would show me the ad-free page first given otherwise identical quality. Of course Google will never do anything like that. That's why I'm hoping for an alternative search engine to do so.

[aendruk]

Teclis implements a fun approximation of this. It runs uBlock Origin on results and penalizes according to the number of items blocked.

[dbcurtis]

Imagine a world where the biggest search engine made its money from advertising. In that kind of a world, wouldn’t the search engine primarily be incentivized to show you the results pages with the most advertising, regardless of the quality of the content?

[lupire]

No, because people would stop using the crappy search engine.

That's how the world was before Google.

[dbcurtis]

And that differs from today because.... why?

[lupire]

Anyone who wants attention is motivated to do SEO. Should engines downrank every site that has good SEO? That is, downrank every site that ranks highly?

They already look at things like clickthroughand dwell time and bounce back. If enough people dislike Example.com enough to avoid clicking on it or come back to search after visiting it, the engine learns that it is a bad result.

Maybe the problem is that most people like what you don't like.

[Calavar]

No, they key is to differentiate SEO'd pages with useful content from SEO'd pages with useless content.

This is a game as old as search engines. In 2005, it meant filtering out sites that were just lists of keywords, not coherent sentences and paragraphs. It meant for giving extra points to articles with structure, such as header tags and paragraphs, as opposed to just blobs of text. It meant using PageRank to organically discover which pages real people thought were useful.

It's a much subtler and more difficult problem in 2022, but there are also better tools to do it (big NLG models). It just seems that Google lost interest in quality control at some point.

And I would guess they lost interest in quality control because of Chrome's market penetration. Chrome is a browser monopoly at this point, and with Google being the default search engine on Chrome, they no longer have to give quality results to maintain their search user base. On top of that, they control such a large share of the ad market that any SEO spam website is more likely than not to be using AdSense. Which means they have a financial incentive to deliver page views to SEO spam sites, which tend to have higher ad/content ratios.

[Consultant32452]

This seems like a good fit for a ! solution like duckduckgo. !gov !universities These may already exist.

[ajmurmann]

That stuff definitely helps. That's also why do many now just search Reddit. However, wouldn't it be nice if the search engine could be smart enough to figure that out itself?

[derefr]

The problem is that people clicking+dwelling on something is not highly correlated with it serving their needs.

See: clickbait YouTube videos that show you something you really want to see in the thumbnail, then spend 10 minutes doing something else before showing it, and when you see it it’s a tiny aside with no more context than what you got in the thumbnail. If it’s even in the video at all.

Those videos have both high clickthrough (thus “click bait”) and also high dwell time (from people waiting for the thing they wanted to see to show up.) They do also have high bounceback, but only from people who recognize what’s going on. “A new sucker’s born every minute”, and those suckers will click the video and watch it, because they don’t yet know the principle that this specific kind of enticing thumbnail+title format implies that they won’t find what they want here.

These metrics all measure, effectively, “wanting” rather than “having.” It’s like measuring food by how addictive it is, rather than by how satisfied it makes you. You’ll end up optimizing toward cheetos — literally flavoured air — rather than toward anything that fills your stomach. People might enjoy cheetos while they’re eating them, but if they’re genuinely hungry, cheetos won’t solve their problem — they’ll still be hungry afterward.

[droopyEyelids]

This is so simple its easy to overlook the fact that its also ingenious

[rootusrootus]

> Sounds like a Google problem and apparently only a Google problem.

I want something like webrings to become A Thing again. A user curated search engine. And the users doing the curated need to be vetted. I don't know if this is even possible, but I get tired of having to come to HN to get a human recommendation that is miles better than the algorithmic crap from the current search engines.

[adolph]

There is the "awesome list" phenomena:

Search "medical information awesome list"

https://github.com/NeovaHealth/awesome-health

https://github.com/lalaithan/awesome-health

https://github.com/jeromecc/awesome-health

Fork and make your own!

[lupire]

Web of trust, not web rings.

Post your bookmarks, share to your friends, encourage your friends to do the same. Import those bookmarks into a search engine site fliter extension.

[hattmall]

T100 sites and the like seemed to be the peak of discovering interesting and relevant content to me.

[cecilpl2]

> And the users doing the curated need to be vetted.

It comes back to the age-old question: Who vets the vetters?

[echelon]

Unsolvable since it's a network of fallible humans we attempt to topologically score.

You can make decent attempts, such as academic peer review. Even this system perpetuates its own problems (beta amyloid) and has perverse incentives (publish or perish, falsified results), though.

Semantic web had some good ideas about this. Networks of signed FOAF data attached to articles and posts. You could form a side graph of trust information that you could revoke at any time.

[mitchdoogle]

You do. I imagine people or groups curating lists of pages or sites - they decide what to put on their lists, but you decide to include them in your personal search engine or not. Or you could fork their list and edit as you see fit.

[Cthulhu_]

It's a bit of a conundrum; on the one side, the NHS and (in a different area) MDN are better, more authoritative, etc sources, so Google should promote those. On the other, this would mean that Google can no longer cite neutrality or hide behind "the algorithm", as has been their legal defense against a ton of lawsuits where the suers said one websites should go higher or lower in the rankings.

[gtirloni]

If it's a human curating content or an algorithm doing so, I don't see how that helps Google on a lawsuit. Unless they blame sentience.

The best algorithms adapt to feedback. Surely Google's own algorithm can accept Google's feedback to adjust for flaws in it.

[nradov]

What lawsuit? There is no legal basis for a lawsuit. As a private corporation, Google is free to rank search results however they like regardless of whether that's done by humans or algorithms.

[gtirloni]

You'd think so but Google has essentially a monopoly depending how you look at it and some countries have stricter rules than others.

[nradov]

So what? You haven't cited a successful lawsuit against Google on that issue, or even a plausible legal theory. Have you discussed this with an actual attorney?

[lupire]

Google chooses to rank "authoritative" sites based on its own notion of authoritativity (which they don't share, but they decide).

They implement it as agnostic tuning as much as possible, avoiding single human chery picking sites. They use panels of humans (mturk style) for quality ratings.

Could you imagine the outrage if Google said "The government is always the best source about everything?"

What even would be the point? Use the government search engine for that use case.

[david_allison]

Google already does this. Searching for "YMYL" (Your Money or Your Life) should produce useful results:

> For pages about clear YMYL topics, we have very high Page Quality rating standards because low quality pages could potentially negatively impact a person’s health, financial stability, or safety, or the welfare or well-being of society.

https://static.googleusercontent.com/media/guidelines.raterh...

[andrewia]

I disagree. There is lots of useful information around the internet that is hard to find. A lot of content simply doesn't have the keywords it needs to be discovered.

For example, somebody could search for "best rewards credit card for married couple with normal lifestyle" and find some listicle full of referral links that's a few years old. But the best advice might be in a Reddit or Twitter discussion titled "what's a good card for my P2 with no AF, low MSR, and at least 2CPP" whose replies are terse comments like "CSP?" or "BofA Custom; select CNP transactions as the 3% category and order everything online. they give high CL if you want to AU instead". There's enough jargon and levels of understanding that Google can't find the best advice, only good keyword matches.

I recently had a similar issue when I searched for something well-phrased and generic ("how to stop wood joints from squeaking"). The results were lackluster, but after a few attempts, I found the most helpful results were actually under a specific application ("how to stop a bed frame from squeaking").

So what's my point? It's not Google's fault. They are trying their best of optimize text search, with some fancy word associations and other stuff to help. But it's going to take a lot of effort to make an efficient, scalable, general-purpose AI that can achieve near-human understanding of text and then find the most relevant articles related to that. This "super-Google" would have to comprehend every post and comment on the internet, contextualize the knowledge ("a squeaky bed is caused by squeaky wood joints or fasteners, so this advice is useful for any kind of squeaking wood furniture"), and quickly generate results for every query.

It's not WebMD's fault they have ad revenue that lets them hire writers to SEO their articles with the best keywords. Nor is it Merck's fault that they are using specific language that doesn't cover all the phrases that a person could search. Nor is it Google's for making a search engine without human comprehension. It's just a technological gap that can't be bridged in the present day.

[jleyank]

No, as I suspect the ad revenues are just fine the way they are now. Google isn’t altruistic. They want ad rates.

[winternett]

Google can make or break any online business they want to. It is what it is... We let them get there... That being said, I haven't even tried to use Bing, and it's pretty much impossible to convince me that Microsoft Edge is worth a second look after all the years of MSIE, and how Windows has been slowly devolving over time.

If worst comes to worst, just add "reddit" to your search term, and then all you have to do is determine whether he answers you find look like they came from a human, a spammer, or a corporation.

[Melatonic]

Edge is chromium based now so......

[hhmc]

I'd make the claim (without any real data), that the NHS has _such_ institutional power in the UK that it goes a long way to transcend any SEO shortfall it may have.

[petepete]

It does. Google treats gov.uk and nhs.uk sites as authoritative sources and ranks accordingly. We typically pay no consideration to SEO on non-campaign sites.

It doesn't stop ads from sneaking in above the results though. There's a whole industry of shady companies 'hijacking' and charging extra for free/cheap government services.

[biztos]

What does "non-campaign" mean in this context?

[petepete]

The sites that sit on gov.uk are usually either:

* transactional services - those that let a person do a task, like applying for a driving licence or registering a trademark

* information services - a collection of guidance on a subject, like maximum working hours or data protection responsibilities

There are some that don't really fall into either category and are there to advertise something. They aim to be informational and inspirational. Some examples:

* https://helpforhouseholds.campaign.gov.uk/

* https://www.apprenticeships.gov.uk/

* https://national-security.campaign.gov.uk/

* https://skillsforlife.campaign.gov.uk/

The reason SEO is more important on these is that there's often competition. For example, if I search for 'teach in uk' the Get Into Teaching campaign site will be competing with lots of teacher training colleges and companies, job sites, etc.

[lupire]

Why don't you use SEO on your non-campaign transactional and info sites?

Those are the ones that need protection from fee scammers. It seems you've got it exactly backwards.

[nicoburns]

SEO is what the scammers use. When Google prioritises an authoritative page, that’s making SEO less important.

[nicoburns]

The NHS definitely does rank highly in the UK. Not sure if it ranks above web md, but they’re usually both in the top 5 results if they have a relevant page.

[IshKebab]

It pretty much always ranks above WebMD in the UK. WebMD has a very low rank for me. Mayo Clinic is the site that I normally see tussling for the top spot with the NHS. Some examples randomly plucked from my brain:

1. Vitiligo: NHS: 1, Mayo Clinic: 2, WebMD: 7

2. Bronchiolitis: NHS 1, Mayo Clinic 2, WebMD: 9 (on the second page!)

3. Appendicitis: NHS 1, Mayo Clinic 2, WebMD: 3

4. Myopia: Mayo Clinic 1, WebMD 2, NHS: 3

5. Gastroenteritis: NHS Inform (never seen this before?) 1, WebMD: 2, Mayo Clinic 3, NHS: 4

6. Hodgkin's disease: Mayo Clinic 1, NHS: 2, WebMD: 5

[Cd00d]

I want to thank you for posting this. I was not aware of the Merck Manuals, and have often struggled to identify useful information on WebMD or Mayo Clinic sites.

Today my wife had a specific medical concern that we wanted supplementary information about, and because of your link we had incredibly helpful and credible insight that helped us and provided a deeper understanding of what her physician was saying.

[ricardo81]

> Whether The Google

Indeed. You mention it being SEO but I see the NHS website has a nice site hierarchy and even has schema.org data.

SEO in one sense is to 'help search engines understand a page', maybe the crossing point is where it outranks and appears too much. A site/page ranking well in Google says as much about Google as the site/page.

[lupire]

Google's problem is that it ranks pages, not sites. It will much prefer a crappy site with a crappy page that is highly specific to your search query, instead of showing you a good site related to the idea you are looking for.

It puts far too much faith in its language parser, which doesn't comprehend.

[mitchdoogle]

Sites definitely matter. All indications are that domains have a ranking as well as individual pages

[nonameiguess]

I think this is giving way too much credit to Google. WebMD has existed in some form since 1996 and first went live as WebMD in 1999. They've been the web's most popular health information publisher from the beginning. This was before Google became the web's dominant search engine.

If Google has an issue here, it's that it tries to rank things based on a more or less mathematical look at its actual popularity at the time of indexing. It isn't aiming to promote sites based on quality in order to make them popular. If a site is where other sites are pointing and people are visiting, then Google is going to rank them highly (modulo whatever the hell other secret sauce goes into the algorithm - i.e. punishing slow load times), even if the information content is bad. WebMD was very much already the web's most popular source of health information well before they hit the top of any Google search.

This isn't even a problem specific to the web, let alone Google. Medical textbooks were available at the time Chicken Soup for the Soul was a best seller. Consumers quite often prefer shittier information to better.

[brokenodo]

I’m honestly blown away by the Merck Manuals! How can this resource be so comprehensive and clear yet seemingly never come up in topical searches?

It seems like the most comparable medical version to Cornell’s Legal Information Institute. https://www.law.cornell.edu/

[MisterTea]

> I don't know how precisely they achieve this, but they're doing something right in an SEO sense if The Google continues to put them near the top of results after all these years.

One hand washes the other. WebMD is chock full of google and other ads.

[biztos]

> The Google continues to put them near the top of results after all these years.

WebMD: the Pinterest of Health.

[scarby2]

i wish pintrest could be scrubbed from the internet

[Melatonic]

I still have a super old merck manual I keep just in case of emergencies. Makes me wonder if I should be purchasing one that is up to date.....

[harvey9]

I wonder if UK users see the NHS site further up the rankings?

I'd be curious to know if people are putting in 'NHS' as a search term as well.

[raesene9]

One anecdote from a UK internet user. I just tried googling

long covid

and the top result was NHS. Not sure if it is for other nationalities but they're definitely getting good ranking in the UK

edit: I also tried flu and meningitis and NHS is top for both of those queries too.

[noneeeed]

I consistently see NHS results at or near the top for most symptom searches.

I don't know if that's geography specific.

I just tried it for "swollen ankles" and the NHS came top, followed by webmd.

[n4r9]

Yes, I'm in the UK and will often include "NHS" in the search terms when reading up on health issues or advice.

[bogota]

Not to mention pretty much every health condition you look up will list that you have cancer or some mental illness.

[habosa]

Here in the UK the NHS website has excellent SEO. I land there all the time

[Spooky23]

I disagree with the WebMD hate. It’s not the best site out there, but it isn’t delivering misinformation or otherwise problematic data.

That type of site is difficult in general. People seeking medical advice via online means are always going to struggle.