[frenchyatwork]

I think one of the fundamental things that make search work well about 1-2 decades ago was that web sites would link to each other, and that those links could vaguely correlate with reputation. There were link spammers, but there was actually a some decent organic content as well.

What's happened since then is that almost all the normal "people linking to things they like" has gone behind walled gardens (chiefly Facebook), and vast majority of what remains on the open web are SEO spammers.

[wodenokoto]

Why has blogs and articles stopped linking to things? I'm reading a restaurant review site, and they won't link to the restaurant. The chef name is a link to a list of all articles tagged with the chefs name, rather a wikipedia link or something useful that can tell me who that person is.

[orcasushi]

Average websites goal is now to keep you on them as long as possible. According to some metric folks, the longer you stay on a website the more money you spend there. Linking to another website destroys that metric.

Also if you are going to make a purchase somewhere, any website would try to get a cut of the money you spend by actually sending referral links to the product. So small websites that do not allow this service will not get linked so much.

On a metalevel it is thus that links or connections between items are information. Information is money. And as soon as that became evident links and connections also became more scarce.

[fractalb]

> According to some metric folks, the longer you stay on a website the more money you spend there.

That is really sad. Metric folks inventing metrics for the sake of metrics, which dubiously correlates to profitability of the company.

[toastal]

Yup and developers have been allowing the marketing and product teams to break the back button as well opening every external link in a new window instead so users have to keep something open to their site. You always had middle-click to do this, but now it's being forced on users.

[toastal]

I just noticed even the goobers at GitHub break the back button when you click a project link too. I don't know why people champion this brand when they have dark patterns and shoehorn 'social' functions into the proprietary platform.

[type0]

This is a prisoners dilemma of sorts and the whole free web is loosing in this.

[ijidak]

Because, years ago, linking to lower reputation sites would drain your page rank.

So everyone worried about SEO became afraid to link to anything except:

1) Their own website 2) High reputation sites like NYTimes, etc.

It's sad. Makes it harder to navigate the web.

[chongli]

Bang on. Saying that "there isn't anything out there anymore" is missing the point: Google's algorithms created this situation, intentionally or not. Before Google, people linked to what they wanted and communities would naturally cluster around topics of interest. Google came in and made reputation into a currency which effectively destroyed all these communities through incentivizing selfishness.

[Beldin]

"When a measure becomes a target, it ceases to be a good measure"

-- Goodhart's Law.

Google's algorithms didn't create this situation; people chasing high Google rankings did. Had Google used completely different algorithms yet became equally dominant, people still would have poured their hearts and souls into getting higher rankings.

Basically, an application of the tragedy of the commons. Or: "why we can't have nice things".

[chongli]

But that's taking for granted that Google would have become dominant. Perhaps if they hadn't chosen the algorithm they did then they wouldn't have been as overwhelmingly successful. Instead, I could imagine a world in which there are multiple search engines and none of them are all that good. In fact, that's the world I remember from before Google existed. Search was bad but communities were strong and life was good.

Then Google came along and we all found it a lot more convenient than the bad search engines we were used to. And of course, we all know where that led. In some sense, Google built an 8-lane superhighway and bypassed all the small towns.

We all traded away paradise in exchange for convenience. Now we have neither.

[Beldin]

On the glass-half-full side of this: we're getting those communities again! Here on HN, on reddit, for certain topics on various social media (there are pearls there too), on Mastodon, various blog authors, Ars Technica, Quanta, etc. [1]

It's just fragmented - i.e., catering to a specific group. Because if it isn't, it's awesome for 5 minutes and then monetization rot sets in.

[1] None of these work for everyone; conversely, all of these are seen as great things by some and have people who prefer that one thing over others for its quality.

[salawat]

>Google's algorithms didn't create this situation; people chasing high Google rankings did.

You're technically right. You'd be more right if you said people chased the highest spots on search engines for the widest breadth of queries.

If there were implicit alphabetical ordering of search results I guarantee you'd end up a bias toward A's, Z's or otherwise in people trying to get top spots.

[mrkramer]

>Google's algorithms didn't create this situation; people chasing high Google rankings did.

But lowkey Google incentivized such behaviour by not being open and transparent on how exactly their algorithms work.

[B-Con]

That would have allowed people to artificially chase rankings even faster and more efficiently. It makes the problem worse, not better.

[xbmcuser]

A big problem is that a lot of community content went behind Facebook. Instead of creating webpages or forums people started using Facebook pages and Facebook groups. This is the main reason I have been anti Facebook for over a decade. Not because of privacy reasons as many are but because I saw that Facebook will put the web behind it's closed doors. Even today some of the best reviews about any product or service are usually in enthusiast community forums. But a lot of that activity has gone behind closed doors of facebook and now reddit. Most of the current thriving forums are those that pre existed Facebook.

[wussboy]

Surely there is just a different algo that could bring about better communities?

[jonathankoren]

Different, but not better.

The incentives to game the algo remain. People adapt to the environment.

[kilburn]

That's why mechanism design [1] exists as a field of study. The whole idea of that field is to provide the proper incentives to steer the participants towards your objective. Yes, considering they will try to "game" the system however they can.

I'm pretty sure google could do strictly better (i.e.: better in all reasonable accounts) than they do now if they focused on the users' experience instead of revenue for a couple terms.

[1] https://en.wikipedia.org/wiki/Mechanism_design

[amelius]

> The incentives to game the algo remain. People adapt to the environment.

Perhaps it could work if the algorithm changed its algorithm all the time.

[marcosdumay]

Only if implemented by the monopolist.

People's best chance is stopping using Google and pushing for it to be broken-up.

[ajmurmann]

I wonder if punishing presence of advertisements would filter out most pages that are SEO'd to the max and instead promote "labors of love" type pages.

[mrkramer]

Wouldn't it be reasonable from Google to show how their ranking algorithms work so all webmasters and content creators know how to behave on the web. Now we have black box that's causing confusion and is misdirecting websites and web users.

[wyattpeak]

I don't think so. If the problem is people gaming the system, making it easier to game isn't going to improve the situation. It's not going to put good content creators on a level playing field with spammers, because good content creators simply don't care as much as spammers about search engine gaming.

[rtsil]

But people are already gaming the system, and on any search from product reviews to code snippets, I see SEO-optimized spams populating the top results. Good content creator don't have the time or the technical inclination to reverse-engineer the ranking algorithm (or to brute-force by creating tens of thousands of sites and see what sticks in a giant multivariate test).

Knowing the actual rules might give them a fighting chance, since the bad guys already know these rules anyway.

[freeflight]

> It's sad. Makes it harder to navigate the web.

Some would even say it killed the web by centralizing all the content in the hands of a few [0].

Which is the direct consequence of everybody optimizing to better show up on Google/Facebook/Amazon/Microsoft and ultimately even migrating all their hosting to these companies.

[0] https://staltz.com/the-web-began-dying-in-2014-heres-how.htm...

[ufo]

There aren't as many blogs now as there used to be.

[adventured]

That will get worse yet most likely. Younger people no longer produce public text to the extent they did prior to the the smartphone heavy era. Supply of that blog style content will continue to dwindle as the producers age out. I'm sure there's a stability point it may reach, of course, because some tiny percentage of people will always want to write long-form.

Younger people TikTok, they Instagram, they chat in private conversations with eachother, they occasionally post short messages in walled gardens like Facebook, they YouTube, they listen to music, they watch Netflix & Co. That's what they do. They do not persistently write LiveJournals, Tumblrs, blogs. That pre video/audio-focused era is over and it's not coming back (even if there's occasionally a bubbling up of hipster fakery centered around how cool it is to write text).

[lmkg]

I heard an interesting theory the other day: blog viability declined because Google killed Reader. Which indirectly ends up poisoning Google's biggest well, since blogs are an important source of relevant cross-domain links.

I'm somewhat skeptical, it seems a little too poetic to blame Google's ultimate downfall on a decision that was notably hated at the time. But it's plausible. If you want it to be a conspiracy theory, you can posit that killing off independent blogs was the intent, to convince bloggers to migrate to Google Plus.

[heavyset_go]

Google used to prioritize blogs and original content like forum posts in search results, but they don't anymore.

[type0]

What do they prioritize now, "reputable" news organizations enrolled in Trusted News Initiative?

[heavyset_go]

Blog spam and pages filled with AdSense ads.

[nkrisc]

I find that claim surprising considering how many more people there are simply using the internet at all.

Fewer unique blog domains due to “blogging” sites that aggregate users? Sounds plausible. Fewer people blogging overall? I’m not convinced yet.

[Baeocystin]

I'd believe it. As an IT consultant, I interact with a lot of people who are semi-techs themselves- mostly small business owners who are used to wearing a lot of hats, and also the type to have been motivated to run their own personal blogs about diving/photography/conlangs/quilting/gardening/whatever their personal hobbies are.

Ten years ago, the majority(!) had at least something up and running, where they would post essays, thoughts, whatever came to mind.

Nowadays? All gone. All! When asked why, the answer almost always is along a mix of ever-increasing negative feedback and harassment from randos, and aggressive automated spamming of their forums. Loss of the pseudo-anonymity plays a large role as well. Many have deleted years' worth of work, simply because they are afraid of someone trolling through their posts to find something to harass them with.

I was never a blogger myself, but I am sad about the change. There was a lot of good stuff out there for a while, and sometimes it just plain made me happy to read someone joyfully nerding out on a favorite subject of theirs.

[nkrisc]

I think a lot of people are still writing this kind of content, but you have to look elsewhere for it: Reddit, Facebook, Twitter; to name the obvious ones. It’s also harder to find, but you can find all kinds of personal content written in comments and posts on these sites.

[Baeocystin]

I realize that this is a hard thing to 'prove', but I am personally certain that the amount and quality of such things has dropped significantly from a decade ago.

Not to zero. You can still find things tucked away in a post on reddit or the like. Almost never, as far as I have experienced, on Facebook or its ilk, as the affordances are different. I genuinely think there has been a loss.

[kritiko]

I frequently append site:reddit.com to searches for a niche search term these days. I think a lot of people who would have blogged or commented on blogs are posting there instead.

[bee_rider]

I wonder if they'll do a walled garden after their IPO. I've always found the site pretty useless, outside the 'old.reddit.com' version. On the bright side, maybe this will open up space for one of the federated clones to grow.

[ufo]

I think the bigger issue now is that more content is inside social media "silos" like twitter, instagram or youtube. I don't have the numbers though.

[Volker_W]

Why is this a problem? Can't google index social media silos?

[bluGill]

Which ones. They can index their own, but for the others only the public stuff. Facebook has a lot of things private so nobody can see them except your friends. (they are by no means perfect, but a lot of things are private and only seen by friends - most of it isn't of interest to a search engine anyway but comments of the form "I love X product" could in a perfect world be indexed as a sign of what people find good)

[freeflight]

> I find that claim surprising considering how many more people there are simply using the internet at all.

Most of these many more people are mobile users, where creating long-style text content can be quite bothersome.

What ain't bothersome, with a smartphone, is taking pictures and videos to slap filters over them, alas that's why we are where we are with TicToc, Instagram and Twitter dominating large parts of the web.

It's even noticeable in a lot of online discussions with text outside of these communities; The average length of forum posts feels like it's gotten way shorter over the decades. People have less attention to read anything that looks longer than a few sentences, often declaring it a "wall of text" based on quantity of text alone.

Imho it's a big part of what drives misinformation; Doing any kind of online research on a small phone screen is extremely bothersome compared to the workspace an actual computer/laptop, particularly with multi-monitor, gives.

There's also the difference in attention; When I sit down at my laptop/desktop, I actively decide to spend and focus my attention on that task and device.

While smartphone usage is mostly dominated by short bursts of "can't do anything else right now", I don't chose to take out my phone and surf the web, it's something I do when I'm stuck in some place with nothing else to do and no access to an actual computer.

But for the majority of web-users [0], that smartphone access to the web is all they know, which then ends up heavily shaping the ways they consume and contribute to it.

[0] https://techjury.net/blog/what-percentage-of-internet-traffi...

[denton-scratch]

> TicToc, Instagram and Twitter dominating large parts of the web

For my part, I'm glad these fora aren't indexed well; I don't want my search results dominated by single-sentence posts and photos. In particular, I don't have accounts on any of these services.

I'd be happy if search engines would decline to index sites behind paywalls. Links to Medium, Substack and Washpo are very common, and if the first thing I see is a popup demand for payment, that browser-tab gets closed.

[prox]

I wonder if it would be possible to have a big filter button “commercial” or “non-profit” or something along those lines. So you get results that are not deemed commercial or are.

Don’t know how hard it would be to know which is which. Maybe non-commercial : don’t run ads, don’t sell a product or service and provide information only.

[lethologica]

I wonder if the majority are moving to vlogging instead?

[srg0]

The original Google algorithm was a clever hack but it relied on web being a hypertext, and links being used contextually.

The algorithm made linking valueable. So instead of writing hypertext people tried to create isolated sites and boost their rank. Remember rel="nofollow"?

Eventually bigger sites took over the small ones.

[kyriakos]

Noticed this with online newspapers too to the extend they are reporting about a website or product and don't include a URL to it.

[dpeck]

I agree very much with this. It seems that between the walled gardens and also people being so reluctant to have “their” audience leave their site/page/etc the discoverability of the web has dropped dramatically.

[ZetaZero]

That's an interesting observation. IMO, we stopped linking to good content because Google was good at finding it. Now Google is suffering, and we need to go back to doing more links.

[Mezzie]

Yup, early Google relied on a lot of unpaid , unseen human intervention and choices. I ran some weblinks and curatorial sites during the search wars, and PageRank could only work because there were people behind the sites choosing links based on their usefulness to their audience.

[prepend]

I wish FB would be more open, but since they have all this walled garden info, are they well placed to start a competing search engine? Would be interesting if their activity could help filter out seo hackers.

[kritiko]

FB search seems to have gotten worse and worse. Unless I can remember the specific Group where I saw something, it's very unlikely that I can find it again. And they know which posts I've been highly engaged on...

[snowwrestler]

This explains why running a search engine on the original Google PageRank algorithm would not work as well today as it did back then.

But Google doesn’t run on PageRank anymore. PageRank is merely one of hundreds of signals they use to sort results.

[inlined]

Does this mean that Facebook is the only company well poised to take on google search?