Hacker News new | ask | show | jobs
by erhk 1663 days ago
Trusted curators is a dangerous dependency
5 comments

Trusted consumers are better. The original page-rank algo was organic and bottom-up. But now it's the person not the page. Businesses compete for interaction not inbound links. So if you can make a modern page-rank that follows interaction instead of links and isn't a walled garden then I'd invest.
I could make that work, but what do you mean by "walled garden" in this context?
the business and allies of google - those entrenched interests that limit the current visibility of the web to themselves
That’s why you don’t make it a hard dependency and let people curate their own list of taste makers. They can share and exchange info about who good taste makers are and good one might even charge for access to exclusive flavors.
It is. The alternative is scooping everything and using algos to curate. That seems worse imo.
Perhaps vote on results like on Reddit posts? Gets the junk sites down (and out of the index eventually).
Any open voting system is going to be under serious SEO pressure.

That’s the real issue, Google has indirectly infected the web with junk sites optimized for it. Any new search engine now has a huge hurdle to sort through all the junk and if it succeeds the SEO industry is just going to target them.

A more robust approach is simply pay people to evaluate websites. Assuming it costs say 2$ per domain to either whitelist or block that’s ~300 million for the current web and you need to repeat that effort over time. Of course it’s a clear cost vs accuracy tradeoff. Delist sites that have copies and suddenly people will try to poison the well to delist competitors etc etc.

Adding a gatekeeper collecting rent isn't a solution - the people using SEO are already spending money to get their name up high on the list.
This is money spent by a search engine not money collected from websites. People don’t ever want to be sent to a domain parking landing page for example.

More abstractly SEO is inherently a problem for search engines. Algorithms have no inherent way to separate clusters of websites setup to fake relevance from actually relevant websites. Personally I would exclude Quora from all search results, but even getting to the point your trying to make that kind of assessment is extremely difficult in the modern web. Essentially the minimum threshold for usefulness has become quite high which is a problem as Google continues to degenerate into uselessness.

Given Reddit is notorious for it's problems with astroturfing and vote bots, I don't think this is a particularly promising approach.
Reddit is a heavily gatekeeped community by the mods in regards to specific topics
Reddit is an extreme example of group think. Try posting something pro-Trump (I mean, surely even that guy has a positive thing or two to be said about him) and you'll get banned in some subs. Or you may get banned simply because the mod doesn't like the fact that you don't toe the party line.
Also, vote bots
That just means that you have to curate the people allowed to vote. Otherwise, it would be rule by the obsessed and the search engine optimizers, and the junk sites will dominate the index.

I'm not convinced that Google's recursive AI algos aren't a functional equivalent. They let you vote by tracking your clicks.

Plus, it scales less well than pure algorithmic search. This fight already happened, with a much smaller internet.
It works really, really well for libraries. Research libraries (and research librarians) are phenomenally valuable. I've missed them any time I'm not at a university.

Both curators and algorithms are valuable. This goes for finding books, for finding facts and figures, for finding clothes, for finding dishwashers, and for pretty much everything else.

I love the fact that I have search engines and online shopping, but that shouldn't displace libraries and brick-and-mortar. Curation and the ability to talk to a person are complementary to the algorithmic approach.

> It works really, really well for libraries

It scales extremely poorly. It works very well for situations where there are customers/sponsors are willing to spend lots of money for quality, because then the cost scaling doesn't matter as much; research libraries, Lexis/Nexus Westlaw, etc. all do this, but it's not cheap, and the cost scaling with the size of the corpus sucks compared to algorithmic search.

It is among the approaches to internet search that lost to more purely algorithmic search, because it scales poorly in cost.

+book stores. Curators can use algorithms to help them curate… Google’s SE is taking signals from poor curators imo.
How about just a meritocratic rating? Even here on HN I would appreciate some sort of weight on expert/experienced opinion. Although in theory I like the idea that every thought is judged on its own, the context of the author is more relevant the deeper the subject. That's one of the reasons I still read https://lobste.rs. It has a niche audience with industry experience.
Lobsters is a great example of the benefits _and dangers_ of expert/experienced opinion. Lobsters is highly oriented around programming languages and security and leaves out large swaths of what's out there in computing. That's fine of course, but it creates a pretty big distortion bubble that's largely driven by the opinions of the gatekeepers on the site rather than a more wide computing audience.
Nothing is meritocratic. I think the term came into our lexicons because of a sociologist satirizing society and writing about how awful a “true” meritocracy would be.
> meritocratic rating

That is literally PageRank.

Pagerank was mostly based on inbound links. A popularity contest with some nuance is just that. Nothing is meritocratic including any Google algo.
It's not merely a democratic vote, where the most links wins, but what the algorithm does is evaluate the links based on the popularity of the originating domain. In other words, meritocratic rating.

You can apply the algorithm to any graph, and what it does is find the most influential nodes.

Meritocracy isn’t a thing. The person who coined it was rightfully mocking society. The recent 2019 Meritocracy Trap book goes further into this.

Your explanation is not “meritocratic”. The wealthy and powerful largely stay on top with the nuance you provided. The popular have more popular. are able to make what they link to more powerful and thus more popular. There is no meritocracy there.