[pete_nic]

In reading this article on finance my mind drifts to the refrain on HN that search (Google) is broken and ready to be disrupted. Can consensus mechanisms on distributed systems help create a less spammy, more equitable search engine?

> a consensus mechanism can be considered as a shared system of standards and steps that allows a distributed system (a group of individuals or computers) to accurately, precisely, and reliably coordinate on shared problems.

> Consensus mechanisms (also known as consensus protocols or consensus algorithms) allow distributed systems (networks of computers) to work together and stay secure.

[cinquemb]

Seems like it would need a consensus mechanism on how results can be filtered from some base algo (i.e. anyone could pay [how? tradfi or defi] to weight a site or page on a it down on an index, but if people still click it more than other sources, it could make down weight more expensive to hide it), centralized from a single corporate entity vs decentralized among more actors, and where the indexes are stored/shared/updated and how can the costs of such be sustainable.

[no_circuit]

That sounds exactly like how search works today. Anyone can make a payment to promote their site for a topic which is equivalent to demoting someone else's. Crypto isn't necessary. The ability to anonymize up and down weights probably will make spammy results worse. Unless one is purchasing ads to drive traffic to one's site, what would be the motivation to help maintain indices that help some one else's web search business?

[cinquemb]

> That sounds exactly like how search works today.

Problem is that today what is indexed is under the sole discretion of the one entity generating the index, and the state of the index at any given time has no way for anyone to tell if it was modified outside the boundaries of the code that generated it from the previous timestemp (or if the code itself was changed between time intervals for index generation). The public has no access to these things because they are controlled by individual companies now.

For it to become decentralized, the code used to generate the indecies needs to be public, the state of the indecies at any given time needs to become public and verified by some cryptographic hash, and the software that serves results based on such incecises need to have a way of agreeing on incidies at any given time such that if an index someone searches against is modified, anyone would be able to tell that the server that served the result is byzantine from the network.

I can imagine, as an naive example, compressed tf-idf matricies that correspond to a given hash for search indecies and search interactions (what data the software tracks when someone does a look up against it or wants to increment a weight of anything in the indecies) and gossiped to other connected servers over some time interval, which can verify that at x time, the index was y and the weights matrix was z.

> The ability to anonymize up and down weights probably will make spammy results worse.

Not if the cost in doing so gets more expensive over time for any given result.

> Anyone can make a payment to promote their site for a topic which is equivalent to demoting someone else's.

This process isn't formalized across search indecies, each company that makes one has their own rules, and what the exact rules are at each time delta is not clear to users of search engines. And all rely upon easily censorable financial systems.

> what would be the motivation to help maintain indices that help some one else's web search business?

People or companies running the software could collect fees to maintain the distributed search engine. If they modify the code to the degree it operates outside of the how consensus is defined in the code others are running, other nodes could block that node and their fees could be blocked as defined in their code.