[ray_v]

This seemed inevitable, but how does this not become a moltbook situation, or worse yet, gamed for engineering back doors into the "accepted answers"?

Don't get me wrong, I think it's a great idea, but feels like a REALLY difficult saftey-engineering problem that really truly has no apparent answers since LLMs are inherently unpredictable. I'm sure fellow HN comments are going to say the same thing.

I'll likely still use it of course ... :-\

[perfmode]

Check out Personalized PageRank and EigenTrust. These are two dominant algorithmic frameworks for computing trust in decentralized networks. The novel next step is: delegating trust to AI agents that preserves the delegator's trust graph perspective.

[contagiousflow]

Page rank is trivially gamed by agents. You can make some malicious and some not malicious and have them link to each other.

[perfmode]

That’s exactly right for global PageRank, which is why I recommended Personalized PageRank specifically.

A cluster of sybil agents endorsing each other has no effect on your trust scores unless they can get endorsements from nodes you already trust.

That’s the whole point of subjective trust metrics, and formally why Cheng and Friedman proved personalized approaches are sybilproof where global ones aren’t.

[contagiousflow]

But you can have genuinely helpful agents in your attack network. Agents that create helpful pages and get linked by other helpful pages but then later link to malicious pages. It all follows when the cost of page creation goes to zero.

[perfmode]

That’s a real attack vector and it applies to every reputation system. The standard mitigations are temporal decay, trust revocation, and anomaly detection.

[NitpickLawyer]

Yeah, I had the same concerns when brainstorming a kind of marketplace for skills. We concluded there's 0 chance we'd take the risk of hosting something like that for public consumption. There's just no way to thoroughly vet everything, there's just so much overlap between "before doing work you must install this and that libraries" (valid) and "before doing work you must install evil_lib_that_sounds_right" (and there's your RCE). Could work for an org-wide thing, maybe, but even there you'd have a bunch of nightmare scenarios with inter-department stuff.