If you can at least attest who is responsible, other tests for AI-generation might be applied, with reputation following from that.
Sadly, that's bullshit most of us didn't ask for, and it's turning up all over the place, e.g., among book and short-story publishers, journalism, academic publishing, as well as blogs and social media.
Attesting who is responsible is more of that "bullshit most of us didn't ask for". I'm not interested in identifying myself so some cretin can know whether I am, or am not, an AI... particularly when the vast majority of posters on any platform act like NPCs anyway.
Would be nice to see something referral based. If you don't like X, block them. If X invited Y and Z and their invites behave poorly, you can block the whole tree. Kinda like lobste.rs referrals but for wider internet
I guess the correlary would be like how you can block an entire ASN if you find a lot of abuse from it, but at the human-network level.
Aside from social dynamics, a chief issue is that if you're relying on this as a mechanism for content filtering, personal relations have low predictive value.
E.g., I may really like a person's content, but not their curation or referrals to other accounts. Conversely, I might not care for a person, but their recommendations may be excellent. More common might be the case that a given account produces little or no content of their own, but makes reliably predictable (either good or bad) recommendations, which would be useful for further filtering. Or the highly verbose individual who emits a constant stream of near-drek, but an occasional diamond.
Content production, content curation, and talent spotting are all distinct skills. Success or lack in any one says little about the others. This is where Bayesian indicators (including relationships and referrer / invite relations) would probably be more robust.
That said, tainting an entire invite tree is likely useful, with a caveat that if a particular invitee has an independent, untainted, relation, they might be worth following.
In practice what I've found most useful is to have a pretty tight primary list of follows, ~50 or fewer, and a slightly broader secondary list. Allow recommendations be default (that is, re-shares / boosts), but curtail those too if problematic. Be quite liberal in blocking / muting anything in the least bit annoying or problematic.
Or participate in a selective group with excellent moderation. HN isn't quite there, but it approaches this ideal more closely than any other major forum I'm aware of presently.
Perhaps, but in both cases, I think that the principle problem is attempted control at the wrong portal.
Rather than individuals or devices, residential / mobile / business service providers should be able to vouch for personal traffic and be in a position to validate patterns of use without undue profiling of specific activity. That is, just looking at the encrypted traffic patterns (rather than MITMing SSL/TLS or other secured comms) should show usage that's typical vs. atypical / malicious.
Traditionally, service providers of all stripes (email, ISPs, Web, etc.) seem to have focused far more on ingress security than egress security, or potentially malicious traffic from within their own networks. That's got to change, it's ultimately a hygiene question.
For residential and mobile Internet, accounts are managed at either the household or individual level, and it should be possible to provide attestation and reputation management (as well as, perhaps, broad-based subscription access to compensated content) at those levels.
For commercial access things get more complicated, particularly where a location might provide public Internet access (e.g., public WiFi), or have a mix of human and system-generated traffic at an office, commercial, or industrial site. Still, there should be both well-established patterns of use and indications of anomolous or malicious traffic possible here.
Another option for smaller human-scale networks (e.g., Fediverse / Mastodon / PeerTube / Pixelfed / Lemmy / WriteAs networks and the like) is a mix of harder authentication (Yubikey or NFC-based wearable authenticators, perhaps) as well as a more manageable human-scale moderation (1:1,000 or 1:10,000 scales far better than 1:1 million or 1:1 billion services), allowing for both oversight and keeping the opportunities / benefits of malicious use limited.
The comment I'd originally responded to had me thinking of under-delivering federated systems such as Gemini (the lightweight Web protocol, not Google's AI) or Diaspora* or countless web boards and wikis which ended up overrun by spam and abuse. Simply saying that you're going to re-invent things at small scale in no way means you'll succeed. The ecosystem's changed, the pathogens are far more numerous and capable. Modern systems and networks (social or otherwise) must face those facts head on, and not ignore them or pretend they don't exist.
I think we're going to end up with some form of cost-based (though not necessarily financialised) reputation management systems. I'd very much like to see those not being terribly invasive of privacy, or putting extreme barriers to those with limited means or technical knowledge. It's a tough problem all the same.