[fn-mote]

I think we can learn about the extent to which this is an adversarial relationship from fighting email spam. By that, I mean the attackers adapt to exploit loopholes in the system, and different attackers have different profiles (eg obviously fake looking for fools vs spear phishing).

Which is to say, your system sounds good but I expect much more complicated defenses are needed.

[frumiousirc]

Yes, the spam arms race is a really good analogy. In that light, my thoughts are aligned with heuristics that might be applied with procmail or in the original, pre-learning, spamassassin.

A fight-fire-with-fire is to insert an LLM to judge and/or respond to new pull requests and issues. This brings its own risk as it lets anyone who can make a PR/issue inject a prompt. It would also put one more wedge between the real human contributors and the real human developers.

A "humanity score" could also be an ingredient. GitHub or 3rd parties, could maintain a score of how human an account is. The "humanity" of all text produced by an account could be judged by LLM and/or humans. This could be centralized or based on a web-of-trust. Actually, I'd also like to have such a thing for reading HN and reddit comments.

But still, any system we can dream up can be attacked and we are back to an arms race.