[currymj]

this sounds similar to problems with peer review in academia. it mostly works fine as a guardrail to enforce scholarly norms.

however many institutions want to outsource responsibility for their own high-stakes decisions to the peer review system. whether it's citing peer-reviewed articles to justify policy, or counting publications to make big hiring decisions.

It introduces very strong incentives to game the system -- now getting any paper published in a decent venue is very high-stakes, and peer review just isn't meant for that -- it can't really be made robust enough.

i don't know what the solution is in situations like this, other than what you propose -- get the outside entities to take responsibility for making their own judgments. but that's more expensive and risky for them, so why would they do it?

It feels kind of like a public good problem but I don't know what kind exactly. The problem isn't that people are overusing a public good, but that just by using it at all they introduce distorting incentives which ruins it.

[tptacek]

My basic take is: if "CVE stuffing" bothers you, really the only available solution is to stop being bothered by it, because the incentives don't exist to prevent it. People submitting bogus or marginal CVEs are going to keep doing that, and CNAs aren't staffed and funded to serve as the world's vulnerability arbiters, and even if they were, people competent to serve in that role have better things to do.

The problem is the misconception ordinary users have about what CVEs are; the abuses are just a symptom.

[currymj]

I suspect for both peer review and CVEs, and probably some similar situations I'm not thinking of, it's not just a misconception, it's often more like wishful thinking.

People really want there to be a way of telling what's good and important that doesn't cost them any money or effort. Ironically these systems can sort-of work for that purpose, only if people don't try to use them for that purpose.

[astrobe_]

I think both are instances of Goodhart-Campbell-Strathern's law: "When a measure becomes a target, it ceases to be a good measure."