[ahartmetz]

As for an alternative, how about using the social fabric of researchers and institutes instead? A few centuries of science ran on it before somebody had the great idea to introduce "objective" metrics which made things worse. Reintroducing that today would probably cause a larger spread in the quality of research, which is good: research is kind of a "hit-driven industry" - higher highs are the most important thing. The best researchers will do the best research, probably better without carrot and stick than with.

[KK7NIL]

> As for an alternative, how about using the social fabric of researchers and institutes instead? A few centuries of science ran on it before somebody had the great idea to introduce "objective" metrics which made things worse.

Oh boy, you seem to be missing the forest for the trees. When science was a hobby of the rich, there was no need to measure output. Only when "scientist" became a career and these scientists started demanding government funding (which only really crystallized in the 20th century), then we started needing a way to measure output.

You could try doing away with an objective measure of academic output and replace it with the "social fabric of researchers and institutes" (whatever the fuck that means) instead , but then all you'd have is a good ol' boys club funded by taxpayer money.

[fc417fc802]

If the metric is publication and citation count and funding is awarded by panels of experts, how is that better than cutting out the flawed metric and continuing to award funding via panels of experts? Either it's a good ol' boys club or it isn't but I don't think a horribly flawed metric is going to change that.

That said, as far as I'm aware those metrics aren't explicitly considered by said panels (NIH for example). Any issue in that regard is presumably due to either unconscious bias or laziness on the part of said experts when exposed to such metrics.

[KK7NIL]

> If the metric is publication and citation count and funding is awarded by panels of experts, how is that better than cutting out the flawed metric and continuing to award funding via panels of experts?

I agree it's not perfect but that's still several steps removed from "Billy is one of us, he should get that tenured position" and, as this article shows, it requires openly unethical behavior, which others can recognize and eventually prosecute (even if that isn't being done often enough).

It's almost like saying "well corruption happens anyways so why do we even criminalize it and have public hearings? Just skip those bits and openly auction votes instead".

[fc417fc802]

I interpreted "social fabric" to mean "panel of relevant professionals" which is what we currently have but perhaps you interpreted it differently?

I think most interviews can essentially be described as "Billy is one of us, he should get position X" if one is feeling cynical.

[KK7NIL]

> I interpreted "social fabric" to mean "panel of relevant professionals" which is what we currently have but perhaps you interpreted it differently?

That would be one way to implement his suggestion of getting rid of objective (if flawed) measures of a researcher's performance.

I agree that, inevitably, there will be a subjective human decision in there, but I argue that dropping all objective measures of performance and going just on vibes is kicking the door wide open for corruption, while it's merely cracked right now. And the exploit mentioned in this article is a very public and explicit one, which is why other researchers were so aware of it and it eventually caught up to him. If it all gets moved behind closed doors instead, it will be even harder to detect and prosecute this sort of behavior.

[ahartmetz]

I was thinking of basically "This person is good, we should hire them. Their results are going to improve our institute's reputation."

[dilawar]

What the guarantee is that folks won't abuse this system in the same way they do the citation system? The recommendation letter system is often abused for the pettiest of reasons...

[ahartmetz]

There is no guarantee. The current system is also not a guarantee for good results, though.

[Wobbles42]

This will be a hard argument to make.

The decision makers who are the target audience for these metrics value "objective" data. They value the appearance of being quantitative, but lack the intellectual tools to distinguish between quantitative science and pseudoscience with numbers bolted on.

That's modern bureaucracy in a nutshell.

[Ma8ee]

A few centuries of science of white males. While I agree that the system with ”objective metrics” has a lot of problems, but just removing it would bring us back to the old days when almost all science was done by a few privileged white men.

[ecshafer]

Almost all science was done by "a few privleged white men" because Europe and the Americas were the only places that had modernized with large central states, university systems, and educational systems. Even in that scenario before the "objective metrics" of the post-war system came about, we still had people like Madam Curie and Ramanujan being able to work with stellar results. The idea that somehow academia would stonewall all of the non-whites is absurd.

[Ma8ee]

It’s really hard to come up with better examples of the exceptions that prove the rule than Marie Curie and Ramanujan. How many more names can you come up with?

I’d even argue that still today women and minorities are strongly disadvantaged at many institutions. I’d say that as a white male that recently left academia myself. I have seen how some of my colleagues have been treated.