[wojcech]

So how do you ensure that sciencists stay careful and try to be objective, how do you have any objectivity in allocation of funds etc. without some metric? I hate the publish or perish system, but that is due to wrong metrics. If (purely pulling this out of my as, probably a horrible idea) a flimsy study counted negatively to your index and a faulty one outright wrecked it (with complete ruin if you do not redo/retract), that would probably skew the metric optimisation to more carefully thought out, substantial research

[tdaltonc]

I'm not sure that scientists need to be super objective for science to work. The history of science is so littered with grudges, feuds, and grit in spite of evidence that it's hard to see the ground of objective rationality that many assert is behind it.

And pandemonium models [1] of computation show that the right rules of cummunication can produce astonishing order from chatic self interest. If scientst were forced to pre publish their methods and share all data -- science would probably work better. This is true even if this change had no effect on their behavior, or even if they made every effort to game the system.

This imrpovement to science doesn't ask a committee to define or measure anything ineffable, and it doesn't expect individuals to change thier behavior. It just changes the rule of interaction in a way that better favors the systems epistomological progress.

[1] https://en.wikipedia.org/wiki/Pandemonium_architecture

[TeMPOraL]

Objective scientists are a good thing, but are not - and should never be - considered an essencial component of the system. Any system that assumes honesty and fair play in its participants is doomed to failure. What science was aiming for is getting the true results from the aggregate - thus peer reviews, replication, etc.

Of course the more noise you have, the less efficient the system is, so it's good to incentivize people to do honest, objective work - thus pre-publishing / pre-registering, sharing data and algorithms, etc. are all good and important goals. But so should be changing the metric affecting the aggregate - like making sure scientists are actually incentivized to replicate previous work.

[nonbel]

This was figured out long ago. Before people should believe you know what you are talking about, you need to make precise and accurate predictions about the future (comet should reappear on x date) or perform engineering feats (pull a ship out of the water using only your own manpower).

People just don't like this real solution because it means many (really, most) of the wild speculations they hold as dear dogma (and have been calling science) will need to be reassessed using the correct metric. There will undoubtedly be a lot of egg on many faces.

[bzbarsky]

> how do you have any objectivity in allocation of funds etc. without some metric?

Our need for "objectivity" here is to a large extent driven by the centralization of the funding allocation that http://mcadams.posc.mu.edu/ike.htm warned about and has since come to pass. If funding were more decentralized, lack of objectivity in any particular funding source would be _much_ less of a problem.

[tdaltonc]

Specifically: The prospect of domination of the nation's scholars by Federal employment, project allocations, and the power of money is ever present – and is gravely to be regarded.

[bzbarsky]

Indeed. Since then it has gone from "prospect" to "it's been this way for decades"...

There's lots more in there that usually doesn't get quoted (unlike the bit about the military-industrial complex) and is spot-on prescient.