[hrhdkdlfnrne]

A lot of science today is basically parallel construction.

You start with a sexy story that you know will get you a lot of press, like "promising you will be honest actually makes you behave in an honest way" and then you just make that paper happen, however you can.

Under the publish or perish system, scientists don't have time to actually research the topic, and imagine if it fails to confirm - you just wasted a lot of time and didn't publish anything. Too risky, it's much easier to just fake it till you make it, especially since you know peer reviewers never ever will accuse you of fraud.

Any reviewer accusing a scientist of fraud will just be excluded from the community, since it's very important to uphold the narrative that "scientists are always honest, they never cheat like politicians, which is why we must always trust scientists and never question them".

[thrwyoilarticle]

There's a lot of value being abused in the term 'science'. Science is a highly valued concept but it's the result of following the scientific method, not the output of anyone with a postgrad.

[dr_dshiv]

Science is what scientists do, like politics is what politicians do? Either science can be critiqued as a social construct or it is an unimpeachable Platonic aspiration. I can see both perspectives. But, communicating that science itself is a somewhat messy social phenomena might be better as a long-term message for the public.

[guerrilla]

Politics is definitely not defined as what politicians do and science is, as the GP said, when somone follows the scientific method which is something that happens all the time, far from a Platonic aspiration.

[jbjohns]

Is there such a method?

https://www.discovermagazine.com/planet-earth/the-scientific...

[guerrilla]

Yes, despite the demarcation problem, I think there is.

https://plato.stanford.edu/entries/scientific-method/

And I think the demarcation problem can be in some ways be solved by distinguishing on predictivity.

[dr_dshiv]

Right. Nice article. Tldr; no, not really, "the scientific method" is rhetoric.

Though i will say that they missed Robert Hooke's essay on the scientific method. I swear no one knows about this (even though Hooke was a founding and seminal member of the English Royal Society) because Hooke sounds insane. Who makes titles like this? I love it:

A scheme, or idea of the present state of natural philosophy, and how its defects may be remedied by a methodical proceeding in the making experiments and collecting observations wereby to compile a natural history, as to the solid basis for the superstructure of philosophy

[simonh]

I think it's important to bear in mind that even with the system working perfectly well, and perfect ethics, we should expect to see a lot of papers published with false results.

Lets say there are 200 propositions we want to test and are candidates for publication, that 20 of them are true and that our error rate is 5%. That means when we test the 20 that are true 19 of them will be accurately shown to be correct and 1 will be erroneously found false.

However when the other 180 propositions are tested 5% of them will be erroneously found to be true, that's 9 propositions. This means we will end up with 28 'successful' studies that make it into prestigious journals, about 1/3 of which are false positives.

And as I said, that's if the system works perfectly with no fraud whatsoever. Throw in some human error and it's not surprising if a fair few studies start to look pretty dodgy. Add in some genuine fraud too and you've got a full-on replication crisis with all the trimmings.

[motohagiography]

Fraud like this is next-order dispicable because of the knock-on effects it has on knowledge and policy. I don't know that stating that is going to have an effect on people who engage in fraud as a matter of course or as passive parties to it, but here we are. The incentives here are what makes most research an unsuitable substitute or appeal to authority for pubic policy. It's a proxy for a petty elitism, and it means to "trust science," just becomes scientistic nihilism. After all, these days you don't need popular consent when you have scientists.

Reading reports like this, I can see why there has been so much criticism in recent years of the concept of the banality of evil, and why so many researchers affect concern for the environment, because more than anyone, they seem to understand what it means to be responsible for poisoning an ecosystem and they need to get out in front of those narratives. Sure, it's just a bit of fraud in a journal, just like it's just a bit of PCB or mercury in a lake, and only a minority of the population who will be impacted, but someone has to call it out as nihilism, or we're a party to it as well.

As much as I dislike blockchain ideas, it makes me think a blockchain DAG of metadata about the integrity of published papers for citations, reproducability, and evidence of certain types of fraud would rebalance the incentives a bit.

[atoav]

I think it is important not to extrapolate from a replication crisis in one field (e.g. psychology) to all the other sciences, because the picture painted by this (to my knowledge) doesn't accurately describe the underlying practises.

[btilly]

No, it is exactly the opposite.

The well-documented failings of psychology should cause us to take a closer look at other fields to see if they have similar problems. If they don't, then great. If they do, then fix them.

For example people studying metascience have found that a lot of medical research is of questionable accuracy. It is not as bad as psychology. But it is bad and I'm glad that people are taking this problem seriously.

[atoav]

Good point. We should be careful in other fields as well.

I came at it from another direction here — there are people who are like: "Look at the replication crisis in psychology — this is proof science cannot be trusted in general". So what I meant to argue here is that this conclusion cannot be drawn automatically, not that we shouldn't scrutinize other fields (we should!)

[hrhdkdlfnrne]

In medical sciences is worse. In one attempt to replicate "lamdmark" cancer studies 89% failed to replicate.

https://www.nature.com/articles/483531a

[PaulDavisThe1st]

> A lot of science today is basically parallel construction. You start with a sexy story that you know will get you a lot of press, like "promising you will be honest actually makes you behave in an honest way" and then you just make that paper happen, however you can.

Gigantic accusation, zero evidence.

If this was a paper, rather than an HN comment, I'd say there was every chance that it would be self-illustrating.