[wredue]

And yet, it is still the best we got for also producing highly reliable and correct information.

Personally, I think the “highly” in your statement is quite over exaggerated. Humans can be convinced to produce bad science, for sure, and there are even journals set up by religious orgs that specifically exist to do just that.

But at the same time, science landed humans on the moon.

[parodysbird]

> But at the same time, science landed humans on the moon.

That was engineering. Closely linked to science, but not the same process of inquiry.

[mrguyorama]

Engineering did not discover the Keplerian or Newtonian laws of motion.

[kelnos]

> Personally, I think the “highly” in your statement is quite over exaggerated.

Except that the entire point of the article here is that it's not exaggerated.

> But at the same time, science landed humans on the moon.

Cherry-picking a highly successful, well-known example doesn't prove a point.

[Tainnor]

> Cherry-picking a highly successful, well-known example doesn't prove a point.

There must be hundreds, if not thousands, of successful scientific discoveries that went into something as complicated as the moon landing, and if you still don't think that's convincing, just look at the world around you - which looks just radically different from the world of, say, just a couple hundred years ago.

[consteval]

> Cherry-picking

As if our lifespans and quality of life haven't been drastically improved by modern medicine.

I mean, we can cut people open and replace entire parts of them and they're fine. They don't even get sick anymore - thanks germ theory and aseptic technique! Do you not understand how much of a marvel that is?

Before that, people used to get cuts and scratches and just... die. We can now fully rummage inside an arbitrary person's internal organs.

And don't even get me started on long-term illnesses. High blood pressure and cholesterol has been killing humans since forever, and we have medicine that just fixes that. And now, we're getting medicine to rewire our brains to prevent addiction in the first place (semaglutide)

[mcmoor]

This is what makes me troubled regarding medical science. I've heard tons of things about fraud and unreproducible results but new wonder drugs (that actually worked!) are deployed every year.

[llamaimperative]

Clinical trials in general are extremely, extremely above board. The level of scrutiny is extreme, and the stakes are unbelievably high for pharma companies and the individuals involved. There are better ways for an unscrupulous pharma co to gain an edge.

That said, wonder drugs are few and far between. The GLPs are at least a once-in-a-decade breakthrough, so that’s probably most of the noise you’re hearing (there are a lot of brand names already).

[jtc331]

What about Vioxx?

[refurb]

Voixx was an unknown unknown problem.

Cardiovascular safety was tested in the original trial. It passed. Nothing in the data during development suggested it was an issue. But trials can’t detect everything.

It wasn’t until it got to market did a safety signal pop up. Then retrospective analyses of large data sets proved it.

[llamaimperative]

> in general

No one is under the illusion it’s perfect or ungameable. A drug slipping by every few years is bad and often tragic, but IMO nowhere close to indicative of a systematic problem. It is a system that is worthy of a high degree of trust.

[jackcosgrove]

I'm unfamiliar with Vioxx and whether its approval really was a result of mistakes.

Shouldn't we expect some small percentage of failures in these processes given that they are driven by statistics and confidence intervals? Is that even a failure of the process, or is it a known limitation given how much resources and time we are willing to allocate to the discovery process?

[llamaimperative]

Yes we should expect some small number of failures and so far I agree, I don’t see evidence of a problem that needs fixing.

As patio11 says, the correct amount of fraud in a financial system is not zero, and the correct amount of false positives in drug approvals is not zero.

[consteval]

> Shouldn't we expect some small percentage of failures

Yes, and this is really solved at a more local level. Doctors aren't prescribing new drugs like candy. They, too, are skeptical of their success and will reserve those prescriptions for the most desperate cases. Over years, we (and the doctors) learn how effective these drugs are and what potential side effects they have.

[wredue]

You’re hearing loads about fraud because the anti-intellectual bots are here to make sure you hear about them all the time.

Republicans and Russian bots WANT you to hate science and academia and they have frequent pushes across social media platforms to make sure you do.

[mike_hearn]

> And yet, it is still the best we got for also producing highly reliable and correct information.

It's not. Markets are good at that. They're actually competitive. Academia is good at producing enormous volumes of documents that claim to be information, and may or may not be if you test them. It's "competitive" in a weird way where people don't compete over what's actually true but over who can convince central planning committees to give them money, which is very different.

[Eisenstein]

Why is 'competition' your marker for ability to 'obtain reliable and correct information'? Wouldn't cooperation be necessary along with competition? Entities that only compete with other produce no standards and share no knowledge. The market (with limitations for externalities and to prevent fraud) is good at distribution of limited resources in the most efficient way possible. Why is it being shoehorned into being the solution to production of reliable an correct information?

[mike_hearn]

They're both necessary, you're right. A more precise statement would be something like "competition between groups of cooperators".

You can't separate the production of correct information from the production of goods and services. They're inherently intertwined, the attempt to separate them is how we ended up with such a polluted scientific literature. Having formed a hypothesis as to what is true, you have to test that out in a robust way where you can't easily cheat and you can't easily cheat yourself, even when "yourself" refers to the vast institutional structures that employ you. In other words you need a system that stops you cheating even if that's what would please your boss, your vice chancellor and ultimately the President.

We only seem to have one such system and that's markets. If you cheat then the product or service you provide will be based on beliefs that are false and - eventually - customers will abandon you because the thing you're selling doesn't solve their problem. This is effectively a referendum of the customers. There is no such feedback loop in academia. There are attempts to approximate or emulate it with things like peer review, but they're all shadows of the real thing.

Meanwhile you can encourage cooperation even between competing entities in lots of ways, and it often emerges naturally even in the absence of any specific social policy. Open source collaboration is one obvious example in the tech sector, patents are a more formalized system.

[EGreg]

No, we have better systems now

[wredue]

Yeah. Like “just do your own research” man.

Tell me of a better method to get to the truth. Go on.

[EGreg]

je refuse!