>> Covariates. We adjusted for an extensive number of covariates all measured before or at cohort entry. These included risk factors for AD and potential confounders associated with PDE5I exposure and AD. [...]
The parent post makes a good point that it’s very hard to do this kind of research well. We should default to skepticism. There is a replication crisis on right now.
That means looking at methods, discussing confounding factors and how they were and weren't handled correctly, and what alternatives could have been employed.
There's a similar form of dismissive, passive skepticism that says: science has problems, so it's not even worth looking harder.
Note: I'm assuming good faith on part of parent, so not pointing fingers. But it is a thing I've seen on HN, especially re: biology.
I will comment though: Passive skepticism is important too. At this stage, the bigger problem is treating one study outcome as if it were fact. Dismissing established, well-replicated science is, of course, a problem as well, but at least from what I've seen, the much bigger problem is that you have one correlational study. Popular media runs with it. People make products based on it. The underlying science is nonsense.
The good news is that it's also possible to solve. There is constant progress in science, of course, but if we ignore all science from the past 20 years, we're still left with pretty good science (at least for the level of policy-making and personal decision-making). Simply ignoring anything recent solves the hype problem. And for all the anti-vaxxers and flat earthers, those are still minorities since recognizing established science is actually not especially hard.
> [...] but if we ignore all science from the past 20 years, we're still left with pretty good science
But what if the people of 20 years ago had made that same choice? And the people 20 years before then? Etc.
It's not a tenable long-term position.
> Popular media runs with it. People make products based on it. The underlying science is nonsense.
This has always been the case, though. A majority of people are dumb and/or lazy. Consequently, the quickest way to make a buck off them is the above.
But that trash science gets popularly communicated doesn't opine on the advancement of the academic practice.
> And for all the anti-vaxxers and flat earthers, those are still minorities since recognizing established science is actually not especially hard.
This feels like something we degree pretty strongly on.
(1) I don't think it should ever be easy to tell bad science, (2) I certainly don't think established science should hold an especially privileged place (heliocentrism!), & (3) I do think that the anti-vaxxers and flat earthers deserve a place in any true scientific practices (there's a myriad of ways they generally demonstrate their ineptitude, but ideas should not be verboten).
It's a perfectly tenable long-term position. You're missing two points:
1) People aren't uniform. I read all the papers in my narrow field and evaluate them. On the other hand, in 99% of fields, you and I are a popular audience. We should rely on established science. And in my field, you should also not rely on hot-off-the-press because most of it is nonsense.
2) Science becomes established -- over decades -- when you have multiple corroborating results, replicated, with multiple methodologies. We know the earth isn't flat because we e.g. sent a man to the moon. That's a very different standard of knowledge than Viagra and Alzheimers. Before acting on this, let experts in this field do their thing for another decade and understand the pathways, do an RCT, etc. At that point, it will be 20-year-old fact. Or perhaps it will turn out to be a correlation / causation issue, and you'll never hear about it again. Either way.
Both have places, but science must be in #2 to work at all. That is the point of science.
One of the critical parts of this mindset is separating criticism of work, quality of work, and the individual. Indeed, the more significant the science, the more it needs criticism. There is little (and probably even a negative) correlation between the quality of science and the quantity of criticism.
If I list out the limitations of your work in a blog post, that is not, should not, and cannot be read as a condemnation of your work, and especially not of you as an individual.
Indeed, generally, what that means is I took an interest in your work, found it compelling enough to do a deep dive, and I wrote a blog post because I'm trying to figure out next steps.
There is a pipeline from speculation to hypothesis to theory to fact, and it RELIES on people doing their best to invalidate a piece of work, understand methodological limitations, find alternative explanations, and otherwise poke holes. Once those holes are filled, and there are no more criticisms, you have trustworthy knowledge.
PLEASE attack my work (so long as you do it honestly and correctly; not unhinged emotional attacks). It makes my work better.
If the comment was along the lines of "I've read the paper and think it has some serious problems like x, y and z" then you'd be right.
But this is just someone who's read the title of a BBC article about the paper and basically saying "yeah but science is hard". We know it's hard. That's why we don't all have publications in peer-reviewed journals. This isn't specific to this study. You think published authors in the field don't know this?
I don't think the audience for the comment was "published authors in the field" but "random people in a web forum" who make comments like yours ("Imagine telling someone you made a website and they're like 'cool, but did you even make sure it has the little green padlock?'")
In that context, it's often helpful to remind people that:
- Correlation is not causation
- It takes many replicated results to go from hypothesis to theory
(Which is quite different from "science is hard"). This is doubly-true with the current level of hype and grandstanding in science where primary and secondary sources are often clickbaity and misleading.