|
|
|
|
|
|
by olooney
2456 days ago
|
|
|
Scientists usually try to distance themselves by saying those are soft science or even pseudo-science. This leads to the embarrassment of the demarcation problem[1] which is that no one can give a bright-line rule[2] to distinguish between the "real" science and pseudo-science. All of the demarcation criteria that have been proposed (such as Popper's falsifiability[3]) are inadequate in one way or another. In particular, they don't seem to capture the reasons a scientist would give about why a particular nutrition or social science paper is bad. The scientist would say things like, "Well, your sample size is small and not representative of anything except psych undergrads, you didn't control for age or gender, the participants and experimenters weren't properly blinded, you tested 15 hypothesis and only reported the p-value for the one that was under 0.05, and even that is wrong because you didn't apply Yate's continuity correction on your chi-squared test, AND NONE OF THAT EVEN MATTERS because the effect size you report is too small to be of practical consequence!" Nothing in there about the hypothesis not being testable; yet this is the kind of stuff that really separately the wheat from the chaff. So we're left with a "No True Scotsman fallacy" where have to say that some science is "good" and some is "bad" and the only way to tell is to ask someone knowledgeable to evaluate each paper on a case by case basis. Not terrible useful to the layman. And why do we want any kind of science to automatically get respect anyway? Good science is good because its already been subjected to an incredible degree of scrutiny. It will hold up to a little more. The real problem is disingenuous, bad faith arguments which are allowed to dominate the conversation. The real problem is to teach the general public to distinguish between sincere, good faith arguments and patent bullshit. This is much more difficult than it sounds because bullshit can easily conform to any merely superficial characteristics. [1]: https://en.wikipedia.org/wiki/Demarcation_problem [2]: https://en.wikipedia.org/wiki/Bright-line_rule [3]: https://en.wikipedia.org/wiki/Falsifiability |
|
|
Yes, the checklist would not be all-encompassing or foolproof, and there would likely be revisions to the checklist, and maybe even domain-specific variants, but it would be an extra level caution that the media could report or choose to ignore at their will. Over time, the apparent level of scientific rigour would improve. No, it’s not bulletproof and there will be people who will try to meet the checklist and still present erroneous conclusions as reliable science, but it would be an improvement in the status quo for a layperson who is aware and values said checklist.