| HN Mirror

> Are two studies enough to “burn their credibility to the grounds”?

You're right, I've been arguing lazily. To fix that: it's more than two studies: The authors also submitted different test studies to different peer-review boards. The methodology was identical, and the variable was that the purported findings either went for, or against, the liberal worldview (for example, one found evidence of discrimination against minority groups, and another found evidence of "reverse discrimination" against straight white males). Despite equal methodological strengths, the studies that went against the liberal worldview were criticized and rejected, and those that went with it were not. [1]

This then shows up as e.g. publication bias in favor of a hypothesis [2].

Now you may read the abstract of the study from [1] and think "95% vs 50% approval rate depending on hypothesis? Well, that's not great, but if hypothesis A is true and B is false, then even if A only gets half as many studies, it should easily prevail as true."

There are two problems with that: One, A won't get half as many studies, but far less. The peer-review boards are assembled from, well, peers. On average, they share the biases of the researchers they are reviewing. If they will reject studies they dislike, and lie about why they rejected them [1, abstract of the study], what do you think the odds are that they would propose such studies (and then publish them despite getting results they dislike)? It's not one filter, but two, and publication bias will drown out the true signal.

Two, studies that show the right results will get promoted more, and if any later show up to debunk them, they'll get mostly ignored. The debunked study got promoted by CNN [3], USA Today [4], NPR [5], the Washington Post [6], and less trafficked sites like the World Economic Forum [7] and ScienceNews [8]. It was sent to the Supreme Court in an amicus brief by the American Medical Association and cited in Justice Brown Jackson's dissent [9].

The debunking was, predictably, promoted mostly by right-wing sites, and WSJ and the Economist (and also the Hill, who published both the initial study and its debunking). I'm not holding my breath for the American Medical Association to send an amicus brief arguing for repealing affirmative action, citing the debunking study.

So if you do what the "Never Trust the Science" article criticizes - follow some respectable publication like NPR, or uncritically parrot "the consensus", you'll just be reproducing the biases of NPR, that cherry-pick which studies to feature, or the peers of that field, the same peers that lied about why they rejected studies that went against consensus.

That said, in an ideal world (that still had flawed scientists), the average person (let alone the 50% below average) should probably follow your advice. They won't be able to distinguish quackery and gibberish from real arguments, and lack the self-honesty to disregard everything when they can't understand the arguments anymore.

In practice, the friend-enemy distinction wins out. People see news like "Diversity Statements Required for One-Fifth of Academic Jobs" [10] and "I Cited Their Study, So They Disavowed It" [11] and "White supremacy is a lethal public health issue that predates and contributes to COVID-19" [12], and immediately realize why studies only ever find one type of villain. So when you say

"The process cannot guarantee that every single paper is True. But, if followed, it guarantees that in time it will self correct."

They will think

"The market can stay irrational longer than I can stay solvent."

Science has entered the culture wars [13], and until people believe they're again neutral, trust will be thin.

[1] https://theweek.com/articles/441474/how-academias-liberal-bi... (the study the article references: https://psycnet.apa.org/record/1986-12806-001)

[2] Revisiting the Income Inequality-Crime Puzzle, https://bura.brunel.ac.uk/bitstream/2438/27988/1/FullText.pd...

[3] https://www.cnn.com/2020/08/18/health/black-babies-mortality...

[4] https://www.usatoday.com/story/news/health/2020/08/19/black-...

[5] https://www.npr.org/2020/09/16/913718630/a-key-to-black-infa...

[6] https://www.washingtonpost.com/health/black-baby-death-rate-...

[7] https://www.weforum.org/stories/2020/10/black-babies-in-the-...

[8] https://www.sciencenews.org/article/black-newborn-baby-survi...

[9] https://unherd.com/newsroom/why-did-it-take-four-years-to-de...

[10] https://freebeacon.com/campus/study-diversity-statements-req...

[11] https://manhattan.institute/article/i-cited-their-study-so-t...

[12] https://www.npr.org/sections/coronavirus-live-updates/2020/0...

[13] Or in some cases, like climate science, oil lobbyists have spread the (as far as I can tell false) belief they're in the culture wars, to undermine trust.