[tokenadult]

A mental test, perhaps, to determine if something truly is "obvious": if the fact is reversed, and you can come up with a reasonable sounding explanation for the reverse, then the original fact is not obvious.

This experiment has been tried, by a scientist who presented the exact opposite of various findings to undergraduates, who were readily able to invent rationales for the counterfactual statements, as reported in "A Sociologist’s Apology" (as posted on the website for the book Everything Is Obvious: *Once You Know the Answer).

http://everythingisobvious.com/wp-content/themes/eio/assets/...

[scott_s]

Eliezer Yudkowsky cites the same experiment in the essay I linked, which was my motivation for bringing it up. I'm not proposing an experiment, however, but a mental tool for people who are trying to be honestly rational. And my motivation for why this tool will work is that experiment.

Of course, using this mental tool requires that you are honest with yourself when you try to explain the counterfactual statement.

[SilasX]

Not sure if you were trying to imply something different, but that result would suggest that the students don't have adequate basis for claiming that things in those fields are "obvious" (or of their own confidence levels), not that the community of scientists in those fields lack adequate epistemic basis.