|
|
|
|
|
|
by SuoDuanDao
2176 days ago
|
|
|
I'm reminded of Tetlock and Gardner's excellent book 'Superforecasting', which was essentially a study of people who consistently score at the top of prediction markets. One key thing that these 'superforecasters' had in common was that any new information caused them to update their model of the world, but none caused them to update it very much - typical people making predictions either didn't update their model or updated it too much in response to new facts. I think it makes a lot of sense, when one is trying to identify patterns in information, that it's easy to over- or undervalue novel information. We don't necessarily know what a new fact means, so ignoring it is one common error while paying too much attention to it is another. |
|
|
We also rarely even know if a "new fact" is actually true. So many studies don't replicate that it makes sense to hold off on updating core beliefs whenever "new facts" seem unlikely or in contradiction with previously known (and reliable) facts.
SSC had a nice article (now gone) that discussed this for a scientific theory that had literally hundreds of confirming studies done for it. All wrong. The "new facts" were bullshit. So even with tons of studies, it's reasonable to be skeptical in some situations.
It's also great that, eventually, science was able to figure out the "new facts" were bullshit. Yay, science. But it also means that people aren't being irrational when they don't immediately alter their fundamental beliefs while the ink is still dry, especially "new facts" that seem in contradiction with everything else we know…