[westoncb]

> If the topic of the book isn't something you know well, how well do you think you can judge the coherence of the argument?

Hopefully by the time you have read the book, you have picked up a set of ideas from the field under the discussion and have an understanding of how they relate to one another. So I'm referring to coherence within that set of ideas and relations (which I'll refer to as a 'framework'). To be more specific I'd define 'coherence' as mutual support and lack of contradiction ('mutual support' is like... one point gains strength because of the content of other points.

The probability that the framework is strongly coherent while also in reality being false decreases with the size of the framework. In other words, if the author has made a large number of claims, which are all coherent with one another, the probability that they are true (in the sense of corresponding to something in objective reality) is relatively high. The alternative is that the author went through enormous efforts to create a large, coherent framework which doesn't correspond to something in objective reality. It's just not easy to do that (requires lots of imagination and lots of attention to detail), and it's fairly easy to detect. So either the framework matches something in reality, or the author is a very imaginative and meticulous con artist.

I think the main risk is that the authors trick themselves too, not that it's a fully intentional scam. So it seems like a decent heuristic.