[psyklic]

I prefer references so I have more information available to assess the validity myself. If an author makes assertions without evidence to back them up, then they are much less credible.

Having many recent studies available to back up a hypothesis at least informs us that it is credible, given our knowledge at the time.

[westoncb]

Sure, all things being equal, it's better to have the references than not.

That said, as a practical matter, if the author cites 100 papers, and each paper takes half an hour to judge the reliability of it (assuming you have good command of statistics and scientific methodology, and some domain knowledge, and access the the required journals), we're talking on the order of 50 hours to validate the argument (of course this is a crazy rough estimate)—who is going to do that? You may as well just take the list of papers, try and validate them, and skip reading the book (and this is largely ignoring the fact that most likely your ability to validate is probably not that great in the first place).

Instead, for ~99.9% of readers, the argument presented will end up working solely as an authority argument.

Not sure what the solution is aside from waiting longer, then coming with maybe a handful of extremely reliable authoritative results that the reader can reasonably verify on their own. I'm not sure. I just know reading a book grounded in 'hard facts' because a study is cited on every page is not at all appealing to me.

[retsibsi]

There is a middle ground: spend a few hours verifying a small sample of the citations. This still takes more time, effort, and ability than most readers will have to spare, but it is feasible and would be enough to expose the worst offenders. Of course it can't guarantee that every study cited in the book is reliable, but it can give you a sense of how accurately the author characterises the studies and how carefully (s)he has chosen them. And if you choose your sample carefully, you can directly verify the claims that you care most about.

(Even better if this is done collaboratively, for a larger sample and less time spent per person.)

> for ~99.9% of readers, the argument presented will end up working solely as an authority argument

Agreed, but an author's authority can still be undermined or reinforced by expert opinion. In an ideal world where book reviews were written by qualified, careful people with time to do a thorough job, the existence of citations would be extremely important, even if hardly any ordinary readers looked at them. Obviously we don't live in that world, but sloppy scholarship is often exposed eventually.

[westoncb]

That's definitely a better, more realistic process than what I described.

> Agreed, but an author's authority can still be undermined or reinforced by expert opinion.

That's a good point. Actually, that's probably how I will try to validate this book (or others like it): I'll give the 'okay', once I see some unassociated other researchers confirming that the work is good, conclusions drawn are not a stretch etc.