Hacker News new | ask | show | jobs
by boomboomsubban 2095 days ago
>There have been more studies on CBT than any other form of therapy, and those studies suffer from the same replication crisis that the rest of social science research does.

More studies showing the same results means they aren't suffering from the replication crisis, even if the exact same study isn't being replicated.

CBT isn't for everyone, that is true. And you shouldn't worry or blame yourself if it didn't work, there are more options available. But I wouldn't cast doubt's on it's overall efficacy or claim it's junk science.

2 comments

Seriously?! I never claimed it is junk science.. didn't even allude to it. I certainly wasn't casting doubt on research figures where they have been replicated and proven.

Truth is, some CBT studies have repeatedly passed replication and others yet have failed. CBT studies in lots of areas where it is used have shown relatively low efficacy, but close to alternative options. Even across anxiety disorders, where is it most effective, it is with an average-ish 70-80% success, and often only a few points different with alternative options. Meta studies have shown it to be ineffective within certain sub-cohorts, even in general areas where it is shown to be effective. Lots of CBT studies are performed differently or with different cohorts to how or with whom they are instituted in practice.

None of this means it is junk science, it just means you have to be very careful to understand the precise limitations of the science. You cannot separate the outcomes you like from the replication issue, you have to test it all. You also cannot generalise the very real successes of CBT across areas where they have not been shown to be effective, or shown to be ineffective. It is worth keeping these things in mind.

This information is almost never shown to clients. Clients are told 'CBT works', with the implication (or outright claim) that if it doesn't work for you it is your fault. It is popular with clients and it is popular with the profession, because it sets expectations early, it is cheap, it is targeted, and it works for the majority of people with the majority of issues people seek counselling for (ie. anxiety and depression). If this doesn't work for you, and this is all your counsellor offers, they are doing a massive disservice.

> More studies showing the same results means they aren't suffering from the replication crisis, even if the exact same study isn't being replicated.

Not specific to CBT, but as a general point, replication crises are behind the scenes. Publication bias means only positive results get published, even if it's only the occasional lucky studies (or ones with overt cheating or innocent bad statistics) which come out positive. Researchers know about it and grumble privately, because they wasted months or years of effort before figuring out it's largely false. But this knowledge isn't generally what goes into archival publications.