[Grieverheart]

Most recent studies in psychiatry and social sciences that I have read, even those published in Nature, use sample sizes of around 50 people, which should be considered insufficient. Instead they use various cooked up formulas for correcting for gender/ethnicity etc. Instead, sample sizes should be much bigger. Of course this is not practical or even perhaps impossible. I find this to be a poor excuse, though.

The matter of fact is, we know very little about how the brain works. We certainly have some big picture on how the brain functions, but we are very far from grasping the chemistry.

[mjburgess]

psychiatry =/= psychology =/= social psychology

This is more "everything is the same" anti-expertism.

Clinical psychiatry is not about p-hacking groups of 50.

The treatment of bipolar disorder hasnt been decided on the basis of what happened to the last dozen undergrad students.

>The matter of fact is, we know very little about how the brain works.

Profoundly untrue. Even if it were truth, its very truth would preclude your ability to make this assertion. If we knew nothing, how are you estimating the nothing we know?

It is no mystery what happens when you give a person lithium. Nor some speculative game to know what the effect of MDMA on PTSD patients is. Nor hocus pocus to identify the effects of repeated trauma.

How are you deciding what percentage of "how the brain works" we need to know in order to do psychiatry in a reasonably well-informed way?

Suppose, even, we know now 0.1% of the entire story. Is this sufficient for an evidence-based treatment of psychosis?

It clearly seems to be. Since the treatment of schizophrenia is overwhelmingly more effective than anything based on a study of 50 people.

Your comment may very well apply to "journalistic science" ie., the increasingly prevent, "ive just had an idea, lets get 50 undergrads and a seminar room!".

That is a tiny percentage of science, even if it is the most commonly popularized.

[Grieverheart]

I had a look at this review on bipolar disorder, https://doi.org/10.1136/bmj.e8508 . According to a study referenced in it, 68 randomized trials where used to determine the effect of different drugs compared to placebo. Again the sample size is low and the statistical significance is also small, unless I'm missing something.

You are correct in saying that we don't actually know how much we know about the brain. But the brain is s much more complex than e.g. planets orbiting the sun. There are many components that can get affected by chemical compounds in the short term, but more importantly in the long term. I agree though that only through testing can we get a better understanding. It is just the way it's currently done that I disagree. I would gladly read some research you think is good and does not have the shortcomings I describe.

[dsacco]

> Most recent studies in psychiatry and social sciences that I have read, even those published in Nature, use sample sizes of around 50 people, which should be considered insufficient.

I disagree with your reasoning even if your conclusion is correct about the validity of those studies. Small sample sizes are not intrinsically bad and there’s no magic number to make sample sizes valid. It’s a sliding scale with statistical significance - with a large enough p value a relatively small sample size doesn’t mean the observed phenomenon isn’t there. In that case I would be more concerned about p value hacking or the researchers “presuming the conclusion from the outset” than the sample size.

[always_good]

You can't condemn sample size without also condemning the statistical methodology of an experiment.

"Small sample size thus irrelevant" does not make sense despite often being the top comment on Reddit.

[Grieverheart]

This is correct. Sample size should be dependent on the strength of the effect. As a rule of thumb though, a sample of 50 people is very little. If physics used these kind of sample sizes and confidence levels, we would have discovered multitudes of new particles.