Hacker News new | ask | show | jobs
by BenoitEssiambre 2052 days ago
No. If you look closely at the data you might be able to draw such conclusions but lack of statistical significance often doesn't suggest or imply a small effect. Notice that in this case in particular, the confidence intervals are consistent with very large positive or fairly large negative effects. Don't underestimate the amount of noise often found in studies. Lack of significance usually just means that data is too noisy to tell us anything. If you get significance you get to say: the data is probably not pure noise but the effect could still be very tiny or caused by systematic measurement errors. Null hypothesis testing is pretty useless really.
1 comments

Thanks for your detailed reply! I meant that even this dataset puts a limit on the effect size, if viewed as an "evidence of absence of clear and large effect".

Of course since "everything is correlated" [0] expecting such truly simple signals might be nonsensical/pointless.

I was just lamenting the lack of simple magical treatments basically.

[0] https://www.gwern.net/Everything

That gwern page is excellent!