[dbecker]

It's disappointing to see links to the abstracts on the front page of HN (given the paper itself is behind a paywall).

In this case, the find some non-zero effect, but call it zero because the difference was not statistically significant.

That's likely a reflection of their small sample size rather than evidence for the null hypothesis as the title suggests.

But it's hard to know, or even have an intelligent discussion about it, since the paper itself is behind a paywall.

[shaki-dora]

Sample size is, of course, one of the factors included in any analysis that gives you a p-value, or any other measure of confidence.

In this case, the abstract specifically mentions a "Bayes factor > 25". The Bayes Factor is the likelihood, i. e. the ratio of probabilities for two competing hypotheses. With no information whatsoever, it is 1. A factor of 25 means that the null hypothesis is 25 times as likely to be correct as the theory of ego depletion.

[grandmczeb]

It's on sci-hub: http://sci-hub.tw/10.1111/bjso.12236

[ghkbrew]

> In this case, the find some non-zero effect, but call it zero because the difference was not statistically significant.

Neither the title of the paper nor the abstract say there is zero effect. They accurately and precisely state that the null hypothesis is favored and that the effect is not significantly different than zero.

> That's likely a reflection of their small sample size rather than evidence for the null hypothesis as the title suggests.

That doesn't seem to be the case. From the introduction:

> The Hagger et al. (2010) meta-analysis reported an estimated depletion effect size of d = .62, which we used to conduct a power analysis to establish sample size (n = 33) at 0.80 power. We established n = 35 per condition for our studies, which is slightly larger than the average sample size in the published depletion literature of n = 27 (Lurquin et al., 2016), because the intent of this series of studies was to examine empirically the frequency of null findings in studies similar in size and procedures to those in the published literature. The use of a similar sample size allows for an empirical examination of how frequently the depletion effect occurs when using methods and sample size similar to those in the pre-2013 published literature.

So if their samples sizes are a problem then it's a problem that the rest of the literature has too.

> But it's hard to know, or even have an intelligent discussion about it, since the paper itself is behind a paywall.

Agreed paywalls are unhelpful, but that doesn't mean we can't intelligently discuss what information is available. In addition, I and probably many other people on this forum do have access to many paywalled papers, so exact information is not exactly impossible to obtain.

[mikraig]

In Bayesian analysis, it's possible to actually accept the null, rather than just the reject/not reject that is common to frequentist statistical inference.