[hencq]

I'm not really sure what you're trying to say.

> Rather, 95% statistical significance means, we got this number from a procedure that 95% of the time produces the right thing, but we have no idea whether this particular number we got is correct or not.

I.e. We got this number from a procedure and there's a 5% chance it didn't produce the right thing.

[Fomite]

Nope. It's "If we did this infinitely more times, there's a 5% of those samples wouldn't have significant results". It's a subtle but important distinction.

Though I'm surprised that his advice wasn't "Report confidence intervals at least". There's much more meaningful information in a point estimate and confidence interval than "p < 0.05"

[rcthompson]

Sorry, confidence intervals are just a different presentation of the same information as p-values, and don't contain any more or less information.

[Fomite]

While built off the same information, and it's possible to do an ad hoc significance test off it, confidence intervals tell you more about the spread of estimate. Especially if, as the author is suggesting, you're not even reporting the actual p-value, but just whether or not it's below a particular threshold.