[scottbot]

I agree, lying is bad regardless! My post isn't anti p-values, it's anti poorly-understood-or-performed-statistics. NHST just happens to be the subject of choice, because it's particularly misunderstood. Anyway, I'm less worried about lying, and more worried about accidental inaccuracies, e.g., someone collecting data until they get tired or run out of funding, but running the calculation as though the "intent" was to get exactly that number of observations.

[pseut]

I don't know... this line towards the end suggests you're anti-testing:

"It is for this reason that I’m trying desperately to get quantitative humanists using non-parametric and Bayesian methods from the very beginning, before our methodology becomes canonized and set."

:)

The "the design of the experiment shouldn't matter so much" assertion is usually followed by an appeal to the likelihood principle and a claim that frequentist estimation is misguided. If that's not what you had in mind, apologies. I've never seen it coupled with a claim that the frequentist would then misrepresent their experiment...

If the quoted statement is your goal, I think a more convincing argument is, "we often want a more nuanced way to express uncertainty than classical tests/confidence intervals give us, and... LOOK! We get that for free using Bayesian principles."

As an aside, running out of funding seems like it should give you the same results as a predetermined sample size, as long as the funding isn't conditional on getting interesting (i.e. statistically significant) results, but I'd need to actually do the math to be certain.

[scottbot]

My stance is much softer than that, but I should have made it clearer, because similar arguments are often made in anti-frequentist rants. I think that there is an appropriate place for most statistics used (including NHST under the right circumstances) - and often, the differences between the results are entirely negligible.

My goal is to make people aware of the various stats out there, their benefits and pitfalls, and let people choose whatever is the most appropriate for their needs. Those choices need to be informed, and given that most introductory stats starts with p-values and seems to teach them wrong, that's where this post is aimed.

Regarding your aside, the universe of possible observations in a given experiment assuming 100 trials may be very different than the universe assuming trials until we run out of money, which happened to fall on 100.

[sesqu]

If you want to go down that route, I suggest examining the assumptions of so called nonparametric tests and uninformative priors as future topics.