[gwern]

> I'm not convinced the parametric assumptions made by the standard tests mentioned (including difference in proportions, whether a z-test or Chi-square test) would hold up well for this kind of data, it would take further examination.

I'm not sure what you mean. If all you have is binary data, there's not much you can do but a chi-squared test or binomial regression. As I understand it, a chi-squared is considered nonparametric: https://en.wikipedia.org/wiki/Chi-squared_test https://en.wikipedia.org/wiki/Fisher's_exact_test

[jupiter90000]

Yes, you're right that chi-squared is non-parametric, my mistake. However, even non-parametric tests can have assumptions that should probably be checked into at least for a given data set. It may be robust to the violations the data collected may exhibit, I'm not sure. This article lists the assumptions for chi-squared test: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3900058/