[jupiter90000]

To be fair, a couple answers there mentioned the Chi-squared approach, and a z-test could also be appropriate for testing a difference in proportions. I think your Bayesian approach would make the most sense for this kind of stuff, to examine the posterior especially. There's other things that could be done than what you mentioned too to assess a difference. I'm not convinced the parametric assumptions made by the standard tests mentioned (including difference in proportions, whether a z-test or Chi-square test) would hold up well for this kind of data, it would take further examination. I think a z-test or Chi-square test are a decent starting point, and could be fine to use if the assumptions made for those tests are met (to defend some of the Quora answers). The issue is often tests are used without folks either knowing how to check or ignoring the underlying assumptions, but that isn't an issue with the test itself I'd argue.

[gwern]

> I'm not convinced the parametric assumptions made by the standard tests mentioned (including difference in proportions, whether a z-test or Chi-square test) would hold up well for this kind of data, it would take further examination.

I'm not sure what you mean. If all you have is binary data, there's not much you can do but a chi-squared test or binomial regression. As I understand it, a chi-squared is considered nonparametric: https://en.wikipedia.org/wiki/Chi-squared_test https://en.wikipedia.org/wiki/Fisher's_exact_test

[jupiter90000]

Yes, you're right that chi-squared is non-parametric, my mistake. However, even non-parametric tests can have assumptions that should probably be checked into at least for a given data set. It may be robust to the violations the data collected may exhibit, I'm not sure. This article lists the assumptions for chi-squared test: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3900058/