|
|
|
|
|
by connoredel
3652 days ago
|
|
This is in my opinion the best simple A/B test significance calculator. I like it for two reasons: 1) The multiple testing correction, which many people ignore when testing > 1 variants vs. the control 2) They display not only the p-value, but the confidence interval around the _difference_ between the variants (i.e. the effect size). It's tempting to observe an increase of say 50% that is statistically significant and think that you can be confident about the 50%. This is not the case! You chose your sample size to be confident that the actual effect size is > 0 -- not that you would have a tight confidence interval around whatever difference you observed. Depending on your sample, the lower bound could still be close to 0. |
|