Hacker News new | ask | show | jobs
by ritzaco 840 days ago
The bottleneck with A/B tests isn't tooling, it's statistical ignorance in marketing professionals.

Not claiming you did this, but in your example of "Jack tweeted" vs "Jack just tweeted", it's so often bad statistics or cherry-picking that leads to results like this. Testing null hypothesis is hard, and many people will take a reality like this chart [0] and claim that their A/B test is successful.

I definitely think there's a gap in the market since Google sunset optimize, but $2000 seems kind of steep for something that many people got for free before.

[0] https://i.ritzastatic.com/images/3dcdd822dbd241b1b3ddeeb9540...