Hacker News new | ask | show | jobs
by cperciva 4994 days ago
Since you say web developer... don't you think it's useful for web developers to have some idea how much traffic is needed for A/B testing to produce meaningful results?
2 comments

Hear hear. I think you've done a good job crafting some questions that debunk this hokum.

Also,if you don't think it's applicable to your life, ok. But when you gripe about the questions or take umbrage with the notion that maybe, just maybe, you've forgotten (or never learned) some information that might have some practical application in the real world, you just sound defensive.

I mean, in the first thread there was a guy who was proud of not knowing what big O notation was. C'mon.

Once you decide to do A/B-testing you only need to know enough about probability to know that it is very hard to guess the amount of traffic needed. That's when you google to remind yourself how to do the actual calculations.
Knowing that you don't know how to correctly calculate the traffic required to reach a specific confidence for an A/B test makes you better than all those people who don't even know that they don't know, and just wing it.

(edit: remove multiple calculates!)