[brabel]

> If instead you had a sample size of 2000, then you'd get a very good signal to noise ratio.

In modern statistics, even a small number of samples can be considered enough to get a satisfactorily small error range, as long as the sample is random and representative of the population. I would think 2,000 samples is far more than strictly required if you're able to sample from your target market.

[vharuck]

It's true that people often incorrectly dismiss results with sample sizes below 100. But, for rare events, you really do need large samples. Otherwise, you'll only ever be able to confidently identify massive differences.

[jjk166]

Average results for cold calling are a 4.8% success rate, meaning with 2000 cold calls you'd expect less than 100 hits for a good campaign. This in turn is highly variable with field, a 1% success rate might be high, especially for a product with no pre-existing market presence. And it's not enough to see whether you're above a certain number, you need to know the variance to predict future performance. Maybe the magic number is 1500 or something, the number will no doubt vary based on the peculiarities of the experiment, including the product and the company, but to see a signal with 2 standard deviations of significance, you need a sample size that you would expect to produce about 20 times the expected noise level.