I have a random number function that has an exponentially decreasing probability of generating a given integer within [0, R). So for example, if the range of values is [0, 100), 99 has a 50% probability of being generated, 98 has a 25% chance, and so on.
I’m trying to confirm that if I run this function N times (let’s say 1000), that the frequency of the numbers generated match the expected distribution.
Ok, so the big issue is that statistical tests like the chi-squared test are not designed to show that a sample matches a certain distribution. Statistical tests are designed to show the opposite--"this sample does not match that distribution".
If the sample matches the distribution, by design the p-value is going to be uniformly distributed--i.e. a p-value of 0.01 is equally likely as a p-value of 0.99.
I’m trying to confirm that if I run this function N times (let’s say 1000), that the frequency of the numbers generated match the expected distribution.