Hacker News new | ask | show | jobs
by sanskritabelt 4572 days ago
From: "If we see that 1000 of the 10,000 random iterations had a difference of more than $1.50, we can say that there is a 10% chance that our $1.50 observed difference was due to randomness."

I'm wondering if maybe you're doing way too many simulations in the calibration, do you really need more than a few hundred to a thousand or so? The 0.1 quantile is reasonably well separated from 0, and I would have expected you'd get "good enough" convergence pretty quickly.

Also: "we can compare our computed p-value of 11% to our simulated 10% result to determine whether or not the model is accurate enough." you're getting a full pdf out of the simulations, are you also comparing to the full distribution of your test statistics?