[fgimenez]

I like to borrow from the decision theory field. There are massive frameworks around the "value of information" or "value of clairvoyance"[1].

The best summation of their framework is "Make a decision when the probability of new information changing your mind is minimal."

Of course, we need to quantify "new information" and "minimal", but it's a good framework to consider what new data points would change your existing course.

[1] http://lesswrong.com/lw/85x/value_of_information_four_exampl...