| There are a ton of things to clarify, but I'll take a stab at it. A proper Bayesian who is betting and doesn't want to lose badly and still ends up as a fan of the 'da Bears' has probably got a prior that gives some confident edge to them winning. Belief A could be stated using this joint distribution as the product of their predicted chances of winning each game in order. If it's actually 'likely' then that means you've got a pretty incredible edge on them winning (the size proportional to how much season is left). Belief B is a weird one though. It's a meta-hypothesis about the calibration of your own personal beliefs. The evidence you use to update on this belief is the discrepancy between what you would honestly predict and what actually happened. A proper Bayesian with money on the line would want to recalibrate as best as they could using data already available in order to get the probability of B as low as possible before starting to look at A. So our proper Bayesian first looks over old predictions he has about the Bear's performance, reworking whatever internal understanding of the factors that go into winning in football he has, until he is well calibrated. At this point, his probability for belief A has almost certainly dropped because it's a pretty unlikely thing for a team to just take a thing apart at every single game for the whole season, but if he still ends up with a strong prior on them winning then a single loss, even if it's pretty bad, won't shift it around a lot. In short, he'll think about it a lot, cancel out whatever personal biases he can manage, then bet conservatively unless he has some sort of knowledge that provides a really, really strong edge on them winning. IMO, he's got an inside line with some dirty, dirty men. |
I see a lot of people using "informal" bayesian reasoning (meaning a lot of talk about priors and updating and reference to theorems but never any use of actual distributions beyond super-super-cursory examples applied to trivial situations like the boy/girl thing here or stuff like the monty hall problem).
I don't have any problem at all with bayesian analysis applied in a rigorous setting to a rigorously specified problem (like spam detection and so on).
In an informal setting I'm extremely skeptical of the uses I tend to see b/c there's no careful attempt to clearly delineate which informally-statable hypotheses are valid and which are "invalid" "meta-hypotheses" like the optimism thing.
What you've described here is a way in which someone reasonably smart would eliminate the meta hypothesis, which is fine. In general I wouldn't expect it to be feasible to take a full mental inventory, do a topological sort on your beliefs, and then apply the same procedure; most people most of the time will be running around holding partially-inconsistent beliefs (where "hold" means if you were to ask them to give an estimate of, say, what beliefs they had about what # of their beliefs were likely to wind up revealed to be significantly off in the future, or to give an estimate of what they believe about the frequency with which they'd encounter evidence leading to significant revisions of their beliefs, they'd have an answer on offer which would still have "work to do", the way the unexamined belief that "I'm too optimistic about the bears" really has work to be done).
What I'm curious about is if there's either a clearly-specifiable criteria for which types of beliefs or hypotheses are workable and which are "too meta to work", or there's some kind of theorem guaranteeing that starting out with "inconsistent" beliefs -- in the sense of "meta-hypotheses" like with da bears -- you can apply this algorithm to process evidence and over time you'll converge on beliefs that're at least more consistent than you started with.
It's hard to say much more without getting formal and I'm out of time for now; since I'm mainly concerned with informal use of "bayesian" metaphors it's not hugely critical to formalize this stuff but later I could give it a proper whack.