[frig]

The specific example of 'da Bears' was a weak attempt at humor. I can't easily clarify what I'm getting at and keep this short, and I've got limited time so I'll do the best I can.

I see a lot of people using "informal" bayesian reasoning (meaning a lot of talk about priors and updating and reference to theorems but never any use of actual distributions beyond super-super-cursory examples applied to trivial situations like the boy/girl thing here or stuff like the monty hall problem).

I don't have any problem at all with bayesian analysis applied in a rigorous setting to a rigorously specified problem (like spam detection and so on).

In an informal setting I'm extremely skeptical of the uses I tend to see b/c there's no careful attempt to clearly delineate which informally-statable hypotheses are valid and which are "invalid" "meta-hypotheses" like the optimism thing.

What you've described here is a way in which someone reasonably smart would eliminate the meta hypothesis, which is fine. In general I wouldn't expect it to be feasible to take a full mental inventory, do a topological sort on your beliefs, and then apply the same procedure; most people most of the time will be running around holding partially-inconsistent beliefs (where "hold" means if you were to ask them to give an estimate of, say, what beliefs they had about what # of their beliefs were likely to wind up revealed to be significantly off in the future, or to give an estimate of what they believe about the frequency with which they'd encounter evidence leading to significant revisions of their beliefs, they'd have an answer on offer which would still have "work to do", the way the unexamined belief that "I'm too optimistic about the bears" really has work to be done).

What I'm curious about is if there's either a clearly-specifiable criteria for which types of beliefs or hypotheses are workable and which are "too meta to work", or there's some kind of theorem guaranteeing that starting out with "inconsistent" beliefs -- in the sense of "meta-hypotheses" like with da bears -- you can apply this algorithm to process evidence and over time you'll converge on beliefs that're at least more consistent than you started with.

It's hard to say much more without getting formal and I'm out of time for now; since I'm mainly concerned with informal use of "bayesian" metaphors it's not hugely critical to formalize this stuff but later I could give it a proper whack.

[tel]

That's definitely an interesting space. I think the highly principled side of the Bayesian boat would state that meta-hypotheses are tied into your prior on model building information (Pr(I), etc) and that it needs to be updated alongside everything else. So now if your hypothesis' posterior becomes something like Pr(H, theta, I) the whole business needs to be updated and will include all the meta level intellectual rigor. At this point I feel like I might be walking into the space of Structural Causal Modeling and I'm not too well versed there at all.

In the informal setting though you're only ever likely to be trying to "update" one belief at a time, so, yeah, it definitely requires intellectual care to make sure to follow dependencies. Worse though, is that it should be possible to two have codependent estimations and if you aren't aware of that codependency you won't ever be able to get along.

I think that's all interesting, but I'm not sure it applies to informal situations as well as one might hope. Frequently, Bayesian techniques are only used informally in conjunction with strong rationalist heuristics which help to build these reductionist hierarchies of effects and then allow for clear(er) methodology to find an accurate answer.

Few people thinking carefully and rationally would be willing to bet on their beliefs so long as they know that thy have an outstanding miscalibration. That's why scientists, good scientists anyway, will so often preclude things with disclaimers. They want you to be aware of whatever biases they can before you start to judge their opinions.