[trashtester]

Sure. I understand that part. 99% of humans wouldn't even be able to do it consciously if they tried, so this has to operate on the subconscious level, to the extent that it's an accurate model.

But if you start out with randomized priors, you will never reach 1 or 0 regardless of how much data you expose a bayesian system to. Humans, on the other hand, tend to fall into treating a probability as either 0 or 1 quite rapidly. For this step in particular, there seems to be something like rounding or L1 regularization going on.

Then, once they're stuck in 0 or 1 priors, people often revert to using evaluation similar to Bayes' Theorem again, but in that case the priors can no longer be updated (except through something like a psychological shock, hallucinogen etc).

But, as stated above, you cannot really reach such priors using Bayes' Theorem alone (if we assume the priors are not provided by genes or something that happens before the learning).

[FrustratedMonky]

Along these lines, what is happening when someone walks into a room and sees a snake, but then does a double take and sees that it is a rope.

It seems humans do have miss-identifying visions. Where they categorize something fast but incorrectly, and then might need 'help/time/shock', to kick start re-categorizing. To re-see it again.

Or Like when seeing something for the very first time and they are 'befuddled', can't grasp it. Maybe like Bayes is having to iterate on it much longer.

I'm curious since I'm not all that familiar with Bayes. Is this what you are talking about with 1 or 0. People do make very rapid judgments, then settle on an a 'view'. Then it can take something to make them re-adjust.

[trashtester]

> Is this what you are talking about with 1 or 0.

It mostly has to do with the zeroes. Take Bayes' theorem:

  P(A|B)=P(B|A)P(A)/P(B)

The prior for some hypothesis A is P(A)

If you start with no knowledge, P(A) can be something like 1/n, where n is approximately the number of competing hypothesis.

B is the "data" part, a weighed compbination of all data used to update P(A).

P(A|B) is the posterior probability for A, given the data B. This posterior is then used to update your prior P(A) before you are exposed to new data.

Take the hypothesis "God exists as a male person". Before any data, maybe you give this a 1/4 prior probability. Add the Bible as your only source of data, P(A|B(ible)) may go to 9/10. Add all other religious texts, and maybe it goes to 1/2. Add all of Science, and maybe it goes a bit lower.

Now what happens when we set a probability to 0?

If P(A)=0, then P(A|B) stays at 0 regardless of how much data you pile into B. A prior of 0 makes it completely impossible to convince you that A is true.

Or, lets say you have two competing hypotheses:

  A1: God is exists as a male person:

  A2: Hypothesis A1 is false. (God either does not exist or is not male or not a person).

Since A1 and A2 cover all possible states, P(A1)+P(A2) = 1.

In other words, if you set P(A1) to 1 you simultaneously set P(A2) to 0.

And as above, when you have a prior of 0, it will never get updated using bayesian logic, so regardless of any evidence to the contrary, P(A1) will remain 1.

Only in a situation where P(B|A1) is also 0 could you ever doubt A1. When this happens, a person may live through severe cognitive dissonance or crisis, as if all of reality falls apart. The person will have to choose to discard A1, or to find some reason to ignore the data B.

For instance, if you believe 100% that some organization (any organization, but let's pick Hamas this time) are the good guys (hypothesis A), and some event B (like a massacre of civilians) seems to strongly oppose that belief.

Lets say P(B|A) is very low in this case.

If you have built your existence and identity around P(A)=1.

Now, instead of believing B really happened, you can introduce an alternative (conspiracy theory) view on the data, for instance:

  C: It was really Mossad that tricked Hamas into attacking Israel.

In this case, P(C|A) can be quite a bit higher than P(B|A).

Now, if you extend this, you can even turn this around. If you see A as the DATA not as the Hypothesis, you can set up

  P(B|A) = P(A|B)P(B)/P(A)

  P(C|A) = P(A|C)P(C)/P(A)

While other people take B simply as data, you have now turned it into something to be disproved. And if you are certain that P(A)=1, you can believe almost any conspiration theory C as long as P(A|C) > P(A|B).

And this can cascade. If P(A|B) is small enough, then you can end up being certain that P(B) = 0 and eventually that P(C)=1.

Anyway, the above type of reasoning is at the root of religious and dogmatic thinking. In fact even a single false belief held with a prior probability of 1 can be used (through Bayesian logic) to prove almost anything.

And this is not limited to happening by accident. A skilled demagogue who can trick the audience into accepting (with no room for doubt) a single false or inaccurate claim, can then use to manipulate them into believing almost anything.

(Obviously, in the Hamas/Isreal conflict, this goes both ways.)

[FrustratedMonky]

Maybe it is just an engineering problem.

Humans aren't strictly math computations that can arrive at a absolute 0 or 1.

Maybe humans have a bit of a random number generator that kicks in some noise to avoid reaching absolute 0 or 1.

Allows some re-evaluation.

Maybe that would be option in AI to avoid the problem. During one of the games with AlphaGo it seemed to get stuck and was treading water making really 'plain or lackluster' moves. Until it kind of got unstuck. It seems to be similar?

[trashtester]

> Maybe it is just an engineering problem.

Yes

> Humans aren't strictly math computations that can arrive at a absolute 0 or 1.

Actually, I think it's the other way around. Humans will arrive at a 0 or 1 (being convinced they know the truth), often with quite little support by evidence/data.

Rather, I think this is biology's "engineering solution" to having limited compute power.

> Allows some re-evaluation.

When you do get P=0 or P=1, you stop re-evaluating (when using basically Bayesian logic).

> Maybe that would be option in AI to avoid the problem.

AI can definitely be made to avoid this problem. It's a lot better at doing webs of Bayesian logic than humans are, if we tell it to, and can probably do that even if not explicitly designed for Bayesian logic.

My concern is more about humans.

The common word for people that evaluate P=0 or 1 AND draw the consequences from that, is fundamentalist. If anything, more intelligent people have a higher risk of becoming a fundementalist than others, since a single wrong absolute belief tends to cascade into a larger web of induced false beliefs, simply to keep their view on reality consistent.

Often it's better when people simply allow themselves to hold mutually exclusive beliefs, without caring so much about the contradiction. Like when people hold religious beliefs (maybe that people that don't belive in their God the way they do will go to hell), while NOT trying convert their souls at gunpoint out of "mercy" to prevent eternal suffering.