[lalaland1125]

A "posterior probability for p" for a Bernoulli distribution is meaningless because it's equivalent to a single Bernoulli distribution.

[contravariant]

It's meaningful in the sense that what the model produces is often not the posterior probability.

See my other comment for more detail.

[steppi]

If one only had such an estimate for a single example, this would be true, but in aggregate over many predictions, the uncertainty bands can useful for decision making. This is an active area of research.