[steppi]

This isn’t what your parent is saying. Many machine learning models are capable of producing calibrated probabilities. What Bayesian models give on top of this is that one doesn’t just predict a probability p, but a posterior distribution for p. This allows for estimating confidence bands around p that quantify ones uncertainty in the estimate. This is useful for assessing data drift and detecting anomalous examples. One can also get such uncertainty bands from non-Bayesian methods by considering ensembles of models. See the review paper by Abdar et al. [0] for more info.

[0] https://www.sciencedirect.com/science/article/pii/S156625352...

[lalaland1125]

A "posterior probability for p" for a Bernoulli distribution is meaningless because it's equivalent to a single Bernoulli distribution.

[contravariant]

It's meaningful in the sense that what the model produces is often not the posterior probability.

See my other comment for more detail.

[steppi]

If one only had such an estimate for a single example, this would be true, but in aggregate over many predictions, the uncertainty bands can useful for decision making. This is an active area of research.