How is explicating the prior an advantage? If the prior is arbitrary anyways you could also stick to your unknown prejudice. This shouldn't change any results and if it does you are in trouble anyways, no matter if you explicitly state your prejudice. I'm still suspecting that Bayesian statistics is just kind of a hack to make results look more convincing.
But, this might be negative, because you can’t consciously tweak an unknown prejudice. But, you can tweak a prior until your results support your hypothesis. In that sense, Baysian statistics might be more transparent, but less honest.
True, but the question is if transparency is desirable. I would say it is dangerous for three reasons. First, you might be tempted to tweak your prior until your posterior confirms your hypothesis. Second, using Bayesian reasoning, you make it seem that the first procedure is justified. And third, if everyone does the tweaking for example within in a scientific community, nobody would complain, since everyone automatically would confirm their hypothesis with higher posterior probability.
If by prior you just mean "I know something about it" or "everything happens in context"; then that fine. But if thats what you mean then a diminishingly small number of events have "priors" which can be expressed in a neat analytical form, or be approximated, or even be quantified. This is part of the problem of frame and context that ML v1.0 tried hard to solve.
Recall as well that in the Bayesian approach the model itself is not subject to Bayesian updating: its part of your prior. Except that you never update it. So youre not merely choosing how to update parameters given data; you're also choosing what you're not going to update.
There is always a prior only if you really care about computing probabilities. The implicit assumption in Bayesian data analysis is that you go first to "best possible estimate of probability", then to "decision based on that". My point was that you usually need not do the first step.
Example: I wear a bicycle helmet because it costs me next nothing and it possibly saves my life. I don't do any Bayesian analysis implicitly or explicitly, because on one side there is an outcome with value minus infinity, so it hardly matters what probability I multiply it with.
You don't need to think hard about massively asymmetric payoffs.
Now what if you needed a something like a $5k licence to wear the helmet. Would you feel like thinking harder and analysing further than you did? Most interesting decisions are more like this.
"Possibly saves my life" is your prior there, btw.
An ideal Bayesian would employ the principle of maximum entropy for choosing priors [1], soon to discover the problem of underdetermination [2].
Such a person would suffer a death akin to Buridan's ass [3].
1. https://en.wikipedia.org/wiki/Principle_of_maximum_entropy
2. https://en.wikipedia.org/wiki/Underdetermination
3. https://en.wikipedia.org/wiki/Buridan%27s_ass