[lliiffee]

To build on fny's answer, there is one school of statistics (Bayesian statistics) where there basically is a "right" way to do analysis for any problem, provided you make the necessary assumptions (likelihood and prior) correctly. However, the most common statistical concepts (e.g. p-values or confidence intervals) are not in the Bayesian school

[mattkrause]

Credible interval or highest posterior density intervals are arguably much closer to what most people think a confidence interval are.

I’m not sure there is a single right way to solve any given problem isn a Bayesian way, but it does force you to think more about the problem at hand and make your assumptions explicit.