[scryder]

Statistics arises from a set of axioms, assumed truths, which can be used to prove all other things in the field.

You can take a look at the three axioms people use to justify statistics. If you are willing to accept them, all else that relies on them (without using new axioms) must be true:

https://en.wikipedia.org/wiki/Probability_axioms

This same logic is used to justify development in pure mathematics: choose a set of axioms which you accept as ground truths, and prove things using them. As long as you are unable to prove your axioms are contradictory, and the axiom choice seems acceptable, then the work that you've done (with respect to them) is philosophically justified.

[tw1010]

Statistics and probability are different things. I'm fine with the foundations of probability.

[mturmon]

Just for reference, not everyone is OK with the foundations of probability - what you might call "conventional mathematical probability" as axiomatized by Kolmogorov. See http://www2.idsia.ch/cms/isipta-ecsqaru/ for the most recent in a series of workshops.

One entry into this set of ideas is what Peter Walley has called the "Bayesian dogma of precision" - that every event has a precise probability, that every outcome has a known cost. There are real-world situations when these probabilities cannot be assessed, or may not even exist; same for utilities.

Some examples are in betting and markets (asymmetric information, bounded rationality), and in complex simulation environments having so many parameters and encoded physics that the interpretation of their probabilistic predictions is unclear.

[BeetleB]

Please don't treat probability and statistics as one.