[loup-vaillant]

E.T. Jaynes book, "Probability Theory: the Logic of Science" may come close to what you want. It emphasize that there are rules of thought, which lead to Bayesian statistics. As such, Bayesian statistics aren't just a recipe, but the law.

Now, I can only personally vouch for the first 2 chapters, as I haven't read the rest yet.

[avaku]

The greatness of this book cannot be overstated

[ph0rcyas]

Agree. It is unbelievable - one has to study it to believe it.

[danso]

Going to Amazon right now...

* edit: Doh, no Kindle version. I don't mind paying $90+ for a good book though, just like it to be electronic: http://www.amazon.com/Probability-Theory-The-Logic-Science/d...

[dan_yall]

Free pdf is available here:

http://bayes.wustl.edu/etj/prob/book.pdf

It's always nice to see good things come out of Wash U. (Alum here.)

[Fixnum]

Unfortunately, it's only the first 95 pages.

[gwern]

There must be a fuller version floating around, though; my PDF version has 548 pages and ends with Appendix E, 'Multivariate Gaussian Integrals'.

EDIT: In case anyone wants to make me feel bad about pirating, Jaynes is dead, and besides that, I bought a hardcopy as backup.

[incision]

Well, it's available on Google Books [1], but I don't know about $63 for what appears to be a skewed scan of the print book.

Personally, I searched out a PDF and based on what I've read so far, I'm itching to pull the trigger on Amazon as I'm simply loving what I'm reading.

1: http://goo.gl/UHMBi

[Datonomics]

http://www.naturalthinker.net/trl/texts/Science/Jaynes,%20E....