| "Information theory and statistical mechanics" by E.T. Jaynes [0]. Starting from a Bayesian/information-theoretic perspective, he derived statistical mechanics in just a few pages. This approach finally made stat-mech 'click' for me. It's a paradigm shift. Entropy (in the Bayesian sense) is fundamentally a subjective quantity. Maximizing entropy is just minimizing the assumed information. When dealing with large complex systems over long timescales, the only pieces of information one is justified in assuming are the values of those quantities that are conserved globally by the dynamics (eg, total energy, total mass, etc). The "thermodynamic entropy" S is just the maximum entropy, given a certain set of conserved quantities (an "ensemble") -- it is therefore more or less objective, modulo the conserved quantities. The second law of thermodynamics is just the information processing inequality: you don't have any more information about the future state of the system than you do about the present state. If anything, you have less, because if you have any information about the current state of non-conserved quantities, that information is not valid at other times (assuming you don't have complete information & the ability to fully simulate the dynamics). From this perspective, entropy does not generate the "arrow of time": the argument is symmetric in time. Another from Jaynes worthy of mention: "Prior probabilities" [1] discusses the use of group theory to derive "non-informative" prior distributions by considering the set of transformations that result in equivalent inference scenarios. This is resolved another question I had when studying stat mech: "How come we start with an assumption of uniform probability in phase space as expressed in (x,p) coordinates -- if we transformed to a different coordinate system, it would look like a non-uniform distribution!" The answer is that we assume Galilean invariance. The concept applies outside stat mech as well. Plenty of other interesting stuff [2]. [0] https://bayes.wustl.edu/etj/articles/theory.1.pdf
[1] https://bayes.wustl.edu/etj/articles/prior.pdf
[2] https://bayes.wustl.edu/etj/node1.html |