| Entropy in Statistical Mechanics is a quantity associated with a probability distribution over states of the system. In classical mechanics, this is a probability distribution over the phase space of the system. Two probability distributions with different entropy can both assign finite probability density to the same state, so an increase in entropy does not preclude the possibility of the system returning to its initial state. A great deal of confusion about entropy arises from imagining it as a function of the microstate of a system (in classical mechanics, a point in phase space) when it is actually a function of a probability distribution over possible states of a system. A further wrinkle: Liouville's Theorem [0] shows that evolution under classical mechanics is _entropy preserving_ (because the evolution preserves local phase space density, and entropy is a function of this density). An analogous result applies to quantum mechanics. However, a simple probability distribution parametrized by a few macroscopic parameters rapidly becomes very complex as it evolves in time. When we imagine the entropy of an isolated classical system increasing over time, the meaning is that if we want to model the (very complicated) evolved probability distribution with a simple probability distribution (describable in terms of a few macroscopic parameters), the simple distribution must have entropy greater than or equal to the complex evolved distribution, which is equal to the original entropy before evolution. It's difficult to reconcile the idea that entropy is a function of a probability distribution (not a function of a system's microstate) with the idea that Thermodynamical entropy is an experimentally measurable (kind of...) property of a system. Jaynes' "The Evolution of Carnot's Principle" [1] is the clearest description I've seen of the relationship between Thermodynamic entropy and Statistical Mechanical/Information Theoretical entropy. Many of Jaynes' other papers [2] on this topic are also illuminating. [0] https://en.wikipedia.org/wiki/Liouville's_theorem_(Hamiltoni... [1] https://bayes.wustl.edu/etj/articles/ccarnot.pdf [2] https://bayes.wustl.edu/etj/node1.html |