[rssoconnor]

While this is a reasonable historical explanation of entropy, and explains that we don't gain net energy from the sun, it still misses the mark on what entropy is now known to be.

Entropy isn't a property of an object, or a system or things in physics. Entropy is a property of our _description_ of systems. More precisely it is a measure of how poorly a given specification of a physical system is, i.e. given description of a systems, typically the pressure / volume / temperature of a gas or whatnot, how many different physical systems correspond to such a description.

In particular, _thermodynamic entropy is Shannon entropy_.

In the case where the description of state specifies a volume of phase space wherein a physical state lies within, then the entropy is the logarithm of the volume of this fragment of phase space. If we take this collection of states and see how they evolve in time, then Liouville’s theorem says the volume of phase space will remain constant.

If we want to build a reliable machine, i.e. an engine, that can operate in any initial state that is bounded by our description, and ends up win a final state bounded by some other description, well, in order for this machine to preform reliably, the volume of the final description needs to be greater than the volume of the description of the initial state. Otherwise, some possible initial states will fail to end up in the desired final state. This is the essence of the second law of thermodynamics.

I want to emphasis this: entropy exists in our heads, not in the world.

E.T. Jaynes illustrated this "5. The Gas Mixing Scenario Revisited" in https://www.damtp.cam.ac.uk/user/tong/statphys/jaynes.pdf where two imaginary variants of Argon gas are mixed together. If one engineer is ignorant of the different variants of Argon gas, it is impossible to extract work from the gas, but armed with knowledge of the difference (which must be exploitable otherwise they wouldn't actually be different) work can be extracted.

Knowledge _is_ power.

Taking an extreme example, suppose we have two volumes of gas at different volumes / pressures / temperature. We can compute how much work can be extracted from those gases.

But, suppose someone else knows more than just the volume / pressure / temperature of these gases. This someone happens to know the precise position and velocity of every single molecule of gas (more practically they know the quantum state of the system). This someone now gets to play a the role of Maxwell's demon and separate all the high velocity and low velocity molecules of each chamber, opening and closing a gate using their perfect knowledge of where each particle is at each moment in time. From this they can now extract far more work than the ignorant person.

In both cases the gas was identical. How much useful work one can extract depends on how precise one's knowledge of the state of that gas is.

[ko27]

Entropy is very much "real" and it exists outside of our mind. The resolution to Maxwell's demon is that knowledge of every particle's state is not free, you need to increase the system's entropy by obtaining knowledge more than you can ever eliminate by opening chamber doors.

If it only existed in our minds and not in physical reality that would mean it would be possible to construct a device that decreases global entropy on average.

[pcwelder]

This is eye opening. Thanks a lot for this comment and linking the pdf. I loved E.T. Jayne's Probability Theory book, so looking forward to reading this pdf too.

[kgwgk]

There are a few chapters of an unpublished book on thermodynamics here: https://bayes.wustl.edu/etj/thermo.html

This article is also interesting: "THE EVOLUTION OF CARNOT'S PRINCIPLE" https://bayes.wustl.edu/etj/articles/ccarnot.pdf

Building on these ideas, the first five chapters of this (draft of a) book from Ariel Caticha are quite readable: https://www.arielcaticha.com/my-book-entropic-physics

[rssoconnor]

I want to add this quote that I was originally looking for, and at last found in Jaynes's "Where do we Stand on Maximum Entropy?" (page 237 of http://ndl.ethernet.edu.et/bitstream/123456789/33178/1/R.%20...)

---

    In the Summer of 1951, Professor G. Uhlenbeck gave his
    famous course on Statistical Mechanics at Stanford, and fol-
    lowing the lectures I had many conversations with him, over
    lunch, about the foundations of the theory and current progress
    on it. I had expected, naively, that he would be enthusiastic
    about Shannon's work, and as eager as I to exploit these ideas
    for Statistical Mechanics. Instead, he seemed to think that
    the basic problems were, in principle, solved by the then
    recent work of Bogoliubov and van Hove (which seemed to me
    filling in details, but not touching at all on the real basic
    problems)--and adamantly rejected all suggestions that there
    is any connection between entropy and information.

    His initial reaction to my remarks was exactly like my
    initial reaction to Shannon's: "Whose information?" His
    position, which I never succeeded in shaking one iota, was:
    "Entropy cannot be a measure of 'amount of ignorance,' because
    different people have different amounts of ignorance; entropy
    is a definite physical quantity that can be measured in the
    laboratory with thermometers and calorimeters." Although the
    answer to this was clear in my own mind, I was unable, at the
    time, to convey that answer to him. In trying to explain a
    new idea I was, like Maxwell, groping for words because the
    way of thinking and habits of language then current had to be
    broken before I could express a different way of thinking.

    Today, it seems trivially easy to answer Professor Uhlen-
    beck's objection as follows: "Certainly, different people
    have different amounts of ignorance. The entropy of a thermo-
    dynamic system is a measure of the degree of ignorance of a
    person whose sole knowledge about its microstate consists of
    the values of the macroscopic quantities Xi which define its
    thermodynamic state. This is a completely 'objective' quantity
    in the sense that it is a function only of the Xi, and does not
    depend on anybody's personality. There is then no reason why
    it cannot be measured in the laboratory."

[cubefox]

Hossenfelder says the same in her entropy video. A really interesting hypothesis.