[tshaddox]

> My understanding is that entropy is a way of quantifying how many different ways a thing could 'actually be' and yet still 'appear to be' how it is. So it is largely a result of an observer's limited ability to perceive / interrogate the 'true' nature of the system in question.

When ice cubes in a glass of water slowly melt, and the temperature of the liquid water decreases, where does the limited ability of an observer come into play?

It seems to me that two things in this scenario are true:

1) The fundamental physical interactions (i.e. particle collisions) are all time-reversible, and no observer of any one such interaction would be able to tell which directly time is flowing.

2) The states of the overall system are not time-reversible.

[CaptainNegative]

The temperature of an object is a macroscopic property basically depending on the kinetic energy of the matter within it, which in a typical cup of water varies substantially from one molecule to the next. If before you could guess a little bit about the kinetic energy of a given water molecule based on whether it is part of the ice or not, after melting and sufficient time to equilibrate the location of a particular molecule gives you no additional information for estimating its velocity.

[dynm]

It's tricky when you think of a continuous system because the "differential entropy" is different (and more subtle) than the "entropy". Even if a system is time-reversible, the "measure" of a set of states can change.

For example: Say I'm at some distance from you, between 0 and 1 km (all equiprobable). Now I switch to being 10x as far away. This is time-reversible, but because the volume of the set of states changed, the differential entropy changes. This is the kind of thing that happens in time-reversible continuous systems that can't happen in time-reversible discrete systems.

[im3w1l]

Isn't that kind of what we want entropy to capture though? If a particle darts off into the distance then in theory it might be time reversible, but in practice it's not so simple. If the particle escapes the gravitational pull, the only way it can come back is if it bumps into some other object and pushes that object away. So things will inevitably spread out more and more creating an arrow of time.

This can then be related to the big bang, and maybe it could be said that we are all living of the negentropy from that event and the subsequent expansion.

Getting different entropy values based on choice of units is a very nasty property though. It kinda hints that there is one canonical correct unit (plank length?)

[jdhwosnhw]

I have yet to see differential entropy used successfully (beyond its explicitly constructed-for purpose for calculating channel capacity). Similar to your thought experiment is the issue that the differential entropy value depends on your choice of unit system. Fundamentally the issue is that you cant stick a quantity with units into a transcendental function and get meaningful results out

[dynm]

Yeah, it's quite disturbing that the differential entropy (unlike the discrete entropy) depends on the units. Even worse, the differential entropy can be negative!

Interestingly, the differential KL-divergence (differential cross-entropy - differential entropy) doesn't seem to have any of these problems.