Entropy (differences) are an objective quantity which can be measured, there is no subjectivity about it. It is not which parameters you know it is about which parameters you hold fixed.
Fascinating discussion. I see some parallel here to the Bayesian vs Frequentist view of probability.
They are perhaps both valid points of view depending on the situation.
If you take a frequentist view of an unbiased coin, then the probability that it will land heads on the next flip is objectively, by definition 50%. So the resulting calculation of entropy (log 50% = 1 bit) is also objectively defined. But if your 50% probability represents a subjective belief, the resulting entropy calculation should also be considered subjective, I would think.
Leonard Susskind disagrees with you. See his lectures on statistical mechanics, he is very clear that entropy is a matter of knowledge about the system. It has to be.
Susskind says that entropy is determined by selecting a macro-state. He doesn't claim that the entropy of a macro-state depends on whether we know which macro-state the real system is really in.
If we happen to know, then, sure. For example we could pick a weird-ass observable state, and when we saw it we would know the entropy of the system was low. But the entropy of each macro-state just depends on how many micro-states we define it to contain. It doesn't depend on our knowledge of the system state.
The concept of entropy wasn't invented so that we could calculate entropies of macrostates, it was so that we could calculate entropies of real systems and understand their behaviour. Macrostates are an accounting tool that helps us do this. You seem to be treating the calculation of macrostate entropy as an end-goal in itself, but also allowing yourself to somehow freely choose any macrostate you want. When it comes to applying thermodynamics in practice, you'll have to calculate the entropy of a real, or at least hypothetical, system.
The point of macrostates is that you ought to know which macrostate a given system is in. That's the thing that you know. You don't know which microstate it's in, but you do know which macrostate it's in.
For example, if I say "a cubic metre volume of air at room temperature and pressure", I've described a physical system. I've also described a macrostate.
If you're calculating the entropy of macrostates that are not consistent with a description of a system -- if you've defined your macrostates such that you don't know which macrostate a given system is in -- then in order to calculate that system's entropy you have to sum up over all such possible macrostates anyway, so you haven't saved yourself any work or earned any insights along the way.
So yes, you can calculate the entropy of a macrostate without knowing what macrostate a real system is in, but it kind of sounds like you're arguing that log(x) is not a function of variable x, because log(3) is a constant and log(4) is a constant, and you can divide up any x into constants of your choice.
We seem to be stuck in a loop of explaining basic first-year statistical mechanics back and forth to each other repeatedly. I'm not sure why.
I'm making a pedagogical point. The OP addresses how difficult entropy is to understand. I'm responding to that. We don't need to talk about "knowledge" when you define entropy, or in an initial explanation of entropy. We could, but we could decide not to.
The log(x) example is a good one. First-time students who are learning about logarithms don't need to be told that a logarithm depends on 'knowledge' or on 'information.' It's ok to just tell them how logarithm is defined.
Sure, there is information. I'm saying it's confusing and unnecessary to introduce more big ideas like information, when the topic is "entropy is difficult to understand" or "logarithms are difficult to understand."
Alright. I agree we seem to be stuck in a weird loop where we agree about all the observable facts but somehow are on different wavelengths in spite of that.
And I totally agree, entropy is a property of a macrostate. The information step comes inseparably in when you go from a system description to a macrostate. And you might just shuffle the confusion from not knowing what entropy is to not knowing what a macrostate is.
If you think it's clearer to teach students by explaining that the entropy of a macrostate is an objective property of that macrostate, that's fine. Just don't leave them believing that the entropy of a brick is an objective property of that brick.
Why shd the entropy of a brick not be an objective property (apart from a constant). I mean you can measure it, isnt it basically the integral of C/T dt?
I've been trying to reconcile these perspectives, and I think it really is both. And they are both physically relevant.
Consider the subjective entropy perspective. If you know the exact microstate of a system, then you can in theory play the part of Maxwell's demon. You could have a little gate that you open only for fast particles, and using your knowledge of the microstate, you can predict exactly when they will arrive.
But consider the objective perspective. If you take this very same system and put it in thermal contact with another system, then an objective entropy perspective is the relevant one. Those systems will equilibrize and your subjective knowledge is irrelevant to that process.
I haven't fully wrapped my head around it yet, but I do think that acknowledging both is a step in the right direction at least.
> If you take this very same system and put it in thermal contact with another system, then an objective entropy perspective is the relevant one. Those systems will equilibrize and your subjective knowledge is irrelevant to that process.
The subjective view handles this scenario just fine, though, and makes more accurate predictions than the objective view.
For example, there are systems where some aspects of the original microstate survive thermal contact with another system. We use such systems to store data! I bet your hard drive is in thermal contact with its environment right now! It's very hard to reconcile this with an objective take on entropy.
And there are some systems that will rapidly be scrambled. The subjective perspective has no problem admitting that your knowledge of a system can become inaccurate and useless. Even without thermal contact, you'd need to perform a tremendous amount of (perhaps reversible) computation in order to make a functioning Maxwell's demon with your initial microstate conditions, because the microstate will evolve in time in a complicated way. The subjective view is still totally consistent with entropy of a system increasing over time!
I wrote two replies to this that I both deleted. Then I had a good long thunk, and here's what I came up with.
The temperature of an object can be determined through 1/T = dS / dE. What is this S? How can it exist if you know the system perfectly? And here is where the great insight comes. The thermometer! You apply a thermometer to a system you perturb it! The system may have started in one particular microstate, but the very nature of thermal contact involves random influence. Those random tiny influences from the thermometer allow the object (harddrive in our case) to enter a bunch of microstates with certain probabilities. And that's what S measures.
So our subjective knowledge does actually not matter. (Classically speaking) the system is in a particular microstate we may know it or not, and it still manages to have a temperature. That is due to the states it could hypothetically enter (but haven't yet)!
If we think back to the harddrive and it's contents: Very gently touching a harddrive with a thermometer while not scramble its contents. So we may say that microstates corresponding to different files than the ones you put there are actually not accessible. And they don't contribute to the entropy we used for the temperature.
No, it is subjective. We just only have such blunt instruments for practically measuring states, relative to the gargantuan amount of entropy in most real systems, that the subjective nature of entropy is easy to miss. But in a world where the frontiers of thermodynamics have moved from steam engines to lasers, computers, DNA, and black holes, the difference is increasingly obvious and important.
With steam engines, we got away with treating a volume of gas as having not only a few parameters that we knew and cared about, like mass, temperature and pressure, but we could further deceive ourselves into thinking that those were the only parameters that existed to describe the system. The only parameters that were knowable. But Boltzmann knew better.
Look at Boltzmann's formula, S = kB log W.
For any single particular system you describe to me, W will be 1, and so S will be 0. So it's only if you describe an ensemble of systems -- that is, if you describe a system vaguely, such that I am left to imagine the details -- that we have nonzero entropy. If you ask me to calculate the entropy of that "system", that macrostate, that ensemble, then sure, I'll end up with nonzero entropy. But if I ask you to keep transmitting more data about the scenario, then with each further description, you'll be narrowing the state space and thereby decreasing the entropy.
Look, since the entropy of a macrostate is nonzero, but the entropy of any single microstate which is consistent with that macrostate is zero, it's clear that entropy is not an intrinsic property of any real system. It's a property of how many other possible non-existent systems could be swapped out for the one in front of you, without you noticing the change.
If I swap out the air in your room for an equal volume of air at equal temperature and pressure, you probably won't notice.
If I swap out the hard drive in your laptop for an equal volume of hard drive at equal temperature and pressure, you probably will!
Maybe better to say that the universe does not appear to pick out a single coarse-graining or fine-graining procedure for practically any system.
For instance, following your Boltzmannian example, I think one would notice swapping 1 µm³ of the r/w head and 1 µm³ of the recording surface of a new, freshly powered on HDD more than one would notice substituting the entire HDD for a new one of the same model and turning that on. And here I am already using units of length (cf. "equal volume"), and we know neither units nor lengths are generally picked out by the universe.
Information entropy and statistical mechanical entropy are two different things.
They share the same equation and the same name but they are two unrelated concepts. You have conflated the two. The person you are responding to is referring to statistical entropy.
Basically in this entire thread nobody, including you, is fully grasping the situation.
Fun rabbit hole would start with classic paper by jaynes
Many more recent examples relating bit erasure costs of computation.
Some names to look up if interested include charlie Bennet,Dave wolpert, James crutchfield, Susanne still, for starters.
Edit -- a collection of ideas related to this problem and mixing in "complexity" can be found in SFI proceedings called "Complexity, entropy, and the physics of information"
If you think there is no relation between the different things called entropy apart from the name maybe you're not fully grasping the situation either.
The free energy F = U - TS is the maximum amount of work you can extract from the system. This depends on how much you know about the system. S does indeed depend on what you know about the system.
If two people disagree on the maximum amount of work that could be extracted from a given system (with both of them basing their figure on their own evaluation of S), are there any cases where it would it be impossible to empirically demonstrate that at least the proponent of the lower figure was wrong?
If only changes in S (and F) have measurable consequences, would that not merely mean that assigning an absolute value is an arbitrary choice, which would not mean the same as it being subjective (there could still be an objective conversion between one basis and another, as there is for kinetic energy in different inertial reference frames.)
In the Gibbs Paradox, there is no subjectivity in whether the gases being mixed are the same or different, and no subjectivity in what the change of entropy is in either case. The paradox is that it does not feel right that identity makes an objective difference between the two cases, but the empirically-demonstrable distinction between fermions and bosons shows that this intuition does not hold in general. I believe Von Neumann came up with a QM resolution of the paradox.
> are there any cases where it would it be impossible to empirically demonstrate that at least the proponent of the lower figure was wrong?
Isn't it more interesting to examine a situation where it would be possible to empirically demonstrate that the proponent of the lower figure was wrong?
In that case we could objectively say that one value of S does not yield F for that system (given that F is defined as a maximum), but this would not resolve the general question of subjectivity.
If that doesn't, I'm not sure what would. Maybe it would help if I taboo the word "subjective". Are you familiar with Maxwell's demon? Let's set up a variation of that experiment.
I have a partitioned box full of air at room temperature and pressure in both partitions. There's a frictionless door that can be open and closed by an ultrafast servomechanism. The servo is connected to a computer which will read a very long bitstring from a magnetic hard-drive platter at a high frequency and open the door when the bit is '1' and close it when the bit is '0'.
Admittedly, this mechanism would be hard to construct in practice, but I hope it's clear enough as a thought experiment.
Now if you're familiar with Maxwell's demon, you'll agree that there are particular, albeit rare, joint configurations of gas microstate and hard drive bitstring, such that after the servo has finished its last motion, the gas will have been separated into hot and cold on either side of the partition. This temperature difference can be used to extract work.
For each possible bitstring on the drive, there are certain corresponding microstates of gas that will maximize the free energy extracted by this process.
And for each possible microstate of the gas, there are certain corresponding bitstrings that will maximize the free energy extracted by this process.
(For the vast majority of other combinations of hard drive bitstring and gas microstate, the operation will have no effect).
The claim "entropy is subjective" is basically just an acknowledgement that the energy extractable from the gas is dependent on both the state of the gas itself and also the data written to the hard drive. It means that two experimenters, tasked with writing the initial data on the hard drive to extract as much work as possible from the gas, will have different levels of success depending on whether they know the particular microstate of the gas (and can thus select the corresponding optimal bitstring) or if they don't know the microstate of the gas beyond "a box at room temperature and pressure", and have to guess a bitstring based on only that. And when the operation is successful, we can describe the data on the hard drive as "information about the gas microstate that was used to extract work".
This experiment, of course, is so impractical that it sounds ridiculous. But we can make a more controlled version of it on the small scale, with excited trapped atoms, and actually make it work.
It is not clear to me in which direction you think the question would be resolved, given a situation in which it would be possible to empirically demonstrate that the proponent of the lower figure (for S) was wrong. Maybe that is because I do not see the connection between your thought experiment and this issue: neither of the candidate values for S entail a particular distribution of states, let alone a particular sequence of future times when a molecule will approach the gate in a particular direction.
As I see it, your experiment is a difficult-to-perform way to demonstrate that, due to the inherent randomness of thermal processes, the entropy of a closed system may decrease when the conditions are right. This is explicitly covered in the article (see also "Monkeys typing Hamlet.")
Furthermore, in the case where the microstates of the system are measured in detail and the arrival times and velocities of the molecules at the gate are computed, one must add in the change in entropy resulting from those measurements and calculations. I am pretty sure this has been done, and is in accordance with the 2nd. law.
Your definition of 'subjective' in your penultimate paragraph is contrary to both common usage and what is being discussed in this thicket of threads, and appears to be closer to 'stochastic'. The outcome of the spin of a roulette wheel does not become subjective when different gamblers place different bets on it, or even when someone who has recorded statistics for its outcomes is able to place better-than-random bets.
I think he is confusing the usage of entropy in physics and computer science. In computer science entropy is conditional probability and depends on what we know about a system.
I've thought that for a while, but I'm not a physicist. Do you know any prominent physicists that hold that view? It seems to contradict at least the popular narrative about entropy as a property of a system.
I only have an undergraduate degree in physics, but I think the point you may be missing is that information requires some physical medium to store it.
So entropy is very roughly speaking the property of the system that determines how big a hard drive you need to store a description of that system.
They are perhaps both valid points of view depending on the situation.
If you take a frequentist view of an unbiased coin, then the probability that it will land heads on the next flip is objectively, by definition 50%. So the resulting calculation of entropy (log 50% = 1 bit) is also objectively defined. But if your 50% probability represents a subjective belief, the resulting entropy calculation should also be considered subjective, I would think.