[d-sc]

Entropy is only kinda about probability.

>How can we talk about the uncertainty of a system that is in a single, well known, predictable state.

The system we are usually talking about is orders of magnitude larger than the uncertainty. Think a liter of gasoline in an engine vs the position of a single atom. While we might not know the trajectory of the atom, we have a pretty good idea how the collective atoms that make up the gasoline will function.

[im3w1l]

> we have a pretty good idea how the collective atoms that make up the gasoline will function.

Yes, and that idea is based on theories assuming things are probabilistic. Thus bringing us back to my second paragraph. That falsely assuming things are probabilistic works very well in practice. But it may break down under certain conditions, and there may be loopholes.

Edit to add:

I think this whole thing is very related to pseudo random number generators and keystretching. Maybe computability theory too. If you have a 1gig file generated from a 128bit seed, then the apparent uncertainty (1gig) is much higher than the actual uncertainty (128bit). But since it's so very hard to undo the prng, it will behave very closely to something with 1gig of actual uncertainty.

Edit to add again:

A fun thought experiment is someone "encrypting" chemical energy into heat, putting the key in a box, only to later go back and decrypt it back into chemical energy.

[Majromax]

> If you have a 1gig file generated from a 128bit seed, then the apparent uncertainty (1gig) is much higher than the actual uncertainty (128bit).

More than that, since you also need to specify the algorithm used for the prng. Your true entropy is the 128-bit key plus the Kolmogorov complexity (https://en.wikipedia.org/wiki/Kolmogorov_complexity) of the generating algorithm.

This is also a good example of how information-theoretic entropy can change based on our prior knowledge. If we already know the generating algorithm, then the entropy is just the 128-bit key.

> But since it's so very hard to undo the prng, it will behave very closely to something with 1gig of actual uncertainty.

No, not really. A one-time pad with 1Gb of actual uncertainty is not made weaker by disclosing 128 bits of entropy. However, 1Gb of PRNG data is made weaker (broken) by disclosing the 128-bit key if the algorithm is known.

We can practically equate the two because inverting the 128-bit system is just as impossible (absent a quantum computer large/fast enough to take 2^64 operations) within the age of the universe as inverting the 1Gb system.

[sfink]

Sure, but the thought experiment is more that you write down 128 bits, generate the 1gb, then eat the paper containing the 128 bits. You now have 1gb that behaves a lot like it is 1gb of entropy, even though it still isn't.

[im3w1l]

I think the concept I mentioned, apparent uncertainty, can be formalized by referencing efficient algorithms. Something like "no efficient algorithm can tell apparent uncertainty and actual uncertainty apart".

[Kednicma]

You're not wrong, but the connection's already explored well; we call it "chemistry". Cook a piece of chalk and it'll crumble into powdery lime; mix it with water and air and it'll heat up and set back into hard stone.

The bigger idea here is that, while it's highly improbable that heat is applied to chalk, we can choose to apply heat to chalk. We have the choice of where to direct our energy. This is, by the Free Will Theorem, the same choice that subatomic particles make about how their energy will be measured. The loophole is precisely that if enough particles choose to synchronize their choices, then they can make highly structured choices with rich data for a long period of time; they can integrate information.

When we crumble the chalk, we lose the specific information about the shape of the chalk, and it would be very hard to recreate; this is entropy. But we also gain the specific information about the shape of the cement, and this would be very hard to determine from the prior context of the universe; this is choice and integrated information.

[im3w1l]

I think this is not what I meant. Let me restate your point as I understand it with a more well known example.

Melt a piece of ice and it will turn into water. Cool it and it will turn back into ice. The bigger idea here is that, while it's highly improbable that heat is applied to ice, we can choose to apply heat to ice. (...)

When you are heating the ice, energy and entropy from the environment is stored in the ice. When you are freezing it, energy and entropy from the water is transferred away to the environment.

This is a different thing from what I was talking about.

[shrimp_emoji]

>A fun thought experiment is someone "encrypting" chemical energy into heat, putting the key in a box, only to later go back and decrypt it back into chemical energy.

Unfortunately, I think it's less like encryption and more like compression. And it's actually less like compression and more like deletion. It's impossible to compress information past a certain point (right?), and the amount left in heat seems like it's just been outright deleted.

The Universe wants to debloat its codebase of information: https://youtu.be/8N1BxHgsoOw?t=119

[im3w1l]

You have a common misunderstanding. Heat contains a lot of information. Chemical energy contains almost none. That's a problem because you can't destroy information, so you can't go back to (much more useful) chemical energy.

But if you have "illusory heat" that seems to contain a lot of information, but it's actually just a tiny amount of information (say 128 bit) that has been used stretched into a lot (say 1gig), then with the right key you can see that it actually is not a lot of information. And then you should, in theory, be able to go back to chemical energy.

[gcthomas]

Systems where there are multiple sequential collisions are not fundamentally deterministic. There are quantum-mechanical effects in the dynamics of the collisions between the electrons in the atoms, whose positions are thoroughly probabilistic in their nature.

Quantum physics is not compatible with determinism, so any dynamical system is only precisely describable in probabilistic terms,

[meroes]

There are deterministic qm interpretations though so this can't be absolutely true.