Hacker News new | ask | show | jobs
by agentofoblivion 3553 days ago
Although entropy originated in the context of statistical mechanics, Shannon/Jaynes showed it's a much more general and powerful concept. E.g. Information theory. Entropy is simply the average information in a probability distribution. Jaynes papers are super interesting! He derives famous stat mech results while making no ergodic assumptions and shows they're merely a consequence of statistical inference.

I suspect the concept is already present in economics. It's certainly under the hood of many statistical modeling techniques. For instance, some tree learning algorithms use entropy reduction measures to decide how to split the data for the tree. You can even derive the normal distribution by merely maximizing the entropy under the constraint of known variance.