Hacker News new | ask | show | jobs
by hirundo 998 days ago
> the next generation will see it as the current see the TV - unimportant old boring low-entropy data which is not entertaining anymore.

Don't you mean high-entropy data? High entropy data would be less orderly, more compressible, and have a lower signal to noise ratio ... like TV shows compared to a textbook.
2 comments
inimino 998 days ago
Low entropy data is what's more compressible. High entropy means unpredictable, which could mean high noise (like TV static which is incompressible) or high signal as GP intended.
link
hirundo 998 days ago
Thank you. I just learned about Shannon Entropy and that it grows with information content. The negative sign in the formula is due to the negative log of the normalized population probability.

https://math.stackexchange.com/questions/395121/how-entropy-...

link
inimino 998 days ago
Yes. You can also take the log of the reciprocal probability (or "luck") and then you don't need the negative sign. This value is the expected number of trials you would need for the outcome to occur once. I find this presentation a bit more intuitive. See also Boltzmann entropy S = k log W which takes this form.
link
mywittyname 998 days ago
Entropy is the "signal" within the noise.
link
tines 998 days ago
Not necessarily. Pure noise has high entropy.
link