Hacker News new | ask | show | jobs
by AnotherGoodName 23 days ago
>A distribution over strings has entropy.

You have to limit yourself to each string being independent for that to be true which is a limitation that leads to the impossibility of calculating entropy generically.

1 comments

If you want to get correct answers out of math, it helps to start with correct definitions.

A random variable is a random variable and is not the same thing as a stochastic process. You now seem to be talking about stochastic processes, which are a different thing.

A stochastic process can have an entropy rate:

https://en.wikipedia.org/wiki/Entropy_rate

If you want to calculate entropy "generically" without defining what kind of entropy you're talking about, I think you are not likely to succeed.