Hacker News new | ask | show | jobs
by SAI_Peregrinus 1709 days ago
Shannon entropy works for anything you can assign an alphabet of symbols to. It doesn't need the thing itself to be symbolic, just the description of the thing.
1 comments

That seems like a strong claim. Any proof?
Sure. https://people.math.harvard.edu/~ctm/home/text/others/shanno...

Particularly part 4, The Continuous Channel. No discrete symbols there!

Maybe I’m not reading it properly, but in part 5, where he deals with transmission of continuous signals he doesn’t use his entropy formulation. He uses a distance metric, showing the difference between the sent signal and the received.