[arketyp]

Compression is a good entry to Shannon entropy. What was also eye-opening for me was that the metric was motivated by fulfilling a specification which matches an intuitive notion of information [1], much like how the Kolmogorov axioms were characterized to capture the intuitive notion of probability, or how the Church-Turing thesis defines computation.

[1] https://en.wikipedia.org/wiki/Entropy_(information_theory)#C...