[muhaaa]

Learning and compressing is closely related. When compressing, you extract systematic rules and unsystematic parameters from the original data. When you learn you do the same but unsystematic parameter become noise. If you throw the noise away you get a lossy compressor.

[tnecniv]

Depending on how you define “learning” (there are different basic frameworks like PAC learning) there’s a number of analyses that show learning and compression are equivalent.

Even quite broadly, Bayesian methods can be interpreted as a rate-distortion problem from Information Theory, which is an approach to lossy compression