Hacker News new | ask | show | jobs
by jtsuken 2024 days ago
Can you please define Shannon-style code? (Obviously, you don't have the shannon code, the compression method, in mind).

Also, isn't Kolmogorov complexity uncomputable and you run into multiple "who shaves the barber" issues, when trying to determine it?

1 comments

The compression code can be specified first. If you have a lot of data, the specification will be negligible in length, compared to the code itself. Together, the specification and the Shannon code give an upper bound on the Kolmogorov compmlexity. If this is the shortest known program, we may consider the Shannon code probabilities as our "best explanation" of the data.

You can also get posterior probabilities using a universal prior such as 2^-K(x), but of course, this can only be approximated in the limit of infinite runtime.