|
|
|
|
|
|
by Der_Einzige
1482 days ago
|
|
|
The deep connections between compression and prediction are not always obvious to those not in the field. To illustrate just how much they are the same, here is (at one point SOTA) lossless text compression with GPT-2 https://bellard.org/libnc/gpt2tc.html |
|
|
Is the concept `addition` a compression of the space `(Int, Int, Int)` ?
If you want to say it is, OK for some definition of compression. But that compression isnt "mere" in the modern AI sense, it's "exponentially dense".
In that my concept `addition` can generate arbitrarily large amounts of that decompressed space, which is infinite in size.
There's a kind of trick played in the marketing here: since NNs compress, and since learning "can be seen as compression", NNs learn... no, because NNs aren't "exponentially dense", they're "exponentially large" -- I'd claim, the opposite of learning!