[earthnail]

If the embedding isn’t smaller than the input, how is it compressing information? It might lose information in its mapping to the embedding space, but in my understanding, the definition of compression means it has to use less bits than the original to hold the same information. As such, the embedding space must be smaller.

[usernametaken29]

You’re thinking information content in the neurons but not distributions. D(world of text) is much much much larger than D(transformer). Just because you’re passing information through a smaller channel doesn’t mean the original distribution necessarily isn’t smaller. Tokenisers are the numbers but then you have V^n where V is your vocabulary and n is the sequence length as a subset of the whole distribution of possible text. Now you pass that through something with sufficient capacity, which compresses the input distribution (you loose bits), and then recreate from your lossy distribution. AutoEncoders are not trained this way because they simply don’t converge if you try to pass smaller input to produce larger embeddings, but they’re doing the same thing. Another way is to think of it is that LLM layers act like binary classifiers via soft max for the input distribution (there were a slew of papers comparing LLMs to SVMs). Essentially during your training you decide what part of the input distribution is going to be part of the embedding. In my humble opinion, while the training diverges, these models produce the same manifold induced by the original distribution.