[d110af5ccf]

Information loss, or the result of useful computation? VAEs exist after all.

[chriswarbo]

LLMs can't reason about spelling, e.g. asking for a sentence which contains no letter "a"; and can also struggle with rhyming, etc. The most obvious explanation is that they never 'see' the underlying letters/spelling, only tokens.