[lsy]

Two points in response to this:

I think it's a category error to call word embedding in a vector space "semantic" representation when discussing concepts like understanding. Semantics deals with the referents of words, but in this case there are no referents, merely a list of representational tokens which are defined as being "close in meaning" to the original due to proximity in text or some other structural characteristic. We call the embedding "semantic" because it is useful for human semantic purposes as we can mechanize some translations from one vector to another and receive a useful response that we then assign meaning to, but that usefulness doesn't indicate that the machine itself has any access to the referents of the tokens it's processing or semantic understanding. Put more simply, "semantics" does not merely mean the relationship between several ungrounded tokens, but that is all a vector embedding can accomplish.

Secondly, I think in the chess thread, the prompt being "engineered" in the example is extremely complex and constrains the output space sufficiently to produce high-quality results, but you start to wonder at what point the LLM is not doing most of the work. Meanwhile deeper in the thread we learn that even this prompting is not reliable and occasionally requires giving feedback that the move was bad(!) and repetition to achieve good results "the majority of the time in less than 3 tries". You can see where the practical problem arises, if we want to rely on LLMs for answers we don't already know. Claiming that we have a "general" function that "just" requires arbitrarily varying the input over an uncountably large space until you achieve the desired result is akin to saying f(x) = rand() * x is a universal computer as long as you find the right x. The ad absurdum version of the chess example is running Stockfish, sending a prompt that contains the Stockfish move and a request to repeat it, and then claiming that the LLM draws against Stockfish. However as we have seen with tokens like "SolidGoldMagikarp", LLMs are not even yet capable of reliably implementing the identity function, so I am not sure we can even say this.