| > When you speak, what comes first, the idea or the word? Do you first feel a thought inside you, and only after that go searching for the right word to wrap around it? I think we all do. Most people don't actively think when they are talking or writing most of the time. Only during period of thinking that we do think. Most of the time, we just talk words. There is evidence is that we just observe ourselves talking and that the thinking itself is not conscious but it's still a theory. > Now ask the same question about an LLM. For an LLM, it is exactly the opposite. No one has any idea how an LLM come to correct words. LLMs produce numbers (tokens) that end up to coherent words. > So what is an LLM, really? At its core, it is a big pile of words that predicts the next word, using some maths the computers figured out. No, not really. > The words are everything. For an LLM, words are the source, and any meaning is just a byproduct that falls out by accident. The author provides no proof for this and there are very good reasons to think that LLMs have inherent thinking. The rest of the article was just some random thoughts the OP jotted down? I just don't see their coherence to the beginning of the article. |