You may write one word at a time but the grammar of most languages forces their users to know what they’re going to write several words ahead of the current word.
Ok. Why do you think GPT isn't doing that? or is different?
It might be calculating the next word, because you can only write one word at a time, but you can't say that the current next word isn't influenced by a few words ahead. Don't think the current understanding of what LLM's do internally can rule that out.
It might be calculating the next word, because you can only write one word at a time, but you can't say that the current next word isn't influenced by a few words ahead. Don't think the current understanding of what LLM's do internally can rule that out.