[daveguy]

I think a lot of this confidence that the sentence will "work itself out" has to do with being able to frame a general direction of the thought before you start, but not have the precise sentence. It takes advantage of the continual parallel processing humans perform in their brain. Confidence in a simple structure of what you expect to convey. When LLMs are able to generate this kind of dynamic structure from a separate logical/heuristic process + fill in the blanks efficiently, then I think we are getting close to AGI. That's a very Chomsky informed view of sentence structure and human communication, but I believe it is correct. Currently the future tokens are dependent on the probabilistic next token rather than having the outline of a structure determined from the start (sentence or idea structure). When LLMs are able to incorporate the structured outline I think we will be much closer to an AGI, but that is an algorithmic problem we have not been able to figure out and one that may not be feasible until we have parallel processing equivalent to a human brain.