[Too]

If it only generates one word at a time and then repeat the process again, how does it know when to stop?

It feels like this method would create endless ramblings. But we all know you can ask Chatgpt to “summarize in one sentence” and it pulls it off. When speaking yourself you sort of have to think how to finish a sentence before you start it, to explain something cohesively, surely there must be something similar in the AI?

[joshhart]

One of the “words” is a stop token, which represents the ending of text. So you can say the thing that maximizes coherence is to stop right then.