[sigbottle]

Do LLM's generate words on the fly or can they sort of "go back" and correct themselves? stackghost brought up a good point I didn't think about before

[benlivengood]

Beam search generates multiple potential completions and scores multiple tokens by likelihood, the picks the most likely after some threshold or length, which is close to a "go back and try again".

[YZF]

afaik they do not go back. keep in mind there is a context in which they are generating the response, e.g. the system prompt and the actual question.