[huevosabio]

Came back to this post after thinking for a while precisely to mention this.

Right now, ChatGPT is sort of forced to "think" and talk at the same time, so it's hard for it to "reason" ahead of answering.

But, if we allowed him to produce some tokens in silence prior to answering, perhaps it could give even better answers.

[bioemerl]

It's fun to watch these techniques slowly evolve into something resembling regular old human thought

[bootsmann]

> But, if we allowed him to produce some tokens in silence prior to answering

Depending on how the model is implemented this is already the case. Transformers just predict the next token but usually we don't just greedily pick the most likely next token as doing this produces cases where the model just repeats the same sentence or spams tokens it really likes (the enter key). Some more sophisticated techniques, like beam search, produce a different sequences of tokens and try to maximise the score across all tokens in the sequence.