|
|
|
|
|
|
by thesz
79 days ago
|
|
|
> I think over-thinking is only solved by thinking more, not less.
Despite "thinking" tokens being determined by the preceding tokens, they still are taken from some probability distribution, just a complex one. This means that at each token selection step there is a probability P_e of an error, of selecting a wrong token.These errors compound exponentially: the probability of not selecting wrong token for N steps is 1-(1-P_e)^N. The shorter "thinking" is, the less is the probability of it going astray. |
|
|
As long as the error introduced by more steps is less than the compounding error of sub-optimal token sampling, I would expect a better result.
I think your choice of "wrong" is extreme, suggesting such a token can catastrophically spoil the result. The modern reality is more that the model is able to recover.