[badsandwitch]

Has anyone tried to see what the output looks like if the model is never allowed to be uncertain?

For example, whenever certainty drops below a threshold the sampler backtracks and chooses different tokens. Such that at the end every single token had an above threshold certainty.

I doubt it would entirely eliminate undesirable outputs, but it would be interesting.

[eddd-ddde]

Couldn't that just, never get an answer?

Or maybe just says "i don't know" with full certainty.

[zbentley]

That would be extremely useful in some domains.

[mumblemumble]

Perhaps only if you can also be very certain that the output is correct whenever the logprobs don't trigger the filter.

If that's not the case then it might just trigger bad risk compensation behavior in the model's human operators.

[Jerrrrrrry]

You used to get purely determinant near-quotes, but still affected by floating point inaccuracies.