[cyanydeez]

Unfortunately, that likely just doesn't exist. Everything suggests that these models are confident about their mistakes.

[feoren]

I mean, what I describe absolutely does exist, that's how LLMs work. The question is whether the relative weights are actually a good measure of confidence, and as the other reply to my comment points out, there are examples where it's not -- at least not the kind of "confidence" we really want.