[chrsw]

> The ability to say "I don't know" is not a high bar.

For you and I, it's not. But for these LLMs, maybe it's not that easy? They get their inputs, crunch their numbers, and come out with a confidence score. If they come up with an answer they're 99% confident in, by some stochastic stumbling through their weights, what are they supposed to do?

I agree it's a problem that these systems are more likely to give poor, incorrect, or even obviously contradictory answers than say "I don't know". But for me, that's part of the risk of using these systems and that's why you need to be careful how you use them.

[kuschku]

but they're not. Ofyen the confidence value is much lower. I should have an option to see how confident it is. (maybe set the opacity of each token to its confidence?)

[furyofantares]

Logits aren't confidence about facts. You can turn on a display like this in the openai playground and you will see it doesn't do what you want.

[johnnyanmac]

>If they come up with an answer they're 99% confident in, by some stochastic stumbling through their weights, what are they supposed to do?

As much as Fi, from The Legend of Zelda: Skyward Sword was mocked for this, this is the exact behavior a machine should do (not that Fi is a machine, but she operated as such).

Give a confidence score the way we do in statistics, make sure to offer sources, and be ready to push back on more objective answers. accomplish those and I'd be way more comfortable using them as a tool.

>hat's part of the risk of using these systems and that's why you need to be careful how you use them.

Adn we know in 2025 how careful the general user is of consuming bias and propaganda, right?

[otabdeveloper4]

The confidence score is about the likelihood of this token appearing in this context.

LLMs don't operate in facts or knowledge.