[rrrrrrrrrrrryan]

> you are confident as you proclaim ["a", "p", "p", "l", "e", "."] as the obvious answer.

Is it possible for the current generation of LLMs to assign confidence intervals to their responses?

That's my main qualm with ChatGPT so far: sometimes it will give you an answer, but it will be confidently incorrect.

[terramex]

Yes, but it has some issues in latest models.

> GPT-4 can also be confidently wrong in its predictions, not taking care to double-check work when it’s likely to make a mistake. Interestingly, the pre-trained model is highly calibrated (its predicted confidence in an answer generally matches the probability of being correct). However, after the post-training process, the calibration is reduced (Figure 8).

pages 10-11: https://cdn.openai.com/papers/gpt-4.pdf

[askiiart]

I don't know exactly how it works, but using GPT-3 via https://platform.openai.com/playground/, you can have it assign a likeliness score to each word, given all the previous text. That could act as a good confidence score.

Take this with a grain of salt though, I'm far from an expert, and it's been a while since I've played around with that feature.

[harpiaharpyja]

Not an expert myself, but I imagine that generating output that expresses confidence would be distinct from any measure of confidence within the inner workings of GPT itself.

[photochemsyn]

If it's learning from human behavior, this is nothing new. Our society of late has been rewarding confidence over questioning and that's likely reflected in the ChatGPT training corpus.