[riffraff]

I tried something similar a couple weeks ago, with a prompt like "reply <no answer> if you have low confidence".

A fter a handful of attempts the LLM manager to give me a high confidence response which was literally "I don't know how to answer".

Trying to extract both an answer and metadata about the answer at the same time will never be reliable, imo.

Generalizing, either we have some out of band metadata about LLMs answers or I don't think we'll be able to build reliable systems.