[IgorPartola]

I mean basically this is the well known problem with LLMs: they know how to mince words but don’t understand meaning. Again, I think you didn’t present a good simple example. As presented, the Harry Potter problem is just using the wrong tool for the job and isn’t the same as the insurance policy problem.

But at the end of the day an LLM is right 80% of the time while being 100% confident 100% of the time that it gets the right answer. You can increase that 80% but I don’t see how the current breed of LLMs can learn to self doubt enough to keep trying to understand better.