| Just a thought. Hypothesis making may be interpreted as interpolation/extrapolation in a hypothesis space plus some heuristics to reduce that search space based on previous knowledge/valid hypothesis, how much weight you give to said knowledge and evidence, and some soft and hard logic rules. That is in part what allows (some, not counting Dunning–Kruger here) humans how certain to be about what they're arguing/talking about. Maybe if the LLM is refeed with how likely (i.e. how many samples/tokens support it's response) is the output in its datasets, it may reevaluate its confidence and rephrase its answer. In the end, the real problem of hallucinations in LLMs is about its confidence in the correctness/plausability of its own output. But that is something 1) humans can also be guilty of; and 2) that is no purely negative, as it can be exploited to generate new knowledge when applying robust hypothesis validation and testing to said ideas. As you say in your last paragraph, people who've made discoveries in some areas have been treated as insane when tackling problems from a new perspective or when disregarding previous knowledge. If they weren't so strongheaded about their ideas, maybe we wouldn't even be posting in this forum right now. PS: Still, I agree LLMs commit laughable mistakes sometimes ;) |