[0xfae]

Interesting. If the computer can simulate things that a human wouldn't have thought up, great, but if it's considering things that any sane person knows wouldn't work and passing them off as correct, framing it that way is laughable.

That's not brilliant, it's just wrong. If your AI behaves in a way that makes me not trust what it tells me, that's a bug not a feature.

But then again, many discoveries have been made because someone trying something that shouldn't work.

[nomel]

> knows wouldn't work and passing them off as correct, framing it that way is laughable.

I don't think this completely true. Considering something that wouldn't work often still leads to ideas, because it kicks your brain of a rut it may be in. This is why it can be incredibly useful to brainstorm with someone that doesn't have expertise in a topic. They'll say zany things that can be inspiring!

Random word sequences are a pretty common way to get inspiration. Something more "on topic" can't be that bad.

[mrbungie]

Just a thought.

Hypothesis making may be interpreted as interpolation/extrapolation in a hypothesis space plus some heuristics to reduce that search space based on previous knowledge/valid hypothesis, how much weight you give to said knowledge and evidence, and some soft and hard logic rules. That is in part what allows (some, not counting Dunning–Kruger here) humans how certain to be about what they're arguing/talking about.

Maybe if the LLM is refeed with how likely (i.e. how many samples/tokens support it's response) is the output in its datasets, it may reevaluate its confidence and rephrase its answer.

In the end, the real problem of hallucinations in LLMs is about its confidence in the correctness/plausability of its own output. But that is something 1) humans can also be guilty of; and 2) that is no purely negative, as it can be exploited to generate new knowledge when applying robust hypothesis validation and testing to said ideas.

As you say in your last paragraph, people who've made discoveries in some areas have been treated as insane when tackling problems from a new perspective or when disregarding previous knowledge. If they weren't so strongheaded about their ideas, maybe we wouldn't even be posting in this forum right now.

PS: Still, I agree LLMs commit laughable mistakes sometimes ;)