[masklinn]

> People clearly don't understand how LLMs work -

Of course not. Most developers don't understand how LLM work, even roughly.

> Humans do the same thing all the time. Of course it's not always correct!

The difference is that LLMs can not acknowledge incompetence, are always confidently incorrect, and will never reach a stopping point, at best they'll start going circular.

[nerpderp82]

Everything out of an LLM is a confabulation, but you can constrain the output space of that confab by restraining it with proper prompting. You could ask it to put confidence intervals for each one of its sentences (ask it in the prompt), but those will be confabulated as well, but will give it some self doubt, as for now, it hasn't been programmed with any. Probably costs a lot more in power to run it with doubt. :)

Edit, I played around with this. It looks like GPT4 has a guard against asking for this. It flat out refused with the two prompts I gave it to include confidence intervals. Maybe that is a good thing.

[fomine3]

> The difference is that LLMs can not acknowledge incompetence, are always confidently incorrect, and will never reach a stopping point, at best they'll start going circular.

Like a smartass?

[lazide]

At least they don’t control any militaries yet, since the natural outgrowth of that is making reality meet their answers.

Or at least it would be if they were human!