[abnercoimbre]

> you did not find anything worthy of reporting. You were fooled by an AI into believing that.

The author's right. Reading the report I was stunned; the person disclosing the so-called vulnerability said:

> To replicate the issue, I have searched in the Bard about this vulnerability.

Does Bard clearly warn to never rely on it for facts? I know OpenAI says "ChatGPT may give you inaccurate information" at the start of each session.

[xd1936]

Oh yeah. Google has warnings like "Bard may display inaccurate or offensive information that doesn’t represent Google’s views" all over it; Permanently in the footer, on splash pages, etc.

[abnercoimbre]

Well Google has a branding problem with Bard... because everyone knows Google for search. "Surely Bard must be a reliable engine too."

[filterfiber]

> Does Bard clearly warn to never rely on it for facts? I know OpenAI says "ChatGPT may give you inaccurate information" at the start of each session.

I know I shouldn't be, but I'm surprised the disclosure is even needed. People clearly don't understand how LLMs work -

LLM's predict text. That's it, they're glorified autocomplete (that's really good). When their prediction is wrong we call it a "hallucination" for some reason. Humans do the same thing all the time. Of course it's not always correct!

[masklinn]

> People clearly don't understand how LLMs work -

Of course not. Most developers don't understand how LLM work, even roughly.

> Humans do the same thing all the time. Of course it's not always correct!

The difference is that LLMs can not acknowledge incompetence, are always confidently incorrect, and will never reach a stopping point, at best they'll start going circular.

[nerpderp82]

Everything out of an LLM is a confabulation, but you can constrain the output space of that confab by restraining it with proper prompting. You could ask it to put confidence intervals for each one of its sentences (ask it in the prompt), but those will be confabulated as well, but will give it some self doubt, as for now, it hasn't been programmed with any. Probably costs a lot more in power to run it with doubt. :)

Edit, I played around with this. It looks like GPT4 has a guard against asking for this. It flat out refused with the two prompts I gave it to include confidence intervals. Maybe that is a good thing.

[fomine3]

> The difference is that LLMs can not acknowledge incompetence, are always confidently incorrect, and will never reach a stopping point, at best they'll start going circular.

Like a smartass?

[lazide]

At least they don’t control any militaries yet, since the natural outgrowth of that is making reality meet their answers.

Or at least it would be if they were human!