[watutalkinbout]

You can't fool an inanimate object.

An LLM is (effectively) just a really, really elaborate "choose your own adventure" book.

It's not "working through" problems, it's just tracing a route through an pre-defined information space. It's not actually thinking, it just does a good impression of it.

[coldtea]

>You can't fool an inanimate object

You can't constrain foolability to just the animate objects.

Foolability is based on intelligent processing.

To what degree such an intelligence needs to be developed is not defined at all (we can, for example, trivially fool a doog using one of many tricks, and a dog is hardly general intelligence).

Thus far, the inteligent processing we knew concerned animate objects. But we now have developed software/hw combos that exhibit intelligent processing. Is it enough to actually be intelligent or to be fooled?

We don't know. We do know it's enough to appear intelligence, convince people that it is intelligent, and to appear to be fooled.

But even if we think of LLM AI as mere mechanistic process with no emergent intelligence, who said one can not fool a mechanistic process? We can fool even simple pre-LLM gaming AI systems (based on simplistic heuristics) just fine.

[watutalkinbout]

They simply appear to "exhibit intelligent processing". The intelligence is in what created the data. It's a surface that's being traversed in a complex way. The LLM doesn't 'understand' that surface in any way, it just traces a path on it and reguritates apparent understanding. I'll grant it's eerily convincing.

'Fooling' something, essentially to deceive or trick, is defined as causing someone to believe an untruth. LLMs don't hold beliefs (neither do mechanistic processes), and they aren't a someone.

You can widen out the definition of the word but that generally makes language weaker - interestingly, semantic drift is a big issue for LLM's.

We have different opinions, and that's fine. Have a good day.

[Eisenstein]

Please explain why the mechanism of the LLM generating output precludes it from being able to be fooled without using use tautologies or reducing to substrate for explanation.

[watutalkinbout]

'Fooling', essentially to deceive or trick, is defined as causing someone to believe an untruth - at least in British English it does.

LLMs don't hold beliefs (neither do mechanistic processes), and they aren't a someone.

You can widen out the definition words but that generally makes language weaker - interestingly, semantic drift is a big issue for LLM's.