[skeledrew]

Useful for regurgitative pattern generation I can get. The core definition though is it doesn't understand what it's generating. Meanwhile I'm seeing LLMs constantly perform tasks which I'd say requires understanding, like reading the manual for a tool it's never encountered before and then going on to effectively use that tool. That's 2 different kinds of useful.

[dekhn]

I'm of the general belief that something can be unaware and not have any understanding (in the subjective experience of consciousness), while also appearing to do so, and be useful.

That's what Bender doesn't get. Some folks tried techniques different from her favorite techniques and made unbelievably fast progress across a wide range of previously insoluble problems (regardless of whether they satisfy properties Bender believes are required for intelligent systems).

I think it's safe to say that none of the main LLMs have some sort of self-awareness as we think of it in humans, but I also expect that more sophisticated systems in the future could. If I had to guess, they would have significantly more activity going on in the network- not just individual end-to-end top-down forward graph, along with cycles instead of trees, and the neurons themselves would be sigifnificantly more capable (effectively little state machines that run functions on input that passes through). I guess also you'd want to have some sort of rules-based (but statistically trained) execution component managing everything.