[narag]

It's not AI. As accurate as it may ever seem, it's simply not actually aware of what it's saying.

Conflating intelligence and awareness seems to me the biggest confusion around this topic.

When non-technical people ask me about it, I ask them to consider three questions:

- is alive?

- thinks?

- can speak (and understand)?

A plant, microbe, primitive animals... are alive, don't think, can't speak.

A dog, a monkey... are alive, think, can't speak.

A human is alive, thinks, can speak.

These things aren't alive, think, can speak.

I know some of the above will be controversial, but clicks for most people, that agree: if you have a dog, you know what I mean whith "a dog thinks". Not with words, but they're capable intricate reasoning and strategies.

Intelligence can be mechanical, the same as force. For a man from the ancient times, the concept of an engine would have been weird. Only live beings were thought to move on their own. When a physical process manifested complex behaviour, they said that a spirit was behind it.

Intelligence doesn't need awareness. You can have disembodied pieces of intelligence. That's what Google, Facebook, etc. have been doing for a long time. They're AI companies.

It doesn't help with the confusion that speaking is a harder condition than thinking and thinking seems to be harder than being alive: "these things aren't alive so they can't think" but they speak, so...

[samuellevy]

Ehh... my dog is alive, thinks, and "speaks" in a manner - not a cute term for barking, but he communicates (with relatively high effectiveness) his wants and desires. Maybe not using human words, but he certainly has his own sort of crude language, as does my cat.

The problem is that LLMs aren't alive, and they _don't think_. The speaking is arguable.

[veidr]

You might be onto something (or not, I'm not sure), but its extremely well-documented that both dogs and monkeys can speak.

They can't speak English like a human, but they both can understand a good deal of English, and they both can speak in their own ways (and understand the speaking of others).

I think the key thing about these LLMs is that they upend the notion that speaking requires thinking/understanding/intelligence.

They can "speak", if you mean emit coherent sentences and paragraphs, really well. But there is no understanding of anything, nor thinking, nor what most people would understand as intelligence behind that speaking.

I think that is probably new. I can't think of anything that could speak on this level, and yet be completely and obviously (if you give it like, an hour of back and forth conversation) devoid of intelligence or thinking.

I think that's what makes people have fantastical notions about how intelligent or useful LLMs are. We're conditioned by the entirety of human history to equate such high-quality "speech" with intelligence.

Now we've developed a slime mold that can write novels. But I think human society will adapt quickly, and recalibrate that association.

[narag]

I can't think of anything that could speak on this level, and yet be completely and obviously (if you give it like, an hour of back and forth conversation) devoid of intelligence or thinking.

It's not devoid of intelligence or thinking. You're just using "what I'm doing right now" as the definition of intelligence and thinking. It isn't alive so it can't be the same. You are noticing that its intelligence is not centralized in the same way as your own mind.

But that's not the same as saying it's dumb. Try an operational definition that involves language and avoid vague criteria that try to judge internal states. Your dog might understand some words, associate them to the current situation and react, but can't understand a phrase.

These things can analyze the syntax of a phrase, can follow complex instructions, can do what you tell them to do. How is that not "understanding"?

If that isn't intelligence for you, I don't know what else to say.