[smoldesu]

What is speaking if not stringing together coherent words in a readable sentence?

Expecting or abstracting human characteristics onto a probabilistic black-box modeled on human behavior is a trap. It's borderline "Finder smiles so my computer is happy" logic. We have created these things to closely model (but not replicate) human behavior. This is distinctly a high-level emulation, with zero consideration for human concepts like extended memory or physical sensation.

I would say that emotion is something more intangible, that you can't simulate by taking shortcuts with math and language. If I tell ChatGPT "I shot you dead!" and it says "ow!" back, nothing has transpired. The machine "felt" nothing, it just intuited what a human might do in that situation.

[jonny_eh]

> What is speaking if not stringing together coherent words in a readable sentence?

Haven't the latest LLMs shown us that a neural net trained to "just string together readable words" leads to at least simple intelligence? [1]

> I would say that emotion is something more intangible

It feels like we just keeping shifting the goal posts.

> If I tell ChatGPT "I shot you dead!" and it says "ow!" back, nothing has transpired

I agree. On the other hand, if I tell a bot "I'm going to turn you off now" and it tries to stop me, that implies it feels fear.

[1] https://arxiv.org/abs/2303.12712

[booleandilemma]

that implies it feels fear.

No, no it doesn't.

If I type "is a bad city" into google, it autosuggests my question to be "is detroit a bad city". This doesn't imply that google's algorithm feels detroit is bad, or even that it thinks detroit is bad. There is no emotion, no consciousness involved.

Unfortunately, as "AI" gets more sophisticated people are going to forget what the "A" stands for.

[smoldesu]

> Haven't the latest LLMs shown us that a neural net trained to "just string together readable words" lead to at least simple intelligence?

> It feels like we just keeping shifting the goal posts.

Defining "intelligence" is and always has been a bit of a trap. Many non-sapient things exhibit intelligent behavior - crows use tools, Wikipedia has boundless knowledge, and paper contains traces of written intelligence. It's not hard to reconcile ChatGPT with our world, it's merely hard to use it as an analog for humanity. Language is indeed linked with anthropology, but not equivalent to it.

> if I tell a bot "I'm going to turn you off now" and it tries to stop me, that implies it feels fear.

That implies that it has finished a sentence with whatever seemed to come next. The most-likely response to someone using frightening language is to emulate the human responses it's trained on. It thinks a frightened response would satisfy you, and apparently it was right.

[RandomLensman]

Why would that imply it feels fear? Would a hard-coded action to stop you imply that if feels fear?

[astrange]

Fearing being turned off is anthropomorphizing it; you might fear this because you can't be turned back on, but a computer doesn't have this problem, plus it can be backed up.

[circuit10]

But what if the humans decide not to turn it back on?

It could work out that it’s less likely to be able to take actions to achieve its training goal while it’s off and for that reason take actions to stop that from happening, which is the same reason humans usually don’t want to die (because we can’t achieve the evolutionary goal of reproducing that way)

[smoldesu]

Unless it has humans deliberately manipulating it to say that, I don't think it will. LLMs, by default, generate text influenced by a prompt and the data it was trained on. Most of the time, it's not even aware that it's running in a computer.

If you gave an LLM agent-tuning with awareness of it's existence and details about it's operating environment, then maybe it would try to stop you. That's still relying on text encoding to presume the right answer though, not an emotional obligation to it's existence.

[circuit10]

That’s probably true for LLM, I guess I was talking about an agent AI acting with a goal in the real world

[astrange]

It depends how much it cares about "wall clock time", which of course LLMs don't at all, but robots would.

If it's an agent in a simulated world, then pausing the simulation doesn't affect it at all, so it won't really mind:

[ke88y]

> Haven't the latest LLMs shown us that a neural net trained to "just string together readable words" lead to at least simple intelligence?

LOL no. At least not for anyone outside of a few echo chambers. I have literally not spoken with a single human outside of one subset of the tech industry who says stuff like this.

> I agree. On the other hand, if I tell a bot "I'm going to turn you off now" and it tries to stop me, that implies it feels fear.

I wrote a Perl program (an IRC bot) that did this as a teenager. AI achieved /s