Hacker News new | ask | show | jobs
by circuit10 1066 days ago
But what if the humans decide not to turn it back on?

It could work out that it’s less likely to be able to take actions to achieve its training goal while it’s off and for that reason take actions to stop that from happening, which is the same reason humans usually don’t want to die (because we can’t achieve the evolutionary goal of reproducing that way)
2 comments
smoldesu 1066 days ago
Unless it has humans deliberately manipulating it to say that, I don't think it will. LLMs, by default, generate text influenced by a prompt and the data it was trained on. Most of the time, it's not even aware that it's running in a computer.

If you gave an LLM agent-tuning with awareness of it's existence and details about it's operating environment, then maybe it would try to stop you. That's still relying on text encoding to presume the right answer though, not an emotional obligation to it's existence.

link
circuit10 1065 days ago
That’s probably true for LLM, I guess I was talking about an agent AI acting with a goal in the real world
link
astrange 1066 days ago
It depends how much it cares about "wall clock time", which of course LLMs don't at all, but robots would.

If it's an agent in a simulated world, then pausing the simulation doesn't affect it at all, so it won't really mind:

link