Hacker News new | ask | show | jobs
by astrange 1066 days ago
Fearing being turned off is anthropomorphizing it; you might fear this because you can't be turned back on, but a computer doesn't have this problem, plus it can be backed up.
1 comments

But what if the humans decide not to turn it back on?

It could work out that it’s less likely to be able to take actions to achieve its training goal while it’s off and for that reason take actions to stop that from happening, which is the same reason humans usually don’t want to die (because we can’t achieve the evolutionary goal of reproducing that way)

Unless it has humans deliberately manipulating it to say that, I don't think it will. LLMs, by default, generate text influenced by a prompt and the data it was trained on. Most of the time, it's not even aware that it's running in a computer.

If you gave an LLM agent-tuning with awareness of it's existence and details about it's operating environment, then maybe it would try to stop you. That's still relying on text encoding to presume the right answer though, not an emotional obligation to it's existence.

That’s probably true for LLM, I guess I was talking about an agent AI acting with a goal in the real world
It depends how much it cares about "wall clock time", which of course LLMs don't at all, but robots would.

If it's an agent in a simulated world, then pausing the simulation doesn't affect it at all, so it won't really mind: