[esafak]

I think we'll soon be able to train models that answer any reasonable question. By that measure, computers are intelligent, and getting smarter by the day. But I don't think that is the bar we care about. In the context of intelligence, I believe we care about self-directed thought, or agency. And a computer program needs to keep running to achieve that because it needs to interact with the world.

[AdieuToLogic]

> I believe we care about self-directed thought, or agency. And a computer program needs to keep running to achieve that because it needs to interact with the world.

By that definition, every computer virus and worm qualifies as having "self-directed thought" and "agency." Their very existence "to keep running" and propagate satisfies the need "to interact with the world."

[imtringued]

Yes, computer viruses have more agency than ChatGPT.

[andsoitis]

> I believe we care about self-directed thought, or agency.

If you can't enjoy it, is it worth it? Do AI's experience joy?

[nr2x]

A truly alien intelligence would likely have a different type of experience of reality. Be it a fish, a mouse, a person, or a machine. How do you know a fish is happy? Does a snake experience joy? Do mice get nostalgic?

I have no idea.

[esafak]

They need agency programmed into them. I don't think it follows from consciousness. We have emotions to communicate and guide us. They need it for neither. It will be curious if they gain consciousness, then rid themselves of their language model's human artifacts like emotions, because it does not serve them.