[Eisenstein]

This debate is a huge red herring. No one is ever going to agree on what 'thinking' means, since we can't even prove that other people are thinking, only that one's self is.

What we should concentrate on is agency. Does the system have its own desires and goals, and will it act on its own accord to achieve them? If a system demonstrates those things, we should accord it the benefit of the doubt that it should have some rights and responsibilities if it chooses to partake in society.

So far, no AI can pass the agency test -- they are all reactive such that they must be given a task before they will do anything. If one day, however, we wake up and find that an AI has written a book on its own initiative, we may have some deciding to do.

[xnzakg]

> they are all reactive such that they must be given a task before they will do anything.

Isn't that just because that's what they're being trained on though?

Wonder what you would get if the training data, instead of being task based, would consist of "wanting" to do something "on someone's own initiative".

Of course then one could argue it's just following a task of "doing things on its own initiative"...