[defen]

> Technologies tend to reflect the values of their inventors.

Maybe for single-use or "constrained" technologies (to be honest I don't even believe that - how does a B-52 Stratofortress reflect the values of Orville and Wilbur Wright?). But isn't the whole point of generalized AI that it's not like other technologies? Even if "regular" technology reflects the values of its inventors, what reason is there to believe that an AI will? AI is a technology that can use itself.

[visarga]

AI will only have a will of its own it is designed as such, and that means it would have a reinforcement learning system on top of lower sensory and action modules. Even if it is based on RL, it will do what it's reward signals tells it to do.

[defen]

> AI will only have a will of its own it is designed as such

Humans weren't designed to have a will, and yet we seem to have them.

> it would have a reinforcement learning system on top of lower sensory and action modules.

Isn't that what OpenAI is doing with Universe? It's simulated sensory/action modules now but I don't see why they couldn't be hooked up to real ones.

[josh2600]

> Humans weren't designed to have a will, and yet we seem to have them.

I have no idea how you could possibly infer this.

[defen]

Which part? I don't think humans were designed - we're probably the result of an evolutionary process without intentional design - but "humans were designed by God to have free will" would be a counter to my statement, yes.

If your complaint is my claim that we have a will, I'm using the common-sense version encoded into our legal and cultural system. I agree that we don't have a good concept of what intentions are, or how they causally connect to actions, but I do know that for at least some of my actions I experience something called "intent" before I undertake the actions.

My overall point was that the capacity for intent can arise through an evolutionary process without being designed in, but it does rest on the two assumptions I just listed.