[i_dont_know_]

K, I'll play. Let's say that Reinforcement Learners (algorithms/strategies/agents in Reinforcement Learning), let's say that they have some property of 'consciousness' that's similar to humans.

A 'reinforcement learner' gets positive or negative feedback and adjusts its strategy away from negative feedback and towards positive feedback. As humans, we have several analogs to this process.

One could be physical pain... if you put your hand on a stove, a whole slew of neural circuitry comes up to try and pull your hand away. Another could be physical pleasure, you get a massage and lean in to the pressure undoing the knots because it's pleasurable.

If we look at it from this angle, then if we're metaphorically taking the learner's hand and putting it continuously on the stove, this would be problematic. If we're giving it progressively less enjoyable massages, this would be a bit different.

Even more different still is the pain you feel from, say, setting up an experiment and finding your hypothesis is wrong. It 'hurts' in some ways (citation needed, but I think I've seen studies that show at least some of the same receptors fire during emotional pain as physical pain), but putting a human in a situation where they're continuously testing hypothesis is different from a situation where their hands are being continuously burned on a hot stove.

I think, then, that the problems (like they alluded to here) are:

- how can we confirm or deny there is some kind of subjective experience that the system 'feels'?

- if we can confirm it, how can we measure it against the 'stove' scenario or any other human analogue?

- if the above can be measure and it turns out to be a negative human scenario, can we move it to one of the other scenarios?

- even if it's a 'pleasurable' or arguably 'less painful' scenario, do we have any ethical right to create such scenarios and sentiences who experience them in the first place?

[unlikelymordant]

I think this argument would have to conclude that training any RL agent at all is unethical, since updating its weights 'away' from some stimulus could be considered pain.

[i_dont_know_]

I think what I'm getting at is that there are different types of 'pain', some that we'd consider ethical and some that we wouldn't.

Constantly subjecting a person (or some abstract simulation that responds like a person) to the equivalent of continuous bodily pain would be deeply unethical, but, say, giving a person clues towards solving a puzzle would be considered less so.

Overall, I think you're right though, if we somehow discovered we've created simulated people (or sentient beings) then we probably shouldn't use them to solve arbitrary problems.

[n4r9]

Surely not, for the same reason that producing and raising a child is not inherently unethical?

[Aaron2222]

You may joke, but there are people that think having children is unethical[0], and not because of population issues.

[0]: https://www.reddit.com/r/philosophy/comments/27p93c/having_c... (Note I haven't read all of that post, just enough to know it shows at least some people think that way)

[unlikelymordant]

RL agents are exposed to millions upon millions of stimuli, almost all of which will be 'painful' at least initially (according to the websites definition of pain). I think for children negative stimuli are not all painful, pain is a certain small subset of negative stimuli that damage is being done.

[n4r9]

I think children are constantly experiencing rewards and punishments of a sort analagous to reinforcement learning. Until a certain age their brain is hardwired to want to please their parents, so they pay a lot of attention to the parents' faces, and find reward in smiles and find punishment in frowns. After that, they are driven by a certain amount of ego, and find reward in self-accomplishment and find punishment in being told what to do.