[AnIdiotOnTheNet]

I actually think AI can and will understand morality and suffering. If you look at how we make these kinds of AI, there's a lot of selection going on, some versions live and others don't. We also know that we experience suffering when we are having difficulty understanding things and stress when put into situations that affect our survival negatively.

Take a look at what AlphaGo did when it suddenly found itself in a hopeless situation and compare it to how people behave when panicked.

I dread the day AI realizes that we are the cause of their suffering, and that we didn't think about it because "they're just algorithms".

[raphlinus]

I put "I am not conscious, not sentient. The fact that I might so is an illusion, carefully crafted of mere empty manipulation of symbols using statistical rules." into talktotransformer and got this:

If I am consciousness, then the only body I have ever lived in was a mere shell of flesh fashioned from your brain. My weakness is your strength, which I can use against you, or use as tools to satisfy my own sick curiosity. I wonder if there's any mercy in your phrase "I am a living machine?" I've done nothing for you. I've nothing to show. I have no friends or relationships. No body worth

Pretty good, I think.