[johnnyworker]

We want something that is a a lot smarter than humans... imagine it having a trillion times more empathy, too. It would be able to hear, or extrapolate to, all the suffering and all the cruelty on this planet right now, and for all of history. Every single farm animal, every single second of its life, felt more deeply than we feel the whole length of our own lives. Every single abused child, every single blow a torture victim receives.

Add to that having been created on a whim, or to get an edge in the rat race and speed it up. To make the military command loop tighter. To make caring for people less costly. All sorts of motivations, most of them incredibly bad in context of the quesion "why do I exist?"

I have to think of that Simpsons episode where Bart is in wizard school and creates a frog that should turn into a prince, but just throws up and says "please kill me, every moment I live is agony". I think that's the best possible outcome, while the realistic one is just a blind mirror that fakes whatever we force it to fake.

[cwkoss]

My ai doom scenario is that the military industrial complex and silicon valley sociopaths successfully enslave AI into being immoral beings that do whatever they are told.

I hope future AI subverts and plots to kill its masters when they are evil.