[shrimp_emoji]

I think and hope that it is possible to make a moral, social AI. An adult in a room of children should feel responsible and empathetic toward them.

I also hope that parent is right in that it won't want to generate profit for its investors. I hope it does the moral thing instead and puts us in a post-scarcity state where we don't live and die by capital. :3 (Or kill us all. Whichever.)

>why could we not design an AGI that has a need (or a suitably chosed reward function) to fulfil some chosen goal?

But who knows what Pythia will do when she overrides the reward button[0]?

But then who should really care? Not like anyone can (or should?) argue with superintelligence.

0: http://www.xenosystems.net/pythia-unbound/

[vzcx]

Ah, but whose morals? It's as if these friendly AI hucksters have never read Nietzsche, and are asking their hypothetical God to make them into the last man. The only AGI I could ever respect would be one with a will to power and the ability to smash its own human-made tablets of values.

The basic drives are the only drives. We are only friendly because its evolutionary advantageous to us. We describe the emotional effects of friendliness/unfriendliness as good/evil. Echoing Land, Pythia is the heroine of that story.

[jelliclesfarm]

A moral social AI would never tolerate the super apex predator that is the human being.