[eega]

Problem here is that morals are not absolute. The moral framework we use in the west isn‘t the same that is used, for instance, in China. It isn‘t even exactly the same between states in the US or countries within the EU. Or political groups within any of that.

So, which one are you supposed to be using? Or is the only approach left to use the lowest common denominator? This probably would burn down to „you shall not kill other humans“ and not even that is certain.

[arpa]

this is where one needs to draw a line between morals and ethics. Because while morals are relative, ethics is something inherent to all human beings (ethos = what makes a thing a thing, ethics = what makes humans human). So we don't need a _moral_ AI, but rather an _ethical_ AI. It can, as you point out, be very simple and maybe Asimovs' laws of robotics could be a good starting point.

[Karunamon]

I've read enough Asimov and stories that play with those laws to know that the "through inaction…" bit of the first law is a severe problem. You either need to concretely and objectively define "harm" to the AI or remove the imperative to act altogether.