[esjeon]

Pour in more AIs to solve AI problems! I mean, people used to do this with software (more code to the problem), but the strategy hardly worked in the long term. Without solving the actual problem, everything just adds up to more complex issues.

Also, I don't think ethics is a local maxima that can be found through optimization. Basically, it's not an absolute truth of the universe, but a set of arbitrary rules invented by human. I think it's much closer to a chaotic system - which can radically change in value even by a slightest change in the underlying parameters, but is still governed by a set of simple rules. Thus, we would need more symbolically capable systems to process contexts based on the rules of ethics, and we're currently far away from this AFAIK.

[Nevermark]

Rational ethics are just game theory applied to identifying symmetric positive sum rules for a group of N individuals, where N gets large.

They are not generally arbitrary, being the product of math and self-interest.

Where there are multiple equally good alternatives, the choice between them could be arbitrary.