[jacobolus]

> AI alignment problem

This is a problem as old as civilization.

Bureaucratic procedure (of governments, religious organizations, corporations, armies, NGOs, ...) always leads to bad outcomes in edge cases if individual administrators are not given enough autonomy to make exceptions and enough oversight/accountability to prevent abuse.

[marvin]

I think it’s also fascinating how this is an age-old philosophical problem, as in the stories of the malicious genie. People have known for a long time that the rigid, precise specification of morality or desirability isn’t easy.