[marvin]

It’s fascinating to see that we’re _already_ at a point where the AI alignment problem («what morality do we want our automated systems to have») is becoming a practical reality, long before AGI or self-improving AI appears to be imminent.

I think this is a clear, contempirary demonstration that philosophizing around the use of AI (even today) has practical benefits beyond the practice of philosophy as an inherently worthwhile species-wide intellectual pursuit.

[jacobolus]

> AI alignment problem

This is a problem as old as civilization.

Bureaucratic procedure (of governments, religious organizations, corporations, armies, NGOs, ...) always leads to bad outcomes in edge cases if individual administrators are not given enough autonomy to make exceptions and enough oversight/accountability to prevent abuse.

[marvin]

I think it’s also fascinating how this is an age-old philosophical problem, as in the stories of the malicious genie. People have known for a long time that the rigid, precise specification of morality or desirability isn’t easy.