[Rhapso]

Yeah, the constant dissonance with AI safety is that every single AI safety problem is already a problem with large corporations not having incentives aligned with the good of people in general. Profit is just another paperclip.

[ben_w]

Not only but also; they're also every problem with buggy software.

Corporations don't like to kill their own stakeholders; a misplaced minus sign, which has happened at least once*, and your AI is trying as hard as possible to do the exact opposite of one of the things you want.

* https://forum.effectivealtruism.org/posts/5mADSy8tNwtsmT3KG/...

[digbybk]

Is that dissonance or shows that the concept is generally applicable? Human inventions can be misaligned with human values. The more powerful the invention, the more damage it can do if it is misaligned. The corporation is a powerful invention. Super intelligence is the most powerful invention imaginable.