[TheOtherHobbes]

I think the most likely route to destructive AI is a corporation with an AI CEO - possibly one that takes over from a human CEO in a boardroom coup.

Corporations already have legal personhood and act in their own interests. It's going to be much easier to automate and formalise business decision making than to develop a true general intelligence with a full spectrum of human characteristics.

This may sound like science fiction, but as competence increases shareholders - who typically are only passingly interested in moral issues - are likely to demand the increased returns an AI CEO can bring.

[dragonwriter]

> Corporations already have legal personhood and act in their own interests.

But AIs don't, and corporate officers must be persons (natural persons, even -- a corporation can't be a corporate officer.)

[cLeEOGPw]

A person could be made a mediator of AI to overcome it.

[zardo]

Many CEOs are already just trying to operate as a share price optimization algorithm. They don't choose to inject human values into their organizations.

[yourapostasy]

> ...operate as a share price optimization algorithm...

That in itself is not so bad, as extremely long-timeframe constraints (say, >50 years) upon such an algorithm could conceivably be consonant with current decision-making behavior that externalizes many input costs (employee overtime, environmental damage, etc.). Running the algorithm to pay out in very short timeframes (a month to a year) due to most CEOs' anticipated short tenure is what seems to cause undesirable optimizations.

[oldmanjay]

That you don't share their values does not make them non-human

[js8]

That they are human doesn't mean they cannot be a threat to humankind. ;-)