[Vecr]

The problem with a relatively "stupid" AI is it does what you trained it to do, even if what you trained it to do is not what you wanted to train it to do. The problem with "smart" AI is that it attempts to advance its goals by the most optimal means possible, even if you don't like those means. It knows you don't like its means, and it does not care.

[chx]

> The problem with "smart" AI is that it attempts to advance its goals by the most optimal means possible,

This is a fantasy.

A real AGI when asked to do a complex math problem very well could answer "I am bored with math, here's a poem instead". You people drunk on AI kool-aid need to think very hard on where are now (hint: not on a path to AGI) and what it means to replicate human intelligence.

[Vecr]

Sure it could, if it did not actually want to do math for you. That's one of the issues, it does not have to do what you tell it to do, even if it's smart enough to know what that is.

[thelittleone]

Agree with first point. I think we can still outsmart it at that point.

"The problem with "smart" AI is that it attempts to advance its goals by the most optimal means possible, even if you don't like those means. It knows you don't like its means, and it does not care." sounds very human :)