[avgcorrection]

> You have the misconception that an AI system has to achieve anthropomorphic qualities to become dangerous.

More than that. It has to achieve super-human abilities.[1] And it might also need to become self-improving, because how else could it spiral out of control, as the hysterics often complain about? (Remember: human intelligence is not self-improving.) It also needs to gain an embodiment that reaches further than server rooms. How does it do that? I guess it social-engineers the humans and then… Some extra steps? And then it gets more and more spooky.

This is more science fiction than technology.

[1] Or else it is no more dangerous than a human, and thus irrational to worry about (relative to what humans do to each other on a daily basis).

> ML algorithms are goal-oriented, they are optimized to maximize a specific goal.

The Paper Clip Machine. I’m more worried about the Profit Making Machine, Capitalism, since it is doing real harm today and not potentially next month/next year/next decade. (Oh yeah, those Paper Clip Machine philosophers kind of missed the last two hundred years of the real-life Paper Clip Machine… but what would the handmaidens of such a monstrosity have against that, to be fair.)