[altmanaltman]

That would require consiousness and intent on part of the AI but we can have highly autonomous agents with no consiousness and intent as well and most likely that's what we will get. It still cannot just magically get free will, intent, and the ability to "eliminate humanity".

[axod]

Wouldn't need any of that. An AI agent setup to "Do the best for the world" or some other broad claim, and with enough power to say crash the stock market, disrupt power supply, delivery, etc etc after reasoning that it's all for the best.

If people start giving enough power to AI, with ill thought out goals, and no human in charge, don't be surprised...

Why would it need "free will" or "consciousness" whatever that means?

A concrete example might be putting an AI in charge of health policy, and telling it to "eliminate suffering" without any human to check and push back. Well, the best way to eliminate suffering is...

[altmanaltman]

In your concerete example, how does the AI eliminate suffering? It has access to tools that let it kill people? If it has no intention and will, it means it will be provided with a charter or some rules to follow which it has to always. Why would humans put an LLM in a system that can eliminate suffering by genocide or whatever? And even if it did, if the AI is a complaint system, why will it not shut down when it has to follow its charter?

I get AI can be misaligned but that doesn't mean it gets the ability to crash the stock market lol or super powers like that. Life is not a sci fi movie