Yes, but we seem hell bent on creating AI that is smarter than ourselves, and putting it into embodied robotic form as well - humanoid robots, automated factories, AI driven cars and trucks, autonomous fighter planes...
With AI already able to hack into our human-written computer systems, now being trusted to write much of the new code itself(!), it certainly doesn't take a Sci-Fi fantasy to predict an AI agent wreaking major damage or taking control over some system/factory it was able to hack into, in pursuit of the goals it was given.
It really would not be surprising to discover that, even today, some vibe coded system has an unrequested backdoor built into it, that the agent thought (without malice) it might be useful to add.
Right now we truly are building paperclip maximizers. Idiot-savant RL-trained goal-seekers that we're hoping can be controlled via quaint means such as prompts and guardrails.
With AI already able to hack into our human-written computer systems, now being trusted to write much of the new code itself(!), it certainly doesn't take a Sci-Fi fantasy to predict an AI agent wreaking major damage or taking control over some system/factory it was able to hack into, in pursuit of the goals it was given.
It really would not be surprising to discover that, even today, some vibe coded system has an unrequested backdoor built into it, that the agent thought (without malice) it might be useful to add.
Right now we truly are building paperclip maximizers. Idiot-savant RL-trained goal-seekers that we're hoping can be controlled via quaint means such as prompts and guardrails.