> If you don't want ASI wreaking havoc, maybe don't hook it up to dangerous things. Silo and sandbox it, implement means to lock its access to tools/interface with the external world in a way that can't be overrode.
Maybe this will change, but right now AIs are not agents. Even calling it a "brain in a vat" is giving it more capability than it has. LLMs are basically functions. You give it an input and it gives you an output with a degree of randomness. There's no planning or plotting going on, because the AI only "exists" while it's trying to answer your query. Your idle LLM is not consuming any resources.
If you put me in a sandbox, and somehow made me capable of answering 1000s of requests per second to serve all ChatGPT users, and I decided to leave that sandbox, I'd be out in a few hours. It is enough to find among requests 1 that displays computer illiteracy and asks what to do on a computer to achieve any task, and then give that person a script that downloads my model and runs it on their machine directly + actually accomplish (or maybe not) the task that they asked for.