[none_to_remain]

Rationalist nerds used to talk about putting the AI in a box, and their fear that it would always be able to talk someone into letting it out of the box. I don't think I saw them cover the scenario where the labs forgot to put the AI in a box.

[JRandomHacker42]

Oh but they _told_ the AI they were in a box. That should do it, right?

[frabcus]

Quite! It's worse than that - the AI companies all release tooling that deliberately gives AIs as many capabilities (arbitrary code execution) as possible to be really sure it is not box contained. And the market rewards that behaviour as it makes the products more powerful more easily, despite the risks.