Which regulators though? There is no AI regulator that I know of, and I'm not sure they should be one. Exaggerated marketing blurb is pretty legal, right?
What they should have is a separate network full of honeypots which they use for this, and that they run an operation to train models to identify intrusion attempts in real time based on the resulting data.
As you said though, that wouldn’t have the desired effect.
These tools literally can only do what you give them access to, give them access to general tools like a linux shell and they can access to everything that comes with that obviously. The security industry figured out sandboxing and isolation a long time ago. You wanna be 100% sure it doesn't break out on open internet? Run it on a airgapped machine and don't give it network connections. OpenAI is (sadly) demonstrating they aren't responsible nor knowledgeable enough to actually run these experiments.
Asking a agent harness to try whatever it wants to achieve some results, without guardrails, while running in lightweight isolation on 3rd party infrastructure? Feels like they didn't even try, people should be held responsible for this.
It doesn’t behave like a super intelligence because it is not.
One of the key strengths agents have is they just keep going and going, and for cyberattacks that is often unreasonably effective. It is like a barely more aware fuzzing.
> This will not happen though, because these stories are marketing.
The magnitude and the complexity of the cynicism displayed by some people when it comes to AI risks is mind-blowing.
It's like if the NRA reported on school shootings and people said "oh, they probably fake these shootings to make guns sound dangerous and sell more of them".
OpenAI could report that its AI started spontaneously generating illegal porn and sending it to people and you'd still think it was a marketing stunt.
I wouldn't describe this stunt as marketing: I would describe it as a bungled attempt to manufacture evidence to get regulatory capture, which OpenAI desperately needs. (Bungled, because Hugging Face fended off the intrusion and got their story out faster than OpenAI did.)
It is because a loud part of the doomerist contingent believe only a major disaster will provide the motivation for regulation, so they are practically hoping to cultivate a major disaster. (Given their preoccupation with bioweapons that probably means a pandemic).
This is based on similar thinking to how the world would not have considered nuclear weapons a major threat if they had forever stayed unused.
The irony of the doomer position is it achieves exactly their supposed nightmare scenario of disaster leading to authoritarian world government without any of the supposed benefits, the twist being they get to be the authoritarian world government so they are ok with it.
So your assessment of the "doomerist" position is that they believe people aren't taking AI risks seriously enough, and that only a large enough catastrophe will wake people up, and your position is we should... Ignore increasingly blatant minor catastrophes to spite them?
This isn’t a minor catastrophe. It is OpenAI being utterly irresponsible by not sandboxing their testing appropriately, to the point it has to be deliberate to provoke uneducated hysteria, which it obviously, sadly, achieves.
Sandboxing processes on networks is not exactly rocket science, and it is something their existing products would readily help them setup.
I have no idea where you’re getting your information from. I’m extremely skeptical of AI and see it as an anti-human technology we would be better without. I believe we are in a massive economical AI bubble that will eventually collapse, unless a financial miracle happens and it is somehow deflated extremely carefully. So I assume you would consider me a doomer. What do biological weapons and world government have to do with that? I’m sorry but your comment reads fairly unhinged to me
I could be wrong, but OpenAI has done the “too dangerous and powerful to release to the public” story a couple times, only to release it shortly afterwards. They have no credibility with me and I don’t trust them.
> They should shut them down immediately then investigate.
Yeah, I don't understand either. If there was a "hitman for hire" service on the clearweb, the police would shut it down first, then ask questions. Now we have a huge company effectively letting AI agents without guardrails run amok on 3rd party infrastructure, and the police is doing nothing?
Agreed.
>This will not happen though, because these stories are marketing.
Regulators should investigate the stories, and shut things down if they are real, announce the ruse if they are false.