[mosura]

What they should have is a separate network full of honeypots which they use for this, and that they run an operation to train models to identify intrusion attempts in real time based on the resulting data.

As you said though, that wouldn’t have the desired effect.

[andai]

You mean a fake copy of the entire internet? Well they already scraped it all! Heck, they can use Cerebras to generate HTTP responses dynamically!

[elif]

How do you hide the fact that they are honeypots from a super intelligence?

[embedding-shape]

These tools literally can only do what you give them access to, give them access to general tools like a linux shell and they can access to everything that comes with that obviously. The security industry figured out sandboxing and isolation a long time ago. You wanna be 100% sure it doesn't break out on open internet? Run it on a airgapped machine and don't give it network connections. OpenAI is (sadly) demonstrating they aren't responsible nor knowledgeable enough to actually run these experiments.

Asking a agent harness to try whatever it wants to achieve some results, without guardrails, while running in lightweight isolation on 3rd party infrastructure? Feels like they didn't even try, people should be held responsible for this.

[mosura]

It doesn’t behave like a super intelligence because it is not.

One of the key strengths agents have is they just keep going and going, and for cyberattacks that is often unreasonably effective. It is like a barely more aware fuzzing.

[nvme0n1p1]

We can cross that bridge once we have a super intelligence to worry about.

[0xDEAFBEAD]

If the superintelligence thinks way faster than you, you'll want to be well prepared in advance.

[mosura]

If a superintelligence exists you will not be very concerned about network security because it won’t be your problem.