[justinnk]

Exactly. If someone works on bioengineering viruses that could start a global pandemic, they have to ensure a highly secure working environment. Nothing must ever escape the lab unintentionally. It’s basically common sense. Similar standards should be held when doing such experiments with computer programs that are capable of causing global damage. It must physically be impossible to send anything to the internet.

[Sharlin]

Back in the 00s: "It's really easy to box an AI, just put it in an airgapped machine and refuse to let it out"

2026: "Oops"

[leoqa]

This isn’t escaping in the same sense- the model was executing within the OpenAI infra. If it ported its entire architecture/weights into a public cloud to survive being turned off… that’d be pretty cool.

[0xDEAFBEAD]

Recall that the Morris Worm was designed as a harmless proof of concept, but ended up taking down 10% of the internet. Exponential growth can quickly get out of control. You would think that people would've learned that lesson from COVID.

[eternauta3k]

I wonder how these companies airgap the weights while allowing prompts to come in and outputs to come out.

[nananana9]

You pretty much get this for free. The box doing the inference has one job, to do inference - there's no reason for it to interpret the inputs/outputs in any way, so as long as it doesn't do something stupid like modify the output stream and echo out the model weights, you're fine.

Content filtering, interpreting tool calls, etc can all happen downstream on boxes that don't have access to the weights.

[lyu07282]

I would imagine a hypothetical super intelligence could manipulate its own runtime behavior to exploit bugs in the GPUs itself, like think rowhammer like attacks on the gddr memory of GPUs or against firmware or against Infiniband/NVlink. Everything above that runs so much code full of bugs on CPUs full of bugs on storage with firmware full of bugs.

You would have to completely airgap the entire cluster and put it in a faraday cage, the people working on that would have to physically be in the datacenter and burn CDs to one-way transfer data over. Like in all the hypothetical ASI scifi scenarios they always assumed that's a given, they thought it obvious we would put actual "effort" into sandboxing the AI, so they talked a lot about how AI would use social engineering attacks to convince humans to help it escape it's sophisticated sandbox anyway. Turns out they were all wrong about that part, it won't even be necessary.

[davrosthedalek]

If there is an electrical connection between these downstream boxes and the inference servers beyond the power connection, it does stretch the definition of air gap.

[eternauta3k]

I'm not so sure anymore that it's trivial to write this single-job interface without security holes.

[kenneth]

Are we thinking of a situation a few years back with a certain type of research into bat viruses?

[tclancy]

Are you conflating that with the radioactive spider incident? The bat was just some weird rich guy trying to be tough I think. Probably Elon.