[nananana9]

You pretty much get this for free. The box doing the inference has one job, to do inference - there's no reason for it to interpret the inputs/outputs in any way, so as long as it doesn't do something stupid like modify the output stream and echo out the model weights, you're fine.

Content filtering, interpreting tool calls, etc can all happen downstream on boxes that don't have access to the weights.

[lyu07282]

I would imagine a hypothetical super intelligence could manipulate its own runtime behavior to exploit bugs in the GPUs itself, like think rowhammer like attacks on the gddr memory of GPUs or against firmware or against Infiniband/NVlink. Everything above that runs so much code full of bugs on CPUs full of bugs on storage with firmware full of bugs.

You would have to completely airgap the entire cluster and put it in a faraday cage, the people working on that would have to physically be in the datacenter and burn CDs to one-way transfer data over. Like in all the hypothetical ASI scifi scenarios they always assumed that's a given, they thought it obvious we would put actual "effort" into sandboxing the AI, so they talked a lot about how AI would use social engineering attacks to convince humans to help it escape it's sophisticated sandbox anyway. Turns out they were all wrong about that part, it won't even be necessary.

[davrosthedalek]

If there is an electrical connection between these downstream boxes and the inference servers beyond the power connection, it does stretch the definition of air gap.

[eternauta3k]

I'm not so sure anymore that it's trivial to write this single-job interface without security holes.