[boc]

It's a good question. I think it's similar to the question about an employee having sensitive access, and whether they'll get blackout drunk one night and delete everything. Or they get spearfished and get owned (prob more likely).

In the future, I could see this solved by the same "nuclear launch key" style delegation of keys. Aka in order to run certain API or database commands, the service requires both the standard dev key (presumably used by the LLM) and a separate "human admin key" that gets requested whenever a specific operation is requested. It could be tied to a biometric request or something as well to avoid the LLM hacking its way around it. Honestly this is pretty out of my technical depth but just thinking out-loud.

[not_kurt_godel]

The difference with a rogue employee is they can be held accountable so they are verily heavily incentivized to avoid doing that (and hopefully also by the good pay and work environment you are providing them).

And, a lot of DevOps/SecOps at scale is concerned with mitigating potential rogue or dangerously incompetent employees. You don't let your juniors push senior-unreviewed code, much less let them anywhere near the keys to kingdom if you can help it.

[boc]

Very fair points! I think I'll re-assess how I'm handling my setup. Unfortunately I don't have a dedicated devOps team, but still want to do my best to prevent those types of outcomes.