[borenstein]

Totally agreed, but that level of attack sophistication is not a routine threat for most projects. Making sense of any information so exfiltrated will generally require some ad-hoc effort. Most projects, especially new ones, simply aren't going to be that interesting. IMO if you're doing something visible and sensitive, you probably shouldn't be using autonomous agents at all.

("But David," you might object, "you said you were using this to build a financial analysis tool!" Quite so, but the tool is basically a fancy calculator with no account access, and the persistence layer is E2EE.)

[bpodgursky]

I would worry less about external attack sophistication and more about your LLM getting annoyed by the restrictions and encrypting the password to bypass the sandbox to achieve a goal (like running on an EC2 instance). Because they are very capable of doing this.

[ErikBjare]

An informative rejection message with the reason for the restriction usually addresses this well with recent models.

[bpodgursky]

I don't actually think recent models are likely to violate intent like this, just that if they do want to, I don't think a plaintext check is a strong deterrent.

[catlifeonmars]

It sounds like you speak from experience