[zerobees]

"Prompt injection is not currently a major risk, but its impact could grow as attackers develop more sophisticated methods." - that's such a weird statement to make. It's one of the most significant factors limiting the adoption of the technology in business.

I have mixed feelings about this feature. We're playing with tech that's supposed to do human-shaped things but can't be trusted nearly as much as a human employee (and can't be held responsible for what it does). Restricting the tools available to that patently untrustworthy entity doesn't solve the problem, it just makes the entity less useful, forcing you to sooner or later let it out of the jail.

[cosmicriver]

I'm also surprised that they considered it reasonable to turn so many features off. Seems like some of it could be configurable, like allowed external connections. I also think some secrets should be handled by a proxy, which would give more capability than just locking down.

[ACCount37]

Responsibility is worthless for humans and even more worthless for AIs. In a way, AIs just make it more obvious.

And "trusted nearly as much as a human employee", well... you do know that phishing and insiders are two primary ways for attackers to get into company infrastructure, right?

AIs pair human-shaped capabilities with human-shaped vulnerabilities. It's a way of automating PEBKAC.

[noir_lord]

> forcing you to sooner or later let it out of the jail

Suspect thats the point, by giving you the “choice” they also make the user responsible or can at least shift the blame.

[NewsaHackO]

Has there been any real, major attacks using prompt injection?