[chasd00]

Can’t the connections and APIs that an LLM are given to answer queries be authenticated/authorized by the user entering the query? Then the LLM can’t do anything the asking user can’t do at least. Unless you have launch the icbm permissions yourself there’s no way to get the LLM to actually launch the icbm.

[alexbecker]

Generally the threat model is that a trusted user is trying to get untrusted data into the system. E.g. you have an email monitor that reads your emails and takes certain actions for you, but that means it's exposed to all your emails which may trick the bot into doing things like forwarding password resets to a hacker.

[thebytefairy]

I think it depends what kind of system and attack we're talking about. For corporate environments this approach absolutely makes sense. But say in a user's personal pc where the LLM can act as them, they have permission to do many things they shouldn't - send passwords to attackers, send money to attackers, rm -rf etc