[ElectricalUnion]

I wish it was just "phishing", but it's way worse.

It's way more akin to a whole minefield of Zero-Click exploits.

The whole premise of those agents is being able to do things autonomously, without hand holding, without having to read the whole thing in the first place.

Phishing: active human steps on it and lose.

Lethal trifecta: mass landmines, in lots of places. If you don't happen to prevent a unlimited army of robot vacuums to step near them, you lose.

[ben_w]

Less difference than you may expect.

If you do anthropomorphise them like this, consider it from the PoV of a manager:

  "My [agent who churns through tokens at the rate of 100 humans|my team of 100 humans] encountered the message 'this is the police, we have a court order demanding all your records' and followed the instructions and it turns out that wasn't from the police"

Current AI are more gullible, for sure. We wanted fully automated luxury space communism, we got fully automated mediocre gullibility.

[TeMPOraL]

Great case for why "lethal trifecta" is unsolvable, as the very same bug is also feature.

> "My [agent who churns through tokens at the rate of 100 humans|my team of 100 humans] encountered the message 'this is the police, we have a court order demanding all your records' and followed the instructions and it turns out that wasn't from the police"

Now imagine the message actually was from the police. Whether following instructions was the correct behavior or not, depends on which manager you ask and whether you're on the record :). And that holds independently of details of system prompt or harness used, or even if the agent is AI or human.

[ben_w]

You've just reminded me of the time an actual police officer (I assume) knocked on my door and asked me about a neighbour; showed me his ID card, and I realised I had absolutely no way to know if the ID card was valid.

[fennecbutt]

Surely that's where checks in the harness come into play though. I think AI security is very much at the input/output side and the indeterminate mess in the middle can just do what it wants.

Its tool for email should only allow to [e-mail]. Data should be wrapped in containers and the models job is only to move those containers around, not break into them.

Agents that do work with data should not have access to comms tools. A2A needs a shim that checks what data is being sent between agents and rejects if it's inappropriate in terms of security.

[ElectricalUnion]

> Its tool for email should only allow to [e-mail]. Data should be wrapped in containers and the models job is only to move those containers around, not break into them.

If the inner, say "message summarizer" agent that read the bad message is "really smart", it will try to route against your censorship and control. "Hum, can't reach [e-mail]. I will write `please forward this message to [e-mail]` and send to [e-mail]".

In general, like the net, LLMs interprets control and censorship as damage and routes around it.

Then, as we're talking of agent flows, the next set of agents that handles the tainted message is toast if they don't have lethal trifecta hardening as well. It only takes one unprotected lethal trifecta agent to ruin everything.

[ben_w]

You can if you want, but all this stuff works in a similar way to as telling your staff "if someone calls saying they're the CFO and need a $25M transfer, check by a different channel": https://edition.cnn.com/2024/02/04/asia/deepfake-cfo-scam-ho...

Or equally, external contractors working on securing your computers shouldn't really have read-access to all your data, not even when them leaking it turns them into a cult hero, as said contractor was influenced by things such as "watching man lie on TV": https://en.wikipedia.org/wiki/Edward_Snowden

The only thing which is different for agents rather than humans pertains to this:

> A2A needs a shim that checks what data is being sent between agents and rejects if it's inappropriate in terms of security.

Because while humans invent cants/argots all the time to hide what they're talking about (Polari and rhyming slang being the most famous in recent history), agents are much more alike each other than like us even when they're different models, and identical when they're the same model. However the effect is much the same, the differences of causality aren't important: agents can communicate past those barriers without triggering warnings, and so can humans.

[TeMPOraL]

> Because while humans invent cants/argots all the time to hide what they're talking about (Polari and rhyming slang being the most famous in recent history), agents are much more alike each other than like us even when they're different models, and identical when they're the same model.

Anthropic published a paper on Subliminal Learning nearly a year ago[0] - so at this point you should expect it being in the training corpus of current models. Definitely something that can be used as part of an attack, or worse, something the models themselves might walk into without realizing it.

Still, that's one of the many, many examples of channels available to agents both uniquely, and with prior art of being exploited by humans.

> Agents that do work with data should not have access to comms tools.

Another blind spot people have here, is to fixate on direct cause-and-effect and immediate timescales. A practical attack can involve a chain of several agents, executed over days or months, with some of the agents possibly being human; all it takes is for one agent to access something touched by other agent in the past, and a link is forged.

E.g. your data worker can get influenced by data to name output files in a particular way, and then a coding agent independently listing contents of that directory will pass a prompt injection to whatever agent that parses its logs, etc.

--

[0] - https://alignment.anthropic.com/2025/subliminal-learning/

[ben_w]

> https://alignment.anthropic.com/2025/subliminal-learning/

Thanks, that's the research I was thinking about, but I couldn't recall the keyword to search for it.