Hacker News new | ask | show | jobs
by Gormo 5 days ago
> By design there is no separation between control & data channels in LLMs.

No? MCP integrations are certainly different channels than the prompt stream input by the user. They're usually interface layers that connect to traditional REST APIs and/or CLI tools, and are exposed to the LLM by the agent software, all of which is deterministic code.

Shouldn't it be relatively simple to strip ANSI escape sequences from data originating from MCP connections? Or alert users to their presence, or have granular approval rules that trigger when ANSI sequences are detected?
2 comments
mschuster91 5 days ago
> Shouldn't it be relatively simple to strip ANSI escape sequences from data originating from MCP connections?

The problem is, literally anything can be malicious for an AI. With SQL? Prepared statements are foolproof, no matter what garbage I put into a buffer destined to be stored in a BLOB, there is no way of this leopard biting my face.

As long as there is no separation between prompt, context and unsafe input in LLMs (which I doubt is possible), any LLM is vulnerable to injection attacks. Even if you place a second LLM to filter input for the actual agent LLM, a two-stage exploit can be used (and so on and so on).

link
Gormo 3 days ago
Why wouldn't there end up being a separation between prompt and context, though? Why can't we set up LLM engines to have one input stream for instructions and another input stream for data? We've got lots of effective approaches to mitigate the analogous issues that come from storing data in the same memory that we execute code from.
link
NitpickLawyer 5 days ago
> MCP integrations are certainly different channels than the prompt stream input by the user.

If the result from the MCP call includes something from the outside (i.e. an API call result, some content from a web page, etc.) then you are back to the same problem of everything being context. This is something inherent to LLMs. You can filter for known things as you say, but all you can do is mitigate some attacks, not all. I was replying to someone saying "don't use untrusted inputs". The surface for LLMs is potentially everything, in this case.

link