[sangnoir]

History doesn't repeat itself, but it rhymes: I foresee LLMs needing to separate executable instructions from data, and marking the data as non-executable.

How models themselves are trained will need to be changed so that the instructions channel is never confused with the data channel, and the data channel can be sanitized to avoid confusion. Having a single channel for code (instructions) and data is a security blunder.

[not2b]

As you say, LLMs currently don't distinguish instructions from data, there is one stream of tokens, and AFAIK no one knows how to build a two-stream system that can still learn from the untrusted stream without risk.

[dikei]

Even human cannot reliably distinguish instructions from data 100% of the time. That's why there're communication protocol for critical situations like Air Traffic Control, or Military Radio, etc...

However, most of the time, we are fine with a bit of ambiguity. One of the amazing points of the current LLMs is how they can communicate almost like human, enforcing a rigid structure in command and data would be a step back in term of UX.