[Terr_]

I'm saying that with this kind of use-case, that problem doesn't exist: The prompt is nothing interesting an attacker couldn't already guess, and knowing it provides an attacker no real benefit.

Since the LLM is just helping the user arrange their choices of input, it is no more vulnerable to things like SQL injection than if someone had made a big HTML form.

[nomel]

My question to that person was "How can you accept non trivial user input without the risk of jailbreak?", in the context of their idea of using one "correctly", without severely limiting the use of LLM. I agree with you.

The problem space of replacing small text boxes is definitely in the realm of "trivial" user input. And not caring about a jailbreak is different than preventing one. But, not caring about a jailbreak is the only sane approach where LLM can really remain useful. That's fine, as long as it's understood. Allowing jailbreaks, in your system, without negative consequences, doesn't mean it's not "correct", which they seemed to be claiming.