[ramoz]

One concern nobody likes to talk about is that this might not be a problem that is solvable even with more sophisticated intelligence - at least not through a self-contained capability. Arguably, the risk grows as the AI gets better.

[NitpickLawyer]

> this might not be a problem that is solvable even with more sophisticated intelligence

At some level you're probably right. I see prompt injection more like phishing than "injection". And in that vein, people fall for phishing every day. Even highly trained people. And, rarely, even highly capable and credentialed security experts.

[chasd00]

"llm phishing" is a much better way to think about this than prompt injection. I'm going to start using that and your reasoning when trying to communicate this to staff in my company's security practice.

[ramoz]

That's one thing for sure.

I think the bigger problem for me is the rice's theorem/halting problem as it pertains to containment and aspects of instrumental convergence.

[choldstare]

this is it.

[hakanderyal]

Solving this probably requires a new breakthrough or maybe even a new architecture. All the billions of dollars haven't solved it yet. Lethal trifecta [0] should be a required reading for AI usage in info critical spaces.

[0]: https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/

[ramoz]

Right. It might be even as complicated as requiring theoretical solutions or advancements of Rice's and Turing's.

[venturecruelty]

Oh, I love talking about it. It makes the AI people upset tho.