|
I owe you an apology too: I took your comment and, instead of focusing 100% on the thing you were trying to argue and discovering the nuance, I pattern-matched a more surface-level read to the flawed reasoning about LLMs I see a lot, including on HN, but one that I know you do not share. Thank you for elaborating here and in other branches of this discussion. I now see that you were reading my take as encouraging a view that "humans can be prompt-injected too, therefore LLMs are not that different from humans, and we already allow humans to do X", which indeed is very worrying. The view I have, but failed to communicate, is more like "humans can be prompt-injected too, but we have thousands of years worth of experience in mitigating this, in form of laws, habits, customs and stories - and that's built on top of hundreds of thousands of years of honing an intuition - so stop thinking prompt injection can be just solved (it can't), and better get started on figuring out LLM theory of mind fast". > I really want to get across to people that there are practical failings today that need to be taken seriously regardless of whether or not they think that "prompt injection" is just SQL-injection #2. I agree with that 100%, and from now on, I'll make sure to make this point clear too when I'm writing rants against misconceptions on "prompt engineering" and "prompt injection". On the latter, I want to say that it's a fundamentally unsolvable problem and, categorically, the same thing as manipulating people - but I do not want to imply this means it isn't a problem. It is a very serious problem - you just can't hope someone will solve "prompt injection" in general, but rather you need to figure out how to live and work with this new class of powerful, manipulable systems. That includes deciding to not employ them in certain capabilities, because the risk is too high. |