[ethanbond]

Right, but humans don’t scale that way, so the threat is completely different.

This is like saying a nuclear weapon accident is not that scary because you can also have a microwave malfunction and catch on fire. Sure you can —- but the fact it’s not a nuke is highly relevant.

[TeMPOraL]

No, I'm saying that securing against "prompt injection" is like saying you want to eliminate fission from physics, because you're worried about nukes. That's not how this reality works. Nuclear fission is what happens when certain conditions are met. You're worried about nukes? Stop playing with nukes. I'm not saying they aren't dangerous - I'm saying that you can't make them safer by "eliminating fission", as it makes no physical sense whatsoever. Much like "securing against prompt injections" in language models, or a GAI, or in humans.

[ethanbond]

> Sure, current bleed of LLMs is badly vulnerable to some trivial prompt injections - but I think a good analogy would be a 4 year old kid.

This reads like you’re trying to say “don’t worry about it, humans are vulnerable too and it’s threatening the way a 4 year old child is” not “correct, we cannot prevent nuclear explosions given that we have fission and yes we’re on track to putting fission devices into every single internet-connected household on the planet.”

[lanstin]

There is a reason humans with security clearances can’t just have an arbitrary large number of interactions with foreign nationals, or that good interrogators say they can always get info from people if they talk enough m

[TeMPOraL]

I'm saying "stop trying to solve the problem of consumer market IoT fission bombs by trying to remove fission from physics - this just can't possibly work, and it takes special confusion to even think it might; instead, focus on the 'consumer-market', 'IoT' and 'bomb' parts".

"Prompt injection" is a vulnerability of generic minds in the same sense "fission" is a vulnerability of atoms.