[catoc]

Exactly!

Attempting to handle prompt injections by prompting the model (not to leak sensitive data), is like attempting to stop a fire by burning the area around it

[politician]

My guy, that's actually how fires are stopped.

https://en.wikipedia.org/wiki/Firebreak

https://en.wikipedia.org/wiki/Controlled_burn

[catoc]

Haha, nice. TIL.

So all we need is ‘controlled prompting’ to handle prompt injections :-)

[NateEag]

It's maybe closer to putting a sign saying "The door is locked" on an unlocked bank vault.

It does nothing to improve security, and if someone manages to get inside and see the sign (i.e. "extract the prompt"), it gives them a strong hint there's interesting stuff behind this door.