Hacker News new | ask | show | jobs
by TedDallas 243 days ago
It is like SQL injection. Probably worse. If you are using unsupervised data for context that ultimately generates executable code you will have this security problem. Duh.
1 comments

Worse because there's really no equivalent to prepared statements.
Sure there is. A common way is to have the LLM generate things like {name} which will get substituted for the user's name instead of trying to get the LLM itself to generate the user's name.
Parameterized queries allow you to provide untrusted input to the database in a way that's guaranteed not to be interpreted as instructions.

There's nothing like that for LLMs.

That's what I explained. You are trying to do something with an untrusted name and the LLM will not treat the name as instructions because it doesn't see the actual name.
You mentioned having the LLM generate a placeholder, whereas the important thing is what it accepts. You can feed an LLM nothing but placeholders but that's very limited since it can't see the the actual data in any way. You're really just having it emit a template. Something simple like "make a calendar event for the reservation in this email" could not be done. In contrast, parameterized queries let the database actually operate on the data.
It may be limited but that doesn't mean it's not similar. For example MySQL can't check the weather when given city string as a paramertized query, but that doesn't mean MySQL doesn't have parameterized queries.