Hacker News new | ask | show | jobs
by phyzome 28 days ago
Because the author is wrong, and LLMs don't actually work that way. Prompt injection cannot be fixed. Role boundaries are a bandaid you can apply, but attackers can work around it.
1 comments

You can still build a system that isn't vulnerable by limiting the API the LLM can access. A process consuming untrusted comments for summarisation shouldn't have access to account private data, it should just deliver a summary report. Another process can them scan that and remove/disable links etc.
Sure. But that's not what was being suggested in the comment I responded to.

(There are also problems with what you're suggesting, though, such as the summary report still being ripe for abuse in similar ways as the blog post describes.)

You can structure the system to cause an LLM an answer structured questions, Y/N or from a structured list of options, or with numeric weights against labels, and then use classical parsing to enforce a compliant response submission, and you do that for random batches of comments, then have another LLM summarise the survey response. It is quite possible to extract sentiment and key topics.

I don't think it's useful to throw your hands in the air and say LLM context statistics are ungovernable. Both context hardening and action shielding architectures are addressing this, and in combination with well designed pipelines the risk is controllable.

Sure, what you're describing should be relatively safe from prompt injection. It's a lot less capable than the original, of course.

I'm not saying "it's impossible to get an LLM to do anything safely", just that you have to really scale back on your goal.