Hacker News new | ask | show | jobs
by Tv9m 1226 days ago
> Is prompt injection even a problem worth worrying about?

It depends what API access the AI has. If it's just a chat bot, prompt injection can only reveal facts about its language model. But if the AI has POST access to something, depending on what it is, prompt injection can set off arbitrary human-caused disasters.
1 comments
nradov 1225 days ago
That's not the correct way to do security vulnerability analysis. If an API call can cause a disaster then fix the API. Whether the API consumer is an AI or some other type of system is irrelevant.
link
Tv9m 1213 days ago
> If an API call can cause a disaster then fix the API

By "API" I'm not referring just to publicly facing REST endpoints. I mean things like shell access for system maintenance, that normally only human professionals like you would be given. In the future it's not clear that humans will be able to dominate that role forever.

Hopefully the issues will be recognized while LLM-based agents are still only serving as retrieval systems.

link