Hacker News new | ask | show | jobs
by zzril 19 days ago
What should it do then?

The whole point of LLMs is that you can stop writing rigorous rules in a programming or config language with hard-to-learn syntax, and can resort to natural language instead. You pay for that with the chance for misunderstandings rising to similar levels as in human interaction. That's the tradeoff. Always has been, always will be.
1 comments
Ardren 19 days ago
Have the comments been combined with another story? Because this isn't about a LLM reading and sending contents of the file to be processed, but the agent framework uploading every single file in /home/user to a random server.
link
zzril 19 days ago
Yes, but the underlying problem is a lack of exact specification what the agent is allowed or not allowed to do. It must have access to some (most?) of my files (but I'm too lazy to specify which ones exactly, e. g. by setting appropriate filesystem permissions). It must have access to some servers (but not those that I consider "random", which again I'm too lazy to specify). It is even expected to upload files (e. g. by checking them into git and pushing, or to reason about them in case I'm not running a local version of Grok). But please - not to a server that I deem "random". (And again, I of course won't tell you what makes a "random" host via exact firewall rules or anything like that!)

We have a lot of implicit assumptions when it comes to security. If we leave out the step of formalizing these assumptions into exact rules and instead stick to ambiguous and unclear natural language all the time, then these things will happen.

(Addendum: But indeed, the failure to formalize our assumptions into enforcable rules isn't specific to agents / LLM-based applications. For any desktop app, we have implicit assumptions like "please only read and write your own config files", that we never care to enforce via filesystem permissions, running them in a vm or similar. But with these almighty agents that are supposed to guess our will from just a couple of words, the risk of them violating our unwritten assumptions gets so much higher...)

link