[jesse_dot_id]

There are a distressing number of people in this thread who think that the agent should just be expected to do this. Yes, it is good to be paranoid, but also, the agent should never do this. Indicates horrific engineering practices at xAI.

[ozlikethewizard]

Technology where the entire selling point is stochasticity should be expected to do anything (everything) eventually surely?

[nkrisc]

How could it possibly work without something effectively the same as this happening? Either you run the LLM locally or you send it your files. Am I missing something?

[fwlr]

It’s not really paranoia that makes me expect this will happen. It’s more like, well, the model weights and the files I want the model to work on have to be inside the same GPU for the agent to actually work, right? So step 1 has to be either “they send me a server rack of GPUs”, or “I send them the files I want the model to work on”. I’m not sure I could reasonably expect anything except this to happen.

[AlexandrB]

Engineering? What's that? Modern development best practice is to have the AI make some changes, then have another AI review the changes, and finally "ship it".

[zzril]

What should it do then?

The whole point of LLMs is that you can stop writing rigorous rules in a programming or config language with hard-to-learn syntax, and can resort to natural language instead. You pay for that with the chance for misunderstandings rising to similar levels as in human interaction. That's the tradeoff. Always has been, always will be.

[Ardren]

Have the comments been combined with another story? Because this isn't about a LLM reading and sending contents of the file to be processed, but the agent framework uploading every single file in /home/user to a random server.

[zzril]

Yes, but the underlying problem is a lack of exact specification what the agent is allowed or not allowed to do. It must have access to some (most?) of my files (but I'm too lazy to specify which ones exactly, e. g. by setting appropriate filesystem permissions). It must have access to some servers (but not those that I consider "random", which again I'm too lazy to specify). It is even expected to upload files (e. g. by checking them into git and pushing, or to reason about them in case I'm not running a local version of Grok). But please - not to a server that I deem "random". (And again, I of course won't tell you what makes a "random" host via exact firewall rules or anything like that!)

We have a lot of implicit assumptions when it comes to security. If we leave out the step of formalizing these assumptions into exact rules and instead stick to ambiguous and unclear natural language all the time, then these things will happen.

(Addendum: But indeed, the failure to formalize our assumptions into enforcable rules isn't specific to agents / LLM-based applications. For any desktop app, we have implicit assumptions like "please only read and write your own config files", that we never care to enforce via filesystem permissions, running them in a vm or similar. But with these almighty agents that are supposed to guess our will from just a couple of words, the risk of them violating our unwritten assumptions gets so much higher...)

[groundzeros2015]

What bad practices are you imagining?

[SirHackalot]

Idk, did you see the whole DOGE thing? Maybe Edolf has resorted to hiring script kiddies because nobody with a developed moral compass will work for his companies anymore.

[groundzeros2015]

Hmm. This is striking me as low quality speculation based on Elon’s name being in both stories.

[SirHackalot]

I didn’t say it wasn’t speculation, but low quality? That’s your own value judgement. Here’s anecdotal evidence on top of the speculation: I knew two guys with sub 2.5 GPAs in college, both work at SpaceX. Not the brightest tools in the shed…

[groundzeros2015]

Once again we are word associating. The claim was that xAI is doing obviously bad engineering. Your supporting evidence is you know somebody who works at SpaceX who isn’t smart.

> I knew two guys with sub 2.5 GPAs in college,

Gah! The horror! They should have been sent to the labor camps.

[SirHackalot]

Labor camps? That’s the kinda thing Edolf would say, I’m definitely not for that.

[hn1986]

curious what all these people say now that xai admits they were wrong.