It's an allowlist so not-git would be disallowed by default. Renaming it to a binary that is already listed in the allowlist should fail due to a cryptographic binary signature mismatch.
That's a pretty oddly narrow & specific definition of what LLM harnesses are for & it certainly doesn't match my experience. In fact I'm genuinely struggling to think of any scenario where an agent CLI would need to execute a build artifact. Sure many devs might ask it to for convenience of not having to spend 3 seconds typing & running the command themselves but I can't see it ever being explicitly necessary.
I will note you used the term "harness" which - while terms around LLM tooling is still inconsistently defined - in my experience generally pertains to agentic automations: e.g. LLM CI integrations, etc. Rather than interactive local tooling like OpenCode. So maybe you're referring to that (though even then I still can't think of examples)
> In fact I'm genuinely struggling to think of any scenario where an agent CLI would need to execute a build artifact.
You should look into "testing". It's when you run the code you just wrote to make sure it works.
For example, in LLM harnesses, it's used to ensure the produced code has the desired properties. It is the key element needed to allow agents to continue to work until they have succeeded.
You're definitely referring to fully autonomous harnesses as I suspected - these are generally quite well suited to VM deployment (quite possibly in the cloud) if you're concerned with sandboxing as they don't typically run into the same challenges as an interactive LLM UI.
For ad hoc task/investigation-oriented fully interactive agent UIs the requirements are more complex & there's all a lot of advantages to running locally on host, but e.g. for builds & testing in that context there's less of a requirement for the LLM to have full control over every execution given user presence. Basically the use-case being discussed here is LLMs-as-assistance/pair programmers/active digital colleagues, rather than fully automated autonomous harnesses (legit but separate use-case).
All of the harnesses I cited above run locally and default to testing the code they generate, despite your inability to conceive of it.
It is very obviously the primary use case: automated development interleaved with investigation. Because you need to do both, at the same time, iteratively.
Please don't make me paste those URLs again: they demonstrate exactly what you say does not happen.
I mean, ideally enforce W ^ X, and enforce stricter permissions on executables where the agent has flipped the W to X. My broader point is we should be using operating system primitives instead of regex and wishful thinking.
You can't give shell access and then revoke special parts of it. This is not controversial.
All your criticisms apply to all LLM harnesses - and everyone who genuinely cares about these things is using security on top of the harness, because OBVIOUSLY.
This doesn't require cooperation from the shell. If writable paths are non-executable, and executable paths are non-writable, then executing `cp $(which git) my-new-favourite-binary` doesn't grant you permission to run it.
The part which does require cooperation (from outside of the shell, i.e. from the harness) is the ability to upgrade writable paths to executable, at the cost of writability and whatever other restrictions you place on the binaries generated by your agent.
This kind of W^X invariant is widely deployed in every desktop operating system, and the type of W-X upgrade hook I described is used by every JIT engine.
> You are confusing CPU-level security with filesystem-level security.
> I'm curious - how do you expect an LLM harness to build and test executables without being able to build and execute executables?
W^X applies to file systems as well as page tables, and this is called out in paragraph 4 of the Wikipedia article you didn't read. Respectfully, I'm leaving this conversation at this point because it doesn't seem productive.
You did not hit max depth - you hit a timer designed to prevent rage posting.
My question stands - if you accept that LLM harnesses are designed to allow the building and testing of executables - how do you imagine you can prevent the building and running of executables?
Going from "build and running executables" to that needing to be done on the same system and under control of the LLM is a large leap.
Separation of duties and ephemerality are well-known security mechanisms that many harnesses jump right over in the pursuit of easier UX.
Mingling permissions + trusting vibe-coded security boundaries in the harness itself (developed by folks whose appetite for risk would make even an 00s front-end developer blush) is rolling a handful of dice all at once.
But fundamentally, this is a disagreement between two risk appetites who will never agree: {it works most of the time} vs {it's guaranteed to never fail}