[Twirrim]

The easiest, most guaranteed way to isolate it is to run it in a VM or container where it literally can't do the wrong thing without some kind of full container or VM exit exploit.

It's not hard, it's trivial. Most folks here are constantly working with containers. You know how to run a container with a local directory mounted in it.

For myself, I've been using Lima (https://lima-vm.io/) to reduce even that little bit of extra work. Lima works cross-platform leveraging native virtualisation or containerisation, and has some useful capabilities for using agents.

[_verandaguy]

Generally, I agree!

But it doesn't matter how good a best practice is if the industry doesn't adopt them wholesale; and even then, if your container or VM is configured with inappropriately-permissive passthrough (which, from experience with similar misconfiguration in the past, will widely happen), it could be for naught in many orgs.

That said, I do hope these become the norm if LLMs are here to stay.

[Twirrim]

I can secure myself first, so I'm never at fault. I never want to be the one to accidentally break production.

Beyond that, I just keep advocating for more safe and secure approaches any opportunity I get.

[arvyy]

I used opportunity to learn about devcontainers. I've only recently started using llms and it's possible I'll change my mind later; but so far I quite like the approach in part because it 2-for-1 also gives benefit of easy to setup coding env for people who don't care about ai.

[SpaceNoodled]

I seem to recall reading about agents already breaking out of containers.

[_verandaguy]

Last time I read about this, this was due to the well-known pitfall of UID mapping across container boundaries.

It's a common misconfiguration and one of the footguns available through containers, which I don't say a wholesale condemnation of the technology, but certainly as a UX facet that could use reevaluation.

[Twirrim]

I've been biasing towards VMs myself just out of caution, but maybe that's just extreme paranoia.

The way I look at it is similar to how I'd look at any hypothetical employee. How do I ensure the agent only has access to the minimum possible they need to get their job done?

That means no access to git repositories (no pushes on my behalf means it can't accidentally nuke git history, something there is anecdotal evidence of agents doing). It can make local changes in git only and I will take responsibility for pushing them. No access to the wider internet beyond what I deem acceptable. No permissions to access any internal APIs except what I provide (and not using my credentials).

In one case, I have a tool that has a set of dangerous commands alongside a large number of safe ones. I don't even have it installed in the agent's VM. I run an MCP that is a simple python wrapper around the tool on the host side, and expose it to the agent in the VM, so that it can only possibly run a strict safe subset that I can trust it with access.