[TacticalCoder]

What's the point of microVMs for running agents?

Are you guys literally spinning up agents where a 100 ms boot time vs a 3 seconds boot time makes a difference?

I'm asking because I understand the appeal of micro VMs but every time the subject comes up people talk about "isolating agents": what's wrong about isolating agents in a regular VM (or in a container which, itself, is in a VM)?

FWIW I've got my stuff nicely isolated in regular VMs that are regularly up for hours and hours.

It's like the microVMs boots in 100 ms, then the agent does... What? And exits after another 100ms and now you need to launch another one?

What's the use case of "microVMs to isolate agents"?

[victorbjorklund]

I imagine you can have a situation where you let an agent run in a shared env but to access certain tools you spin up a VM just for the tool call duration and then shut it down again. Let’s say you wanna allow the agent to write and run code then you need it to run it somewhere safe

[vmg12]

Microvms are better for the provider VM provider. They use less memory and have a smaller attack surface. Also starting in 100ms means you don't need to add a bunch of async machinery when launching the vms.

[0xbadcafebee]

This is for people who want both faster execution, and better security isolation for agents/subagents. It is a different use case than yours

[TacticalCoder]

I understand that but micro VMs don't provide better security isolation than regular VMs.

So that leaves faster boot times.

Faster boot times and then the agent does what? And at how many token/s? And what's the "time to first token" anyway?

How do the time to first token and then the token/s inherent limitations of LLMs not totally dominate the running time?

I just don't get the use case.

[nok22kon]

imagine installing an agent in slack at a company with 1000 employees, and you want each request to have its own VM for data analysis, downloading repos and working on them, ...

regular VMs just use too much memory, a typical ubuntu uses 512 MB as a baseline