[ATechGuy]

VM is not the right abstraction because of performance and resource requirements. VMs are used because nothing exists that provides same or better isolation. Using a throwaway VM for each AI agent would be highly inefficient (think wasted compute and other resources, which is the opposite of what DeepSeek exemplified).

[tptacek]

To which performance and resource requirements are you referring? A cloud VM runs as long as the agent runs, then stops running.

[ATechGuy]

I mean performance overheads of an OS process running in a VM to (vs no VM) and additional resource requirements for running a VM, including memory and additional kernel. You can pull relevant numbers from academic papers.

[transpute]

A linear bar graph comparing compute/memory requirements?

  - OS process
  - virtual machine
  - LLM inference

Could have longevity as PC master race meme template.

[tptacek]

OK. Thanks for clarifying. I think you're pretty wrong on this one, for what it's worth.

[vineyardmike]

Is “DeepSeek” going to be the new trendy way to say to not be wasteful? I don’t think DS is a good example here. Mostly because it’s a trendy thing, and the company still has $1B in capex spend to get there.

Firecracker has changed the nature of “VMs” into something cheap and easy to spin up and throw away while maintaining isolation. There’s no reason not to use it (besides complexity, I guess).

Besides, the entire rest of this is a python notebook. With headless browsers. Using LLMs. This is entirely setting silicon on fire. The overhead from a VM the least of the compute efficiency problems. Just hit a quick cloud API and run your python or browser automation in isolation and move on.

[ATechGuy]

I think you are assuming that inference happens on the same machine/VM that executes code generated by an AI agent.

[tptacek]

I'm not even talking about Firecracker; for the duration of time things like these run, you could get a satisfactory UX with basic EC2.