[selcuka]

Aren't LLMs commodity products these days? It's the same thing as running this on a $7 VPS that you don't "own".

I don't think switching to a different provider, or running an open one locally would affect the response quality that much.

[ekianjo]

The LLM is the key element here, not the 7 dollars VPS... The model itself has cost billions of dollars to train and of the service shuts down or is interrupted for some reason your fancy setup breaks like nothing.

[selcuka]

> The model itself has cost billions of dollars to train

But that has nothing to do with this use case, right? By the same logic, Linux has millions of man-hours went into it but we can use it for free on a $7 VPS.

> service shuts down or is interrupted for some reason your fancy setup breaks like nothing

No, it doesn't. That's what I meant by commodity. You can switch to another service and it will work just fine (unless you meant that all LLM providers might cease to exist).

Also note that they have a $2/day API usage cap, meaning that they are willing to spend $60+/month for the LLM use. If everything else fails, they can use those funds to upgrade the VPS and run a local model on their own hardware. It won't be Sonnet-4.6-level, but it will do. It just doesn't make sense with current dollar-per-token prices.

[ekianjo]

> Linux has millions of man-hours went into it but we can use it for free on a $7 VPS.

Bad analogy. I don't need an API to run Linux.

[chatmasta]

> The LLM is the key element here

No, the key (novel) element here is the two-tiered approach to sandboxing and inter-agent communication. That’s why he spends most of the post talking about it and only a few sentences on which models he selected.