[simonw]

That $200/month is already more like $4,000/month if you have to pay full API pricing - "enterprise" companies for example. That drops the equivalent to 10 months.

(I'd be surprised if that local rig really can drive the equivalent of $4,000/month of API spend though, given that a local rig can run prompts in parallel a lot less effectively than Anthropic's many data centers.)

[fweimer]

I think the decode phase of inference typically uses local compute resources poorly due to the very small batch size. If you can run many inference tasks in parallel, this will make local inference more competitive to centralized inference, not less.