[monodeldiablo]

$0.18/hr is the (massively) subsidized price of AI services. Once these companies are required to turn a profit for their investors, they'll raise the price. Then the math doesn't look so lopsided. We're already seeing this process unfold with token windows and ad rollout.

[joegibbs]

It's not that subsidised, this is just wishful thinking. You can run a local model like Qwen for equivalent prices. You might see it go up to $0.50/hr but you're definitely not going to see it at $22

[monodeldiablo]

I do run open models locally, but let's not fool ourselves into thinking that they're functionally competitive. I'm extremely skeptical of anybody claiming they've obviated a $22/hr job with an open model. Qwen is a big step down in capability. I can play with something like k2.5 for awhile, but if I want real work done I'm going back to a frontier model, which has significant runtime requirements for inference.

You're also ignoring the cost of purchasing and amortizing dedicated hardware in your local model example.

It's not an apples-to-apples comparison.

[kcb]

Inference isn't really that expensive, its the training of new foundational models that is. With whatever highly optimized setup the big providers are using, they should be able to pack quite a lot of concurrent users onto a deployment of a model. Just think too, it's very possible their use case would be served just fine by a 100B model deployed to a $4,000 DGX Spark.