[jcgrillo]

Yes, it's amazing how quickly so many tech companies have hitched their tooling to these big AI vendors seemingly without any thought towards whether they'll still exist a year or three or five from now. Insane behavior. To the (debatable!) extent that AI coding tools are useful at all wouldn't it be a hell of a lot smarter to self-host? At least that way you have some control over QoS, and a stable, predictable result... Or maybe nobody cares about that kind of thing anymore? What happened to basic business math in this industry?

[twoodfin]

The basic business math is (to start) software companies realizing that spending $10k, $20k, $50k (more ?) per year, per developer for current models at current token rates might not be particularly insane, given the value return.

Models are likely going to keep getting better, and as costs go down, demand is likely to rise faster.

[jcgrillo]

> as costs go down

Huh? Why would that happen? Indications are that costs will likely go up, especially if currently vendors are selling tokens at a loss.

[twoodfin]

The main operational expense of a million LLM tokens is pennies of electricity.

Even if you generously depreciate the GPU and other hardware, it’s hard to believe inference at scale in April 2026 isn’t highly profitable.

[jurgenburgen]

> The main operational expense of a million LLM tokens is pennies of electricity.

I think you meant dollars of electricity.

[twoodfin]

I don’t think so.

https://www.theregister.com/2024/03/18/nvidia_turns_up_the_a...

A Blackwell 8X node consumes about 15kw, let’s up that to 50kw to generously account for cooling and everything else.

A US kWh is something like $0.20, so running that node for an hour costs ~$10.

Nvidia got 30,000 parallel TPS out of DeepSeek-R1 on that node:

https://developer.nvidia.com/blog/nvidia-blackwell-delivers-...

So that $10 buys you over 100M tokens or … pennies per million.

I’m sure these numbers are off, but not by an aggregate two orders of magnitude.