[infecto]

Inference costs are largely break even and profitable. Now yes there is absolutely questions on the training side.

You could turn off the switch today though and most of your leading models could keep the lights on.

[algorithmmonkey]

I keep seeing everyone guessing what the margin is on inference. You say that it's largely break even. We have this person in the thread claim 80% margin (https://news.ycombinator.com/item?id=45462442).

These numbers are complete make believe.

[infecto]

Why would they be make believe? I am generalizing across multiple companies and simplifying it as much as possible as we won’t know all the cost buckets but we do know at current costs the bare inference cost of keeping the machines running is being covered.