[dathanb82]

If "inference is cheap," why is OpenAI spending a ton getting Broadcom to design custom AI chips that make inference cheaper? Reports suggest their custom silicon isn't all that good for training, it's all to make inference more efficient. That shouldn't be necessary if inference is already quite cheap.

[petra]

A large part of the market will be ad based. For that, having the lowest cost inference is useful.

Also for agent doing r&d, cheaper tokens allows doing more, which is always good.