[treis]

Relative to the current usage demand for tokens is effectively unlimited. If the price of tokens go down people will send more tokens to compensate. We are very very far away from a cost per token where people run out of things they want to send through an LLM.

[dgellow]

You’re describing the issue. The problem isn’t that datacenters are under utilized, it’s that they are used to generate something that has a value going down over time, while being financed through debt with the assumption their revenue grow over time. That’s where you have a mismatch.

To simplify let’s assume a given datacenter can generate a fixed number of token per second (in reality that depends on the model tokenizer, and other model specific factors). And let’s say it is used at full capacity, split between inference and training. If people are mostly using the frontier models the datacenter might be profitable, with enough margin to cover training of future frontier model, operating costs, and repay the debt used for construction.

However if the token price goes down and the demand doesn’t change you now risk to run the datacenter at loss, with risk to not be able to repay your debt.

If price reduce and demands increase, you can reduce the part dedicated to training, but eventually run the datacenter at full capacity to generate tokens that are becoming less and less valuable over time

[treis]

The value is not going down. The cost is.