Hacker News new | ask | show | jobs
by teravor 14 days ago
we reached a point where there will always be demand for tokens.

suppose the current frontier models are final and there is no more progress. it still wouldn't matter because the cheaper the tokens the more demand there will be which will drive the silicon demand onward.

consider what you can do if a billion tokens cost cents: you could create a scaffold that would literally burn ungodly numbers of tokens on every conceivable angle. you could create scaffolds that don't just generate the next token but perform MCTS then at the end you keep the best results.

another way to look at it, we know that there is a document in the space of all possible documents which holds an answer to every conceivable solvable problem. but the combinatorics prohibit random exhaustive search. LLM's make exhaustive search somewhat more tractable, because there will probably exist a prompt/chain of thought which can yield it (especially when you can always get experts to poke it with cheap ideas).

creating silicon fabs is probably the most reliable investment at the moment, because there will always be room around the sun. orbital launch is probably less reliable because ocean floors still have a lot of room but it will depend on nuclear regulatory innovations. amusingly, had SpaceX actually gone up it would actually draw parallels to the dotcom bubble as being too soon.

note that tokens aren't just text anymore, MCP is very effective. if nothing else with billions of cheap tokens you will be able to exhaustively create endless virtual worlds with just Unreal Engine MCP - this is the absolute floor. entertainment without end.

and this all with just the current generation models frozen in time. but suppose only the intelligence is frozen, suppose training still works. if it's cheap enough you can always just dump compute on RL on the MCP directly. and the MCP can be something like silicon design...

4 comments

There is no evidence that billions of tokens will ever cost cents. The raw infrastructure cost alone is massive. That's why you see services now charging usage based pricing, because continuing to subsidize inference costs is unsustainable long term.
cents per billion is only 3 orders of magnitude reduction in cost. I can see 6 or so happening in the next decade or so.

1. Like any commodity, I expect the price to converge to the marginal cost. The marginal cost is the price of electricity which is currently 10% or so of the cost of inference.

2. I expect the cost of electricity to go down an order of magnitude. Solar power and batteries are getting cheaper fast.

3. As the price of electricity starts dominating costs, you can drastically reduce costs by by taking advantage of electricity price variations. Build data centers all around the world and answer requests from the part of the world where the sun is shining.

4. Moore's law

5. Algorithm improvements

6. Moving from general purpose chips to chips that are highly optimized for inference.

> we know that there is a document in the space of all possible documents which holds an answer to every conceivable solvable problem.

So you think reality is static and not an ever growing ever changing process?

>we reached a point where there will always be demand for tokens.

yes but who will pay for them.

I fear it will be tax payers and consumers of high end products... everyone else else can go eat dirt.

How is this related to the post?

    > How is this related to the post?
it argues that AI cannot be a bubble?
You are saying that AI is useful.

That doesn't mean there isn't a bubble. The internet is useful and we still had a dot com crash.

    > The internet is useful and we still had a dot com crash.

    "we reached a point where there will always be demand for tokens."

is it your position that the dotcom crash would have happened had there been demand for more internet at the time?
vibecoding is as much demand for AI tokens as pets.com was for early internet ecommerce. Kimi K3 is already crashing this.