| we reached a point where there will always be demand for tokens. suppose the current frontier models are final and there is no more progress. it still wouldn't matter because the cheaper the tokens the more demand there will be which will drive the silicon demand onward. consider what you can do if a billion tokens cost cents: you could create a scaffold that would literally burn ungodly numbers of tokens on every conceivable angle. you could create scaffolds that don't just generate the next token but perform MCTS then at the end you keep the best results. another way to look at it, we know that there is a document in the space of all possible documents which holds an answer to every conceivable solvable problem. but the combinatorics prohibit random exhaustive search. LLM's make exhaustive search somewhat more tractable, because there will probably exist a prompt/chain of thought which can yield it (especially when you can always get experts to poke it with cheap ideas). creating silicon fabs is probably the most reliable investment at the moment, because there will always be room around the sun. orbital launch is probably less reliable because ocean floors still have a lot of room but it will depend on nuclear regulatory innovations. amusingly, had SpaceX actually gone up it would actually draw parallels to the dotcom bubble as being too soon. note that tokens aren't just text anymore, MCP is very effective. if nothing else with billions of cheap tokens you will be able to exhaustively create endless virtual worlds with just Unreal Engine MCP - this is the absolute floor. entertainment without end. and this all with just the current generation models frozen in time. but suppose only the intelligence is frozen, suppose training still works. if it's cheap enough you can always just dump compute on RL on the MCP directly. and the MCP can be something like silicon design... |