[K0balt]

GPUs are fundamentally a terribly inefficient way to achieve our goal, the calculations we are doing with them are difficult problems that fall out of physics for free. Tech like thermal wells could give us 2t models the size of micro-SD cards that consume milliwatts. There are already commercially available ASIC LLM models (the model is burned into silicon) that use 100x less power than running it on a GPU, while allowing a single card to serve thousands of users at once.

There is a lot of low hanging fruit here as models stabilize, and the more ambitious tech that might take a decade or so to land offers efficiencies 10-100x compared even to biological systems. (Most of it requires cryogenic temperatures though). A forward looking tech investment might be inexpensive, tiny, efficient cryogenic cooling systems optimized for desktop or portable use.