The costs are so staggering it's becoming clear we're going to be priced out of owning computers altogether. Self-hosting frontier models is a corporation's choice, not an individual's.
It's not like people ever owned iPhones to begin with. They're Apple's computers, Apple's just generously allowing their customers to use them, and only on their terms. A monthly iPhone subscription is just the end game.
The memory prices put computers into the same category as cars. Expensive but still realistic.
It's these 200kUSD+ nodes with a dozen or more data center class GPUs that are killing the self-hosting dream. The cheapest nodes are is in the same price category as literal houses.
Either we get some competition in this space, or computing will go back to their roots as ivory tower big iron.
HBF will probably save the small inference nodes. (Eventually, the initial price will obviously be in the stratosphere.)
Long term, HBF shouldn't be more than 4x as expensive as commodity flash, and Kimi K3 is an interesting target for a system using it given how aggressively it compresses the KV-cache. An inference box with ~1.5TB of HBF and ~48GB of DRAM should be able to be built for less than a couple of grand, and get something near to 100tok/s on full Kimi K3 for a single token stream.