[SwellJoe]

Yes! Microsoft made buckets and buckets of money on products with a unit cost that approaches zero. Hardware was vastly less profitable and there's a graveyard full of PC and server makers that didn't survive. Even the most profitable PC makers (Dell and Compaq during the rise of Microsoft) didn't make anything like Microsoft money.

I don't know how to apply those lessons to AI, as long as AI requires so much hardware to operate. If small models actually get capable enough, the shape of the industry changes drastically.

[fragmede]

How capable do they need to be? You can run distillations on a Pixel 10 Pro today, it's just not packaged up neatly yet.

[SwellJoe]

I'm super enthusiastic about small models, but let's be realistic. A distillation is not the whole model (and, in fact, a lot of the small distillations on HuggingFace are worse than the base model...most of the Qwen 3.6 Opus/Fable/whatever distillations get weirder on some dimensions than Qwen 3.6 alone, as I understand it).

There are little models that are very good for their size. I say nice things about Gemma 4 damned near every day. But, I'm not writing code with it. I am using it for finding security bugs, though, as the 31b variant is outrageously good at it for its size: https://swelljoe.com/post/gemma-4-exceeds-expectations/ and I'm also using it as a base for my own training experiments, specifically the 12b which is small enough to train a LoRA for on my local hardware so I don't have to rent cloud GPUs. The 12b QAT can run on your Pixel 10 Pro today and is frightfully smart for its size, and has great vision capabilities.

But, I keep saying "for its size". You have to be realistic about what tasks these self-hosted models can do. They are getting better though. Gemma 4 31b is competitive with models 10 times its size from a year ago. That's remarkable, and indicates where things are going.