[bigyabai]

Lowering the cost of hardware still won't solve the issue. HBM and DDR5 was never cheap, even before the shortages, so selling a full inference system is beyond the acceptable price range for most casual customers.

We're going to see Apple and Google compete over services and AI/OS integration instead, it will probably be years before your OEM takes local models seriously.

[tanseydavid]

Apple and Google (via smartphones) are in literally everyone's pocket.

Running KIMI on a phone is not possible today and I agree with you that it will "probably be years before..." it is.

But how many years do you guess? I personally do not think it will take even 10 years for the situation to be commonplace.

[root_axis]

> But how many years do you guess? I personally do not think it will take even 10 years for the situation to be commonplace.

IMO it won't be possible for the foreseeable future. There's essentially zero possibility that phones will gain the hardware capacity to run today's Kimi, so the only other alternative is to squeeze the power of today's Kimi into something that can fit on a smartphone, which also seems fairly unlikely considering the current rate of progress.

[JoeAltmaier]

A bold prediction. Phones have gigabytes today. There are famous laws of growth that put terabytes at just a few years away - perhaps 10 isn't too bad an estimate?

[root_axis]

> Phones have gigabytes today.

Phones have zero HBM today.

> There are famous laws of growth that put terabytes at just a few years away

I assume you're referring to Moore's law here, but if you are, it doesn't really apply to HBM in the same way, especially in a smartphone form factor where LPDDR is the only practical option due to heat and energy constraints, and a variety of architectural complexities specific to HBM that make a TB of it in a smart phone something far beyond what we can hope for in any timeline we can project today.

[aleqs]

Phones with HBM are already in development, not that HBM is some sort of strict requirement.

[JoeAltmaier]

capability has been advancing faster than raw FLOPS

[bigyabai]

> I personally do not think it will take even 10 years for the situation to be commonplace.

Do you personally remember how far smartphones progressed in the past 10 years? It's not as long a time as you think it is, the limits of what a smartphone GPU is capable of did not substantially change in that time. Nor did the amount of onboard RAM that we include in the package. This is true even for Nvidia's ARM SOCs, frankly.

Apple, Microsoft and Google all eventually want to enforce OS-level lock-in for the most profitable AI services (eg. their own). It's much more attainable and profitable to use that lock-in to sell you exclusive service integration, the local AI revolution probably won't begin on their hardware.

[Danox]

It won’t take 10 years, 3 years maybe 4 years, depends on the next two hardware generations that and whether or not the current memory fiasco is solved.

[fragmede]

Only if you narrowly define AI as talking with a chatbot. Today, on an iPhone, there are a number of features that only work due to some sort of a model running locally. OCR and intelligent object selection in a photos, and summarization of texts are local models running on the iPhone hardware, they're just not a chatbot. Yes, there's also cloud backing various features, but the idea of running models locally isn't foreign to Apple. On Google's side, the Pixel 10 Pro is powerful enough to run quantized local chatbot models locally today. Local translate is a model, and has been for a while. Both corporations are going to sell whatever customers are willing to pay for, in money or via ads, and if local models get good enough that customers actually are willing to pay for it, I have no doubt that it's Apple and Google will go that direction. It's Google that's releasing Gemma models for download, and they're a big enough organization that the left hand doesn't know what the right hand is doing, so the one conspiracy theory that they'll never do local models because they only want to profit off hosted models is too simplistic.