[repeekad]

Meta is selling their now excess compute, other compute has been on the market for a while. The current hardware cost bubble is temporary, especially once people are forced to pay the real inference price instead of majorly subsidized subscriptions.

[program_whiz]

yep, all those coders paying $200-$500 per day to use claude once subsidy ends will be seriously rethinking how much they really want to vibe-code "rewriting X in rust". Helping people write word docs, recipes, and emails isn't going to justify $15K per month subscriptions either.

[apitman]

> Helping people write word docs, recipes, and emails isn't going to justify $15K per month subscriptions either.

Those things can all be done today on a $250 used video card and pennies of electricity

[root_axis]

Not really. Models that can run on a $250 card may be able to produce a decent word doc sometimes, but the fidelity of such models is so poor that it's no longer very attractive for general use on those types of tasks.

[andrekandre]

i might have agreed with you 4 months ago, but today with open source models i am surprised as anyone with the progress compared to frontier... so much so that i also picked up one of those $250 cards!

[apitman]

Have you tried qwen3.6-27b or Gemma 4? gemma4-26b-a4b is particularly impressive for its balance of speed and performance

[gf000]

Then I buy a very expensive GPU rack and timeshare it with a couple of friends.

[km3r]

The subscription is certainly subsidized but its no where near 100x cheaper than API prices.

Heck, most large enterprise moved to usage based billing and are still happily paying for it. They are force multipliers for your top talent, and when a top engineer is being paid $500k a year, doubling their output for $500/day is a no brainer.

[mlrtime]

Yes, it's close to 100x and it's easy to calculate this.

Take a $10 claude sub and fill it with Fable/Opus for a month (meaning use all your tokens) and then use one of the many token/session recording tools (I like agentsview).

It will show you how much your token cost is at current rates if using an api. I use the $20 plan and I don't use all my tokens, and am around 50x.

[bigyabai]

Lowering the cost of hardware still won't solve the issue. HBM and DDR5 was never cheap, even before the shortages, so selling a full inference system is beyond the acceptable price range for most casual customers.

We're going to see Apple and Google compete over services and AI/OS integration instead, it will probably be years before your OEM takes local models seriously.

[tanseydavid]

Apple and Google (via smartphones) are in literally everyone's pocket.

Running KIMI on a phone is not possible today and I agree with you that it will "probably be years before..." it is.

But how many years do you guess? I personally do not think it will take even 10 years for the situation to be commonplace.

[root_axis]

> But how many years do you guess? I personally do not think it will take even 10 years for the situation to be commonplace.

IMO it won't be possible for the foreseeable future. There's essentially zero possibility that phones will gain the hardware capacity to run today's Kimi, so the only other alternative is to squeeze the power of today's Kimi into something that can fit on a smartphone, which also seems fairly unlikely considering the current rate of progress.

[JoeAltmaier]

A bold prediction. Phones have gigabytes today. There are famous laws of growth that put terabytes at just a few years away - perhaps 10 isn't too bad an estimate?

[root_axis]

> Phones have gigabytes today.

Phones have zero HBM today.

> There are famous laws of growth that put terabytes at just a few years away

I assume you're referring to Moore's law here, but if you are, it doesn't really apply to HBM in the same way, especially in a smartphone form factor where LPDDR is the only practical option due to heat and energy constraints, and a variety of architectural complexities specific to HBM that make a TB of it in a smart phone something far beyond what we can hope for in any timeline we can project today.

[aleqs]

Phones with HBM are already in development, not that HBM is some sort of strict requirement.

[JoeAltmaier]

capability has been advancing faster than raw FLOPS

[bigyabai]

> I personally do not think it will take even 10 years for the situation to be commonplace.

Do you personally remember how far smartphones progressed in the past 10 years? It's not as long a time as you think it is, the limits of what a smartphone GPU is capable of did not substantially change in that time. Nor did the amount of onboard RAM that we include in the package. This is true even for Nvidia's ARM SOCs, frankly.

Apple, Microsoft and Google all eventually want to enforce OS-level lock-in for the most profitable AI services (eg. their own). It's much more attainable and profitable to use that lock-in to sell you exclusive service integration, the local AI revolution probably won't begin on their hardware.

[Danox]

It won’t take 10 years, 3 years maybe 4 years, depends on the next two hardware generations that and whether or not the current memory fiasco is solved.

[fragmede]

Only if you narrowly define AI as talking with a chatbot. Today, on an iPhone, there are a number of features that only work due to some sort of a model running locally. OCR and intelligent object selection in a photos, and summarization of texts are local models running on the iPhone hardware, they're just not a chatbot. Yes, there's also cloud backing various features, but the idea of running models locally isn't foreign to Apple. On Google's side, the Pixel 10 Pro is powerful enough to run quantized local chatbot models locally today. Local translate is a model, and has been for a while. Both corporations are going to sell whatever customers are willing to pay for, in money or via ads, and if local models get good enough that customers actually are willing to pay for it, I have no doubt that it's Apple and Google will go that direction. It's Google that's releasing Gemma models for download, and they're a big enough organization that the left hand doesn't know what the right hand is doing, so the one conspiracy theory that they'll never do local models because they only want to profit off hosted models is too simplistic.