[AaronAPU]

That’s too strong of an assertion.

Local models aren’t deterministically equivalent in capabilities to foundation models. Home computers are turing complete; just like a mainframe. They are just slower. Often not slower enough to matter.

[sandworm101]

Most people are ok with slower. An AI that lets you edit a family picture, in say 30 seconds, locally is preferable to one that is instantaneous but requires you to submit that picture to examination/storage/training/sale in someone else's AI ecosystem. If i want to crop my ex out of family photos, i should not have to first give that photo to Microsoft. If want an LLM to write a book report for me, i dont want it also alerting my school. And if i write a memo for a client, and i want an LLM to check the spelling, i dont want that memo leaked either.

[Pxtl]

I'd like to think so but the existence of Google and Apple and Microsoft's cloud based photo tools with phone integration suggests that's false.

You could run a pretty good home server on $50 of gear and yet we never saw any real adoption of OwnCloud/NextCloud style products as an alternative to Google Drive/Photos or Apple Cloud.

Why should LLM/Transformers be any different? Especially when you need a proper expensive GPU to run them instead of a Raspberry Pi?

[thewebguyd]

Apple's photo tools run on device, and they'll probably ship more on device foundation models at WWDC too.

On-device AI is going to be important, I think. It doesn't have to take the form of a chatbot UI to be useful.

[com2kid]

After the latest round of cloud storage price increases my non technical wife has been asking if we can do local backups instead...

[parineum]

> Most people are ok with slower. An AI that lets you edit a family picture, in say 30 seconds, locally is preferable to one that is instantaneous but requires you to submit that picture to examination/storage/training/sale in someone else's AI ecosystem.

Maybe if you ask them that question, but if you show them two products, they'll definitely prefer the faster one. 30 seconds is a long time to watch a progress bar.

[spwa4]

Plus there's the other question. If this thing is slower ... what's the price? The desktop/mini-pc version of this is $3000, after all. At this performance level what is an acceptable price for the laptops?

People definitely aren't going to accept more expensive + slower ...

[sandworm101]

Fast and public, or slow and private. Not everyone wants, or is allowed to, share their data with the AI world. And do not doubt that every bit shared with an AI service will be used for training.

[parineum]

The question here is about markets though. Not everyone wants x but if the vast majority of people want y, x is going to be niche and expensive.

You don't think the commercials of Google's AI photo features aren't going to have an impact on Apple users of their phones can do a worse version of that feature and it takes longer?

[robotresearcher]

It’s completely technically possible to have cloud services where customer data is opaque to the provider. Some of Apple’s services are like this already, for example.

I think there’s a sweet spot currently with munging your data blindly on the server so that your client device battery still lasts all day.

Meanwhile Apple and others push on with making client side models more efficient so that eventually the server costs and complexities go away.

[fg137]

This.

If asked to choose between photo editing done within 3s using cloud provider vs an average of 30s using local compute, most consumers will choose the former without hesitation.

Most users' usage is also going to fall nicely in the free tier of a typical freemium pricing model, like ChatGPT today.

People who talk endlessly about local inference have no idea about user workflows and usability.

[dominotw]

dont want to share my pics with "cloud services"

[wolvesechoes]

You may not, but experience shows that most people are just fine sharing the most personal stuff not only with cloud services, but with hole world through anti-social media.