[maeil]

> i don’t think they need to win the on device market.

The second Apple comes out with strong on-device AI - and it very much looks like they will - Google will have to respond on Android. They can't just sit and pray that e.g. Samsung makes a competitive chip for this purpose.

[SimianSci]

I think Apple is uniquely disadvantaged in the AI race to a point people dont realize. They have less training data to use, having famously been focused on privacy for its users and thus having no particular advantage in this space due to not having customer data to train on. They have little to no cloud business, and while they operate a couple of services for their users, they do not have the infrastructure scale to compete with hyperscaler cloud vendors such as Google and Microsoft. Most of what they would need to spend on training new models would require that they hand over lots of money to the very companies that already have their own models, supercharging their competition.

While there is a chance that Apple might come out with a very sophisticate on-device model. The problem here is that they would only be able to compete with other on-device models. The magnitude of compute needed to keep pace with SOA models is not achievable on a single device. It will take many generations of Apple silicon in order to compete with the compute of existing datacenters.

Google also already has competitive silicon in this space with the Tensor series processors, which are being fabbed at Samsung plants today. There is no sitting and praying necessary on their part as they already compete.

Apple is a very distant competitor in the space of AI, and I see no reason to assume this will change, they are uniquely disadvantaged by several of the choices they made on their way to mobile supremacy. The only thing they currently have going for them is the development of their own ARM silicon which may give them the ability to compete with Google's TPU chips, but there is far more needed to be competitive here than the ability to avoid the Nvidia tax.

[dagmx]

There’s an easy solution here: Apple isn’t trying to compete with the big models everyone else is running. They’re betting in the opposite direction that many small models is a better value ad for their customers. And they can call out to other services as needed for the larger stuff.

I’m in the camp that this is the right call for consumers, instead of trying to compete on the large model side. They’ve yet to deliver on their full promise, but if they can, it’s the place where I think more of the industry will go (for consumers)

And regarding Google’s mobile tensor chips, they are infamously behind all other players in the market space for the same generation of processor. They don’t share the same advantages they do in the server space.

[whimsicalism]

training bigger models gets you small models for free plus a higher upper bound in capabilities.

Apple just isn’t very capable in this space, not sure what’s so hard to accept

[mike_hearn]

Apple have trained their own foundation LLM.

[whimsicalism]

hardly even qualifies for ‘fast follow’, more like ‘surprisingly slow follow’

their models aren’t even that good. sorry apple fanboys but the talent isn’t there

[simonw]

"having famously been focused on privacy for its users and thus having no particular advantage in this space due to not having customer data to train on"

That may not be as big a disadvantage as you think.

Anthropic claim that they did not use any data from their users when they trained Claude 3.5 Sonnet.

[whimsicalism]

sure but they certainly acquired data from mass scraping (including of data produced by their users) and/or data brokering aka paying someone to do the same.

[mark_l_watson]

It is likely Apple can get additional data by creating synthetic data for user interactions.

About 7 years ago I trained GAN models to generate synthetic data, and it worked so well. The state of the art has increased a lot in 7 years, so Apple will be fine.

[SimianSci]

For a while there I would have been in agreeance with you, but the thought that models can be trained purely on synthetic data has shown to be wrong on multiple levels. Synthetic data needs to be reviewed by individuals to ensure data quality, significantly reducing the speed at which an organization can adopt training data. Reasonable engineers would suggest that the answer to this is to have other language models review the synthetic data, but we have seen that this is what leads to model collapse due to compounding issues around hallucinations.

At best Synthetic data is a "slow follow" for training a model due to the need for human review, but a competitive model, it does not make.

[whimsicalism]

yeah i’ve never understood the outsized optimism for apple’s ai strategy, especially on hn.

they’re a little bit less of a nobody than they used to be, but they’re basically a nobody when it comes to frontier research/scaling. and the best model matters way more than on-device which can always just be distilled later and find some random startup/chipco to do inference

[msabalau]

Theory: Apple's lifestyle branding is quite important to the identity of many in the community here. I mean, look at the buy-in at launch for Apple Vision Pro by so many people on HN--it made actual Apple communities and publications look like jaded skeptics.

[dagmx]

Oh please, this is the classic “everyone who chooses differently than myself is <superficial/dumb/misinformed>” argument that a lot of people use when it comes to tech nerd identity politics.

Is it really that hard to imagine people have different viewpoints, and decisions than yourself without being painted as vapid, airheads?

[whimsicalism]

I work in this industry, been working professionally on transformers since 2018.

The level of optimism for Apple AI capabilities on here is wrong. I can imagine people having wrong viewpoints, but it is wrong.

[maeil]

For clarity, I was only talking about the hardware side, not the software one. I don't think the models matter too much, by the time the hardware is ready there will be open models that Apple can take and modify to their liking.

Besides, did Anthropic and e.g. Mistral inherently have such troves of data to train on that Apple doesn't? For the last 6 months, Anthropic has had the SOTA model for the average production usecase.

> Google also already has competitive silicon in this space with the Tensor series processors, which are being fabbed at Samsung plants today. There is no sitting and praying necessary on their part as they already compete.

Intel had a much bigger advantage with x86, and look where we are now. I find it hard to believe that creating a good AI chip isn't a much smaller challenge than it was to do Apple Silicon. The upcoming SE uses their in-house 5G modem, another huge hardware achievement that no one else has been able to do.

With that in mind, how can you bet against Apple when it comes to designing chips at this point? It's not like Amazon et al aren't producing their own AI chips too. Let alone all of the startups like Cerebras. That indicates the moat and barriers are likely much lower than Apple Slicion or the 5G modem.

If I'm talking nonsense, do correct me.

[reportingsjr]

The Android on chip AI is and has been leagues better than what is available on iOS.

If anything, I think the upcoming iOS AI update will bring them to a similar level as android/google.

[petra]

But given inference time compute, to give a strong reply reasonably fast, you'll need a lot of compute, very rarely used.

Economically this fits the cloud much better.