[rock_artist]

While I'm happy with Apple introducing this abstraction. my main concern was with local models.

I'd love using Gemma4 as an example. but thinking of a user. if 10 Apps each uses same model and downloads it, the phone will be bloated.

I still didn't understand if Apple provided a way for multiple apps uses same on-device model (without tricky namespaces and permissions).

I didn't see anything suggesting that's the case.

[scosman]

I think that's what they are trying to avoid. If you need on-device intelligence, their pitch was "The model the device already has is best", and if you need something more specific an adapter (aka, a fine-tune/lora) is best.

They were wrong when their on-device model was way behind. They still might be right in the long term.

While multiple app I use might need Gemma 4 E4B, I use dozens of apps and app devs can choose from hundreds of models. A shared cache might reduce size a little when there's overlap, but the core problem still exists. If each app chooses a model disk and memory-swapping explode.

Its probably be better for device manufacturers to bake in a default. I'm not proposing they limit you from using others, but one shared default might be best developer/user experience for 99% of apps.

- Being warm in memory is the single biggest perf speedup you can get, and a default is much more likely to be warm.

- "Best model" is usually "best model for this device" given both RAM and compute. A developer can't test every device but Apple can/will.

- Each model needs to be optimized for the hardware (what's running on ANE, what's running on Metal, what's running on CPU). The default gets optimized.

- If you need custom model, a Lora is probably best (30MB, benefits from all of the above)

You could say the default should be swappable, but that's more a linux ideal than an Apple one so I doubt we ever see that. Plus there are real downsides: intentional or not, prompts end up optimized to the model they are developed for, so swapping the default system model would degrade every app.

[scotty79]

But models aren't universally best, especially small ones. For text Gemma is great. For vision qwen3.6 is amazing.

[scosman]

I think the point is: if every app chooses the best model for their use case, my phone is hosed (disk, memory swap, memory). A good-enough default might be better for the user than the each app having the best possible.

[jtfrench]

That's a great opportunity for Apple to provide a universal unique model ID protocol and some shared storage space to allow devs to register models.

[alwillis]

Check out “Bring an LLM provider to the Foundation Models framework” - https://developer.apple.com/videos/play/wwdc2026/339

[rock_artist]

I see an id based ability suggesting `modelId`. but in current docs I cannot find any context to it. The other limit is that it suggests Swift Packages. but I'm not seeing any model management hints similar to Docker/Ollama/etc where:

- Application can ask for specific model, if available use it. if not, ask to download it (or try some fallback / alternative)

- User can manage models. So as a user I can clean unused models (and for non-techie have something similar to offloading apps when unused for some period of time).

[klausa]

The apps can use the system provided on-device model using the same framework and APIs; but there's no affordances to deduplicate custom models between apps.

[satvikpendem]

That is exactly what foundation models are, yes. Same in Android with AICore which uses Gemma underneath, apps can query the LLM and receive responses back rather than bundling in their own model.

[trvz]

Do you guys not have phones (with at least 1TB of storage)?

[rock_artist]

Who’s “you guys” a developer from Bay Area? A student with a MacBook Neo? Or John Appleseed who bought basic iPhone 17e?

[JBiserkov]

https://knowyourmeme.com/memes/do-you-guys-not-have-phones

[exitb]

I'm guessing it's a BlizzCon joke.

[mft_]

I have a Mac with 4TB of storage but it’s still annoying when every new AI app I try installs its own virtual environment with a fresh copy of Python, PyTorch, other duplicate libraries, and then models on top of that.

[DrScientist]

As an occasional python user I'm always amazed and frustrated that it seems that the only way to be able to use/build anything is to create a whole separate environment.

And now given everybody now does this I guess the incentive to stop breaking stuff reduces even further.

Might as well have static binaries.

[simondotau]

The meme phrase “it’s fractally wrong” applies to the entire python ecosystem, IMHO. Virtual environments are just another layer of this fractal wrongness in the layer cake of ecosystem awfulness.

It’s a nice language though.

[kstrauser]

That’s exactly how NPM works, and how Cargo works by default. You can make npm install stuff globally, but that’s not recommended except for things like CLI tooling. Cargo builds every project in its own separate targets/ directory unless you manually configure it to share that dir between builds. In both cases, the default is to isolate your current project from everything else on the system.

The main difference is that Python use to make you have to know that the virtualenv existed. Now `uv run` and `poetry run` abstract that away so you don’t have to interact with it if you don’t want to.

[whstl]

I have a couple small apps that have a (non-LLM) model, and originally the models and code were in PyTorch, built by Python devs.

The original plan was to ship Python. However I found out I can migrate them to CoreML, and now it's a model file + Swift code. I got some massive performance improvements as well.

Of course, this doesn't work at all for non-Mac environments, but it was nice to be able to do it. (Also doesn't solve the duplicate large models problem)

[hedora]

It’d be nice if there was a standard like ~/.local/llm/hugging-face-name.gguf or something.

Python heaviness is a more fundamental problem.

[ac29]

If you use uv, python apps use a shared cache which helps a lot.

[fragmede]

No? iPhones don't come standard with that much storage.

[JBiserkov]

https://knowyourmeme.com/memes/do-you-guys-not-have-phones

[mohamedkoubaa]

Ok but don't expect Anthropic to help with local models, that'll be something apple rolls out themselves if at all

[taneq]

Sounds ripe for block-level deduplication. :D Or an API that lets you request a model and handles caching.