did miss it until now, cool to see it on device and first party. as soon as it lands I will see the impact on apfel.
but i definitely feel flattered, either my little project inspired them or that I reached the same conclusion at a similar time as a team at apple that "hey, this is totally missing"
but only when it is actually available we will see if it's a clean drop-in vs. just "chat-completions-ish".
one of my learnings from apfel is that is is very easy to get a kinda openAI api compatible server, and a lot of work to get it really totally compatible. sometimes i wonder if even the openai implementation of openai's api is openai api compatible to the core....
> sometimes i wonder if even the openai implementation of openai's api is openai api compatible to the core….
It's a similar situation with "Arca-Swiss compatible" tripod plates in photography. There is really no such thing — Arca-Swiss didn't make a standard, so they didn't have to stick to it themselves, and while most things using this "standard" fix to most things, some things just won't fit, or won't stay put. Everyone implements it, and if they don't, people complain "why didn't you just put an Arca standard foot on it?" and then you have to sit them down and tell them.
They've said there are limits, and increased limits for those on iCloud+ ... so it seems that Apple is in the selling LLM access game now. I don't think there are any details yet on the nature of those limits, and whether they can be increased as required etc.
Agreed. The idea of a system wide (and platform wide) on device model being a core part of OS APIs is very appealing. I do like my software more piecemeal, generally, but when it comes to Apple, I really love a lot of the out-of-the-box offerings they have. Just giving software access to something they know exists on these platforms and can use for various small (and likely increasingly large) gen AI tasks is so appealing.
Thanks apfel looks useful! I have been experimenting with Apple's foundation models for almost a year and they are useful for embedded applications. I have been taking a deeper dive into local agentic coding tools (starting with 'little-coder --model ollama/gemma4:12b-it-qat') and I put together a tiny free book with some setup advice that might save people a few minutes of setup time: https://leanpub.com/read/local-coding-agents
I have been fairly much pissed off at the "hype in hyperscaler" AI growth (data center environmental and other societal costs) and I support anything we can do to promote local and private AI.
Thanks for building this! Something I grab on a regular basis, especially for doing simple education of folks about the basics of using LLMs by showing something that's not just a chatbot.
Are you surprised they apparently didn’t adopt your idea and add an OpenAPI compatible endpoint in Core AI, even if just as a testing tool? I am.
I also really want to hear more about their containerisation/seatbelt strategy now that they are offering MCP support. Not seen any news about Darwin inside their containers system.
(Apfel is a cool project; it’s been the only thing tempting me to upgrade to Tahoe)
Here's what you get when you run it... https://gist.github.com/robgough/7893602895e7580117475076198...