[rishabhjain1198]

For people interested in AI research, there's nothing new here.

IMO they should do a better job of referencing existing papers and techniques. The way they wrote about "adaptors" can make it seem like it's something novel, but it's actually just re-iterating vanilla LoRA. It was enough to convince one of the top-voted HackerNews comments that this was a "huge development".

Benchmarks are nice though.

[lolinder]

> For people interested in AI research, there's nothing new here.

Was anyone expecting anything new?

Apple has never been big on living at the cutting edge of technology exploring spaces that no one has explored before—from laptops to the iPhone to iPads to watches, every success they've had has come from taking tech that was already prototyped by many other companies and smoothing out the usability kinks to get it ready for the mainstream. Why would deep learning be different?

[csvm]

Prototyping tech is one thing; making it a widely adopted success is another. For instance, Apple was the first to bring WiFi to laptops in 1999. Everyone laughed at them at the time. Who needs a wireless network when you can have a physical LAN, ey?

[steve1977]

> For instance, Apple was the first to bring WiFi to laptops in 1999. Everyone laughed at them at the time. Who needs a wireless network when you can have a physical LAN, ey?

From https://en.wikipedia.org/wiki/AirPort:

"AirPort 802.11b card"

"The original model, known as simply AirPort card, was a re-branded Lucent WaveLAN/Orinoco Gold PC card, in a modified housing that lacked the integrated antenna."

[gumby]

That was also how lucent’s access points worked.

[collinmanderson]

It was pretty neat how you could take the PCMCIA card out of an AirPort or WaveLAN/Orinoco and stick it in a laptop and it would just work.

[Matl]

On the other hand, people who laughed at them removing the 3.5mm jack can still safely laugh away.

[throw0101d]

> On the other hand, people who laughed at them removing the 3.5mm jack can still safely laugh away.

Then laugh at Samsung and their flagship line of phones as well, since they haven't had headphone jacks for a while now. "After Note 10 dumps headphone jack, Samsung ads mocking iPhone dongles disappear" (2019):

* https://www.cnet.com/tech/mobile/after-removing-headphone-ja...

"Samsung is hiding its ads that made fun of Apple's removal of headphone jack":

* https://www.androidauthority.com/samsung-headphone-jack-ads-...

[tivert]

>> On the other hand, people who laughed at them removing the 3.5mm jack can still safely laugh away.

> Then laugh at Samsung and their flagship line of phones as well, since they haven't had headphone jacks for a while now. "After Note 10 dumps headphone jack, Samsung ads mocking iPhone dongles disappear" (2019):

I totally do. One of the problems with Apple is the industry seems to mindlessly ape their good and bad decisions. Their marketing has been so good, many people just assume whatever they do must be the best way.

[barbecue_sauce]

At the time I felt like Apple was getting rid of the 3.5mm jack as a potential bottleneck for future iPhone designs (as one of the limiting aspects of form factor), but there still doesn't seem to be anything design-wise to justify it, even several years later. It is very clear now that it was merely to encourage Air Pod adoption.

[monooso]

> One of the problems with Apple is the industry seems to mindlessly ape their good and bad decisions...

That's not a problem with Apple.

[deanishe]

> One of the problems with Apple is the industry seems to mindlessly ape their good and bad decisions.

That's hardly Apple's fault, but it is so annoying. Hardly any of my cables have proper reinforcement sleeves any more.

Apple started making awful cables that snapped at the plugs, and everybody else just copied them.

[skeaker]

We absolutely do laugh at both already.

[krick]

I would gladly laugh, but it's nearly impossible to buy a good phone now. TBH I don't care that much about my phone not having a 3.5mm (even if I would need to use wired headphones, which is very rare now, I can use an USB adapter for that), but there are basically no phones without this stupid hole in the display, or with a good dedicated (not under screen) fingerprint scanner (because who needs that, when you can have face recognition, right?) All top-line phones are like $1500 now, but still are considered like disposable products that are naturally expected to be changed every 2 years. Batteries are not removable, yet devices are not actually (reliably) waterproof.

And maybe I'm wrong, but somehow it feels each improvement like that was actually pioneered by Apple. In the dreamworld of free-market enthusiasts this should have made Apple bankrupt or iPhone a very niche consumer device, but in the real world everything just became iPhone. There are some rare exceptions, but these are either outright experimental and gimmicky (because being different is their identity), or just bottom-of-the-line products that have these "intentional defects" that should make you chose the more expensive option.

[joshstrange]

This is such a tired talking point. Use a (lightning|USB-C)->3.5mm adapter or use bluetooth.

[astrange]

And your experience for phone use cases will be better, because walking with wired headphones in gives you nasty telephonic effects (sound transmission along the cable) and they get tangled up.

[bigstrat2003]

The two aren't mutually exclusive. Those who wish to use Bluetooth headphones can happily use them, while those who prefer wired can continue to use them. There's no reason smartphone manufacturers shouldn't support both.

[jb1991]

Interesting that you suggest laughing at their decision to remove the headphone jack, when it was actually just the first of an industry-wide shift that has done so by other companies.

[bildung]

Was that really the case? I remember they were mocked for e.g. offering wifi only, firewire only etc., while the respective removed alternatives were way more common.

[blihp]

In the consumer space at least, WiFi was nowhere to be seen on a typical PC when Apple adopted it. Same with USB. So while it technically originated and existed elsewhere, there was no serious traction on it prior to Apple adoption.

What you say is also true: many people weren't ready to ditch the old when Apple decide to deprecate it.

[josephg]

This has been true for ages. They were the first to ditch the floppy disk drive and later cd drive in their computers. Both choices were very controversial at the time.

[paulmd]

They were also the first to usb-c nirvana - they were shipping laptops with thunderbolt in 2011, and moved to usb-c in 2016, giving you four full-capability outputs at a time when most laptops had one, sometimes, and it took another 5 years before most laptops adopted at least one as standard and premium laptops sometimes had two.

(People look down on the move to usb-c which I don’t quite get, everyone seems to fawn over usb-c in other contexts but macbooks, amirite!?!?. Yes it’s nice to have a hdmi port but fundamentally if you buy into this vision that usb-c does everything but you also want to use a bunch of legacy ports (vs thunderbolt video, thunderbolt networking, etc) then obviously you’re going to have to have dongles, and people supposedly buy into that vision in other contexts. Apples implementation of that vision was fundamentally at least a decade ahead of the curve there, if you’re going to do that you want lots of ports and you want every port to do everything, not “this one is the only one that can charge fast”, “that one doesn’t have video output”, “if you run both ports they drop to some weird lower capability because you’re dividing the controller”. Those complaints are the things people don’t like about the base-tier M-series processors today, and apple’s previous models solved that problem long before anyone else did.)

Hell until very very recently a lot of the time the competition didn’t even have thunderbolt/Pcie tunnelling… you got 10gbps usb-c and a grab-bag of charging and display features, and you’re gonna like it. That’s still the case with motherboards and it’s literally only with this years’ release that we’re finally getting usb4/thunderbolt as standard on high-end boards. Literally more than a decade from when apple started putting thunderbolt on laptops, almost a decade from the era of 4x tb3 full-spec ports.

… and reminder that in classic usb fashion, usb4 still doesn’t even guarantee Pcie tunneling support. So really it can still be just a normal usb-c 10gbps connection in a silly mustache and trench coat. Even on the next-gen stuff. What’s the term for doing an ok, moderately competent but not even exceptionally good job while your competition repeatedly shoots themselves in the face, again? But it’s by design - the intent is deceiving and manipulating the customer into buying last year’s junk, it’s working as intended for USB-IF’s real customers and stakeholders.

https://en.wikipedia.org/wiki/USB4#cite_ref-auto2_15-0

[IOT_Apprentice]

Apple was first with 64 bit iPhone chips. Remember Qualcomm VP at the time claimed it was nothing. Apple Silicon for M1 was impressive for instant in low power high performance.

[lolinder]

Those are both still (major) incremental improvements to known tech, not cutting-edge research. Apple takes what other companies have already done and does it better.

[sunshinerag]

all cutting-edge research other companies are supposedly doing are also incremental. Depends on your vantage point.

[prmoustache]

But last at bringing a calculator on the iPad =)

[viscanti]

No one has ever brought a native (not 3rd party) calculator to the iPad before. Apple is the first.

[froggit]

> No one has ever brought a native (not 3rd party) calculator to the iPad before. Apple is the first.

I'm not sure I understand... Considering apple makes the ipad, wouldn't all ipad calculators other than apple's be 3rd party by definition?

[jeanlucas]

> For people interested in AI research

I think he is pointing out for people interested in research.

OTOH, it is interesting to see how a company is applying AI to customers at the end. It will bring up new challenges that will be interesting from at least an engineering point of view.

[rmbyrro]

I think you misinterpreted OP's comment. Apple makes it sound like there's smth new, but there isn't. They don't have to innovate, but it's good practice to credit who've done what they're taking and using. Also to use the names everyone else is already using.

[marci]

The strange thing is Apple did mention (twice) in the article that their adapters are loras so I don't understand OP's comment.

[aceazzameen]

I gathered from OP's "huge development" comment he was talking about others people's popular perception that it wasn't a lora.

[steve1977]

> Also to use the names everyone else is already using.

That would be a very un-Apple thing to do. They really like to use their own marketing terms for technologies. It's not ARM, it's Apple Silicon. It wasn't Wi-Fi, it was AirPort. etc. etc.

[gumby]

> It wasn't Wi-Fi, it was AirPort. etc. etc.

FWIW the term “airport” predated the name “wifi” — in those days you had to otherwise call it IEEE 802.11.

And the name as great: people were buying them like crazy and hiding them in the drop ceiling to get around the corporate IT department. A nice echo of how analysts would buy their own apple II + visicalc to…get around corporate IT.

I’m OK with Apple using “apple silicon” as the ARM is only part of it.

Just commenting on your two examples; in general I agree with your point.

[steve1977]

As far as I know, both the AirPort trademark and the term Wi-Fi got introduced in 1999 (could be that AirPort was a couple of weeks earlier)

[gumby]

Airport came out in the beginning of the year, maybe January, at Macworld. WECA renamed themselves the WiFi Alliance a few years later, but the name (and trademark) had to exist for it to be worth them doing so.

WECA reminds me of other "memorable" names of the era, my favorite being PCMCIA, though VESA is another fave.

[astrange]

> It wasn't Wi-Fi, it was AirPort.

Except in Japan, where it's AirMac. And China, where it's WWAN not Wi-Fi.

[Tijdreiziger]

See also: FireWire, iSight, Retina, FaceTime, etc.

[KerrAvon]

None of these really fit the pattern. Apple invented FireWire, called it FireWire, and other companies chose to call it different things in their implementations (partly because Apple originally charged for licensing the name, IIRC). iSight is an Apple product. FaceTime is an Apple product. Retina is branding for high-resolution displays beyond a certain visual density.

[steve1977]

"Apple invented FireWire" is maybe not fully accurate (but actually a good example of the point here).

Wikipedia: FireWire is Apple's name for the IEEE 1394 High Speed Serial Bus. Its development was initiated by Apple[1] in 1986,[3] and developed by the IEEE P1394 Working Group, largely driven by contributions from Sony (102 patents), Apple (58 patents), Panasonic (46 patents), and Philips (43 patents), in addition to contributions made by engineers from LG Electronics, Toshiba, Hitachi, Canon,[4] INMOS/SGS Thomson (now STMicroelectronics),[5] and Texas Instruments.

What might be interesting in this regard is that Sony was also using its own trademark for it: "i.LINK".

[caseyy]

> Apple has never been big on living at the cutting edge of technology

There was such a time. Same as with Google. Interestingly, around 2015-2016 both companies significantly shifted to iterative products from big innovations. It's more visible with Google than Apple, but here's both.

Apple:

- Final Cut Pro

- 1998: iMac

- 1999: iBook G3 (father of all MacBooks)

- 2000: Power Mac G4 Cube (the early grandparent of the Mac Mini form factor), Mac OS X

- 2001: iPod, iTunes

- 2002: Xserve (rackable servers)

- 2003: Iterative products only

- 2004: iWork Suite, Garage Band

- 2005: iPod Nano, Mac mini

- 2006: Intel Macs, Boot Camp

- 2007: iPhone and Apple TV

- 2008: MacBook Air, iPhone 3G

- 2009: iPhone 3Gs, all-in-one iMac

- 2010: iPad, iPhone 4

- 2011: Final Cut Pro X

- 2012: Retina displays, iBooks Author

- 2013: iWork for iCloud

- 2014: Swift

- 2015: Apple Watch, Apple Music

- 2016: Iterative products only

- 2017: Iterative products mainly, plus ARKit

- 2018: Iterative products only

- 2019: Apple TV +, Apple Arcade

- 2020: M1

- 2021: Iterative products only

- 2022: Iterative products only

- 2023: Apple Vision Pro

Google:

- 1998: Google Search

- 2000: AdWords (this is where it all started going wrong, lol)

- 2001: Google Images Search

- 2002: Google News

- 2003: Google AdSense

- 2004: Gmail, Google Books, Google Scholar

- 2005: Google Maps, Google Earth, Google Talk, Google Reader

- 2006: Google Calendar, Google Docs, Google Sheets, YouTube bought this year

- 2007: Street View, G Suite

- 2008: Google Chrome, Android 1.0

- 2009: Google Voice, Google Wave (early Docs if I recall correctly)

- 2010: Google Nexus One, Google TV

- 2012: Google Drive

- 2013: Chromecast

- 2014: Android Wear, Android Auto, Google Cardboard, Nexus 6, Google Fit

- 2015: Google Photos

- 2016: Google Assistant, Google Home

- 2017: Mainly iterative products only, Google Lens announced but it never rolled out really

- 2018: Iterative products only

- 2019: Iterative products only

- 2020: Iterative products only, and some rebrands (Talk->Chat, etc)

- 2021: Iterative products only, and Tensor Chip

- 2022: Iterative products only

- 2023: Iterative products only, and Bard (half-baked).

[vile_wretch]

Some of your choices and what you consider iterative/innovative are strange to me. For 2009, a chassis update for the iMac and a spec/camera bump for the iPhone doesn't seem particularly innovative especially in comparison to say the HomePod in 2017 or Satellite SOS in 2022.

Also small correction but iTunes (as Soundjam MP) was originally third-party software and Final Cut was acquired by Apple.

[caseyy]

Yes, it's not perfect.

About iTunes: I did not know that! Thank you.

About iterative/innovative: I considered hardware and software that became household names or general knowledge to be significant innovations. It is not rigorous, I tried to include more rather than less. Still, on some years these companies mostly did version increases for their hardware and software, like new iOS and macOS versions, and that was it. Those years I marked as iterative.

I included a few too many iPhones, although when I wrote that, my thought process was that these phones were pivotal to how iPhones developed. I should have included the original iPhone, and iPhone 3G — the first iPhone developed around the concept of an app platform and with an App Store. This has undoubtedly been a big innovation. iPhone 4 and 3Gs, perhaps, should not have been included.

It's loose and just to illustrate a general trend, individual items are less important, we could all pick slightly different ones. But I believe the trend would remain.

[KerrAvon]

You're missing Apple Silicon, which has had a huge impact across the entire industry even if random soccer dad doesn't know about it -- if any one thing is responsible for Intel's marketshare collapsing in the future, the M series of processors is it.

[AshamedCaptain]

> if any one thing is responsible for Intel's marketshare collapsing in the future, the M series of processors is it.

If anything, that would be AMD. But I'd guess they're both more worried about the entire desktop & laptop market shrinking way more than anything Apple does.

[barbecue_sauce]

No Newton?

[caseyy]

Missed it, but should have been included for 1998. A very good example.

[lolinder]

The iMac refined a form factor that dated back to at least Commodore. The iBook came after decades of iteration by other companies on laptops. The Cube was just a PC with a more compressed form factor. The iPod came a few years after other commercial digital media players. Etc, etc.

Note that I'm not saying that there's anything wrong with their approach or that they didn't make real improvements. I'm just saying that Apple has never produced any successful product that would count as "new" to someone interested in cutting-edge research. They've always looked around at things that exist but aren't yet huge successes and given them the final push to the mainstream.

[caseyy]

It depends on the definition of "new". With some definitions, we may claim that nothing is ever new — we may say computers started with the Antikythera mechanism and abaci, or maybe before. With other definitions (like "new" as in a "new for most people") we will see that Apple has brought about many new things. So we need to agree on the definition.

I used the definition of new somewhere between "new for most people", "newly popular", and "meaningfully advanced from the previous iteration". With such a definition, I think you can agree with me.

[KerrAvon]

In the consumer space, I'm not sure I can think of any examples from anyone ever that are examples of cutting-edge research at the time. It's hard to build consumer products on the bleeding edge. You'd be releasing phones today using CHERI, for example, which is not quite ready for prime time.

[com2kid]

> - 2000: Power Mac G4 Cube (the early grandparent of the Mac Mini form factor), Mac OS X

The Next Cube being a very obvious inspiration here.

Mac OS X was a needed step, and entire books have been written about its creation. While it has some innovative pieces, it was very much a do or die situation for Apple, not brought on by innovation so much as the need to survive. (I'm sure BeOS fans will argue that BeOS was the real innovative OS. ;) )

> - 2001: iPod, iTunes

iPod was a, very well done, refinement of the existing MP3 device category. iTunes' innovation was the licensing deal they got with record companies, that is what really surprised everyone.

> - 2002: Xserve (rackable servers)

Not sure how this is an innovation? Rack mount servers had been around a long time.

> - 2004: iWork Suite,

Microsoft literally had a product called Microsoft Works that was originally released in 1988 and came shipped on tons of home PCs.

> - 2005: iPod Nano, Mac mini

The iPod Nano was cool, the Mac mini was a wonderful feat of engineering and cost reductions.

> - 2006: Intel Macs, Boot Camp

Necessity brought this about.

> - 2007: iPhone and Apple TV

This is a perfect example of Apple entering an existing product category and doing an amazing job of execution. Palm, Blackberry, and Microsoft were already releasing very capable smart phones, but none of them bothered polishing the product (MS and Blackberry focused on corporate sales, end user experience was not the top priority) and while Apple did push a lot of technology forward to make the iPhone (notably screen tech and using capacitive touch screens), their main innovation was realizing they could get customers to pay for a cell phone. For those who don't remember, prior to the iPhone, most customers got their cellphone for "free" from their cellular provider in return for agreeing to a 2 year contract. Apple realized if they made a really nice product, that people would buy it.

Apple also did some really cool, and now largely forgotten about, positioning here involving the iPod Touch, where the iPod Touch had access to the full App Store and became entry level "kids toy" devices that got people into the ecosystem.

Heck arguably the App Store was a larger innovation than the phone.

Fun fact: Microsoft had an App Store ready to launch for Windows Mobile (pre Windows Phone 7) but it was scrapped at the last minute because an exec thought that "no way would phone users ever pay for apps".

(When I joined MS in 2006 the source code was still laying around in the Windows Mobile source tree!)

Apple TV was arguably too early at this point in time, I'd say it didn't really take off until later generations when more streaming media was available.

But innovative? Web TV was out in the late 90s (!!) and Microsoft tried to do Media Center PC's since 2002. Heck for awhile with Xbox 360, Microsoft basically owned the "TV smart device" market segment. (and they released the Xbox One as a media streaming device and sort of forgot that it was also a games console... oops)

As with most products, Apple just did a really good job of it, but Roku has dramatically outplayed everyone else in the market by getting embedded directly into cheap TVs sold at Costco.

> - 2010: iPad, iPhone 4

iPad is/was an amazing product, and it succeeded thanks to great apps.

It was also a refinement of Tablet PCs which have been around since the late 80s/early 90s.

Apple was willing to do what Microsoft wasn't, break all back-compat and make a really good single purpose device. Microsoft's tablets (some of which are really damn nice!) were always hamstrung because Microsoft never could go all in on abandoning existing x86 software. (The closest attempt being Windows 8 RT, which managed to make the perfect set of compromises to anger everyone!)

> - 2015: Apple Watch, Apple Music

The first generation Apple Watch was... meh. Now, I say this as someone who was working on a direct competitor - I am still not sure how it has such a miserable battery life and why such a massively overpowered CPU and GPU still dropped frames.

I am not sure what is innovative about Apple Music, vs every other streaming music service.

[throwaway4good]

I thought the news of them using Apple Silicon rather than NVIDIA in their data centers was significant.

Perhaps there is still hope of a relaunch of xserve; with the widespread use of Apple computers amongst developers Apple has a real chance of challenging NVIDIA's CUDA moat.

[pjmlp]

Not at Apple's price points.

[throwaway4good]

I think NVIDIA has the highest hardware markup at the moment.

[Hugsun]

You get considerably more ML FLOPS per dollar in a 4090 than any mac. It seems like the base M2 MAX is at roughly the same price point. It does grant you more RAM.

Quadro and Tesla cards might be a different story. I would still like to see concrete FLOPS/$ numbers.

[hajile]

They don't need the entire mac. Their cost per Max chip is probably $200-300 which beats the 4090 by a massive margin and each chip can do more than a 4090 because it also has a CPU onboard.

4090 peaks out at around 550w which means they can run 5+ of their Max chips in the same power budget.

A 4090 is $2000. Apple can probably get 5 chips on a custom motherboard for that cost. They'll use the same amount of power, but get a lot more raw compute.

[Hugsun]

> Their cost per Max chip is probably $200-300 which beats the 4090 by a massive margin...

That's true. I was talking about end user pricing.

> ...each chip can do more than a 4090 because it also has a CPU onboard.

That's a strange thing to say. It has a CPU, correct. It makes the chip more versatile but for data center ML tasks it doesn't really matter. A 4090 chip also has much more ML relevant compute per chip. So apple's chips can't really "do more than a 4090" in any relevant way.

Of course apple pays less for their in house made chips vs external products. That comparison doesn't seem relevant to the context, e.g. they're not going to challenging CUDA with internal chips.

They might get more compute per watt though. My guess is that nvidias datacenter chips are competitive in that space, but that's another story.

[eek2121]

The GPU in the M-series is much slower than a 4090. 4060-4070ish performance at best, and it varies quite a bit.

[throwaway4good]

The M2 is a chip designed to be in a laptop (and it is quite powerful given its low power consumption). Presumedly they have a different chip or at least completely different configuration (RAM, network, etc.) in their data centers.

[mrweasel]

The interesting point here is that developers targeting the Mac can safely assume that the users will have a processor capable of significant AI/ML workloads. On the Windows (and Linux) side of things, there's no common platform, no assumption that the users will have an NPU or GPU capable of doing what you want. I think that's also why Microsoft was initially going for the ARM laptops, where they'd be sure that the required processing power is available.

[skohan]

There was a rumor floating around that Apple might try to enter the server chip business with an AI chip, which is an interesting concept. Apple's never really succeeded in the B2B business, but they have proven a lot of competency in the silicon space.

Even their high-end prosumer hardware could be interesting as an AI workstation given the VRAM available if the software support were better.

[esskay]

Of course you do, Apple's selling mobile SOC's not high end cards. That doesn't mean they're incapable of making them for the right application. You don't seriously think the server farms running on M4 Pro Max chips do you...

[pjmlp]

Depends on which card one is talking about.

[throwaway4good]

Maybe. It is not really obvious how much you for the AI accellerator part of their offerings. For example the chips in iPhones are quite powerful even adjusted for price. However for some cases - like the max chip in the macbooks or the extra ram - their pricing seems high - maybe even nvidia high.

[bayindirh]

I mean, even Apple can't match the markups nVidia has right now. If you break a GPU in your compute server, you wait months for a replacement, and the part is sent back if you can't replace it in five days.

Crazy times.

[c0balt]

Enterprise offerings tend to differ. You can get a replacement NVIDIA GPU via a partner, like Lenovo, in 2-3 weeks. And that's on the high side for some support contracts.

[bayindirh]

That's from HPE, for an Apollo 6500.

[grecy]

I'm wondering how my electricity they will save just from moving from Intel to Apple Silicon in their data centers.

[derefr]

I think the thing they're saying that's novel, isn't what they have (LoRAs), but where and when and how they make them.

Rather than just pre-baking static LoRAs to ship with the base model (e.g. one global "rewrite this in a friendly style" LoRA, etc), Apple seem to have chosen a bounded set of behaviors they want to implement as LoRAs — one for each "mode" they want their base model to operate in — and then set up a pipeline where each LoRA gets fine-tuned per user, and re-fine-tuned any time the data dependencies that go into the training dataset for the given LoRA (e.g. mail, contacts, browsing history, photos, etc) would change.

In other words, Apple are using their LoRAs as the state-keepers for what will end up feeling to the user like semi-online Direct Preference Optimization. (Compare/contrast: what Character.AI does with their chatbot response ratings.)

---

I'm not as sure, from what they've said here, whether they're also implying that these models are being trained in the background on-device.

It could very well be possible: training something that's only LoRA-sized, on a vertically-integrated platform optimized for low-energy ML, that sits around awake but doing nothing for 8 hours a day, might be practical. (Normally it'd require a non-quantized copy of the model, though. Maybe they'll waste even more of your iPhone's disk space by having both quantized and non-quantized copies of the model, one for fast inference and the other for dog-slow training?)

But I'm guessing they've chosen not to do this — as, even if it were practical, it would mean that any cloud-offloaded queries wouldn't have access to these models.

Instead, I'm guessing the LoRA training is triggered by the iCloud servers noticing you've pushed new data to them, and throwing a lifecycle notification into a message queue of which the LoRA training system is a consumer. The training system reduces over changes to bake out a new version of any affected training datasets; bakes out new LoRAs; and then basically dumps the resulting tensor files out into your iCloud Drive, where they end up synced to all your devices.

[Hugsun]

There is no way they would secretly train loras in the background of their user's phones. The benefits are small compared to the many potential problems. They describe some LoRA training infrastructure which is likely using the same capacity as they used to train the base models.

> ...each LoRA gets fine-tuned per user...

Apple would not implement these sophisticated user specific LoRA training techniques without mentioning them anywhere. No big player has done anything like this and Apple would want the credit for this innovation.

[wmf]

I don't think the LoRAs are fine-tuned locally at all. It sounds like they use RAG to access data.

[derefr]

Consider a feature from earlier in the keynote: the thing Notes (and Math Notes) does now where it fixes up your handwriting into a facsimile of your handwriting, with the resulting letters then acting semantically as text (snapping to a baseline grid; being reflowable; being interpretable as math equations) but still having the kind of long-distance context-dependent variations that can't be accomplished by just generating a "handwriting font" with glyph variations selected by ligature.

They didn't say that this is an "AI thing", but I can't honestly see how else you'd do it other than by fine-tuning a vision model on the user's own handwriting.

[Hugsun]

I didn't see the presentation but judging by your description, this is achievable using in-context learning.

[wmf]

For everything other than handwriting I don't think the LoRAs are fine-tuned locally.

[derefr]

Well, here's another one: they promised that your local (non-iCloud) photos don't leave the device. Yet they will now — among many other things they mentioned doing with your photos — allow you to generate "Memoji" that look like the people in your photos. Which includes the non-iCloud photos.

I can't picture any way to use a RAG to do that.

I can picture a way to do that that doesn't involve any model fine-tuning, but it'd be pretty ridiculous, and the results would probably not be very good either. (Load a static image2text LoRA tuned to describe the subjects of photos; run that once over each photo as it's imported/taken, and save the resulting descriptions. Later, whenever a photo is classified as a particular subject, load up a static LLM fine-tune that summarizes down all the descriptions of photos classified as subject X so far, into a single description of the platonic ideal of subject X's appearance. Finally, when asked for a "memoji", load up a static "memoji" diffusion LoRA, and prompt it with the that subject-platonic-appearance description.)

But really, isn't it easier to just fine-tune a regular diffusion base-model — one that's been pre-trained on photos of people — by feeding it your photos and their corresponding metadata (incl. the names of subjects in each photo); and then load up that LoRA and the (static) memoji-style LoRA, and prompt the model with those same people's names plus the "memoji" DreamBooth-keyword?

(Okay, admittedly, you don't need to do this with a locally-trained LoRA. You could also do it by activating the static memoji-style LoRA, and then training to produce a textual-inversion embedding that locates the subject in the memoji LoRA's latent space. But the "hard part" of that is still the training, and it's just as costly!)

[janekm]

That's going to be something similar to IPAdapter FaceID: https://ipadapterfaceid.com Basically you use a facial structure representation that you'd use for face recognition (which of course Apple already compute on all your photos) together with some additional feature representations to guide the image generation. No need for additional fine-tuning. A similar approach could likely be used for handwriting generation.

[gokuldas011011]

I believe this could be achieved by providing a seed image to the diffusion model and generating memoji based on it. This way fine tuning isn't required.

[throwthrowuknow]

I think you’re misunderstanding what they mean by adapting to use cases. See this passage:

> The adapter models can be dynamically loaded, temporarily cached in memory, and swapped — giving our foundation model the ability to specialize itself on the fly for the task at hand

This along with other statements in the article about keeping the base model weights unchanged says to me that they are simply swapping out adapters on a per app or per task basis. I highly doubt they will fine tune adapters on user data since they have taken a position against this. I wonder how successful this approach will be vs merging the adapters with the base model. I can see the benefits but there are also downsides.

[rvaish]

Easel has been on iMessage for a bit now: https://apps.apple.com/us/app/easel-ai/id6448734086

[avidphantasm]

Very little of the “AI” boom has been novel, most has been iterative elaborations (though innovative nonetheless). Academics have been using neural network statistical models for decades. What’s new is the combination of compute capability and data volume available for training. It’s iterative all the way down though, that’s how all technologies are developed.

[sigmoid10]

Most people don't realize this, but almost all research works that way. Only the media spins research as breakthrough-based, because that way it is easier to sell stories. But almost everything is incremental/iterative. Even the transformer architecture, which in some way can be seen as the most significant architectural advancement in AI in the past years, was a pretty small, incremental step when it came out. Only with a lot of further work building on top of that did it become what we see today. The problem is that science-journalists vastly outnumber scientists producing these incremental steps, so instead of reporting on topics when improvements actually accumulated to a big advancement, every step along the way gets its own article with tons of unnecessary commentary heralding its features.

[astrange]

The "bitter lesson of machine learning" means that you actually can't do anything novel; it won't work as well as just doing the simple thing but bigger.

(So there is room left if you're limited by memory or budget.)

[w10-1]

> What’s new is the combination of compute capability and data volume available for training

This is the important part.

My advisor said new means old method applied to new data or new method on old data.

Commercially, that means price points, i.e., discrete points where something becomes viable.

Maybe that's iterative, but maybe not. Either way, once the opportunity presents, time is of the essence.

[WiSaGaN]

This gives me the vibe of calling high resolution screens as "retina" screens.

[dishsoap]

I don't see anything wrong with that at all. They've created a branding term that allows consumers to get an idea of the sort of pixel density they can expect without having to actually check, should they not want to bother.

[necovek]

Except that everyone has different visual acuity and different distance they use the same devices at, and in the end, "retina" means nothing at all.

But this is exactly the type of marketing Apple is good at, though "retina" is probably not the most successful example.

[theshrike79]

If your "visual acuity" is so good that you can see the pixels of a retina-branded display from the intended viewing distance, you might need to be studied for science.

[necovek]

If your visual acuity is 20/10, you'd roughly need 3600 pixels vertically to not notice any pixelation if Bill Otto did the calculations right at https://www.quora.com/What-resolution-does-20-10-vision-corr...

20/10 is rare but can easily be corrected to with glasses or contacts.

You also left that "intended viewing distance" hanging there, without at all acknowledging what that is at a minimum?

[jackothy]

It's not so impossible to spot flaws if you're using worst-case testing scenarios. Which are not worthless because such patterns do actually pop up in real world usage, albeit rarely.

[kolinko]

Examples?

[ngcc_hk]

Agreed. It is not high resolution as such, but high resolution that the user can relate to - like cannot see the pixel.

Still remember the hard time using Apple newton in a conference vs the palm freely on loan in a Gartner group conference. Palm solved a problem, even though not very Apple … user can input on a small device. I kept it, on top of my newly bought newton.

It is the user …

[pyinstallwoes]

Still no manufacturer compares to the quality of apple screens and resolution …

[Malcolmlisk]

Those screens are produced by samsung.

[PaulRobinson]

By your logic, I own a Foxconn smartphone with a FreeBSD-based OS. If you bought a Porsche, would you call it a Volkswagen?

[lostmsu]

Perhaps you should.

[mensetmanusman]

Part of the screen is, yes. Apple designs the full stack and sources new technology from multiple suppliers including Samsung.

[astrange]

Except for when it's LG, Sharp or BOE.

[dwaite]

Or AMD calling monitors which meet quality and feature requirements 'FreeSync'

Or Intel calling USB4 devices and cables which meet quality and feature requirements 'Thunderbolt 5'

Compared to, say, manufacturers who aren't willing to meet any certification requirements or to properly implement the standards at play saying they have "USB-A 3.2 2x2 ports" on their motherboards.

Retina doesn't carry the same weight as an industry certification effort like thunderbolt, but it still informs people that a screen actually meets some sort of bar without them having to evaluate pages of tech specs, and reviews saying whether the tech specs are accurate or have undocumented caveats.

Finally, establishing such certifications are difficult - look at the number of failed attempts at creating industry quality/feature marks in the television market.

[viktorcode]

Retina means high pixel density, not high resolution. And there are very few standalone displays on the market which can be called “retina”, unfortunately.

[necovek]

Interestingly, I would venture to say that DPI is a measure of resolution: that's the way it's still used in printers or scanners, for example (600 dpi). And retina instead means high angular resolution, or pixels having small arc measures from "appropriate" distances.

The term "resolution" transitioned gradually to mean number-of-individual-lines-rendered horizontally and vertically for displays, but really, the idea is how many dots can you "resolve" (or "resolving power"): a "high resolution" screen or screen mode had a larger number of individual pixels being drawn, which meant that the density is higher. You never talk about a scanner having a resolution of 7200x3600 even if it can scan 12"x6" at 600dpi.

So really, in an informal conversation, I believe both are fine. If you want to be extremely precise, you missed the mark: width and height in pixels is the sanest way to call what you refer to as "resolution".

[necovek]

See also https://en.wikipedia.org/wiki/Display_resolution:

> For device displays such as phones, tablets, monitors and televisions, the use of the term display resolution as defined above is a misnomer, though common. The term display resolution is usually used to mean pixel dimensions, the maximum number of pixels in each dimension (e.g. 1920 × 1080), which does not tell anything about the pixel density of the display on which the image is actually formed: resolution properly refers to the pixel density, the number of pixels per unit distance or area, not the total number of pixels. In digital measurement, the display resolution would be given in pixels per inch (PPI).

[threeseed]

It's a huge development in terms of it being a consumer-ready, on-device LLM.

And if Karpathy thinks so then I assume it's good enough for HN:

https://x.com/karpathy/status/1800242310116262150

[rishabhjain1198]

The productization of it (like Karpathy mentioned) is awesome. But I think the URL for that would be this maybe? [link](https://www.apple.com/apple-intelligence/)

[Cthulhu_]

Thing is, Apple takes these concepts and polishes them, makes them accessible to maybe not laypeople but definitely a much wider audience compared to those already "in the industry", so to speak.

[marcellus23]

They refer to LoRA explicitly in the post.

[rishabhjain1198]

Although I caught that on the first read, I found myself questioning when I read the adaptors part, "is this not just LoRA...".

Maybe it's my fault as a reader, but I think the writing could be clearer. Usually in a research paper you would link to the LoRA paper there too.

[monkeydust]

Feel Apple should have just focused on their models for this one and not complicate the conversation with OpenAI. They could have left that to another announcement later.

Quick straw poll survey around the office, many think their data will be sent off to OpenAI by default for these new features which is not the case.

[scosman]

I think you’re referring to my comment about this being huge for developers?

Just want to point out I call this launch huge, didn’t say “huge development” as quoted, and didn’t imply what was interesting was the ML research. No one in this thread used the quoted words, at least that I can see.

My comment was about dev experience, memory swapping, potential for tuning base models to each HW release, fine tune deployment, and app size. Those things do have the potential to be huge for developers, as mentioned. They are the things that will make a local+private ML developer ecosystem work.

I think the article and comment make sense in their context: a developer conference for Mac and iOS devs.

Apple also explicitly says it’s LoRA.

[franzb]

This isn't about AI research, it's about delivering AI at unimaginable scale.

[throwthrowuknow]

180 million users for chatgpt isn’t unimaginable but it does exceed the number of iPhone users in the United States.

[frompom]

Do you have the same expectations for any company launching hardware that they cite the various papers related to how the tech was developed? EVERY piece of tech announced by ANY company relies on a variety of research out there yet it doesn't seem expected that every time anyone launches something they cite the numerous papers related to that. Why would products/services in this category be any different?

[talldayo]

> Do you have the same expectations for any company launching hardware that they cite the various papers related to how the tech was developed?

If they try to market it with a seemingly unique or yet-unheard of name, then yeah. It is nice knowing what the "real world" name of an Apple-ized technology is.

Just ignoring it and marketing the technology under some new name is adjacent to lying to your audience through omission.

[dwaite]

> Just ignoring it and marketing the technology under some new name is adjacent to lying to your audience through omission.

They don't market technology, they market solutions. E.g. afib detection on Apple Watch, rather than calling it a BNNS using a custom-built sensor for one-wire EKG.

This is the document where they describe how the solution works, and they clearly state adapters work based on LoRA.

[arvinsim]

> The way they wrote about "adaptors" can make it seem like it's something novel, but it's actually just re-iterating vanilla LoRA.

That's a classic Apple strategy though.

[gigglesupstairs]

Was there anything about searching through our own photos using prompts? I thought this could be pretty amazing and still a natural way to find very specific photos in one’s own photo gallery.

[mazzystar]

Run OpenAI's CLIP model on iOS to search photos. https://github.com/mazzzystar/Queryable

[gigglesupstairs]

Yes, exactly this. I have had this for a while and works wonderfully well in most cases but it’s wonky and not seamless. I wanted a more integrated approach with Photos app which only Apple can bring to the table.

[avereveard]

Which is in turn just multimodal embedding

Besides I could do "named person on a beach in August" and get the correct thing in photos on Android photos, so I don't get it.

It's amazing for apple users if they didn't have it before. But from a tech stand point people could have had it for a while.

[azinman2]

Photos has had this for a while with structured natural language queries, and this kind of prompt was part of the WWDC video.

[theshrike79]

The difference is that Apple has been doing this on-device for maybe 4-5 years already with the Neural Engine. Every iOS version has brought more stuff you can search for.

The current addition is "just" about adding a natural language interface on top of data they already have about your photos (on device, not in the cloud).

My iPhone 14 can, for example, detect the breed of my dog correctly from the pictures and it can search for a specific pet by name. Again on-device, not by sending my stuff to Google's cloud to be analysed.

[fauigerzigerk]

They have been trying and failing to do a tiny little bit of this. It's so broken and useless that I've been uploading all my iCloud photos to Google as well, for search and sharing.

[theshrike79]

If you like Google using your personal photos for machine learning, that's your option. Now they have your every photo, geotagged and timestamped so they can see where you have been and at what times. Then they of course anonymise that information into an "advertiser id" they tag on to you and a sufficient quantity of other people so they can claim they're not directly targeting anyone.

I prefer Apple's privacy focused option myself.

[fauigerzigerk]

>If you like Google using your personal photos for machine learning, that's your option.

It's a trade-off between getting the features I need and the price I have to pay. All else being equal I do prefer privacy as well. Unfortunately, all else is not equal.

>I prefer Apple's privacy focused option myself.

It's only an option if it works.

[lhl]

I think your conclusion is uncharitable or at least depends on how deep your interest in AI research actually is. Reading the docs, there are at least several points of novelty/interest:

* Clearly outlining their intent/policies for training/data use. Committing to no using user data or interactions for training their base models is IMO actually a pretty big deal and a differentiator from everyone else.

* There's a never-ending stream of new RL variants ofc, but that's how technology advances, and I'm pretty interested to see how these compare with the rest: "We have developed two novel algorithms in post-training: (1) a rejection sampling fine-tuning algorithm with teacher committee, and (2) a reinforcement learning from human feedback (RLHF) algorithm with mirror descent policy optimization and a leave-one-out advantage estimator. We find that these two algorithms lead to significant improvement in the model’s instruction-following quality."

* I'm interested to see how their custom quantization compares with the current SoTA (probably AQLM atm)

* It looks like they've done some interesting optimizations to lower TTFT, this includes the use of some sort of self-speculation. It looks like they also have a new KV-cache update mechanism and looking forward to reading about that as well. 0.6ms/token means that for your average I dunno, 20 token query you might only wait 12ms for TTFT (I have my doubts, maybe they're getting their numbers from much larger prompts, again, I'm interested to see for myself)

* Yes, it looks like they're using pretty standard LoRAs, the more interesting part is their (automated) training/re-training infrastructure but I doubt that's something that will be shared. The actual training pipeline (feedback collection, refinement, automated deployment) is where the real meat and potatoes of being able to deploy AI for prod/at scale lies. Still, what they shared about their tuning procedures is still pretty interesting, as well as seeing which models they're comparing against.

As this article doesn't claim to be a technical report or a paper, while citations would be nice, I can also understand why they were elided. OpenAI has done the same (and sometimes gotten heat for it, like w/ Matroyshka embeddings). For all we know, maybe the original author had references, or maybe since PEFT isn't new to those in the field, that describing it is just being done as a service to the reader - at the end of the day, it's up to the reader to make their own judgements on what's new or not, or a huge development or not. From my reading of the article, your conclusion, which funnily enough is now the new top-rated comment on this thread isn't actually much more accurate the the one old one you're criticizing.

[steve1977]

You know what company you are talking about here?

[rvaish]

reminds me of Easel on iMessage: https://easelapps.ai/

[Spooky23]

Those people aren’t looking at Apple.

They seem to have a good model for adding value to their products without the hold my beer, conquer the world bullshit that you get from OpenAI, et al.

[kfrzcode]

"AI for the rest of us."

[wkat4242]

Except Apple isn't really for the rest of us. Outside of America and a handful wealthy western countries it's for the top 5-20% earners only.

[jahewson]

Approximately 33% of all smartphones in the world are iPhones.

[rvnx]

60% in the US

[dwaite]

Maybe the suspicion is the top 5% carry six smartphones?

[throwaway2037]

Japan and Taiwan are both more than 50% iOS.

Ref: https://worldpopulationreview.com/country-rankings/iphone-ma...

[whynotminot]

Who do you think this presentation is geared toward?

[theshrike79]

In the EU the market share is 30%

[d1sxeyes]

Yes but not evenly distributed, BeNeLux, Germany, Austria, and Nordic countries have a lot of iPhone users, while moving further east (or south) you see lower market share. Maybe it’s “two handfuls” of wealthy western countries rather than just one, but I think OPs point holds true.

[theshrike79]

https://worldpopulationreview.com/country-rankings/iphone-ma...

Poland, Greece, Hungary and Bosnia-Herzegovina are the only ones under 20% (and maybe a few others).

OTOH Britain is over 50% as is Sweden. Finland, the land of Nokia is over 35%.

[XajniN]

One average American user is probably worth 5-10 average European users.

[theshrike79]

(I've dabbled in mobile games)

Yes. Americans are THE most valuable customer base, y'all use insane amounts of money on mobile crap.

[kolinko]

In Poland it’s 33%

[d1sxeyes]

Poland seems to be 14%: https://gs.statcounter.com/os-market-share/mobile/poland

[elbear]

In Romania it's 24.7%

[d1sxeyes]

Huh interesting, I missed that. You’re right (actually I see even 25.5%).

[chuckjchen]

This sounds like every newcomers to the stage except for big players like Apple.