[luk212]

Very Apple-ish approach to AI catch up: wrap an external tool in a privacy architecture, embed into the OS and productize the orchestration layer.

It will be interesting to see if the Private Cloud Compute + on-device routing can make third-party model capabilities feel like a first-party system without leaking user context to the model provider.

If Apple handles the Google-Apple boundary right, this will be an elegant move on their part, otherwise it will feel like Apple Intelligence with a just a privacy-polished frontend for Gemini.

[Quothling]

> wrap an external tool in a privacy architecture

Wouldn't it be more accurate to call Apple's architecture data protection rather than privacy? As an European citizen in a post Snowden world I would be surprised if any of my data on Apple services was actually kept private from the US government, and Apple certainly wants to own a lot of data/metadata about you. Gotta have Siri listening for carplay and so on. I would aboslutely trust Apple not to sell my data as a commodity though.

> If Apple handles the Google-Apple boundary right, this will be an elegant move on their part, otherwise it will feel like Apple Intelligence with a just a privacy-polished frontend for Gemini.

I'd say this is spot on. At least if what Microsoft is doing with Copilot Cowork is anything to go by. Cowork is not a privacy-polished as much as it's an Enterprise compliant polish to make Opus 4.8 run "safely" in your enterprise organisation. So far Microsoft is winning the AI war in non-tech enterprise with this, especially here in the EU. If Apple manages to do this for the private market that will be great for them.

I'm not personally sold on what an AI should do on my phone though. I use a lot of AI professionally, but I haven't even turned on Bixby or whatever the Samsung AI is called.

[miki123211]

From an EU perspective, Microsoft is doing data protection, Apple is doing data privacy.

Microsoft's approach to data is basically "we promise nobody else but you and your government can access it, we can but we pinky swear we won't." This promise is mostly enforced at the legal layer and through legal consequences, not technical safeguards. If they think they can get away with it (or are forced to get away with it by the US government), there's nothing stopping them from using your data in whatever way they want.

When they can, Apple designs their systems so that they physically don't even have the capability to use your data, even if it's processed on their own servers. They're not privacy maximalists like Signal is, they care more about user experience, but they do aim for the highest level of privacy you can get while still having a good experience, and when they do need to make sacrifices, they typically let you opt into the privacy features if you really want to.

I'm far more inclined to believe that Microsoft is secretly (or not so secretly) collaborating with the US government than that Apple is.

[tguedes]

There has been anecdotal statements/blogs from Apple employees about the data privacy. They have said building some internal capabilities or user facing features are extremely difficult or impossible because they aren't able to access user data at the level required.

[the_mar]

From the information presented, the privacy case is not that your data is only accessible to you (which arguably can have a backdoor) but that the data is NOT stored at all, so it's not possible to build a backdoor. I know there are probably other ways around it, but it's my understanding is that no data is kept on any server when the response is sent back to your device

[antiframe]

I don't think the Snowden allegations were about employee access to user data.

[ziofill]

Do you have any examples?

[tguedes]

There is a comment in this thread from an alleged Apple employee that said that, but it doesn't seem like it's possible to send a link for a specific comment. Over the years I've seen comments and blogs posted here in Hacker News reaffirming the same thing.

But to answer your question directly, I don't have any links for those blogs or comments

[rusk]

> it doesn't seem like it's possible to send a link for a specific comment

Click on where it says how long ago the comment was posted

For example https://news.ycombinator.com/item?id=48461367

[tguedes]

I'm replying again because another commenter showed how to link to a comment

https://news.ycombinator.com/item?id=48456081

[JimDabell]

> Wouldn't it be more accurate to call Apple's architecture data protection rather than privacy? As an European citizen in a post Snowden world I would be surprised if any of my data on Apple services was actually kept private from the US government, and Apple certainly wants to own a lot of data/metadata about you.

Your conception doesn’t seem to match PCC at all. The whole point of it is that nobody can access the data, not even the people running the servers.

https://security.apple.com/blog/private-cloud-compute/

[Quothling]

I don't trust a single US tech company to keep my data private from the US government. Maybe I need a tinfoil hat, but I don't feel like I'm unjustified in this based on the history going back to echelon. Not that this is a particular jive at the USA, my own government (Danish) actively pushes for mass surveillance and non-functional e2e encryption.

There is still a difference though. Google will sell my data and use it for all sorts of things. Though I've obviously accepted that since I have had a Samsung flip phone since Apple made their iPhones too big for my pockets.

[jesseendahl]

This part of their requirements for how PCC is architected directly addresses your concern:

“Verifiable transparency. Security researchers need to be able to verify, with a high degree of confidence, that our privacy and security guarantees for Private Cloud Compute match our public promises. We already have an earlier requirement for our guarantees to be enforceable. Hypothetically, then, if security researchers had sufficient access to the system, they would be able to verify the guarantees. But this last requirement, verifiable transparency, goes one step further and does away with the hypothetical: security researchers must be able to verify the security and privacy guarantees of Private Cloud Compute, and they must be able to verify that the software that’s running in the PCC production environment is the same as the software they inspected when verifying the guarantees.”

[brianmcnulty]

They do this by allowing you to download all of the components (minus data cryptexes containing the model weights) and run it on your own Apple silicon chip (you can put your computer in recovery mode and use csrutil to enable research guest operating systems)

I think what is concerning is that they are expanding into Google Cloud and NVIDIA to run with it too with their versions of confidential compute, which if I remember correctly are not as well verified as Apple PCC and a little harder for researchers to get their hands on.

Apple uses a key ceremony process where no single party has access to all the keys required to sign hardware, meaning in theory they can’t just sign malicious hardware. However, I’m not sure how Google and NVIDIA play into this and I don’t think they’ve provided much detail on it. I think it seems a little rushed to get the features out since they fucked up with initial Apple Intelligence release.

[tguedes]

From my understanding of the architecture, Apple and Google have basically developed a fork of Gemini that is built to run on Apple's PCC. There is no data being sent to any Google servers.

From this MacRumors article:

"The new architecture centers on Apple Foundation Models co-developed with Google, which Apple says are adapted to run both on-device and on servers through its existing Private Cloud Compute infrastructure."

And

"The company reiterated that Apple Intelligence relies on on-device processing and Private Cloud Compute, with a promise that user data is only used to execute the immediate request and is not accessible to Apple or third parties. Apple added that outside experts can verify those privacy guarantees "at any time.""

[criley2]

What does verify mean?

Can they verify the private cloud is completely immune to nationstate actors, has no zero-day vulnerabilities, is completely bulletproof in a court of law and can never be compelled to secretly share info with government(s), etc?

I think the users fear here is real. "We did good due diligence at the consumer level" and "we're completely immune to nationstate hackers and clandestine legal cases" are very different things.

[brookst]

You should read the paper.

Like any good security paper, it doesn’t assert immunity to particular parties. Instead, covers things like how PCC attests that the running software image is identical to the publicly-available, forensically-studied one.

Fear is real for sure, but don’t let fear be an excuse to lose rigor in thinking.

[saagarjha]

How are security researchers going to have access to the Nvidia GPUs that will be running this?

[brookst]

It’s a fair concern, but the only way to reconcile a belief that Apple is sharing data from PCC with anyone (including themselves) is to assert the whole PCC thing is a massive fraud.

Which it could be, but given both breadth of claim and Apple’s strong incentives not to be caught lying about something so massive, I’d want something more than vibes to take the idea seriously.

[Geee]

There's no guarantee against data exfiltration, because the data leaks happens through tool calls, which are not made from the PCC, but from your own device.

E.g. "the user asks if their Bitcoin private key is unique, let's make a web search".

Combined with prompt injection attacks, it's quite easy for an attacker to craft a prompt which sends your private data through any supported tool call (web search, database search, email, app APIs, etc.). Everything is wide open for the attacker / or yourself accidentally to exfiltrate your data.

[JimDabell]

That doesn’t make sense in this context – the point of PCC is so you know somebody isn’t snooping on your information when you send it to the servers. The person I was responding to seemed to think that Apple would be looking at that information.

[Geee]

You're right, but also "PCC is very secure" might give a false sense of security, considering that there might be other associated vulnerabilities in these kinds of systems.

[fragmede]

Which is a good point. set a Bitcoin wallet private key in an obvious place on your system, and then setup a monitor (on another system) to notify you if its contents gets stolen.

Doesn't prevent the exfiltration but at least you'll know when it does.

[shaky-carrousel]

And we have to believe that it's not backdoored because they say so? That's incredibly naïve.

[JimDabell]

No. I provided the link so you could read more about it.

[9dev]

I have read it. The entire trust hinges on several critical points, such as trusting secure boot.

You remember when the NSA injected itself in TLS termination at all major cloud providers? You remember when several giant automotive corporations built elaborate detection of testing scenarios to fake emissions? You remember room 641A?

I have no real way to tell if this is security Theater or meaningful protection. None of us has,

[shaky-carrousel]

That's "because they said so" but with more words. Sorry, but a pretty blog post is not proof enough.

[paulluuk]

> I use a lot of AI professionally, but I haven't even turned on Bixby or whatever the Samsung AI is called.

I know this was just a small aside, but man do I hate Bixby and other phone AIs. They are so frustratingly difficult to turn off, and turning them on accidentally is as simply as holding the wrong button for a few seconds, such as when your phone is in your pocket. Very frustrating design.

[Quothling]

I never turned Bixby on, so it never really bothers me except for when I update and it want me to accept something which I decline. I turned the button off, I forgot what I switched it to but holy hell was that annoying.

[paulluuk]

I never turned Bixby on either, but on my phone is comes pre-installed and pre-turned-on, and you can't uninstall it, only "deactivate" it.

[embedding-shape]

I just want to be able to turn things off without affecting completely other unrelated things. Like Siri and Carplay, makes no sense I need one activated to use the other, just a trick to get people to avoid disabling Siri.

[giancarlostoro]

> I would be surprised if any of my data on Apple services was actually kept private from the US government

Outside of law enforcement having a warrant, Apple's efforts against CSAM, or their Chinese data centers, I've not heard of Apple doing any of what you assume in a post-Snowden world. iMessage is supposed to be end to end encrypted, and there was a few years ago that whole scandal where Apple wouldn't unlock a literal terrorists cell phone for the FBI.

The FBI had to reach out to... a third party to unlock the phone (I forget the name of the firm that did it - Cellebrite maybe?) for them, what's funny is they spent a lot of money on it, when the rest of the world pointed out that the very specific iOS version in question had known vulnerabilities they could have found online for free (or cheaper?).

[dofm]

Is there a meaningful Google-Apple boundary in operation?

They are buying the right to distill their own Gemini models and run them in their data centres (or at least data centres they control); unless I am missing something, this isn't going to be infrastructure that Google has operational control over.

[janalsncm]

If Apple is running the inference from Apple iPhones and Apple data centers then Apple has operational control. Google’s influence ends the moment they hand the weights over to Apple.

[impulser_]

They are using Google Cloud.

https://security.apple.com/blog/expanding-pcc/?linkId=100000...

"Now, we are collaborating with Google and NVIDIA to run new Apple Intelligence workloads on Google Cloud, extending our industry-leading PCC privacy commitments to third-party data centers for the first time."

[btown]

Per that link: I think there's an interesting question about whether a nefarious actor who's infiltrated a cloud provider with physical access to machines that are running signed operating systems, with signed binaries, with TDX remote attestation, and with hardware supply chain verification, has the ability to break the privacy guarantees of a tenant with Apple's sophistication.

Certainly, one could tamper with the hardware, but could one do it in a way that wouldn't get that machine immediately flagged, removed from the routing pool, and told to wipe its memory immediately, by a watchtower (perhaps even the routing layer itself) that runs in a separate secure Apple datacenter?

[Cassell]

Those datacentres would be in the same position of trust as a VPN provider in that the data must be unencrypted at points in the process.

They could be making it very safe, and the things apple says they are doing would make it as safe as possible, but as a user there is no way of verifying the claims.

[freedomben]

> as a user there is no way of verifying the claims

I think this sums up what it's like to be an Apple user pretty well. With their heavy proprietary and closed approach, all users can do is "trust" them.

[brookst]

Have you read the PCC whitepapers? Are you saying the user-facing verification methods in them are insufficient, or vulnerable, or just false?

[rasz]

>nefarious actor who's infiltrated a cloud provider

Google is buying that compute from xAI aka Musk

[RobMurray]

Apple could simply be ordered to include a hardware backdoor, and legally be prevented from talking about it. Everything else in the architecture could work exactly the way they claim in the PCC paper.

[zelon88]

Spoiler alert; Google is the nefarious actor.

[impulser_]

I think the last thing Google wants to do is get on the bad side of their largest partners.

[SoftTalker]

Why bother with all that cloak and dagger stuff when they can just buy the data? You believe Apple and/or Google isn't selling it? I have some land in Florida I'd like to talk about.

[appplication]

Having worked at Apple, I will say I firmly believe they do not sell data. I worked in data science and we had the shittiest inference because we had essentially no access, even internally, to longitudinal or cross-app user data. Best we had was 15 minute rotating sessions for a single app. There are internal teams dedicated to deanonymizing data to try to narrow down users - if they can successfully do so, and relevant fields that lead to deanonymization get permanently purged from internal logging.

I can’t speak to the current architecture but Apple has shown a consistent willingness to sacrifice access to user data in the name of selling privacy instead at a premium price (you could argue precisely because no one of their competition have any meaningful posture on this). I do believe they are quite serious in their commitment to that, as they have found this strategy to be more valuable than the data itself.

[boringg]

Apple's incentives don't align to sell private data as their whole thing is privacy. They do that they tank their business. If you have proof that they are doing it -- I'd love to see it. (*3rd party actors from an app re-selling data doesn't count)

Google is 100% doing that because thats their entire incentive for the business. They sell low cost software / subsidized hardware on the grounds that you pay with your sharing data. That's the implied cost.

Show me the incentives - I will show you the outcomes.

[cheriot]

Apple/Google make less money if they sell the data because their ad product would no longer have an advantage. So no, I don't think they do that.

[materielle]

That’s not so special, though? There’s a difference between Google infra running Google services.

Versus any F500 company running their services on GCP.

It’s a bit whacky to think about because Apple will operate Google owned software on GCP. But it should be sandboxed just the same.

I’m not making a normative privacy argument here. Just pointing out that this is cloud business as usual. Perhaps it’s interesting Apple is doing it, but basically everything else is already using either AWS or GCP at this point.

[airstrike]

I think the difference is scale. This is Apple, so it's an enormous amount of devices. And it's a seamless experience, to the user, going from local model to cloud models.

So the question about which model Apple was going to use and where has been highly anticipated, especially by the likes of OpenAI and Anthropic. Imagine if either one could say they have Apple as their customer?

Apple certainly has the cash to burn if they wanted to train their own model, but it also always seemed out of their core competency. This is a major win for Google.

So "business as usual" but with huge implications for the AI ecosystem in general.

[Someone]

Google Cloud, but, the way I read it, not Google’s AI offerings. They, basically, hire Google servers to run their software on it.

They also (claim to) ensure those servers run only software they have approved to run on it.

(Part of their software are models derived from Google Gemini, but that’s orthogonal to this)

[fauigerzigerk]

>(Part of their software are models derived from Google Gemini, but that’s orthogonal to this)

You're right that it is orthogonal to the privacy promises Apple makes to its own users.

The moralistic and righteous undertone in their marketing material is questionable though given that these Apple services might not exist if Google didn't exploit Gemini app user data on Android the way it does.

That's fine with me. Users have a choice here. In fact, it's a big improvement over the search deal with Google where Apple sends its own users directly to Google.

[huslage]

They are not _only_ using Google Cloud. They continue to build and invest in their own datacenters. It's not a binary choice.

[impulser_]

Yeah, but the models are running in Google Cloud which makes sense they are based on Gemini.

[huslage]

They appear to be running them on both GCP and in their datacenter.

[dofm]

That is news — I guess not very surprising that they'd need more data centres than before.

But again there is no Apple-to-Google transfer in the inference in the sense of the comment I was originally replying to (I am not suggesting you're implying otherwise, obviously)

But I stand happily corrected where I said they aren't in the picture at all.

That is an interesting press release because it outlines what they would have had to do with any data centre they were outsourcing to.

[impulser_]

This is probably why Google had to rent compute from SpaceX. They needed to free up NVIDIA GPUs for Apple so they probably moved internal workloads to SpaceX compute.

[didibus]

Google likely won't rent compute from SpaceX, they have a substantial share of SpaceX (they own 5% of it) and need the IPO to be valued highly, so to prop up the IPO stock, they made this announcement, but if you read the fine print, both SpaceX and Google are allowed to cancel it at any time, as-in, after they cash out from the IPO.

[ezfe]

iCloud already uses Google Cloud, so that still doesn't change the operational boundaries of where data goes

[LoganDark]

I hope they are still using PCC hardware rather than running private data through third-party servers.

[dofm]

Right — I suppose I mis-phrased my first sentence a bit, because I guess it can be interpreted as me saying the boundary is blurred, when what I was trying to write is: in operation there is nothing crossing any boundary; Google are not in the picture.

[ProAm]

But google is paying SpaceX/xAi for compute... so...

[al_borland]

As someone who doesn't use Android, they showed a lot of integration into the apps, which I think is where the real magic happens, and it's not something I can do with any 3rd party chatbots today (that I'm aware of). I also don't know that I would trust the other 3rd parties with the access required to pull it off.

[xattt]

If anything, having Shortcuts built-in at the OS level was a very long play that’s going to pay off.

[dep_b]

And Intents

[toddmorey]

I'm interested in how it feels to use: whether there is any context leaking, as you mentioned, if it introduces latency, and whether there are any pricing implications? I know they weighed a variety of factors, including the smaller models, but cost had to be a big concern, too... I feel like Google is the only provider giving away so much AI inference for free.

This will further blur the picture about when and how consumers / employees are supposed to pay for AI services. For example, they showed consumer rather than coding tasks, but could you select five files and ask Siri to write a Python script or a small app? Will enterprises just disable Siri AI functionality, or will they be able to route it through their own AI auditing and providers?

[aorloff]

Oh yes Apple "wraps" their AI in a "privacy architecture" -- you can't use Carplay unless you turn it (Siri) on.

[ezfe]

Yes, but Siri can be turned off from invocation without turning off CarPlay. You can disable the side button and Hey Siri while leaving Siri "on."

[burgreblast]

No disrespect for your valuable discovery but this attitude of “it’s possible, if you do these non-obvious steps” feels a lot like victim blaming in UI.

If Apple (or anyone else) wanted to make a feature used, they can. For everyone else, if Siri is off CarPlay doesn’t work. And that’s by design.

Not the design of “ooh if Siri is off then voice in CarPlay won’t work” (warnable), but punishment if Siri is off.

Again this pattern isn’t Apple only but it’s bad everywhere.

[ezfe]

It feels cynical to see this as a punishment when it's such a specific use case that does demonstrate deep integration with Siri. Maps, Messages, etc. use Siri for their interactions.

I am sure that there was a meeting where they decided what to do when Siri was off and somebody decided (very possibly with ulterior motives) not to split the feature set - all or nothing. However I don't think the challenge they were faced with in this hypothetical meeting was an easy one.

The alternative is you open the Messages app and you can't send messages. You open Maps and you can't get directions (unless parked). Sure, I get that they could show a screen saying "Sending messages is not available when Siri is disabled" but now you're hitting error messages while driving.

Anyways, the main reason people would disable Siri is accidental activation, and Apple provides all the toggles needed to avoid that without disabling the core components needed for CarPlay.

[jasonmp85]

How do you expect to use a nav system while driving without voice control?

[drusepth]

Before you start driving, at stop lights, while waiting in lines, etc.

I don't know how it works on CarPlay but when I turn my car on I have a bunch of suggested addresses (home, work, parents, recent Maps searches, etc) that I just touch-to-go. Having to use voice every time you want to navigate not only sounds unnecessary, but cumbersome.

[geden]

Oh so exactly like Apple Maps via Carplay then.

[yunwal]

Yep, shouldn't require Siri at all

[doikor]

Put in the destination before start driving?

CarPlay does not work at all if you have not enabled Siri. As in it won’t even connect.

[TylerE]

Since when? I've used Carplay for years and I've NEVER had Siri enabled. I despise voice control.

[intrikate]

I purchased an iPhone 15 a few months ago and ended up making this discovery myself. CarPlay would refuse to launch unless I enabled Siri. I didn't do any of the Siri setup, or anything but the app would hard refuse to launch unless I went and toggled on Siri. Maybe that's different depending on your make/model, or the specific infotainment system in your car, but in my '21 Kia Forte, Siri is a very hard requirement.

[koolala]

Never enabled it or never disabled it? Do you remember disabling it?

[TylerE]

Never enabled it. I always explicitly skip the "Setup/Enable siri" step on every update.

[00kevn]

No sense of adventure

[preg_match]

You don’t need to enable Gemini or voice assistant on android to use android auto. Some functionality is lost, of course, but you can still navigate and play music.

[wuliwong]

I almost never use Siri in my car and use a "nav system" every time I am driving.

[az_reth]

I set it before I start moving, and then don't touch it while in motion.

[adastra22]

I never use Siri when using CarPlay? I’m confused by your question.

[aorloff]

If you disable Siri, you cannot connect to Carplay

[adastra22]

Not disabled. I just never use Siri.

[artursapek]

I use Carplay all the time and I didn't even realize it has voice control. I just set things up on my phone and drive.

[c1sc0]

DIt disn’t even occur to me that I need voice control. Stop car. Enter destination. Drive. How hard is that?

[Klayy]

Good luck telling Siri to drive to Hronská Breznica. Outside of English speaking countries voice control is very hit and miss (mostly miss)

[jazzyjackson]

“Sorry, I don’t know where you are”

[scosman]

> just a privacy-polished frontend for Gemini

I'd use this.

I'd rather have strong privacy guarantees, but this is still good.

[boxed]

The most Apple-ish thing would be to produce a great platform, enable third parties to do something on top of it, and take a cut.

I REALLY wish they'd do that with voice assistants.

[paulddraper]

All I know is that Siri is a terrible user experience.

[Tagbert]

Consider that they described the development process as taking Siri down to bare metal and rebuilding on a new architecture. I don’t think our previous Siri experience will be particularly relevant.

[antiframe]

They decided to use the Siri branding, so I am going to do as they tell me an associate it with my prior Siri experience. That's a brands sole purpose, to give a buyer an indication of a product before they buy it. (Or in my case, given my experience with the brand, not buy it).

[paulddraper]

As long as it’s not the same folks I guess

[dev_l1x_be]

This is post Jobs era typical Apple.

[ActorNightly]

I absolutely adore the historical revisionism that apple cares about privacy.

Run your router through a linux laptop as a proxy so you can capture traffic, connect any apple device to your router, and see the vasts amount of data your device sends to apple.

Apple DGAF about privacy, they want your data as much as anyone else, their only thing is that they should be the only ones to get it and then other people have to pay them for it, rather than your device sending the data to the 3d party directly.

And if you think your data is secure, reminder that The Fappening was all done targeting apple devices.

[Cider9986]

Apple added e2ee and created the most complete end to end encrypted cloud ecosystem to prevent that from happening again.

[ActorNightly]

Do you trust them to actually implement that correctly, and also do you trust them not to share your data with others, and if so, why?

[Cider9986]

I don't use it, but yes I think that it works because they have everything to lose and nothing to gain. They definitely could share whatever data they have with whoever, which is why you use e2ee.

At the end of the day you are trusting Apple with pretty much everything if you use this service because they make most of your phone, the entire operating system, host the update servers, etc.

[ActorNightly]

>At the end of the day you are trusting Apple with pretty much everything if you use this service because they make most of your phone, the entire operating system, host the update servers, etc.

Correct but this is not privacy.

Privacy is when you have a phone does not contact external services, or at the least, allows you to control which services it can contact.

With Apple, you don't have this. Sure e2ee to Apple exists, but if China decides to put enough pressure on Apple and request your data from Apple if you visit China, its very likely they will give it to them, and you have no option to opt out.

[Cider9986]

Apple doesn't run iCloud in China.

We were originally talking about if they implemented e2ee correctly and obviously e2ee gives you significant privacy. Your documents are still private if you encrypt them locally and then upload to google drive, no matter what connections google drive makes.

Privacy is a spectrum and Apple gives you the most privacy of any big tech company.

[DANmode]

Is that secret code for “rate-limited auth on the Find My API”?

[DANmode]

I’m not breaking NDA if I never signed one/was nowhere near the company or situation, you guys can fuck yourselves lol ^_^