[DanyWin]

I see your point. We have been creating content to democratize Confidential Computing, which is a field leveraging hardware-based (instead of software based like Homomorphic encryption) solutions to protect data in use.

I have a video from a webinar here: https://youtu.be/a2nprLS6bSA?t=1882, we have some examples in our blog https://blog.mithrilsecurity.io/privacy-voice-ai-with-blinda..., and we will release a series where we show to use secure enclaves by building a KMS with secure enclaves.

I don't necessarily agree with your statement regarding deployment on laptop. Not everyone has the skill/hardware to deploy such models, and providing simple APIs to leverage those, especially if the model is complex, could bring a lot of value to users in our opinion. We have seen hospitals wanting a simple API to do speech to text for medical voice notes and they just want an app on their old phones. I hardly see them deploying a 1B Whisper model for this use case.

Using BlindAI would allow them to have state-of-the-art AI, without having to worry about showing their data to us.

[danShumway]

I guess I should say to be fair, getting the user input and model output onto a single set of hardware with even semi-reasonable guarantees that it won't be examined in an unencrypted state is a heck of a lot better than what OpenAI is doing, so I don't necessarily want to say it's a bad business model -- there are lots of services I rely on that make privacy promises to me that aren't based on complete E2EE setups. What you're describing is absolutely an improvement over OpenAI's privacy.

I think my quibble is that (to me) the privacy terms on the homepage feel like they're suggesting something stronger than a secure enclave. My first thought when I saw it was "huh, somebody figured out homomorphic encrypted models." I'm not sure I'd have the same objections if I hadn't gotten that impression to start with.

----

My suspicion with local models is less that they'll run on a cell phone or a low-end laptop, and more that at the point they can run on high-end consumer hardware, they'll also be cheap enough to run that hosting an LLM might become a commodity service. In the same way that a hospital might not want to host its own website or data records, I'm still wondering how the companies involved in that stuff don't eventually just turn into generic hosting companies. And again, to be fair here, if the security model you're talking about holds up, maybe that's a model that can be applied to multiple products or hosting offerings. It does sort of feel like the main thing you're advertising here is the generic enclave, not the AI specifically.

And who knows, maybe I'm just completely wrong about all of it, I'm not an expert. But I sort of suspect that it's going to get progressively easier to host these kinds of models in the future.

[DanyWin]

The thing is that what they say on the homepage has to be trusted and cannot be verified. At best they put contractual commitments but no one will know what happens behind the scenes.

With attestation of secure enclaves (https://blog.mithrilsecurity.io/confidential-computing-expla..., sorry it's a bit old and not tech enough we will update it), you can have technical proof that people will respect what they say contractually. I don't think OpenAI is using any real Privacy Enhancing Technologies, and even if they did you have no actual proof they are doing anything (unless they use secure enclaves).

I agree, ideally you would like a purely mathematical solution like homomorphic encryption but truth is we might not see that before years or more (public key cryptography is not known to be fast).

Not everyone has access to high-end consumer hardware, and just maintaining the software/hardware stack on premise is complicated, so imagine having to manage thousands of device. It is not impossible unless you are Apple/Google, and even if you are it's not perfect. By sending model on the device it is quite easy to reverse engineer it, so not only your IP gets stolen easily but people can start making adversarial attacks.

Yes enclaves are a generic solution. In the end it's a bunch of level hardware primitives. But to have something that is truly fast, secure and easy to use, you need to focus on a use case to serve your users. We have chosen to focus on AI because we love AI and think it's a first niche market that is relevant, especially today.

I am not sure about going more on device / on premise. You can benefit from huge scaling effects by relying on managed services that are easier to maintain, patch, and deploy.