[FrasiertheLion]

You can use Tinfoil for inference, which lets you use the model in the cloud while getting similar privacy as running locally: https://tinfoil.sh/inference.

Disclaimer I'm the cofounder. This works by running the model inside a secure enclave (using NVIDIA confidential computing) and verifying the open source code running inside the enclave matches the runtime attestation. The docs walk you through the verification process: https://docs.tinfoil.sh/verification/verification-in-tinfoil

[cataflutter]

Worth noting that NVIDIA confidential computing and similar schemes have been compromised and shouldn't be relied upon if it really matters. See https://tee.fail/ and similar.

[Frannky]

I was interested in trusted execution environments and how safe they were. If you look on google scholar and start reading, they seem super vulnerable. The feeling is that the industry has no better option and that they are a way to tell customers they are safe when they're not

[itemize123]

with physical access right?

[7777332215]

Hi there I use your service. It's great. But I have a few requests... Please support crypto payments...? Also you are missing some open source models (qwen 30b 3a, Deepseek 4 flash).

[FrasiertheLion]

Unfortunately we don’t support crypto payments at this time as we use Stripe.

We try to add models selectively as we have to be mindful about our compute allocation. Is there a specific reason why you need those two models (and our models such as Kimi K2.6, GLM 5.1, Deepseek V4 Pro, Gemma 4 amongst others) don’t suffice for your use case?

Feel free to email me at [e-mail] and happy to continue the conversation there.

[100ms]

Tinfoil looks super interesting! Do you have load balancers in front of the trusted compute stack? Looked at a design like this in a different space and the options for ensuring privacy in a traditional "best practice" architecture seemed very limited

[FrasiertheLion]

Yes we do, but the load balancer also runs inside the enclave and is attested: https://github.com/tinfoilsh/confidential-model-router

In turn, that attests the model enclaves, for instance, see https://github.com/tinfoilsh/confidential-deepseek-v4-pro. The model repo/release that the model router attests is included in the attestation config, which creates a chain of trust.

Also see https://docs.tinfoil.sh/verification/attestation-architectur...

[jdasdf]

While that does sound interesting, I don't see any benefit for me.

It would still ultimately exfiltrate the data outside of my control, and frankly i don't trust any "secure enclave" tech.

As far as i'm concerned physical access is root access, and for any private stuff that is wholly unacceptable.

[FrasiertheLion]

Very reasonable if you have the resources to run it locally and certainly the best option.

But we created Tinfoil because not everyone has that capability especially when it comes to larger models, and it still doesn’t solve for the situation where you’re building a service for your end user and you want to lock yourself out of accessing their data. In those cases, this is the second best thing you can do.

The technical walkthrough section on this blog that we co-wrote with one of our customers walks through the various attack surfaces: https://www.workshoplabs.ai/blog/private-post-training

We weave in many mitigations against attacks, but it depends on what class of attack it is.

If there are specific attacks you are concerned about, happy to provide an answer if it’s something we can address or not.