[FrasiertheLion]

Enclaves have a property that allows the hardware to compute a measurement (a cryptographic hash) of everything running inside it, such as the firmware, system software such as the operating system and drivers, the application code, the security configuration. This is signed by the hardware manufacturer (Intel/AMD + NVIDIA).

Then, verification involves a three part approach. Disclaimer: I'm the cofounder of Tinfoil: https://tinfoil.sh/, we also run inference inside secure enclaves. So I'll explain this as we do it.

First, you open source the code that's running in the enclave, and pin a commitment to it to a transparency log (in our case, Sigstore).

Then, when a client connects to the server (that's running in the enclave), the enclave computes the measurement of its current state and returns that to the client. This process is called remote attestation.

The client then fetches the pinned measurements from Sigstore and compares it against the fetched measurements from the enclave. This guarantees that the code running in the enclave is the same as the code that was committed to publicly.

So if someone claimed they were only analyzing aggregated metrics, they could not suddenly start analyzing individual request metrics because the code would change -> hash changes -> verification fails.

[olejorgenb]

Thanks for explaining :)

> First, you open source the code that's running in the enclave, and pin a commitment to it to a transparency log (in our case, Sigstore).

This means you have reproducible builds as well? (source+build-artifacts is signed)

Also - even if there are still some risk that the link is not 100% safe, maybe it's safe to assume vendors like yourself going through all that trouble are honorable? (alternatively - they are very curious of what "paranoid" people would send through LLMs :sweatsmile:)

[FrasiertheLion]

We don't have reproducible builds because we attest the full OS image that we run, which is the Ubuntu image. Unfortunately bit-by-bit reproducible binaries for OS images is kind of an unsolved problem, because it requires the hundreds of package maintainers across all dependencies to eliminate any sources of non-determinism in the compilation. Things like timestamps and file reordering are very common and even one of these changes the entire hash.

So we do the next best thing. We decide to trust Github and rely on Github Actions to faithfully execute the build pipeline. We also make sure to pin all images and dependencies.