[YesBox]

Im a encryption noob. Less than a noob. But something I've been wondering about is how can homomorphic computing be opaque/unencryptable?

If you are able to monitor what happens to encrypted data being processed by an LLM, could you not match that with the same patterns created by unencrypted data?

Real simple example, let's say I have a program that sums numbers. One sends the data to an LLM or w/e unencrypted, the other encrypted.

Wouldn't the same part of the LLM/compute machine "light up" so to speak?

[saagarjha]

I don’t actually think an LLM is a good way to sum numbers, but it is a pretty good example to explain the phenomenon you’re interested in. When you run an LLM, you essentially take your input and matrix multiply it with the weights to get an output. This happens regardless of what the input is or what the model “needs” to do. So, to some extent, the same part of the machine is being used every time even though the results seem very different.

(Of course, the reality is much more complicated than this; you can trace things like power and in theory track individual activations with great difficulty and then do interpretability to see what the model is doing. But hopefully it illustrates that you can usually take some sort of special operation and turn it into a process that does the same operation on different data.)