[bangaladore]

I'll concede you are correct that whether the key is extractable or not doesn't really matter if the GPU eventually will eventually need to store the decrypted model in memory.

However, if NVidia or similar was serious about securing these models, I'd be pretty sure they could integrate the crypto in hardware multipliers / etc such that the model doesn't need to be decrypted anywhere in memory.

But at this point there isn't much value in deploying models to the edge. Particularly the type of models they would really want to protect as they are too large.

The types of models deployed to edge devices (like the Apple ones) are generally quite small and frankly not too difficult (computationally) to reimplement.