[j2kun]

IMO a big unanswered question in FHE (which is my field so I care about the answer) is key management. If you have a service with lots of different users, and each user requires 100 MiB or more in evaluation key material, simply loading that key material from disk onto the GPU for each user's request adds a nontrivial overhead on the order of magnitude of the FHE computation itself. Not to mention the storage cost (for a 1B-user service, each requiring 1 GiB of evaluation key material, requires 1 EiB of storage, which is a hard sell).

So it seems kind of reasonable to me that if you want to showcase your fast inference for its own sake, you can sidestep the remaining engineering problems around key management by giving the user pre-computed key material corresponding to hot-loaded evaluation keys.

[wocc]

You're identifying one of the (many) underappreciated cost in the RLWE FHE stack. The evaluation key problem doesn't scale, and no amount of hardware acceleration fixes a per-user storage and loading overhead that grows with the security parameter.

[j2kun]

I think one counterpoint here is that you can still get value in using FHE in settings that don't involve scaling the number of users (e.g. B2B, public/private collab).

[wocc]

agreed. The omission of what bit security they are using is of note and they Key missing component of the overhead equation. Private key is better :)