[itemize123]

tinfoil doesn't seem to distinguish between cached and input tokens. I think that would make it untenably expensive

[traceroute66]

> tinfoil doesn't seem to distinguish between cached and input tokens.

Correct.

Privatemode correctly distinguish[1][2].

But with Tinfoil it appears to be a fundamental architectural limitation that they cannot.[3]

[1] https://www.privatemode.ai/pricing [2] https://www.privatemode.ai/blog/secure-prompt-caching [3] https://docs.tinfoil.sh/resources/caching

[3s]

Hi, cofounder of Tinfoil here. We're working on implementing prompt caching. It just requires more work than with traditional, non-private inference and requires some careful considerations to actually implement it.