[manojlds]

Is there some easy to understand source / paper about how this caching works?

[xihajun]

https://arxiv.org/pdf/2311.04934

[danielmarkbruce]

Ask chat gpt to explain how K-V caching works. What they are doing is essentially the same thing, with a few more engineering details.