Predictive Speculative KV Replication for Bursty LLM Inference

[shreybirmiwal]

https://github.com/jwlaboratory/bite-the-bullet

[msp26]

the hand drawn diagrams and highlights are charming

[shreybirmiwal]

thank you so much

[holtkam2]

Awesome animations!

[shreybirmiwal]

thanks a lot