Hacker News new | ask | show | jobs
by red2awn 253 days ago
A lot of optimizations in LLMs now are low hanging fruits inspired by techniques in classical computer science. Another one that comes to mind is paged KV caching which is based on memory paging.