|
|
|
|
Show HN: MinLlama – Llama 3.2 inference in ~100 lines of NumPy
(github.com)
|
|
2 points
by timothygao
38 days ago
|
|
|
I built minLlama because I wanted a Llama implementation that was easy to understand and hack for KV cache compression research. There is also a PyTorch and Jax version in ~140 lines. Would be interested in feedback from people who have written transformer implementations before, are there any implementation "tricks" that I'm missing (e.g, cleaner KV cache for PyTorch/Jax or rope tricks)? |
|
|