Hacker News new | ask | show | jobs
by Const-me 32 days ago
> decode (GEMV) is memory bound

Decode with batch size 1 is GEMV. Batching makes the decode GEMM too.