[zzzoom]

Prefill (GEMM) is compute bound, decode (GEMV) is memory bound.

[Const-me]

> decode (GEMV) is memory bound

Decode with batch size 1 is GEMV. Batching makes the decode GEMM too.