Hacker News new | ask | show | jobs
by midulan 11 days ago
i'm getting similar toks on my 36gb m3 max. new omlx release (0.5.2) has dedicated decode kernels for the bonsai models — see if this improves perf for you.