Hacker News new | ask | show | jobs
by embedding-shape 22 hours ago
> suspect it’s better than say a 9b or 12b model

Whaaat, a 27b model might be better than 9b or 12b model? What would make you do such an outrageous claim?

1 comments

Sorry I should have clarified - I meant that a ternary 27b model would outperform a non-quantized or 8 bit quantized 9 or 12b model - which it is generally close to (or much smaller than) in size. So yeah the comparison I was trying to make was between models of equivalent size or models that could run on similarly sized hardware.
Ah yeah, that makes a ton more sense :) I mean, what you said earlier also makes sense but was too obvious, now it makes sufficient sense, thanks for explaining :)