[andy99]

Is the 2B measured like that as well? I did use it with llama.cpp and noticed it ran slower than I expected.

That's the danger of too much abstraction, it's easy to have big gaps in one's understanding of what's really going on.

[rasbt]

Yes, it's somewhat similar to the 2B model as it uses the same vocabulary size.