[skavi]

and the repo for this project: https://github.com/microsoft/BitNet

[sinuhe69]

The demo they showed was full of repeated sentences. The 3B model looks quite dense, TBH. Did they just want to show the speed?

[newswasboring]

3B models, especially in quantized state, almost always behave like this.