[lnenad]

Cool demo, but even the smallest model does 2.5 tps on my 7900xtx which is not great.

[christkv]

Hmm I get 44,7 tokens/s on the smallest 1.7b one in the browser on my m1 max.

[k3liutZu]

I get 9-10 tps on my MacBook Pro. Though the small model seem really limited

[whstl]

Interestingly, I get 44 tps on Safari, MacBook Pro base model.

[Tade0]

60tps in Chrome here - 14" MBP M4. Safari loads 3% of the model and then stops - repeatedly.

Basically zero using a 7700S in Chrome on Ubuntu though. Must be not working at all, as the performance difference can't be that large.

[LorenDB]

18.8 tps on an 11th gen mobile i7 here.