Hacker News new | ask | show | jobs
by popol12 1201 days ago
Using which model ? On a pretty mid range i5 11th gen I'm getting 0.35 token/s, using the 7B model. Haven't tried the bigger models.