Hacker News new | ask | show | jobs
by popol12 1204 days ago
0.35 words/s on my 11th gen i5 with 7B model (framework laptop)

not so bad !
1 comments
fguerraz 1204 days ago
How long did you have to wait for it to load? On my machine it's been running for 15mins, I'm still waiting for a prompt...
link
popol12 1202 days ago
You get the full answer after completion, so it’s normal if you don’t get an output immediately

I computed the speed by doing speed=number of words/ total run time

link
kristianp 1204 days ago
How much RAM do you both have?
link
popol12 1202 days ago
32gb of ram and 64gb on NVME swap
link