Hacker News new | ask | show | jobs
by talldayo 756 days ago
Gemma 2B and Phi-3 3B, if you run them at Q4 quantization. I wouldn't bother with anything larger than 4B parameters; you're just not going to be able to reliably expect an end-user to run that size of model on a phone yet.