Hacker News new | ask | show | jobs
by staticman2 475 days ago
Smaller, dumber models are faster than bigger, slower ones.

What model do you find fast enough and smart enough?
1 comments
Matl 475 days ago
Not OP but I am finding the Qwen 2.5 32b distilled with DeepSeek R1 model to be a good speed/smartness ratio on the M4 Pro Mac Mini.
link
bastardoperator 474 days ago
I'm running the same exact models.
link
a1o 475 days ago
How much RAM?
link
Matl 474 days ago
It takes between 22GB-37GB depending on the context size etc. from what I've observed.
link
a1o 474 days ago
Thanks!
link