Hacker News new | ask | show | jobs
by chrisweekly 6 days ago
Obtaining that 64GB RAM is a meaningful obstacle for many.
2 comments
simonw 6 days ago
I'm still amazed that you can run LLMs of this quality on a machine that costs less than $3,000.

I used to assume that anything GPT-4 equivalent or higher would need $30,000+ of server-class hardware.

That said... gemma-4-12b-qat is 7.15GB on disk so should run reasonably well in 16GB, that takes it down to MacBook Air territory https://lmstudio.ai/models/google/gemma-4-12b-qat

link
verdverm 5 days ago
Second this notion. After picking up an OEM Spark and running qwen36moe/dense, I was thoroughly impressed with what such small models can do and the (reasonable) speeds you can get. I'm back to using open weight models via an API (wanted more capability for the time being), but will be getting more hardware soon (re: ds4-flash and the fable shot heard round the world)
link
frollogaston 6 days ago
Not just RAM, VRAM, right? Though they're one and the same on the Mac.
link