Hacker News new | ask | show | jobs
by dchest 361 days ago
I'm using it on MacBook Air M1 / 8 GB RAM with Qwen3-4B to generate summaries and tags for my vibe-coded Bloomberg Terminal-style RSS reader :-) It works fine (the laptop gets hot and slow, but fine).

Probably should just use llama.cpp server/ollama and not waste a gig of memory on Electron, but I like GUIs.
1 comments
minimaxir 361 days ago
8 GB of RAM with local LLMs in general is iffy: a 8-bit quantized Qwen3-4B is 4.2GB on disk and likely more in memory. 16 GB is usually the minimum to be able to run decent models without compromising on heavy quantization.
link
hnuser123456 361 days ago
But 8GB of Apple RAM is 16GB of normal RAM.

https://www.pcgamer.com/apple-vp-says-8gb-ram-on-a-macbook-p...

link
minimaxir 361 days ago
Interestingly it was AI (Apple Intelligence) that was the primary reason Apple abandoned that hedge.
link
arrty88 361 days ago
I concur. I just upgraded from m1 air with 8gb to m4 with 24gb. Excited to run bigger models.
link
diggan 360 days ago
> m4 with 24gb

Wow, that is probably analogous to 48GB on other systems then, if we were to ask an Apple VP?

link
vntok 360 days ago
Not sure what Apple VPs have to do with the tech but yeah, pretty much any core engineer you ask at Apple will tell you this.

Here is a nice article with some info about what memory compression is and how it works: https://arstechnica.com/gadgets/2013/10/os-x-10-9/#page-17

It's been a hard technical problem but is pretty much solved by now since its first debut in 2012-2013.

link
pxc 360 days ago
I've heard good things about how macOS handles memory relative to other operating systems. But Linux and Windows both have memory compression nowadays. So the claim is then not that memory compression makes your RAM twice as effective, but that macOS' memory compression is twice as good as the real and existing memory compression available on other operating systems.

Doesn't such a claim... need stronger evidence?

link
dchest 361 days ago
It's 4-bit quantized (Q4_K_M, 2.5 GB) and still works well for this task. It's amazing. I've been running various small models on this 8 GB Air since the first Llama and GPT-J, and they improved so much!

macOS virtual memory works well on swapping in and out stuff to SSD.

link