[pjdesno]

Not mentioned in this is the issue of memory scaling.

DRAM price per GB has been roughly flat for well over a decade - consumer prices hit $4/GB in 2011, and have fluctuated around there ever since - most of the drop in real cost since then has been due to declining value of that $4. Prices for large enterprise/hyper scalers are probably similar, as it’s a low-margin commodity market.

Two sockets gets you more memory channels and more DINNS, but as memory price causes the RAM/CPU ratio to drop, and single-channel bandwidth increases with DDR5, that becomes less important.

Of course that’s one of those things you can’t really say to customers, kind of like “you don’t really need 250hp in a passenger sedan”.

[tbrownaw]

I thought that most of the reason that Apple's newer cpus are supposed to be so good for LLMs is that the integrated memory lets them have more channels than usual?

[wmf]

Yes, but that has nothing to do with servers which is the topic of this thread.