Hacker News new | ask | show | jobs
by mips_avatar 29 days ago
The cool thing about the 3090s is the RAM bandwidth. Token generation is mostly bottlenecked on memory bandwidth. Dual 3090s have 1.87 TB/s memory bandwidth (0.936 TB/s each), vs the M5 Macbook pro with only 0.3 TB/s (max chip has up to 0.63 TB/s but it's a $10k machine at that config).

This translates to qwen 27b actually working fast enough for useful work on dual 3090s and being painfully slow on Macbook Pros. Also if you're running a big model on a macbook pro the UI gets laggy and the keyboard gets hot. Much better to run dual 3090s in your basement and connect to them from your Macbook.
2 comments
CobaltFire 29 days ago
$4.8k for 48GB Max (what the parent said). Half of your quote.

Even a 128GB is $6.8k today. Still only 2/3 your quote.

Bandwidth is relevant (I have both a 5090 and an M4 Max 128GB Studio, so have direct comparison right here), but quote the cost appropriately!

link
mips_avatar 29 days ago
You need the 128gb ram config to get the 614 GB/s bandwidth (which is $6999), you could skip out on upgrading the storage to save money but at that point I think most people upgrade the storage too at which point it's $8-10k + tax.
link
CobaltFire 29 days ago
No? Any M5 Max with the upgraded GPU has the full bandwidth, which includes the 48GB model the original poster mentioned. Same as the M4 Max, where only the trimmed part had a lower bandwidth.

Why are you throwing in extra cost for something thats not necessary? I know multiple people with 128GB Macs and none of us upgraded the storage. Especially not on a Studio (which isn't currently available).

I will say that their $3k number is off. I somehow missed that, and its too low.

link
mips_avatar 29 days ago
I made a mistake, there is a $5k config with high memory bandwidth. The Max chip has two tiers (I incorrectly thought the tiers were based on memory capacity), you need the higher tier Max GPU upgrade (+$300) to get the 614 GB/s memory bandwidth but you don't need to upgrade the RAM to 128 GB to get the full memory bandwidth. So to get the 614 GB/s you need to upgrade to the max chip + upgraded GPU, but you can spec it at only 48gb if you want. So the total for an m5 max with 614 GB/s memory bandwidth is $4999-$9999 depending on config.

Still 3x lower memory Bandwidth than a dual 3090 setup which you can build for $3k with parts from facebook marketplace and run in your basement.

link
titanomachy 29 days ago
The bandwidth argument is compelling, do we have benchmarks for these models? I’m curious what it translates to in tokens per second
link
mips_avatar 29 days ago
I benchmarked mine for a deep research workload I was running. Concurrency 1 is the speed you'd get if you're chatting with one agent,

2x3090 (has an nvlink bridge though it didn't seem to matter hugely for inference)

Qwen 3.6 27b int4: Concurrency 1: 68 tok/s output Concurrency 32: 363 tok/s output Prompt processing speed: 1520 tok/s

Qwen 3.6 35ba3b int4: Concurrency 1: 150 tok/s output Concurrency 32: 1083 tok/s output Prompt processing speed: 4324 tok/s

Macbook Pro m3 36gb RAM: Qwen 3.6 27b int4: Concurrency 1: 18 tok/s output didn't measure the other metrics and it was a slightly different benchmark.

link
titanomachy 28 days ago
Yeah that’s a huge difference. I don’t think I want to interactively use any model with 18 tps.
link