[piskov]

They went for local LLM route and for that high-bandwidth memory is a must.

Consider it a low-price alternative to mac mini or nvidia’s box.

This can also be chained though not as effectively as macs for example (those have thunderbolt for interconnect)

[awiesenhofer]

> low-price alternative to mac mini

This starts at 1099$ per the article, so the mac mini is the low-price alternative here.

[kllrnohj]

This starts with 32GB of RAM, so no the mac mini isn't the low priced alternative at that point.

Also fully spec'd at 128gb is $2000, which only gets you a 64gb mac mini

[TiredOfLife]

The max config with 128gb ram is half the price of mac and 8tb ssd is 800 and not 2000

[MindSpunk]

If you can find me a Mac Mini with 32GB of RAM I'd love to see it.

[zamadatix]

https://www.apple.com/shop/buy-mac/mac-mini/apple-m4-chip-wi...

Select 32

It's not exactly an out of the park win though as it's only $100-$200 cheaper and it trends poorly the farther from comparing base models you go (particularly after 64 GB, for which you need to leave the Mini family for the Studio). By the time you get to 128 GB, which you'd want for the 70B class AI models, you end up back at the original statement.

[MindSpunk]

Huh. Ate my words then. I briefly checked Apple's page and didn't see them.

However those are the base M4 chip and not the M4 Pro. You need the M4 Pro to get competitive in GPU compute numbers for a more like to like comparison. The M4 Pro mini comes out at $1800 with the 48GB option, or $2000 for 64GB. For the same price the Framework machine gets you 128GB.

[zamadatix]

I think you're probably right comparing to the M4 Pro would make more sense but keep in mind you don't really need much compute, it's just that the M4 Pro has memory bandwidth more similar to the AI Max 395+ while the "normal" M4 doesn't.

Every large AI model is heavy memory bandwidth constrained to the point my 9800X3D (with the extra L3 cache) and 128 GB/s memory attached is only 60% utilized running a 32 GB model in CPU only mode (no NPU, iGPU, or GPU offload enabled). Really small AI models can start to be compute bound but, at that point, you don't really need the 32 GB of memory anymore and probably just want a normal GPU.