Hacker News new | ask | show | jobs
by andy_ppp 248 days ago
It can run larger models quite slowly but lacks matmul acceleration (included in the M5) that is very useful for context and prompt performance at inference time. I will probably burn my budget with an M5 Max with 256gb (maybe even 512gb) memory, the price will be upsetting but I guess that is life!
1 comments
sgt 248 days ago
Yes! I think smaller models on the M3 Ultra is interesting enough, but now with matmul/ tensors on M5 Ultra or Max, with decent unified mem, it will be a gamechanger.

I can easily imagine companies running Mac Studios in prod. Apple should release another Xserve.

link
andy_ppp 247 days ago
Yes completely, my guess is M6 will have external GPUs perfect for AI accelerators at home and in datacenters.
link