[brookst]

…and for models that require 64GB of VRAM? 120GB of VRAM?

You can get a 128GB UMA mac for less than a single 48GB a100, let alone a single 96GB a100.

I think Apple got incredibly lucky here, but I don’t see how the PC world catches them any time soon. We’ve all known that UMA is theoretically better for ages, but Apple’s timing couldn’t be better. And scale economies mean they can sell the same chip to people who need 100GB of system RAM and people who need 100GB of VRAM.

If they can get their GPU / neural performance up and sort out their terrible relationship with academic research, they could snipe ML away from nvidia. It seems very unlikely, but it’s kind of stunning that it’s even in the realm of possibility.

[smoldesu]

> they could snipe ML away from nvidia.

If Nvidia announced tomorrow that they were cancelling every datacenter deal they had, open-sourcing CUDA and publishing their entire patent library to the creative commons, I would still not believe you.

This is a fun project for people with Apple Silicon machines who want to participate in the AI happenings, but I don't think you can warp it into a call for Nvidia's head. Let's wait until Apple pulls the curtains on their rackmount Mac Pros, so we can compare it with Nvidia's ARM server offerings: https://www.nvidia.com/en-us/data-center/grace-cpu/

[brookst]

Whoa, who’s calling for nvidia’s head? Not me.

My point was that the PC architecture of separate system and GPU memory is hitting a wall that means inefficiency and higher prices.

I have little doubt that Nvidia’s attempted acquisition of ARM was in part because nvidia recognized this. I expect they are exploring other UMA approaches. But it will be hard in the fragmented, not-vertically-integrated model.

Apple’s advantage here is one platform that can scale: it is hard to imagine Grace and similar running Windows on developer’s desktops. Maybe!

But my point was that, shockingly, Apple has a chance here. A small chance, as I said, but I don’t think anyone (including Apple) saw just how soon UMA was going to become a competitive advantage.

[smoldesu]

Nvidia doesn't need to acquire ARM to sell systems with unified memory. The Tegra boards are all mixed-address-space systems, and CUDA lets you manipulate memory over PCI. They see the writing on the wall, which is why they've been building systems for the past decade that reflect this philosophy.

If you think it's hard to imagine Nvidia hardware running on a developer desktop, wait until you hear about what happened when Macs tried entering the server market.