[Maursault]

> Local LLMs do best on big GPUs, which aren’t an option on Macs. What Apple has done with the M-series silicon is really impressive, but it’s not as fast as big, dedicated GPU silicon.

It'd be pretty silly for Apple to cram in big, dedicated GPU when 99.9% of their customers don't care and don't want to pay for it, especially considering that anyone that does want big, dedicated GPU can outboard as much GPU as they want.[1] And many seem to think that the onboard GPU along with Neural Engine should be adequate for local LLM.

[1] https://support.apple.com/en-us/HT208544

[thoughtsimple]

Only on Intel Macs which are going away soon (only 2019 Mac Pro is remaining outside of refurbished.)

Apple Silicon doesn't currently have any provision for external graphics cards.

[Maursault]

There are two issues.

     1) Apple Silicon won't support PCIe. 

     2) Apple doesn't want it to.

#2 means Apple is taking nVidia and AMD head on in the GPU space. Apple wants to control everything, and allowing these competitors on their platform is giving away too much. Because Apple Silicon scales better than competitors' hw, the desire for third party GPU is probably going to evaporate within a few generations of Apple Silicon. I mean, we'll see, but that is my best guess, because it seems like that was an intentional decision rather than oversight.

[bhj]

AS supports Thunderbolt just fine. Isn't the lack of eGPU support due to lack of ARM drivers for them?

[Maursault]

To suggest it is merely lack of drivers is an oversimplification. There is a chip errata that prevents PCIe GPUs from working properly on Apple Silicon: the architectures are not compatible. Apple Silicon GPU drivers are deeply integrated into the system. Due to this integration, only graphics cards that use the same GPU architecture as Apple Silicon could be supported, and there just aren't any, and I don't see how there could be unless Apple developed one and released it.

[GeekyBear]

> To suggest it is merely lack of drivers is an oversimplification.

Not really. When you hook up an external PCI chassis with a graphics card inserted, it sees the PCI expansion slot and the GPU just fine, it just doesn't have a driver for the GPU.

[Maursault]

Not really. Software drivers alone will not get it done, at least not adequately (Asahi-related developers may come up with a software solution for Asahi, but it will necessarily degrade performance, so the effort is likely to be abandoned).

Thunderbolt supports PCIe, and for most devices Apple Silicon does also, but GPU is different enough from audio interfaces and NVMe that it isn't just a "load the driver and plug it in" situation. GPU is vastly more complex than other PCIe devices. Apple Silicon and x86 architectures are not compatible, so GPU for x86 is not going to work with Apple Silicon with merely a software driver.

It's going to take hw translation and other technology that is not yet available for Apple Silicon, thus Apple's recent patent applications,[1] showing that Apple is either exploring supporting outboard GPU or locking anyone else out from their method of doing so, but either way is no guarantee they'll complete development or release, because it seems just as if not more likely the roadmap for Apple Silicon GPU performance will outpace nVidia and AMD GPUs.

But, again, claiming, "it just doesn't have a driver for the GPU" is a staggering oversimplification.

[1] https://image-ppubs.uspto.gov/dirsearch-public/print/downloa...

[nordsieck]

It'll be interesting to see what happens with the Mac Pro.

It could be that Apple does away with it entirely and the Mac Studio is the new Pro.

Or they might make a machine with PCIe support, but make it so expensive that only people with a serious need get access to it.

Or something else.

[Maursault]

> It could be that Apple does away with it entirely and the Mac Studio is the new Pro.

Or the Mac Pro will be released without PCIe GPU support, and Apple will be able to leverage increases in Apple Silicon GPU performance to eliminate any need or desire for PCIe GPU, drawing away high end GPU customers from nVidia and AMD and locking them into Apple Silicon and the Apple ecosystem.

[nordsieck]

> Or the Mac Pro will be released without PCIe GPU support, and Apple will be able to leverage increases in Apple Silicon GPU performance to eliminate any need or desire for PCIe GPU, drawing away high end GPU customers from nVidia and AMD and locking them into Apple Silicon and the Apple ecosystem.

Maybe. But graphics cards aren't the only thing people put into PCIe slots.

[Maursault]

> Maybe. But graphics cards aren't the only thing people put into PCIe slots.

Right, so there will be PCIe slots for expansion, they just won't support PCIe GPU, just like Thunderbolt PCIe expansion chassis for Thunderbolt now. It isn't the PCIe slots that break compatibility, its the difference in architecture between x86 and Apple Silicon that makes them incompatible.