[talldayo]

> Metal is the answer. Everything else is just implementation detail as GP said.

You can say this as long as you want, Nvidia makes money hand-over-fist supporting CUDA alongside OpenCL and DirectX. It's all just business to them - they don't have to play the same game as Apple because they're just not quite so petty with the ecosystem politics.

Look at MacOS, for example. Plenty of legacy software never was supported in Metal, it's "implementation detail" never manifested. It wasn't even really used in AI either until Apple upstreamed their own MPS hacks into Pytorch and people got BERT et. al. working, and even that was a pint-sized party trick that you could do on a Raspberry Pi. Apple themselves aren't even using their own servers for serious inference either, because you can't. It's gotta be offloaded to a lower-latency platform.

It's not just that Metal as a platform has failed it's users, although it's certainly contributed to developers giving up on Mac hardware for serious compute. Apple's GPU design is stuck in iPhone mode and they refuse to change their approach with Apple Silicon desktop hardware. It was Apple's big bet on NPUs that hamstrung them, not an implementation detail, and if you don't believe me then wait and see. Xserve didn't tear down the 1U market, Asahi didn't upend Linux HPC, and Metal isn't going to upend AI compute any more than DirectX will. This is the same "Apple will get 'em next year" quote we always hear when they fuck up, and they never actually seem to swallow their pride and take notes.

[astrange]

Apple are using their own servers for inference, that's the whole private cloud compute thing. Siri and other things use models and probably aren't running on it (though it's not announced), but those are older.

> Apple's GPU design is stuck in iPhone mode and they refuse to change their approach with Apple Silicon desktop hardware.

Looks competitive to me.

https://venturebeat.com/ai/you-can-now-run-the-most-powerful...

[talldayo]

> Apple are using their own servers for inference, that's the whole private cloud compute thing.

Not for everything, though. Any ChatGPT/OpenAI-based inference request is being sent to Nvidia GPUs that run models too large for even the biggest Mac servers. You cannot refute this simply because Apple does not sell DGX-like server products. Even the rackmount Apple Silicon is still orders-of-magnitude off on the kind of performance you can get from a 1u GPU rack.

> Looks competitive to me.

When compared on equal grounds, Apple doesn't even have a GPU that beats Nvidia's 30XX series on power efficiency: https://browser.geekbench.com/opencl-benchmarks

If it "looks competitive" to you, then I invite you to look closer than just qualitative evidence. Apple's 3nm desktop designs are losing in straight-shot comparisons with Nvidia's 8nm products.