[throwaway4good]

I thought the news of them using Apple Silicon rather than NVIDIA in their data centers was significant.

Perhaps there is still hope of a relaunch of xserve; with the widespread use of Apple computers amongst developers Apple has a real chance of challenging NVIDIA's CUDA moat.

[pjmlp]

Not at Apple's price points.

[throwaway4good]

I think NVIDIA has the highest hardware markup at the moment.

[Hugsun]

You get considerably more ML FLOPS per dollar in a 4090 than any mac. It seems like the base M2 MAX is at roughly the same price point. It does grant you more RAM.

Quadro and Tesla cards might be a different story. I would still like to see concrete FLOPS/$ numbers.

[hajile]

They don't need the entire mac. Their cost per Max chip is probably $200-300 which beats the 4090 by a massive margin and each chip can do more than a 4090 because it also has a CPU onboard.

4090 peaks out at around 550w which means they can run 5+ of their Max chips in the same power budget.

A 4090 is $2000. Apple can probably get 5 chips on a custom motherboard for that cost. They'll use the same amount of power, but get a lot more raw compute.

[Hugsun]

> Their cost per Max chip is probably $200-300 which beats the 4090 by a massive margin...

That's true. I was talking about end user pricing.

> ...each chip can do more than a 4090 because it also has a CPU onboard.

That's a strange thing to say. It has a CPU, correct. It makes the chip more versatile but for data center ML tasks it doesn't really matter. A 4090 chip also has much more ML relevant compute per chip. So apple's chips can't really "do more than a 4090" in any relevant way.

Of course apple pays less for their in house made chips vs external products. That comparison doesn't seem relevant to the context, e.g. they're not going to challenging CUDA with internal chips.

They might get more compute per watt though. My guess is that nvidias datacenter chips are competitive in that space, but that's another story.

[eek2121]

The GPU in the M-series is much slower than a 4090. 4060-4070ish performance at best, and it varies quite a bit.

[ribit]

You need to consider this in the context of the relevant task. Nvidia GPUs have extremely high peak performance for GEMM, but when working with LLMs, bandwidth (and RAM capacity) becomes the limiting factor. There is a reason why real ML-focused datacenter Nvidia GPUs use much wider RAM interfaces and a much higher price point. The M2 Ultra might not have the raw compute, but it has a lot of RAM and large caches.

[hajile]

If they can get 5 4070s for the price and power of one 4090, that's a win for them as they'll get more performance per dollar and per watt.

[throwaway4good]

The M2 is a chip designed to be in a laptop (and it is quite powerful given its low power consumption). Presumedly they have a different chip or at least completely different configuration (RAM, network, etc.) in their data centers.

[mrweasel]

The interesting point here is that developers targeting the Mac can safely assume that the users will have a processor capable of significant AI/ML workloads. On the Windows (and Linux) side of things, there's no common platform, no assumption that the users will have an NPU or GPU capable of doing what you want. I think that's also why Microsoft was initially going for the ARM laptops, where they'd be sure that the required processing power is available.

[qwytw]

> The interesting point here is that developers targeting the Mac can safely assume that the users will have a processor capable of significant AI/ML workloads

Also that a significant proportion (majority?) of them will have just 8 GB of memory which is not exactly sufficient to run any complex AI/ML workloads.

[skohan]

I believe MS is trying to standardize this, in the same way as they do with DirectX support levels, but I agree it's probably going to be inherently a bit less consistent than Apple offerings

[InsomniacL]

That sounds like a big issue, but surely assuming for either case is bad.

I expect OS's will expose an API which, when queried, will indicate the level of AI inference available.

Similar to video decoding/encoding where clients can check if hardware acceleration is available.

[treprinum]

How does it help me (with maxed out M3 Max) that Apple might have some chip in the future right now? I do DL on A6000 and 4090, not waiting until Apple produces a chip someday that is faster than 1650 in ML...

[sofixa]

That's probably where Microsoft's "Copilot+ PCs" come in.

[skohan]

There was a rumor floating around that Apple might try to enter the server chip business with an AI chip, which is an interesting concept. Apple's never really succeeded in the B2B business, but they have proven a lot of competency in the silicon space.

Even their high-end prosumer hardware could be interesting as an AI workstation given the VRAM available if the software support were better.

[jitl]

> Apple's never really succeeded in the B2B business

Idk every business I’ve worked and all the places my friends work seem to be 90% Apple hardware, with a few Lenovo issued for special case roles in finance or something.

[esskay]

Of course you do, Apple's selling mobile SOC's not high end cards. That doesn't mean they're incapable of making them for the right application. You don't seriously think the server farms running on M4 Pro Max chips do you...

[pjmlp]

Depends on which card one is talking about.

[throwaway4good]

Maybe. It is not really obvious how much you for the AI accellerator part of their offerings. For example the chips in iPhones are quite powerful even adjusted for price. However for some cases - like the max chip in the macbooks or the extra ram - their pricing seems high - maybe even nvidia high.

[bayindirh]

I mean, even Apple can't match the markups nVidia has right now. If you break a GPU in your compute server, you wait months for a replacement, and the part is sent back if you can't replace it in five days.

Crazy times.

[c0balt]

Enterprise offerings tend to differ. You can get a replacement NVIDIA GPU via a partner, like Lenovo, in 2-3 weeks. And that's on the high side for some support contracts.

[bayindirh]

That's from HPE, for an Apollo 6500.

[grecy]

I'm wondering how my electricity they will save just from moving from Intel to Apple Silicon in their data centers.