[mrweasel]

The interesting point here is that developers targeting the Mac can safely assume that the users will have a processor capable of significant AI/ML workloads. On the Windows (and Linux) side of things, there's no common platform, no assumption that the users will have an NPU or GPU capable of doing what you want. I think that's also why Microsoft was initially going for the ARM laptops, where they'd be sure that the required processing power is available.

[qwytw]

> The interesting point here is that developers targeting the Mac can safely assume that the users will have a processor capable of significant AI/ML workloads

Also that a significant proportion (majority?) of them will have just 8 GB of memory which is not exactly sufficient to run any complex AI/ML workloads.

[talldayo]

Easy solution; just swap multiple gigabytes of your model to SSD-based ZRAM when you run out of memory. What could possibly go wrong?

[skohan]

I believe MS is trying to standardize this, in the same way as they do with DirectX support levels, but I agree it's probably going to be inherently a bit less consistent than Apple offerings

[pjmlp]

DirectML can use multiple backends.

[InsomniacL]

That sounds like a big issue, but surely assuming for either case is bad.

I expect OS's will expose an API which, when queried, will indicate the level of AI inference available.

Similar to video decoding/encoding where clients can check if hardware acceleration is available.

[treprinum]

How does it help me (with maxed out M3 Max) that Apple might have some chip in the future right now? I do DL on A6000 and 4090, not waiting until Apple produces a chip someday that is faster than 1650 in ML...

[sofixa]

That's probably where Microsoft's "Copilot+ PCs" come in.

[pjmlp]

Plus DirectML, wich as the name implies, builds on top of DirectX, allowing multiple backends, CPU, GPU, NPU.