[jiggawatts]

In practice it just means that the processor includes a matrix multiply unit somewhere.

For example, some of the latest Intel Xeon CPUs have 8x 1KB registers that can perform only one operation: matrix multiply (and accumulate results). See: https://en.wikipedia.org/wiki/Advanced_Matrix_Extensions

Generally this is very similar to the Vista-era Microsoft requirements for PC manufacturers, where it was a logo requirement to include a GPU that could be used to accelerate composition in the desktop window manager, such as transparency effects and blur. (Prior to that, low-end PCs had "graphics cards" that were fixed pipeline and not programmable /general purpose.)

Now Microsoft is forcing PC manufacturers to do the same kind of thing, but instead of a GPU it's now an "NPU" that they have to include. This can be a CPU instruction set, a co-processor, or a GPU baseline capability. The requirement is 40 teraoperations per second.

IMHO, 40 TOPS is way too low, and doesn't focus enough on memory bandwidth. Also, that 40 is total across CPU+GPU+NPU, which means in practice it'll require fiddly optimisation to get anywhere near that level of performance.

Windows Vista had the same issue, where many low-end laptops especially would struggle and spin up their GPU fans just from desktop workloads, let alone gaming...

[samspenc]

"40 TOPS is way too low", I'm curious about this, from what I've read, Apple's M3 maxes out at 18 TOPS, and the newer M4 at 38 TOPS, so to me it sounds like even the entry-level Copilot+ PC is going to beat Apple's M3/M4 family. Am I misunderstanding.

[wmf]

It's possible that Microsoft's target is too low and Apple's performance is also too low. However when you find yourself saying that the entire industry is wrong you might want to stop and think.

[jiggawatts]

An NVIDIA 4070 puts out 836 "AI TOPS" using 200 watts of power. So the 40 TOPS target is about 10 watts of power draw equivalent, assuming it uses a similar silicon logic tech to what NVIDIA used. With a more modern process, this is about 5 watts.

For an ultra-mobile tablet or laptop, this is... reasonable, I suppose.

For a desktop, it's quite a bit behind the current-gen tech, let alone "the future".

Like I was saying, this is aiming for where the puck was, not where the puck will be.

It's the same thing as Vista GPU requirements. Vendors will do the bare minimum that ticks the checkbox, but in practice it'll be useless garbage.

Modern AIs require > 1 TB/s memory bandwidth, > 128 GB memory capacity, and > 1,000 TOPS of compute to be really usable locally, not just technically capable of "running".