At least for Nvidia and AMD desktop GPUs/cards, the priorities are:
#1 (by far) "can it play [hot game of the moment] in 4K?"
#2 (maybe) "is the fan noisy in desktop mode?"
#3 (for nerds) "can I run LLMs on it?"
I don't think top efficiency in desktop mode is on anyone's list, and even if it were, it would be hard to come up with a design that uses hundreds of watts when running top-tier games or LLMs, but also uses as little as possible when idle?
No, rather that they focused on peak performance, i.e., "churn out the most raw throughput at 400W+", rather than "get as close as possible to 0W when idle or in common uses".
Very different metrics - the former is about optimizing your architecture for pushing the most operations, the latter is about being able to power as many things off as possible.
TL;DR: Low power draw for laptops and phones is about who can reduce to the lowest performance, with the most hardware turned completely off while still just barely performing the task at hand. Completely different ballgame.
Peak performance happens at peak power draw and is a matter of having as much hardware as possible pushing as many operations as possible at any given point without spontaneously combusting. Those who have the most advanced manufacturing process, or architecture with the most execution units and best able to keep all units busy wins.
Peak power efficiency is about being able to turn as much hardware off as possible, and having lower quiescent current ("leakage power"), with bigger, beefier chips naturally having higher quiescent currents.
What is talked about here is about gating hardware such that you can shave off milliwatts or microwatts when the system is completely idle, by taking tasks that otherwise use slightly larger blocks that would have to remain on and moving them to smaller, more dedicated blocks. For example, being able to play a video with most render capabilities powered down because the display server can take the output of a hardware video decoder block and feed it straight into a display controller plane.
I meant peak performance vs peak performance though. An M1 Ultra uses 25% of the energy of an RTX3090 but gets 70% of the FPS. And in synthetic tests the M1 Ultra can get as close as 95% (!).
Nvidia (and AMD and Intel) just suck at efficiency. There is no excuse for such a performance-per-watt delta. The same is true in CPU land.
Once the first ARM Steam Deck launches gamers will realize they've been had for a decade.