[smoldesu]

A proper hardware encoder will absolutely smoke CPU decoding. Switching from OpenGL software rendering to NVENC/VAAPI reduced power consumption considerably for me. Nvidia and Apple's hardware-based decoders are both top-notch, with Intel's also being fairly decent. Like the other commenter said, I've heard mixed things about the AMD decoders (not to insinuate that their GPUs/iGPUs are bad, though).

As for machine learning, most metrics right now suggest that conventional GPGPU calculations scale with power consumption. If you want high-performance ML without specialized hardware, your only choice is high-wattage hardware.