[niftich]

Note that the article is about innovation and performance gains in general-purpose processors slowing, and about the increasing shift onto specialized computing engines by those who seek further performance gains for specific workloads.

This was foreshadowed with the Netburst not being able break 4 GHz in 2004-2005, and CPUs having to shift to multi-core. This bought "classic" CPUs more time, but CUDA showed up in 2007 and GPUs went from strictly specialized computing engines to general-purpose (in research, if not yet in the home). CPUs have also been steadily gaining SIMD extensions.

Now GPUs are showing promise for NN workloads, but in environments where the stack is tightly controlled, NN co-processors are showing up. This is because tightly controlling the stack has the benefits of being able to optimize and harmonize software and hardware, and interop outside of the stack (and in some cases, stack longevity) is not a factor.

The article isn't truly about how more and more computing environments tightly control their stack, but that mechanism does play a part in the design choices that result.

[petermcneeley]

The article discusses the current trend but you hint at the possibility of a trend reversal. The CPUs are more and more multicore with wider SIMD. Many now have dynamic clock rates or low clock rates. The GPUs which use to actually have fixed functions now more and more are just GPGPUs. This to me is a convergence.