CUDA came out in 2007. Wikipedia puts the start of the GPU-driven 'deep learning revolution' in 2012 [1] and people have been putting GPUs into their supercomputers since 2012 as well [2]
I find it strange that Intel has basically just left the entire market to nvidia, despite having 10-15 years warning and running their own GPU division the whole time.
Competing with Nvidia on Gaming GPU wasn't something Intel were keen to do after their failure with i740. The Gaming market wasn't as big, and you are ultimately competing on Driver optimisation, not on actual hardware.
CUDA and Deep Learning may have started in 2007 and 2010. But their usage, or their revenue potential was unclear back then. Even in 2015, Datacenter revenue was less than one eighth of gaming revenue. And rumours of Google AI Processor ( now known as TPU ) started back in 2014 when they started hiring. In 2021, Datacenter is roughly equal to Gaming revenue, and are expected to exceed them in 2022.
Intel sort of knew GPGPU could be a threat by 2016 / 17 already. That is why they started assembling a team, and hired Raja Koduri in late 2017. But as with everything Intel in post Pat Gelsinger era, Intel was late to react. From Smartphone to Foundry Model and now GPGPU.
They created the Xeon Phi[1] for that niche. It was spun out of Larabee[2]. I presume they will be taking advantage of their coming GPU architecture for more going forward.
They tried to check many, some, maybe possibly more of the boxes with the Xeon Phi, and it kinda seems like things simply didn't go their way.
Cuda wasn't as flexible, and the payoff wasn't as big in 2010 or so as it is now.
I've never used a phi, but i can see where they were coming from i think. No need for a full rewrite like Cuda (maybe). The hardware is also more flexible than a GPU, but that turned out to be less important than they thought it might be.
this isn't true. the phi was extremely complex to program for, and it was not simply a port of standard x86 code. it required you to pay attention to multiple levels of memory hierarchy, just as the GPU did.
CUDA and Deep Learning may have started in 2007 and 2010. But their usage, or their revenue potential was unclear back then. Even in 2015, Datacenter revenue was less than one eighth of gaming revenue. And rumours of Google AI Processor ( now known as TPU ) started back in 2014 when they started hiring. In 2021, Datacenter is roughly equal to Gaming revenue, and are expected to exceed them in 2022.
Intel sort of knew GPGPU could be a threat by 2016 / 17 already. That is why they started assembling a team, and hired Raja Koduri in late 2017. But as with everything Intel in post Pat Gelsinger era, Intel was late to react. From Smartphone to Foundry Model and now GPGPU.