[sealeck]

I think it's a bit hard to say that this is definitively true: people have always been interested in running linear algebra on computers. In the absence of NVIDIA some other company would likely have found a different industry and sold linear algebra processing hardware to them!

[senbrow]

Almost certainly not at the scale of the consumer gaming industry, however!

[willis936]

Google is making millions of TPUs per year. Nvidia ships more gaming GPUs, but it's not like multiple orders of magnitude off.

[senbrow]

I'm willing to bet TPUs wouldn't be nearly as successful or sophisticated without the decades of GPU design and manufacturing that came before them.

Current manufacturing numbers are a small part of the story of the overall lineage.

[hparadiz]

It's pretty interesting that consumer GPUs started to really be a thing in the early 90s and the first Bitcoin GPU miner was around 2011. That's only 20 years. That caused a GPU and asic gold rush. The major breakthroughs around LLMs started to snowball in the academic scene right around that time. It's been a crazy and relatively quick ride in the grand scheme of things. Even this silicone shortage will pass and we'll look back on this time as quaint.

[danielmarkbruce]

Of course you are right, but in addition they wouldn't have even made them if GPUs hadn't made ML on CPU so relatively incapable. Competition drives a lot of these decisions, not just raw performance.

[danielmarkbruce]

You are missing his point. They very likely don't start building TPUs if there were no GPUs.

[willis936]

I'm not missing the point. If you recall your computer architecture class there are many vector processing architectures out there. Long before there was nvidia the world's largest and most expensive computers were vector processors. It's inaccurate to say "gaming built SIMD".

[danielmarkbruce]

You are missing the point - it's an economic point. Very little R&D was put into said processors. The scale wasn't there. The software stack wasn't there (because the scale wasn't there).

No one is suggesting gaming chips were the first time someone thought of such an architecture or built a chip with such an architecture. They are suggesting the gaming industry produced the required scale to actually do all the work which lead to that hardware and software being really good, and useful for other purposes. In chip world, scale matters a lot.

[willis936]

The Cray-1, which produced half a billion USD in revenue in today's dollars, at a time when computing was still science fiction, did not demonstrate scale? I just can't take you in good faith because there has never been a time when large scale SIMD computing was not advanced by commercial interests.