[tverbeure]

What does Microsoft make? You download bits over a wire and store them on your personal SSD. They don’t make SSDs so it should be trivial for other companies to quickly catch up.

For the sake of argument, let’s reduce Nvidia (my employer but I don’t speak for them) to just a silicon vendor.

You’re not only ignoring the work of thousands who design the stuff that goes on that silicon, the billions of $ in R&D, you’re also dismissing the decisions about what to put in that design, way before other people do it. Some of those decisions were made years before the actual implementation started. It happens to be something a few people at Nvidia are very good at.

[rnikander]

Yes, my short post may sound dismissive, but I work in tech and I'm aware that our industry has lots of legitimate creativity and intellectual property. Of course we also have hype, marketing, BS, and herd stampedes. Some aspects of GPU/parallel design are commonly understood and other companies make such chips. So I guess I was just wondering if Nvidia had a secret sauce, are they ahead of others, and if so by what measure? I work more in software, and I'm not in a position now to evaluate their hardware on the merits.

[tverbeure]

I think your biggest mistake is ignoring the amount of software that runs on top of the hardware.

There were times in the past where others had a higher number of FLOPS, but it didn’t make a dent in the usage of CUDA.

My standard assumption about anything in tech (and beyond) is that you don’t win on first principles alone (“parallel design is well understood”). It’s the details that make you stand out.

Bill Daly gave a great presentation at Hot Chips a few weeks ago that explains how Nvidia’s AI performance was only in small parts due to process improvement. The rest was doing things smarter: matrix sparsity, different number representations and so forth.

https://youtu.be/rsxCZAE8QNA?si=IZneUQQ1J7Gtu81H