|
|
|
|
|
|
by tverbeure
978 days ago
|
|
|
I think your biggest mistake is ignoring the amount of software that runs on top of the hardware. There were times in the past where others had a higher number of FLOPS, but it didn’t make a dent in the usage of CUDA. My standard assumption about anything in tech (and beyond) is that you don’t win on first principles alone (“parallel design is well understood”). It’s the details that make you stand out. Bill Daly gave a great presentation at Hot Chips a few weeks ago that explains how Nvidia’s AI performance was only in small parts due to process improvement. The rest was doing things smarter: matrix sparsity, different number representations and so forth. https://youtu.be/rsxCZAE8QNA?si=IZneUQQ1J7Gtu81H |
|
|