[jsiepkes]

There was a time when Intel seemed unbeatable. In 2000 they had a 500 billion USD valuation. That's almost a trillion dollars in today's (2024) USD. Today they are valued at 90 billion USD and Broadcom was thinking about buying them...

My point is these things don't seem to last in tech.

[loa_in_]

Over 23 years almost every person working there probably either moved on or changed roles. Corporations are made of people. It lasted as long as it should.

[jeffbee]

Let's don't ignore context. In 2000 JDS Uniphase was worth $125 billion. A lot of things were "worth" a lot of money in the year 2000. Anyway Intel got their ass handed back to them in 2003 with the AMD Opteron. Today is not Intel's first struggle.

[ryao]

What made Intel seem unbeatable was its process node advantage. Nvidia does not have fabrication plants, so it is able to get the best process node from whoever has it. Nvidia is therefore not vulnerable to what befell Intel.

What makes Nvidia seem unbeatable is that Nvidia does the best job on hardware design, does a good job on the software for the hardware and gets its designs out quickly such that they can charge a premium. By the time the competition makes a competitive design, Nvidia has the next generation ready to go. They seem to be trying to accelerate their pace to kill attempts to compete with them and so far, it is working.

Nvidia just does not do the same thing better in a new generation, but tries to fundamentally change the paradigm to obtain better than generational improvements across generations. That is how they introduced SIMT, tensor cores, FP8 and more recently FP4, just to name a few. While their competitors are still implementing the last round of improvements Nvidia made to the state of the art, Nvidia launches yet another round of improvements.

For example, Nvidia has had GPUs on the market with FP8 for two years. Intel just launched their B580 discrete GPUs and Lunar Lake CPUs with Xe2 cores. There is no FP8 support to be seen as far as I have been able to gather. Meanwhile, Nvidia will soon be launching its 50 series GPUs with FP4 support. AMD’s RDNA GPUs are not poised to gain FP8 until the yet to be released RDNA 4 and I have no idea when Intel’s ARC graphics will gain FP8. Apple’s recent M4 series does have FP8, but no FP4 support.

Things look look less bad for Nvidia’s competitors in the enterprise market, CDNA 3 launched with FP8 support last year. Intel had Gaudi 2 with FP8 support around the same time as Nvidia, and even launched Gaudi 3. Then there is tenstorrent with FP8 on the wormhole processors that they released 6 months ago. However, FP4 support is no where to be seen with any of them and they will likely not release it until well after Nvidia, just like nearly all of them did with FP8. This is only naming a few companies too. There are many others in this sector that have not even touched FP8 yet.

In any case, I am sure that in a generation or two after Blackwell, Nvidia will have some other bright idea for changing the paradigm and its competition will lag behind in adopting it.

So far, I have only discussed compute. I have not even touched on graphics, where Nvidia has had many more innovations, on top of some of the compute oriented changes being beneficial to graphics too. Off the top of my head, Nvidia has had variable rate shading to improve rendering performance, ray tracing cores to reinvent rendering, tensor cores to enable upscaling (I did mention overlap between compute and graphics), optical flow accelerators to enable frame generation and likely others that I do not recall offhand. These are some of the improvements of the past 10 years and I am sure that the next 10 years will have more.

We do not see Nvidia’s competition put forward nearly as many paradigm changing ideas. For example, AMD did “smart access memory” more than a decade after it had been standardized as resizeable bar, which was definitely a contribution, but not one they invented. For something that they actually did invent, we need to look at HBM. I am not sure if they or anyone else I mentioned has done much else. Beyond the companies I mentioned, there are Groq and Cerebras (maybe Google too, but I am not sure) with their SRAM architectures, but that is about it as far as I know of companies implementing paradigm changing ideas in the same space.

I do not expect Nvidia to stop being a juggernaut until they run out of fresh ideas. They have produced so many ideas that I would not bet on them running out of new ideas any time soon. If I were to bet against them, I would have expected them to run out of ideas years ago, yet here we are.

Going back to the discussion of Intel seeming to be unbeatable in the past, they largely did the same thing better in each generation (with occasional ISA extensions), which was enough when they had a process advantage, but it was not enough when they lost their process advantage. The last time Intel tried to do something innovative in its core market, they gave us Itanium, and it was such a flop that they kept doing the same thing incrementally better ever since then. Losing their process advantage took away what put them on top.

[baal80spam]

> In any case, I am sure that in a generation or two after Blackwell, Nvidia will have some other bright idea for changing the paradigm and its competition will lag behind in adopting it.

This is the most important point. Everyone seems to think that Nvidia just rests on its laurels while everyone and their dog tries to catch up with it. This is just not how (good) business works.

[Jlagreen]

Nvidia once rested on their laurels and that was the Geforce FX over 20 years ago. Jensen was so pissed, he literally screamed at every person in the company.

But he also made sure that resting won't ever happen again. Andy Grove from Intel once said that only the paranoid survive and I bet Jensen is the most paranoid CEO alive. You won't see him in public that way because Jensen in public is Nvidia marketeer.

Nvidia has also understood early on how important marketing and brand recognition is. They learned it the hard way with the utter failure of their first chip the NV1 which was a technological master piece which no one wanted. Witht the bad GeForce FX Nvidia even made marketing videos to make fun of themselves and that helped. ATI didn't crush Nvidia as much as expeceted because of such activities despite having the ultra superior Radeon 9700/9800 series back then.

[xnx]

Nvidia has been very smart to be well prepared, but the emergency of Bitcoin and AI were two huge bits of good luck for them. It's very unlikely that there is another once-in-a-lifetime event that will also benefit Nvidia in that way. Nvidia will be successful in the future, but it will be through more normal, smart business, means.

[ryao]

Those are both computational challenges. Nvidia is well positioned for those due to their push into HPC with GPGPU. If there is another “once-in-a-lifetime” computational challenge, it will likely benefit Nvidia too.

[quesera]

I would never bet against the market finding a use for higher performance products.

~"Good fortune favors those who are prepared to take advantage of it", etc.

[singhrac]

I've been using Gaudi chips for a little bit and they are totally fine (and the software stack is even pretty good, or at least the happy path is mostly covered for me). For example I set up training with autocasting, activation checkpointing, fused ops, profiling etc., without too much trouble. I'll write a long blog post about it soon but I think their issue with the Gaudi chips is simply making enough and convincing people to buy them before Falcon Shores (which will, I think, be Xe slice based, so more like a better PVC chip than a Gaudi).

In summary the software story was very surprisingly better than I expected (no Jax though).

[notreallyauser]

> What made Intel seem unbeatable was its process node advantage. Nvidia does not have fabrication plants, so it is able to get the best process node from whoever has it. Nvidia is therefore not vulnerable to what befell Intel.

It's able to get the best process node from /whoever is willing to sell it to Nvidia/: it's vulnerable (however unlikely) to something very similar -- a competitor with a process advantage.

[bfrog]

Exactly this, hard to grok why people think somehow the fab is the boat anchor around Intel's neck. No, it was the golden goose that kept Intel ahead until it didn't.

BK failed to understand the moat Intel had was the Fab. The moat is now gone and so is the value.

[chrsw]

Intel didn't have a software stack moat.

[michaelt]

In 2000 Intel had a huge software moat: Microsoft Windows, and the large install base of x86-only software.

Rich webapps hadn't been invented. Smartphones? If you're lucky your flip phone might have a colour screen. If you've got money to burn, you can insert a PCMCIA card into your Compaq iPAQ and try out this new "802.11b" thing. Java was... being Java.

Almost all the software out there - especially if it had a GUI, and a lot of it did - was distributed as binaries that only ran on x86.

[gary_0]

So many devs are too young to remember a time before you would expect to just download some open source and compile it for x86/amd64/arm/emscripten/etc and be good to go. In the old days, if you didn't want to write that library code yourself, chances are all your AltaVista search would turn up was a guy selling a header file and a DLL and OCX[0] for $25. If you were lucky!

A vast amount of code was only intended to compile and run on a single OS and architecture (circa 2000, that was usually x86 Win32; Unix was dying and Wintel had taken over the world). If some code needed to be ported to another platform, it was as good as a from-scratch re-write.

[0] in case you wanted to use the thing in Visual Basic, which you very well might.

[chrsw]

>In 2000 Intel had a huge software moat

"had". That's what helped prop up their monopoly but it didn't last. These days if can't run your software on another architecture, like ARM, you can run at least on AMD. AMD can basically run the same software as Intel. This isn't the situation for NVIDIA vs everyone else, so far.

[acdha]

Other than the huge amount of enterprise software which was only supported on Intel, most of the high-end server business below the mainframe level after the mid-90s, and the huge install base of x86 software keeping everyone but AMD out? Even their own Itanium crashed and burned on x86 compatibility.

[ryao]

Then there were software libraries and the Intel C/C++ Compiler that favored Intel. They would place optimized code paths that only ran on Intel hardware in third party software. Intel has stopped doing that in recent years as far as I know (the MKL has Zen specific code paths), but that is a fairly recent change (maybe the past 5 years).

There were also ISA extensions. Even if Intel had trouble competing on existing code, they would often extend the ISA to gain a temporary advantage over their competitors by enabling developers to write more optimal code paths that would run only on Intel’s most recent CPUs. They have done less of that ever since the AVX-512 disaster, but Intel still is the one defining ISA extensions and it historically gained a short term advantage whenever it did.

Interestingly, the situation is somewhat inverted as of late given Intel’s failure to implement the AVX-512 family of extensions in consumer CPUs in a sane way, when AMD succeeded. Intel now is at a disadvantage to AMD because od its own ISA extension. They recently made AVX-10 to try to fix that, but it adds nothing that was not already in AVX-512, so AMD CPUs after Zen 3 would have equivalent code paths from AVX-512, even without implementing AVX-10.

[rasz]

>Intel C/C++ Compiler that favored Intel

Thats where Nvidia learned to "optimize" Cuda software path. Single threaded x87 FPU on SSE2 capable CPUs.

https://arstechnica.com/gaming/2010/07/did-nvidia-cripple-it...

https://www.realworldtech.com/physx87/3/ "For Nvidia, decreasing the baseline CPU performance by using x87 instructions and a single thread makes GPUs look better."

They doubled down that approach with 'GameWorks' crippling performance on non Nvidia GPUs, Nvidia paid studios for including GameWorks in their games.

[YetAnotherNick]

NVidia has software moat for specialized applications but not for AI, which is responsible for most of their sales now. Almost everyone in AI uses pytorch/jax/triton/flash attention and not CUDA directly. And if Google can support pytorch for their TPU and Apple for their M1 GPU, surely others could.

[david-gpu]

> NVidia has software moat for specialized applications but not for AI, which is responsible for most of their sales now. Almost everyone in AI uses pytorch/jax/triton/flash attention and not CUDA directly

And what does pytorch et al. use under the hood? cuBLAS and cuDNN, proprietary libraries written by NVidia. That is where most of the heavy lifting is done. If you think that replicating the functionality and performance that these libraries provide is easy, feel free to apply for a job at NVidia or their competitors. It is pretty well paid.

[YetAnotherNick]

Did you read the last part? Pytorch uses drivers, and drivers exists for Google's TPU and Apple's M1 GPU as well and both works pretty well. I have tested both and it reaches similar MFU as Nvidia.

[gajjanag]

Maybe on a particular model/dataset but extremely unlikely in general. Again, like another commenter pointed out: if you truly believe it isn't that hard we would love to hire you at Meta ;)

[YetAnotherNick]

Yes some operations are not supported in MPS/TPU and falls back to slower CPU. But for common architectures like transformers and convnets, it works very well for all the datasets.

I never claimed it was easy. I meant in my opinion it is in the order of 10s of millions dollars of investment, not a trillion dollar CUDA moat that people comment here.

[CharlieDigital]

Are M1 GPUs available for data center deployment at scale? Are Google TPUs available outside of Google? Can Amazon or Microsoft or other third parties deploy them?

Anyone that wants off the shelf parts at scale is going to turn to Nvidia.

[Wytwwww]

So Nvidia's moat is mainly hardware and not software?

[YetAnotherNick]

That's the point I am making. And the reason Amazon or Microsoft can't deploy them is the hardware, not CUDA.

[caycep]

Intel didn't do a lot of things...

[KennyBlanken]

The Pentium math bug, Puma cablemodems, their shitty cellular modems that are far worse than Qualcomm's, gigabit chipset issues, 2.5GB chipset issues, and now the 13th/14th gen CPUs that destroy themselves.

And we just gave them billions in tax dollars. Failing upwards...

[rasz]

Atom C2000 waves from behind all this crowd!

[brirec]

I still have five C2758 nodes running just fine, though!

[mensetmanusman]

Nvidia is helping power the tool that destroys the software moat.

[akira2501]

Interesting that it's entirely failed to do that so far.

Nvidia is helping power the next generation of big brother government programs.

[mensetmanusman]

Aren’t we like 0.000001% in the journey?

[akira2501]

That is the largest goalpost move I've ever seen, but sure, at that order of magnitude anything is plausible.

[tester756]

> and Broadcom was thinking about buying them...

I'm thinking about buying Nvidia

(this is bullshit)