[beebmam]

It seems deeply overvalued to me, as it's pretty likely that NVIDIA's competitors will have parallel computation hardware that competes well enough. I'm personally not a fan of NVIDIA's drivers or the reliability of their hardware.

[dave_sullivan]

> I'm personally not a fan of NVIDIA's drivers or the reliability of their hardware.

As compared to AMD or Intel? I wish there was real competition to Nvidia but there isn't. I'm not a fan of their defacto monopoly but they do have the best product on the market and their competition has been asleep for 10 years. AMD and Intel barely knew what deep learning was 10 years ago (and certainly did not appreciate the opportunity) and Nvidia was already investing heavily.

[TillE]

Ten years ago, Intel was the leader in chip fabrication technology. Things can change fast.

Intel has already been making good low-end GPUs (with lousy but rapidly-improving drivers). If they're smart, they'll keep at it.

[hatenberg]

Intel wasted the last decade because they elected a shareholder optimizing CEO who chose to prioritize shareholder payouts and attempts at controllig the ecosystem rather than pushing it forward over investments.

Now they're completely outclassed by TSMC and have to partner with UMC to compete.

[relativ575]

The point is that thing can change. Intel was king a decade ago. TSMC is on the summit at the moment. Intel can be back in a few years if their plan works out:

https://www.xda-developers.com/intel-roadmap-2025-explainer/

[HDThoreaun]

As compared to a hypothetical product that is better than whats available today. With literally a trillion dollars on the line I find it very difficult to believe no one will come and scoop this opportunity up. The real value in GPUs is the datacenter segment which largely didnt exist, certainly not in that state it is today before the LLM take off. Takes time to develop products but theyll arrive eventually.

[rezonant]

Well there's not really a trillion dollars on the line. This is a valuation, not revenue.

All this says is that AI and LLMs are extremely over hyped, and the market believes Nvidia's tech is the only viable supplier of the platform LLMs run on.

These are things we already knew, so it's not surprising the market is quadrupling what it thinks Nvidia is worth.

[HDThoreaun]

Yes there is a trillion dollars on the line here. Market cap is what matters to investors end of the day, everything else is just influencing it. Yes the market is saying Nvdia is the only supplier for AI and LLM hype, but its also saying they will continue to be the only supplier long term and that seems deeply flawed to me.

[rezonant]

While there's certainly a feedback cycle involved, market cap is decided by investors in the large. The stock price is just reflecting buys and sells of those investors. People believe AI has a lot of money and hype in it, and Nvidia supplies necessary equipment for AI, so they buy Nvidia. As a result the stock price goes up, and the market capitalization goes up accordingly.

As a thought experiment, imagine buying 50% of a company on the open market and then seeing the price go up accordingly as the market does, then saying it must be even more valuable than I thought! And buying the other half of the company at the higher price. You caused that "value" by buying.

No one investor has a trillion dollar opportunity, and no competitor does either. Making an assumption that a competitor will come along and zero out Nvidia because they have a better AI chip is not rational. For one the value of Nvidia isn't solely based on this, and by the time you've made your AI chip the market is going to have changed.

Considering how insanely inflated the public's expectations of what LLMs can do, despite the fact that they are only a mirage of intelligence, it would probably be foolhardy to build a new AI chip to replace them.

I suppose for AMD they could see their stock increased by such a margin if they could just produce a chip that convinced the market, but were they not already trying to do that?

[HenriTEL]

Yep, I remember 8 years ago Intel trying to sell us their CPU solutions for AI. It was hard to make things work and in the end performance were not there. We were comparing against consumer gpu like the GTX 1080.

[pas]

> Nvidia was already investing heavily

that sounds very interesting, can you link/share/describe some details on it? how much did they invest? how? into CUDA? what else?

[throw0101b]

> how much did they invest? how? into CUDA? what else?

CUDA 1.0 was released in 2007:

* https://insidehpc.com/2007/07/nvidia-releases-cuda-10/

* https://developer.nvidia.com/cuda-toolkit-archive

From SIGGRAPH 2007, "GPU computing with NVIDIA CUDA":

* https://dl.acm.org/doi/10.1145/1281500.1281647

"NVIDIA: The Era of the Personal Supercomputing":

* https://www.nvidia.com/content/events/siggraph_2007/supercom...

Before AI/ML was hot, and before even the Bitcoin paper was released. NVidia was investigating/experimenting/investing in the concept before there was any kind of 'killer app' for it.

[pjmlp]

And even better, NVidia understood not everyone wants to use plain old C for their GPGPU coding, and early on staring with CUDA 3.0 in 2010, introduced C++ support and PTX.

Later on they acquired PGI, which thanks to PTX, had C, C++ and Fortran compilers, thus adding Fortran into the mix.

Followed along by all the IDE, graphical debuggers tooling and library ecosystem.

Meanwhile Intel and AMD were doing who knows what at Khronos stuck in their "C is good enough" mentality, and barely released useful developer experiences.

[dave_sullivan]

My statements based on high level meetings I had at the time with all 3 companies when I had started an early neural network PaaS company and was looking for them to invest. Nvidia knew what they were talking about and were already moving that direction, Intel heard about deep learning somewhere but didn't believe there was anything there, and AMD didn't know anything about anything.

[wslh]

Seems like a tale already told on "Only The Paranoid Survive" by Andrew Grove. Now Jensen should add some chapters. BTW I just discovered that their web page/brand is fully invested in AI, the title is: "World Leader in Artificial Intelligence Computing".

I am curious about your PaaS because we were analyzing that business for fun first. A small thread here in HN: https://news.ycombinator.com/item?id=39329764

[dave_sullivan]

> I am curious about your PaaS because we were analyzing that business for fun first.

Here's an old tutorial video of the product: https://youtu.be/V2q9hVdi80w

We were doing cool things and on the cutting edge but ultimately couldn't make a business out of it and weren't talking to the right people.

[fnordpiglet]

I’m not sure I would bet an ads/spyware company with admittedly deep deep pockets would beat a specialized hyper focused company at its core business, especially as they have deeper pockets now with 30 years of experience in the sub sector. If Nvidia were wandering in the desert like Cisco I could believe it. But Nvidia isn’t, and I don’t believe Google, Amazon, or others will beat Nvidia at their own game. (Given my time in FAANG, I also speak from inside knowledge of how deeply f’ed these companies are - making your own arm chip or network adapter isn’t the same thing as taking on high end designers at their own game)

[microtherion]

It's interesting that you would name drop Cisco. They exploded in market valuation in 1999/2000, having a near-monopoly on infrastructure that was in high demand due to the Web boom. Nvidia similarly profited from the Crypto/AI boom, but I wonder whether that is bound to end similarly.

[fnordpiglet]

All things end, and Nvidia will probably decay at some point. But they’ve had a long and strong run.

[rgmerk]

Have you worked at a large (say 500+ employee) company that you would say isn't "deeply f'ed"?

I once read a fascinating corporate history of Xerox, and that company became deeply, deeply f'ed up in ways that are of their time but do have strong parallels to the issues I understand that FAANG, particularly Google, have.

[dehrmann]

> Have you worked at a large (say 500+ employee) company that you would say isn't "deeply f'ed"?

You get a weird insider bias where all you see are bug reports and problems, so you get the impression the product is shit, even if 99% of customers love it.

[eru]

> Have you worked at a large (say 500+ employee) company that you would say isn't "deeply f'ed"?

Perhaps Google until around 2010? Or Goldman Sachs until the 1990s?

[Horffupolde]

Could you please share any links or pointers for that? Sounds interesting.

[rgmerk]

One of them was called Xerox: American Samurai, and dates from 1986 (I read it when I was a teenager in the early 1990s, I think). There was another one from the early 1990s, I believe.

This book lauded Xerox's success at reforming its corporate culture, regaining a strong position in the photocopier market and even spent a chapter detailing their success in getting into electronic typewriters!

With the benefit of hindsight the company didn't survive its core product losing all relevance any better than Kodak did, but that was still some way in the future (if foreseeable by the 1980s).

That said, much of the material about a hugely bloated organisation, with a sclerotic bureaucracy and lots of cushy middle managers assembled through a previous period of explosive growth, turning out poor-quality product, sounds very reminiscent of some of what we now hear about the current big tech companies.

The title reflects another American obsession at the time - the idea that the US was "losing" to Japan.

[pb7]

>Given my time in FAANG, I also speak from inside knowledge of how deeply f’ed these companies are

Please tell us more of your expertise and deep insight, FAANG employee #1,908,680.

Google has developed their own chips. Apple has developed their own chips. It’s really not that hard if your pockets are deep enough or the bottom line checks out.

When Apple launches their AI offering this year, it’s not going to need NVIDIA.

[fnordpiglet]

They have, over many years and not against a company at the top of their game and with considerable help. Apples move is an ARM chip, not an Apple chip. Googles chips are competitive in spaces like network infrastructure. TPUs are illustrative of their inability to provide a viable alternative.

Apple isn’t going to launch with an Nvidia killing alternative, I’ll bet you $1,908,680 it’s backed by Nvidia.

They will likely use their neutral chips for local models, but their data center stuff will be 100% Nvidia.

[AnthonyMouse]

The interesting question isn't whose hardware they use for the launch, it's what the public-facing software API looks like. Apple isn't likely to directly expose CUDA. At which point they're free to swap out the hardware with whatever they want at any time.

Also, Apple has a longstanding dislike for Nvidia and even if they weren't going to design their own chips at launch, they could be using AMD.

[KaoruAoiShiho]

Do you have any knowledge of what it runs on? I was pretty sure it was NVIDIA (accessed through cloud providers as a customer).

[riku_iki]

TPU is a thing and well competitive already for a long time?..

[kccqzy]

TPUs are made primarily to satisfy Google's own needs (especially YouTube), not necessary what GCP customers need.

[fnordpiglet]

Except they come tied to GCP and don’t offer as high precision training and inference. These are pretty major disadvantages.

[tuyguntn]

don't underestimate market forces.

Same could have been told about Intel, but Apple anyway beat them in some ways and took away big market share

[AnthonyMouse]

AMD has taken more CPU market share from Intel than Apple.

But the weird thing about the "Nvidia will remain undefeated forever" theory is that it seems to assume they have some kind of permanent advantage.

Nvidia was well positioned to make an early investment in this because they had the right combination of existing technology and resources. Other companies would have had to invest more because they were starting from a different place (e.g. Microsoft), or didn't have the resources to invest at the time (AMD).

But now the market is proven and there are more than half a dozen 800 pound gorillas who all want a piece of it. It's like betting that Tesla will retain their 2022 market share in electric car market even as the market grows and everyone else gets in. Maybe some of the others will stumble, but all of them? Apple, AMD, Google, Intel, Microsoft, Amazon and Facebook?

[eru]

> It's like betting that Tesla will retain their 2022 market share in electric car market even as the market grows and everyone else gets in.

Yes. Especially once you take the Chinese electric car companies into account, that are already outselling Tesla.

[AnthonyMouse]

And even a Tesla optimist would presumably admit that maintaining a third of the EV market would be a huge win for Tesla, as the EV market becomes "the car market" going forward. Maybe that won't happen, but it's at least within the realm of possibility -- maybe some of the existing carmakers stick the transition and some of them fail, but Tesla remains the biggest one, that's not impossible.

But maintaining the 80% they had a few years ago, much less 100%? That's not optimism, it's fantasizing.

[eru]

Agreed.

Btw, I was talking about global electric car production. I don't know whether Tesla ever did 80% of global electric car sales?

[fnordpiglet]

Intel isn’t top of their game and haven’t been for many many years.

[AnthonyMouse]

Intel knows how to make software and libraries for their hardware, which is the thing people keep lamenting about AMD. Intel's current GPUs are mediocre but priced competitively for what they are, and Intel having more competitive hardware in the future is not implausible.

Which could lead to Intel realizing the opportunity they have. Create decent libraries that work across every vendor's GPUs. In the short term this helps AMD at the expense of Nvidia, which in itself helps Intel by preventing Nvidia from maintaining a moat. In the medium term Intel then has people using Intel's libraries to write code that will work on their future GPUs and then their problem is limited to producing competitive hardware.

[nalllar]

Their biggest competitor in the GPU space is AMD who have spent years chasing deadlock issues that won't stay fixed, playing whackamole, in between trying to do actual driver development. Following the amdgpu mailing list is not fun.

[nicce]

> I'm personally not a fan of NVIDIA's drivers

A bit funny thing is that they are essentially software company which focuses on software quality, if you look at the stats how many people are working with software over there. And still it is not good enough?

[stubish]

I'd welcome a lot fewer software developers, so I get drivers rather than the bloated driver lifestyle synergy experience with cloud mess. So many companies spend so much money developing software that alienates and aggravates their customers. I blame HP, who by my recollection started it with printer drivers that required a custom installer and a CD.

[rgmerk]

Not having done any GPU programming...is most of the code out there tied to the Nvidia architecture? I mean, if AMD or somebody else builds a better mousetrap, how much work would it be to switch?

[gitgud]

A lot of projects have trouble moving off of Nvidia's proprietary CUDA platform. An example is this [1] ML repo, the issue has been open for years...

[1] https://github.com/alicevision/AliceVision/issues/439

[cowsandmilk]

Most of the AI and ML related programming uses frameworks that abstract away what brand of GPU you are using. They could care less whether it is nvidia or AMD.

[latchkey]

This is true for making things just work, however to really squeeze performance out of a GPU, you need to go lower level and that is tied to the architecture.

[virtue3]

This has happened before and it will probably go the same way. Software and compilers will make up the difference, or hardware will become so cheap and ubiquitous it wont super matter.

In 3-5 years what will a 10% performance difference matter to you? Then calculate how much that 10% performance difference is going to cost in real dollars to run on nvidia hw and then the fun math should start.

[latchkey]

Performance for GPUs isn't just speed, but also power efficiency. The complexity of GPUs doesn't lend itself to just being solved with better tooling. They are also not going to get cheaper... especially the high end ones with tons of HBM3 memory.

Given that data centers only have so much power and AI really needs to be in the same data center as the data, if you can squeeze out a bit more power efficiency so you can fit more cards, you are getting gains there as well.

When I was mining ethereum, the guy who wrote the mining software used an oscilloscope to squeeze an an extra 5-10% out of our cards and that was after having used them for years. That translated to saving about 1 MW of power across all of our data centers.

Let me also remind you that GPUs are silicon snowflakes. No two perform exactly the same. They all require very specific individual tuning to get the best performance out of them. This tuning is not even at the software level, but actual changes to voltage/memory timings/clock speeds.

[eru]

You are right to worry about power efficiency. Though do keep in mind that power is also fungible with money, especially in a data centre.

I suspect a lot of AI inference (thought probably not the majority) will happen on mobile devices in the future. There power is also at a premium, and less fungible with money.

[spenczar5]

We are so, so, so far away from compilers that could automatically help you, say, rewrite an operation to achieve high warp occupancy. These are not trivial performance optimizations - sometimes the algorithm itself fundamentally changes when you target the CUDA runtime, because of complexities in the scheduler and memory subsystems.

I think there is no way that you will see compilers that advanced within 3 years, sadly.

[ric2b]

And yet I see tons of stuff requiring CUDA.

[dheera]

except AMD produces nothing remotely close to the compute capability of an H100. they only compete at the gaming card level.

[edward28]

They released the MI300 which beats a H100 but is close to a H200.

[saberience]

Actually, this hasn't been shown yet. AMD showed some "on paper" specs which could "theoretically" be faster than a H100, but they didn't show any practical tests which could be recreated by third parties. Also, some of the tests AMD ran on H100's were deliberately not using the correctly optimized software, massively slowing down the H100 performance...

As a result there is a lot of sceptism as to whether it's actually faster in real world scenarios. There are a few articles explaining this situation. Here is one: https://www.forbes.com/sites/karlfreund/2023/12/13/breaking-...

[teaearlgraycold]

Google is actually working on 1st party datacenter chips. Although I doubt those would be user-facing GPUs.

[Laremere]

Google Pixels have a "neural core", and they have edge TPUs in addition to their data center TPUs. However Google hardware seems much less poised to take immediate advantage of the AI gold rush.

[jeffbee]

The last decade of Google datacenter engineering can be viewed as an elaborate plan to avoid paying either Intel or Nvidia any more money than was really necessary.

[fnordpiglet]

And, yet.

[what_ever]

Yet, what?

[kccqzy]

Yet, Google's customers still demand Google to pay Nvidia exorbitantly.

[teaearlgraycold]

Google has plenty of first party servers.

[fnordpiglet]

And intel.

[ein0p]

Nobody other than Anthropic and a few other large cloud customers like that really cares what Google is working on. They bet the farm on a DL framework (Jax) with 2% market share. That’s a very deep hole to climb out of, particularly considering the popularity of PyTorch in generative AI space.

[eru]

Feel free to short sell NVIDIA?