[smokefoot]

I mean, I don’t know how long the NVIDIA moats can hold. With this much money at stake, others will challenge their dominance especially in a market as diverse and fragmented as advanced semiconductors.

That’s not to say I’m brave enough to short NVDA.

[mark_l_watson]

I think that NVIDIA’s moat is the US government. Remember our government’s efforts to prevent the use of Huawei cell infrastructure in Europe and around the world?

I am a long time fan of Dave Sacks and the All In podcast ‘besties’ but now that he is ‘AI czar’ for our government it is interesting what he does not talk about. For example on a recent podcast he was pumping up AI as a long term solution to US economic woes, but a week before that podcast, a well known study was released that showed that 95% of new LLM/AI corporate projects were fails. Another thing that he swept under the rug was the recent Stanford study that 80% of US startups are saving money using less expensive Chinese (and Mistral, and Google Gemma??) models. When the Stanford study was released, I watched All In material for a few weeks, expecting David Sack’s take on the study. Not a word from him.

Apologies for this off-topic rant but I am really concerned how my country is spending resources on AI infrastructure. I think this is a massive bubble, but I am not sure how catastrophic the bubble will be.

[heavyset_go]

> Remember our government’s efforts to prevent the use of Huawei cell infrastructure in Europe and around the world?

The US is burning good will at an alarming rate, how long will countries keep paying a premium to be spied on by the US instead of China?

[mark_l_watson]

I think the answer to your question is ‘not for very long.’ I frequently have breakfast with a friend who is a retired math professor and he is an avid investor in the stock market. We talk a lot about how long the US stock market will keep increasing in value. We don’t know the answer about the stock market, but it is fun to talk about. We both want to start easing out of the stock market.

[rsynnott]

The main competitors to Huawei in cell network stuff are mostly European (Nokia and friends), not American.

[ivape]

They are heavy into AI investing but will tell people AI startups are just toy apps (Chamath). That podcast is full of crooks. I’d be willing to give them a pass as bunch of old white guy techies that just love to talk about tech, but they are literally at the dinner table with Trump and Musk.

This country used to have congressional hearings on all kinds of matters from baseball to the Mafia. Tech collusion and insider knowledge is not getting investigated. The All-in podcast requires serious investigation, with question #1 being “how the fuck did you guys manage to influence the White House?”.

Other notes:

- Many of them are technically illiterate

- They will speak in business talk , you won’t find a hint of intimate technical knowledge

- The more you watch it, the more you realize that money absolutely buys a seat at the table:

https://bloximages.chicago2.vip.townnews.com/goskagit.com/co...

(^ Saved myself another thousand words)

[hbarka]

Remember that time in history when Chamath thought he found gold in SPACs. Hubris is easily forgotten or forgiven.

[anonymousDan]

You say 95% failed like it's a bad thing - a 5% success rate sounds reasonable to me in terms of startups!

[GoatInGrey]

It's not startup success rate, it's application of the technology at companies. Meaning that 95% of the time that AI is applied to a work problem, it fails to generate material value over existing methods.

[nikkwong]

Sacks has always been absolutely disingenuous and interested in pedaling his own interests over the interests of the common good. As a total Trump shill he talks out of both sides of his mouth at the same time & accuses the left of things that he has no problem with when he or his own party does it.

Anyone who's listened to him (even those who align with him politically) for an extended period of time can't help but to notice so obviously so self interested to the point of total hypocrisy—the examples of which are too many to begin to even wanting to enumerate. Like—take the Trump/Epstein stuff, or the Elon/Trump fallout—topics he would absolutely lose his sh*t over if these were characters on the left. I find it hard to believe anyone actually ever took him seriously. Branding myself as a fan of his would just be a completely self-humiliating insult to my intelligence and my conscience IMO.

[giancarlostoro]

> a week before that podcast, a well known study was released that showed that 95% of new LLM/AI corporate projects were fails.

I mean. I think some of us knew this. There's a lot of issues with AI, some psychological, some are risk adverse individuals who would love to save hours, weeks, months, maybe years of time with AI, but if AI screws up, its bad, really bad, legal hell bad, unless you have a model with a 100% success rate for the task, it wont be used in certain fields.

I think in the more creative fields its very useful, since hallucinations are okay, its when you try to get realistic / look reasonably realistic (in the case of cartoons) that it gets iffy. Even so though, who wants to pay the true cost of AI? There's a big uphill cost involved.

It reminds me a lot of crypto mining, mostly because you need an insane amount to invest into before you become profitable.

[dworks]

"Your margin is my opportunity" as someone said. Certainly Google must have plans to sell its chips externally with this much up for grabs?

[heavyset_go]

They make more money using them themselves or renting out their time to others.

[mark_l_watson]

I was also wondering if Google would try to make profit from selling TPUs, but they probably won’t because:

At least for me, Google has some real cachet and deserves kudos for not losing money selling Gemini services, at least I think it is plausible that they are already profitable, or soon will be. In the US, I get the impression that everyone else is burning money to get market share, but if I am wrong I would enjoy seeing evidence to the contrary. I suspect that Microsoft might be doing OK because of selling access to their infrastructure (just like Google).

[alephnerd]

There's no point selling TPUs when you can bundle TPU access as part of much more profitable training services. The margins are much higher providing a service as part of GCP versus selling.

[mark_l_watson]

I agree. Amazon and I think Microsoft are also working on their own NVIDIA replacement chips - it will be interesting to see if any companies start selling chips, or stick with services.

[alephnerd]

From what I'm hearing in my network, the name of the game is custom chips hyperoptimized for your own workloads.

A major reason Deepseek was so successful margins wise was because the team heavily understood Nvidia, CUDA, and Linux internals.

If you have an understanding of the intricacies of your custom ASIC's architecture, it's easier for you to solve perf issues, parallelize, and debug problems.

And then you can make up the cost by selling inference as a service.

> Amazon and I think Microsoft are also working on their own NVIDIA replacement chips

Not just them. I know of at least 4-5 other similar initiatives (some public like OpenAI's, another which is being contracted by a large nation, and a couple others which haven't been announced yet so I can't divulge).

Contract ASIC and GPU design is booming, and Broadcom, Marvell, HPE, Nvidia, and others are cashing in on it.

[coredog64]

I wouldn't be surprised if a fair portion of Amazon's Bedrock traffic is being served by Inferentia silicon. Their margins on Anthropic models are razor thin and there's a lot of traffic, so there's definitely an incentive. Additionally, every model that's served by Inferentia frees up Nvidia capacity for either models that can't be so served or for selling to customers.

[Mistletoe]

Do you have a link or references showing Google isn’t losing money on Gemini?

[mark_l_watson]

Earning report does not break out profit from Gemini separately, but this is still useful https://abc.xyz/assets/34/fa/ee06f3de4338b99acffc5c229d9f/20...

A long time ago I worked as a contractor at Google, and that experience taught me that they don’t like things that don’t scale or are inefficient.

[brazukadev]

That's the same as saying that Google is winning the AI race because they don't like losing. They won't win anything if we are in a bubble that burst tho

[GoatInGrey]

A hypothetical AI bubble bursting doesn't mean that every single AI vendor fails completely. Like the Dot-Com Bubble, the market value drops precipitously and many companies fold, but because the market value does not fall to zero, the survivors (i.e. Amazon) still win.

[hiddencost]

Fabrication is the bottle neck. They can't even meet internal demand.

[mrktf]

As long as only TMSC is only top performance chip producer and it is possible to reserve all it manufacturing capacity for one two clients the NVIDIA will hold without problem...

My opinion, the problems for NVIDIA will start when China ramp up internal chip manufacturing performance enough to be in same order of magnitude as TMSC.

[impossiblefork]

But all sorts of people get their things fabbed by TSMC.

Cerebras get their chipped fabbed by them. I assume Eucyld will have their chips fabbed by them.

If there's orders, why would they prefer NVIDIA? Customer diversity is good, is it not?

[nebula8804]

TSMC and NVIDIA's relationship has gone back for more than 20 years. In the NVIDIA biography they talk about how TSMC really helped NVIDIA out early on when other suppliers just couldn't meet the quality and rate demands that NVIDIA aspired to. That has led to a strong relationship where both sides have really helped each other out.

[impossiblefork]

Yes, but other are still getting chips from them. I think it's just a matter of having enough demand.

[re-thc]

> If there's orders, why would they prefer NVIDIA? Customer diversity is good, is it not?

Money talks. Apple asked for first dips a while earlier (exclusively).

[impossiblefork]

But other people are literally getting their things fabbed by them.

AMD are, Cerebras are, I assume OpenChip's and Euclyd's machines will be.

[re-thc]

> But other people are literally getting their things fabbed by them.

Sure, but in my example Apple got access exclusively for a few months to a newer node, which would make a world of difference if you compete in the same space.

[user34283]

I'm not knowledgeable about this, but I wonder how important performance really is here.

Wont it be enough to just solder on a large amount of high bandwidth memory and produce these cards relatively cheaply?

[alephnerd]

> but I wonder how important performance really is here.

Perf is important, but ime American MLEs are less likely to investigate GPU and OS internals to get maximum perf, and just throw money at the problem.

> solder on a large amount of high bandwidth memory and produce these cards relatively cheaply

HBM is somewhat limited in China as well. CXMT is around 3-4 years behind other HBM vendors.

That said, you don't need the latest and most performant GPUs if you can tune older GPUs and parallelize training at a large scale.

-----------

IMO, Model training is an embarrassingly parallel problem, and a large enough cluster leveraging 1-2 generation older architectures that is heavily tuned should be able to provide similar performance to train models.

This is why I bemoan America's failures at OS internals and systems education. You have entire generations of "ML Engineers" and researchers in the US who don't know their way around CUDA or Infiniband optimization or the ins-and-outs of the Linux kernel.

They're just boffins who like math and using wrappers.

That said, I'd be cautious to trust a press release or secondhand report from CCTV, especially after the Kirin 9000 saga and SMIC.

But arguably, it doesn't matter - even if Alibaba's system isn't comparably performant to an H20, if it can be manufactured at scale without eating Nvidia's margins, it's good enough.

[TylerE]

Isn’t memory production relatively limited also?

[TSiege]

They are currently doing this. It’s part of their Made in China 2025 plan

[StopDisinfo910]

> That’s not to say I’m brave enough to short NVDA.

Their multiples don't seem sustainable so they are likely to fall at some point but when is tricky.

[re-thc]

> Their multiples don't seem sustainable so they are likely to fall at some point but when is tricky.

They've been trying really hard to pivot and find new growth areas. They've taken their "inflated" stock price as capital to invest in many other companies. If at least some of these bets pay off it's not so bad.

[xbmcuser]

google has already started offering its TPUs to other neocloud providers

[xnx]

I hadn't heard that. Source?

[xbmcuser]

https://www.datacenterdynamics.com/en/news/google-offers-its...

[xnx]

Interesting. I read that as Google is using colocation to host its TPUs. I don't think Google is selling its TPUs like Nvidia sells H100s.